2023-11-13T10:34:49.038 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-13T10:34:49.039 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203 --owner mchangir --name mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} -- /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/orig.config.yaml 2023-11-13T10:34:49.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T10:34:49.154 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203 branch: wip-mchangir-testing-20231110.052303 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} email: mchangir@redhat.com first_in_suite: false job_id: '7456203' kernel: flavor: default kdb: true sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-20231110.052303 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 35f0debb3cc28955b7132ef4c509cd138a09d56b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 35f0debb3cc28955b7132ef4c509cd138a09d56b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-20231110.052303 sha1: 35f0debb3cc28955b7132ef4c509cd138a09d56b owner: mchangir priority: 1000 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9746 sha1: 35f0debb3cc28955b7132ef4c509cd138a09d56b sleep_before_teardown: 0 subset: 768/1024 suite: fs suite_branch: wip-mchangir-testing-20231110.052303 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_35f0debb3cc28955b7132ef4c509cd138a09d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 35f0debb3cc28955b7132ef4c509cd138a09d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmDbJx+U9GmeFR4wwiA9x34wyOtfCZEFig5NaMF0vaUMTTjcj/SfrLykbTjtRWCfPw3oIGybvtHApcHSzVhUZg= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJpsKYQqUYDjhRd5uNMPwBwJ509eIRR2VHJ+32ddcRdjFkfLmbJY/0edFsI/2jlUR0AIUdrZhPsuw93QGjPHEZY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-13_10:27:15 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714 2023-11-13T10:34:49.154 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_35f0debb3cc28955b7132ef4c509cd138a09d56b/qa; will attempt to use it 2023-11-13T10:34:49.155 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_35f0debb3cc28955b7132ef4c509cd138a09d56b/qa/tasks 2023-11-13T10:34:49.156 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-13T10:34:49.157 INFO:teuthology.task.internal:Checking packages... 2023-11-13T10:34:49.178 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '35f0debb3cc28955b7132ef4c509cd138a09d56b' 2023-11-13T10:34:49.178 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-13T10:34:49.179 INFO:teuthology.packaging:ref: None 2023-11-13T10:34:49.179 INFO:teuthology.packaging:tag: None 2023-11-13T10:34:49.179 INFO:teuthology.packaging:branch: wip-mchangir-testing-20231110.052303 2023-11-13T10:34:49.180 INFO:teuthology.packaging:sha1: 35f0debb3cc28955b7132ef4c509cd138a09d56b 2023-11-13T10:34:49.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-20231110.052303 2023-11-13T10:34:49.316 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7288.g35f0debb 2023-11-13T10:34:49.317 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-13T10:34:49.323 INFO:teuthology.task.internal:no buildpackages task found 2023-11-13T10:34:49.323 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-13T10:34:49.338 INFO:teuthology.task.internal:Saving configuration 2023-11-13T10:34:49.352 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-13T10:34:49.357 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-13T10:34:49.387 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 10:28:47.875774', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmDbJx+U9GmeFR4wwiA9x34wyOtfCZEFig5NaMF0vaUMTTjcj/SfrLykbTjtRWCfPw3oIGybvtHApcHSzVhUZg='} 2023-11-13T10:34:49.410 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 10:28:47.874354', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJpsKYQqUYDjhRd5uNMPwBwJ509eIRR2VHJ+32ddcRdjFkfLmbJY/0edFsI/2jlUR0AIUdrZhPsuw93QGjPHEZY='} 2023-11-13T10:34:49.410 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-13T10:34:49.416 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-13T10:34:49.417 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-13T10:34:49.417 INFO:teuthology.run_tasks:Running task console_log... 2023-11-13T10:34:49.522 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8c43e08b80>, signals=[15]) 2023-11-13T10:34:49.523 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-13T10:34:49.529 INFO:teuthology.task.internal:Opening connections... 2023-11-13T10:34:49.530 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T10:34:49.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:34:49.630 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T10:34:49.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:34:49.714 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-13T10:34:49.721 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-11-13T10:34:49.739 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-11-13T10:34:49.739 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-11-13T10:34:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2023-11-13T10:34:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2023-11-13T10:34:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2023-11-13T10:34:49.795 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2023-11-13T10:34:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2023-11-13T10:34:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2023-11-13T10:34:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-13T10:34:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-11-13T10:34:49.796 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2023-11-13T10:34:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-13T10:34:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2023-11-13T10:34:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T10:34:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-13T10:34:49.797 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T10:34:49.798 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-11-13T10:34:49.822 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2023-11-13T10:34:49.839 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2023-11-13T10:34:49.840 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2023-11-13T10:34:49.896 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2023-11-13T10:34:49.896 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2023-11-13T10:34:49.896 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2023-11-13T10:34:49.897 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2023-11-13T10:34:49.897 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2023-11-13T10:34:49.897 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2023-11-13T10:34:49.897 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-13T10:34:49.897 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2023-11-13T10:34:49.897 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2023-11-13T10:34:49.898 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-13T10:34:49.898 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2023-11-13T10:34:49.898 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T10:34:49.898 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-13T10:34:49.898 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T10:34:49.899 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2023-11-13T10:34:49.922 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-13T10:34:49.930 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-13T10:34:49.935 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-13T10:34:49.936 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2023-11-13T10:34:49.938 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2023-11-13T10:34:49.953 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-13T10:34:49.959 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-13T10:34:49.960 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2023-11-13T10:34:49.995 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2023-11-13T10:34:50.021 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-13T10:34:50.078 INFO:teuthology.run_tasks:Running task kernel... 2023-11-13T10:34:50.092 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'} 2023-11-13T10:34:50.092 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}}, timeout 300 2023-11-13T10:34:50.093 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-13T10:34:50.093 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-13T10:34:50.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T10:34:50.109 DEBUG:teuthology.task.kernel:sha1 for mon.b is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-13T10:34:50.110 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-13T10:34:50.110 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2023-11-13T10:34:50.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T10:34:50.111 DEBUG:teuthology.task.kernel:sha1 for mon.a is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-13T10:34:50.111 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-13T10:34:50.112 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-11-13T10:34:50.165 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-361.el9.x86_64 2023-11-13T10:34:50.165 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-13T10:34:50.166 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-13T10:34:50.166 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-13T10:34:50.169 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-361.el9.x86_64 2023-11-13T10:34:50.170 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-13T10:34:50.170 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-13T10:34:50.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-13T10:34:50.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum upgrade -y linux-firmware 2023-11-13T10:34:50.532 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum upgrade -y linux-firmware 2023-11-13T10:34:52.343 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 16 MB/s | 6.3 MB 00:00 2023-11-13T10:34:52.702 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-11-13T10:34:53.693 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 8.3 MB/s | 2.0 MB 00:00 2023-11-13T10:34:54.302 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2023-11-13T10:34:54.751 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 7.9 MB 00:00 2023-11-13T10:34:57.901 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 7.9 MB 00:02 2023-11-13T10:35:02.918 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 18 MB 00:06 2023-11-13T10:35:05.343 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 2.8 MB/s | 18 MB 00:06 2023-11-13T10:35:07.827 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 5.8 MB 00:01 2023-11-13T10:35:09.135 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 14 MB/s | 5.8 MB 00:00 2023-11-13T10:35:09.462 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2023-11-13T10:35:10.669 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2023-11-13T10:35:11.614 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 19 MB 00:02 2023-11-13T10:35:13.731 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 6.7 MB/s | 19 MB 00:02 2023-11-13T10:35:16.495 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2023-11-13T10:35:18.445 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-11-13T10:35:18.992 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-11-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-11-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-11-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-11-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-11-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi016.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-11-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-11-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-11-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 1 Package 2023-11-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 326 M 2023-11-13T10:35:19.004 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-11-13T10:35:20.757 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2023-11-13T10:35:20.768 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2023-11-13T10:35:20.768 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2023-11-13T10:35:20.769 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2023-11-13T10:35:20.769 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2023-11-13T10:35:20.769 INFO:teuthology.orchestra.run.smithi184.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-11-13T10:35:20.770 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:35:20.770 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2023-11-13T10:35:20.770 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2023-11-13T10:35:20.771 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 1 Package 2023-11-13T10:35:20.771 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:35:20.771 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 326 M 2023-11-13T10:35:20.772 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2023-11-13T10:35:22.721 INFO:teuthology.orchestra.run.smithi016.stdout:linux-firmware-20230926-140.el9.noarch.rpm 88 MB/s | 326 MB 00:03 2023-11-13T10:35:22.722 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-11-13T10:35:22.723 INFO:teuthology.orchestra.run.smithi016.stdout:Total 88 MB/s | 326 MB 00:03 2023-11-13T10:35:22.723 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-11-13T10:35:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-11-13T10:35:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-11-13T10:35:24.454 INFO:teuthology.orchestra.run.smithi184.stdout:linux-firmware-20230926-140.el9.noarch.rpm 88 MB/s | 326 MB 00:03 2023-11-13T10:35:24.455 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2023-11-13T10:35:24.456 INFO:teuthology.orchestra.run.smithi184.stdout:Total 88 MB/s | 326 MB 00:03 2023-11-13T10:35:24.456 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2023-11-13T10:35:24.555 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2023-11-13T10:35:24.555 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2023-11-13T10:35:24.989 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-11-13T10:35:24.990 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-11-13T10:35:26.636 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2023-11-13T10:35:26.637 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2023-11-13T10:35:31.137 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-11-13T10:35:31.417 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-13T10:35:31.513 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-13T10:35:32.341 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2023-11-13T10:35:32.504 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-13T10:35:32.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-13T10:35:32.731 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-13T10:35:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-11-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi016.stdout: linux-firmware-20230926-140.el9.noarch 2023-11-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-11-13T10:35:33.362 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on mon.a... 2023-11-13T10:35:33.362 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-13T10:35:33.509 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ 2023-11-13T10:35:33.509 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-13T10:35:33.513 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi016.front.sepia.ceph.com to complete... 2023-11-13T10:35:33.801 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-13T10:35:33.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-13T10:35:34.174 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-13T10:35:34.175 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:35:34.175 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2023-11-13T10:35:34.175 INFO:teuthology.orchestra.run.smithi184.stdout: linux-firmware-20230926-140.el9.noarch 2023-11-13T10:35:34.175 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:35:34.175 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2023-11-13T10:35:34.471 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on mon.b... 2023-11-13T10:35:34.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-13T10:35:34.846 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ 2023-11-13T10:35:34.846 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-13T10:35:34.849 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi184.front.sepia.ceph.com to complete... 2023-11-13T10:37:29.791 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13 10:37:29 URL:https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [320585487/320585487] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-13T10:37:29.792 INFO:teuthology.orchestra.run.smithi184.stderr:FINISHED --2023-11-13 10:37:29-- 2023-11-13T10:37:29.792 INFO:teuthology.orchestra.run.smithi184.stderr:Total wall clock time: 1m 55s 2023-11-13T10:37:29.792 INFO:teuthology.orchestra.run.smithi184.stderr:Downloaded: 1 files, 306M in 1m 54s (2.67 MB/s) 2023-11-13T10:37:29.794 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on mon.b... 2023-11-13T10:37:29.794 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-13T10:37:31.647 INFO:teuthology.orchestra.run.smithi184.stdout:Verifying... ######################################## 2023-11-13T10:37:31.709 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing... ######################################## 2023-11-13T10:37:31.789 INFO:teuthology.orchestra.run.smithi184.stdout:Updating / installing... 2023-11-13T10:37:38.923 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-13T10:37:52.908 INFO:teuthology.orchestra.run.smithi184.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-13T10:38:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13 10:38:13 URL:https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [320585487/320585487] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-13T10:38:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:FINISHED --2023-11-13 10:38:13-- 2023-11-13T10:38:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:Total wall clock time: 2m 40s 2023-11-13T10:38:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:Downloaded: 1 files, 306M in 2m 39s (1.92 MB/s) 2023-11-13T10:38:13.165 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on mon.a... 2023-11-13T10:38:13.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-13T10:38:14.060 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-13T10:38:14.060 DEBUG:teuthology.orchestra.run.smithi184:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-13T10:38:14.131 INFO:teuthology.orchestra.run.smithi184.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:14.131 INFO:teuthology.orchestra.run.smithi184.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:14.131 INFO:teuthology.orchestra.run.smithi184.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:14.131 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:14.132 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-13T10:38:14.132 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-13T10:38:14.132 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-13T10:38:14.132 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-13T10:38:14.132 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-13T10:38:14.133 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-13T10:38:14.133 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-13T10:38:14.133 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-13T10:38:14.133 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-13T10:38:14.133 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-13T10:38:14.134 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-13T10:38:14.134 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-13T10:38:14.134 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-13T10:38:14.134 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-13T10:38:14.134 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-13T10:38:14.135 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-13T10:38:14.136 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-13T10:38:14.136 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-13T10:38:14.136 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-13T10:38:14.136 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-13T10:38:14.136 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-13T10:38:14.137 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-13T10:38:14.137 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-13T10:38:14.137 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-13T10:38:14.137 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-13T10:38:14.138 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-13T10:38:14.138 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-13T10:38:14.138 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-13T10:38:14.138 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-13T10:38:14.138 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-13T10:38:14.139 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-13T10:38:14.139 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-13T10:38:14.139 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-13T10:38:14.139 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-13T10:38:14.139 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-13T10:38:14.140 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-13T10:38:14.140 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-13T10:38:14.140 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-13T10:38:14.140 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-13T10:38:14.140 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-13T10:38:14.141 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-13T10:38:14.141 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-13T10:38:14.141 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-13T10:38:14.141 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-13T10:38:14.141 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-13T10:38:14.142 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-13T10:38:14.142 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-13T10:38:14.142 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-13T10:38:14.142 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-13T10:38:14.142 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-13T10:38:14.143 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-13T10:38:14.143 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-13T10:38:14.143 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-13T10:38:14.143 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-13T10:38:14.143 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-13T10:38:14.144 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-13T10:38:14.144 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-13T10:38:14.144 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-13T10:38:14.144 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-13T10:38:14.144 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-13T10:38:14.145 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-13T10:38:14.145 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-13T10:38:14.145 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-13T10:38:14.145 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-13T10:38:14.146 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-13T10:38:14.146 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-13T10:38:14.146 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-13T10:38:14.146 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-13T10:38:14.146 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-13T10:38:14.147 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-13T10:38:14.147 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-13T10:38:14.147 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-13T10:38:14.147 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-13T10:38:14.147 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-13T10:38:14.148 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-13T10:38:14.148 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-13T10:38:14.148 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-13T10:38:14.148 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-13T10:38:14.148 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-13T10:38:14.149 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-13T10:38:14.149 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-13T10:38:14.149 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-13T10:38:14.149 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-13T10:38:14.149 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-13T10:38:14.150 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-13T10:38:14.150 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-13T10:38:14.150 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-13T10:38:14.150 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-13T10:38:14.151 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-13T10:38:14.151 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-13T10:38:14.151 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-13T10:38:14.151 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-13T10:38:14.152 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-13T10:38:14.152 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-13T10:38:14.152 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-13T10:38:14.152 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-13T10:38:14.152 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-13T10:38:14.153 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-13T10:38:14.153 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-13T10:38:14.153 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-13T10:38:14.153 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-13T10:38:14.153 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-13T10:38:14.154 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-13T10:38:14.154 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-13T10:38:14.154 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-13T10:38:14.154 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-13T10:38:14.154 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-13T10:38:14.155 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-13T10:38:14.155 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-13T10:38:14.155 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-13T10:38:14.155 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-13T10:38:14.156 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-13T10:38:14.156 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-13T10:38:14.156 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-13T10:38:14.156 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-13T10:38:14.156 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-13T10:38:14.157 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-13T10:38:14.157 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-13T10:38:14.157 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-13T10:38:14.157 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-13T10:38:14.157 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-13T10:38:14.158 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-13T10:38:14.158 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-13T10:38:14.158 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-13T10:38:14.158 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-13T10:38:14.158 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-13T10:38:14.159 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-13T10:38:14.159 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-13T10:38:14.159 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-13T10:38:14.159 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-13T10:38:14.159 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-13T10:38:14.160 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-13T10:38:14.160 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-13T10:38:14.160 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-13T10:38:14.160 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-13T10:38:14.160 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-13T10:38:14.161 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-13T10:38:14.161 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-13T10:38:14.161 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-13T10:38:14.161 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-13T10:38:14.161 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-13T10:38:14.162 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-13T10:38:14.162 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-13T10:38:14.162 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-13T10:38:14.162 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-13T10:38:14.162 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-13T10:38:14.162 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-13T10:38:14.163 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-13T10:38:14.163 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-13T10:38:14.163 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-13T10:38:14.163 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-13T10:38:14.165 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-13T10:38:14.165 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-13T10:38:14.165 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-13T10:38:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-13T10:38:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-13T10:38:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-13T10:38:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-13T10:38:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-13T10:38:14.167 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-13T10:38:14.167 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-13T10:38:14.167 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-13T10:38:14.167 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-13T10:38:14.167 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-13T10:38:14.168 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-13T10:38:14.168 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-13T10:38:14.168 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-13T10:38:14.168 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-13T10:38:14.168 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-13T10:38:14.169 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-13T10:38:14.169 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-13T10:38:14.169 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-13T10:38:14.169 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-13T10:38:14.169 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-13T10:38:14.170 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-13T10:38:14.170 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-13T10:38:14.170 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-13T10:38:14.170 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-13T10:38:14.170 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-13T10:38:14.171 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-13T10:38:14.171 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-13T10:38:14.171 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-13T10:38:14.171 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-13T10:38:14.171 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-13T10:38:14.172 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-13T10:38:14.172 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-13T10:38:14.172 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-13T10:38:14.172 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-13T10:38:14.173 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-13T10:38:14.173 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-13T10:38:14.173 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-13T10:38:14.173 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-13T10:38:14.173 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-13T10:38:14.174 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-13T10:38:14.174 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-13T10:38:14.174 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-13T10:38:14.174 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-13T10:38:14.174 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-13T10:38:14.175 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-13T10:38:14.175 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-13T10:38:14.175 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-13T10:38:14.175 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-13T10:38:14.175 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-13T10:38:14.176 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-13T10:38:14.176 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-13T10:38:14.176 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-13T10:38:14.176 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-13T10:38:14.176 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-13T10:38:14.177 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-13T10:38:14.177 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-13T10:38:14.177 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-13T10:38:14.177 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-13T10:38:14.177 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-13T10:38:14.178 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-13T10:38:14.178 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-13T10:38:14.178 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-13T10:38:14.178 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-13T10:38:14.178 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-13T10:38:14.179 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-13T10:38:14.179 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-13T10:38:14.179 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-13T10:38:14.179 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-13T10:38:14.179 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-13T10:38:14.180 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-13T10:38:14.180 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-13T10:38:14.180 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-13T10:38:14.180 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-13T10:38:14.180 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-13T10:38:14.181 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-13T10:38:14.181 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-13T10:38:14.181 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-13T10:38:14.181 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-13T10:38:14.181 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-13T10:38:14.182 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-13T10:38:14.182 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-13T10:38:14.182 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-13T10:38:14.182 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-13T10:38:14.182 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-13T10:38:14.183 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-13T10:38:14.183 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-13T10:38:14.183 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-13T10:38:14.183 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-13T10:38:14.183 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-13T10:38:14.184 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-13T10:38:14.184 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-13T10:38:14.184 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-13T10:38:14.184 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-13T10:38:14.184 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-13T10:38:14.184 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-13T10:38:14.185 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-13T10:38:14.185 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-13T10:38:14.185 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-13T10:38:14.185 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-13T10:38:14.185 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-13T10:38:14.186 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-13T10:38:14.186 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-13T10:38:14.186 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-13T10:38:14.186 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-13T10:38:14.187 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-13T10:38:14.187 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-13T10:38:14.187 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-13T10:38:14.187 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-13T10:38:14.187 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-13T10:38:14.188 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-13T10:38:14.188 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-13T10:38:14.188 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-13T10:38:14.188 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-13T10:38:14.188 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-13T10:38:14.189 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-13T10:38:14.189 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-13T10:38:14.189 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-13T10:38:14.189 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-13T10:38:14.189 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-13T10:38:14.190 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-13T10:38:14.190 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-13T10:38:14.190 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-13T10:38:14.190 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-13T10:38:14.190 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-13T10:38:14.191 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-13T10:38:14.191 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-13T10:38:14.191 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-13T10:38:14.191 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-13T10:38:14.191 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-13T10:38:14.192 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-13T10:38:14.192 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-13T10:38:14.192 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-13T10:38:14.192 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-13T10:38:14.193 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-13T10:38:14.193 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-13T10:38:14.193 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-13T10:38:14.193 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-13T10:38:14.193 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-13T10:38:14.194 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-13T10:38:14.194 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-13T10:38:14.194 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-13T10:38:14.194 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-13T10:38:14.194 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-13T10:38:14.195 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-13T10:38:14.195 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-13T10:38:14.195 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-13T10:38:14.195 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-13T10:38:14.195 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-13T10:38:14.196 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-13T10:38:14.196 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-13T10:38:14.196 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-13T10:38:14.196 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-13T10:38:14.196 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-13T10:38:14.197 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-13T10:38:14.197 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-13T10:38:14.197 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-13T10:38:14.197 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-13T10:38:14.198 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-13T10:38:14.198 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-13T10:38:14.198 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-13T10:38:14.198 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-13T10:38:14.198 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-13T10:38:14.199 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-13T10:38:14.199 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-13T10:38:14.199 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-13T10:38:14.199 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-13T10:38:14.199 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-13T10:38:14.200 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-13T10:38:14.200 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-13T10:38:14.200 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-13T10:38:14.200 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-13T10:38:14.200 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-13T10:38:14.201 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-13T10:38:14.201 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-13T10:38:14.201 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-13T10:38:14.201 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-13T10:38:14.201 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-13T10:38:14.202 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-13T10:38:14.202 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-13T10:38:14.202 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-13T10:38:14.202 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-13T10:38:14.202 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-13T10:38:14.203 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-13T10:38:14.203 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-13T10:38:14.203 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-13T10:38:14.203 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-13T10:38:14.203 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-13T10:38:14.204 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-13T10:38:14.204 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-13T10:38:14.204 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-13T10:38:14.204 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-13T10:38:14.204 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-13T10:38:14.205 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-13T10:38:14.205 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-13T10:38:14.205 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-13T10:38:14.205 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-13T10:38:14.205 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-13T10:38:14.206 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-13T10:38:14.206 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-13T10:38:14.206 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-13T10:38:14.206 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-13T10:38:14.206 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-13T10:38:14.207 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-13T10:38:14.207 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-13T10:38:14.207 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-13T10:38:14.207 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-13T10:38:14.208 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-13T10:38:14.208 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-13T10:38:14.208 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-13T10:38:14.208 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-13T10:38:14.208 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-13T10:38:14.209 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-13T10:38:14.209 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-13T10:38:14.209 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-13T10:38:14.209 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-13T10:38:14.209 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-13T10:38:14.210 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-13T10:38:14.210 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-13T10:38:14.210 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-13T10:38:14.210 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-13T10:38:14.210 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-13T10:38:14.211 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-13T10:38:14.211 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-13T10:38:14.211 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-13T10:38:14.211 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-13T10:38:14.211 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-13T10:38:14.211 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-13T10:38:14.212 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-13T10:38:14.212 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-13T10:38:14.212 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-13T10:38:14.212 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-13T10:38:14.212 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-13T10:38:14.213 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-13T10:38:14.213 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-13T10:38:14.213 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-13T10:38:14.213 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-13T10:38:14.213 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-13T10:38:14.214 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-13T10:38:14.214 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-13T10:38:14.214 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-13T10:38:14.214 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-13T10:38:14.214 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-13T10:38:14.215 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-13T10:38:14.215 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-13T10:38:14.215 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-13T10:38:14.215 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-13T10:38:14.215 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-13T10:38:14.216 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-13T10:38:14.216 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-13T10:38:14.216 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-13T10:38:14.216 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-13T10:38:14.216 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-13T10:38:14.217 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-13T10:38:14.217 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-13T10:38:14.217 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-13T10:38:14.217 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-13T10:38:14.217 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-13T10:38:14.218 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-13T10:38:14.218 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-13T10:38:14.218 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-13T10:38:14.218 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-13T10:38:14.218 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-13T10:38:14.219 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-13T10:38:14.219 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-13T10:38:14.219 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-13T10:38:14.219 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-13T10:38:14.219 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-13T10:38:14.220 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-13T10:38:14.220 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-13T10:38:14.220 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-13T10:38:14.220 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-13T10:38:14.221 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-13T10:38:14.221 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-13T10:38:14.221 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-13T10:38:14.221 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-13T10:38:14.221 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-13T10:38:14.222 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-13T10:38:14.222 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-13T10:38:14.222 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-13T10:38:14.222 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-13T10:38:14.222 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-13T10:38:14.223 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-13T10:38:14.223 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-13T10:38:14.223 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-13T10:38:14.223 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-13T10:38:14.223 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-13T10:38:14.223 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-13T10:38:14.224 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-13T10:38:14.224 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-13T10:38:14.224 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-13T10:38:14.224 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-13T10:38:14.225 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-13T10:38:14.225 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-13T10:38:14.225 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-13T10:38:14.225 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-13T10:38:14.226 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-13T10:38:14.226 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-13T10:38:14.226 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-13T10:38:14.226 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-13T10:38:14.226 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-13T10:38:14.227 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-13T10:38:14.227 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-13T10:38:14.227 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-13T10:38:14.227 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-13T10:38:14.227 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-13T10:38:14.227 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-13T10:38:14.228 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-13T10:38:14.228 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-13T10:38:14.228 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-13T10:38:14.228 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-13T10:38:14.228 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-13T10:38:14.229 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-13T10:38:14.229 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-13T10:38:14.229 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-13T10:38:14.229 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-13T10:38:14.229 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-13T10:38:14.230 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-13T10:38:14.230 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-13T10:38:14.230 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-13T10:38:14.230 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-13T10:38:14.230 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-13T10:38:14.231 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-13T10:38:14.231 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-13T10:38:14.231 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-13T10:38:14.231 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-13T10:38:14.232 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-13T10:38:14.232 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-13T10:38:14.232 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-13T10:38:14.232 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-13T10:38:14.232 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-13T10:38:14.233 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-13T10:38:14.233 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-13T10:38:14.233 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-13T10:38:14.233 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-13T10:38:14.233 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-13T10:38:14.234 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-13T10:38:14.234 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-13T10:38:14.234 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-13T10:38:14.234 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-13T10:38:14.234 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-13T10:38:14.235 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-13T10:38:14.235 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-13T10:38:14.235 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-13T10:38:14.235 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-13T10:38:14.235 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-13T10:38:14.236 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-13T10:38:14.236 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-13T10:38:14.236 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-13T10:38:14.236 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-13T10:38:14.236 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-13T10:38:14.237 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-13T10:38:14.237 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-13T10:38:14.237 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-13T10:38:14.237 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-13T10:38:14.237 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-13T10:38:14.237 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-13T10:38:14.238 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-13T10:38:14.238 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-13T10:38:14.238 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-13T10:38:14.238 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-13T10:38:14.238 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-13T10:38:14.239 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-13T10:38:14.239 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-13T10:38:14.239 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-13T10:38:14.239 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-13T10:38:14.239 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-13T10:38:14.240 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-13T10:38:14.240 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-13T10:38:14.240 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-13T10:38:14.240 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-13T10:38:14.240 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-13T10:38:14.241 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-13T10:38:14.241 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-13T10:38:14.241 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-13T10:38:14.241 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-13T10:38:14.241 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-13T10:38:14.242 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-13T10:38:14.242 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-13T10:38:14.242 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-13T10:38:14.242 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-13T10:38:14.242 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-13T10:38:14.243 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-13T10:38:14.243 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-13T10:38:14.243 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-13T10:38:14.243 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-13T10:38:14.244 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-13T10:38:14.244 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-13T10:38:14.244 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-13T10:38:14.244 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-13T10:38:14.244 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-13T10:38:14.245 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-13T10:38:14.245 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-13T10:38:14.245 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-13T10:38:14.245 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-13T10:38:14.245 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-13T10:38:14.246 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-13T10:38:14.246 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-13T10:38:14.246 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-13T10:38:14.246 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-13T10:38:14.247 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-13T10:38:14.247 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-13T10:38:14.247 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-13T10:38:14.247 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-13T10:38:14.247 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-13T10:38:14.248 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-13T10:38:14.248 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-13T10:38:14.248 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-13T10:38:14.248 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-13T10:38:14.248 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-13T10:38:14.249 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-13T10:38:14.249 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-13T10:38:14.249 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-13T10:38:14.249 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-13T10:38:14.249 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-13T10:38:14.250 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-13T10:38:14.250 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-13T10:38:14.250 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-13T10:38:14.250 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-13T10:38:14.250 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-13T10:38:14.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-13T10:38:14.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-13T10:38:14.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-13T10:38:14.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-13T10:38:14.251 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-13T10:38:14.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-13T10:38:14.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-13T10:38:14.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-13T10:38:14.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-13T10:38:14.252 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-13T10:38:14.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-13T10:38:14.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-13T10:38:14.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-13T10:38:14.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-13T10:38:14.253 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-13T10:38:14.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-13T10:38:14.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-13T10:38:14.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-13T10:38:14.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-13T10:38:14.254 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-13T10:38:14.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-13T10:38:14.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-13T10:38:14.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-13T10:38:14.255 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-13T10:38:14.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-13T10:38:14.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-13T10:38:14.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-13T10:38:14.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-13T10:38:14.256 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-13T10:38:14.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-13T10:38:14.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-13T10:38:14.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-13T10:38:14.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-13T10:38:14.257 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-13T10:38:14.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-13T10:38:14.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-13T10:38:14.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-13T10:38:14.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-13T10:38:14.258 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-13T10:38:14.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-13T10:38:14.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-13T10:38:14.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-13T10:38:14.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-13T10:38:14.259 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-13T10:38:14.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-13T10:38:14.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-13T10:38:14.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-13T10:38:14.260 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-13T10:38:14.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-13T10:38:14.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-13T10:38:14.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-13T10:38:14.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-13T10:38:14.261 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-13T10:38:14.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-13T10:38:14.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-13T10:38:14.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-13T10:38:14.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-13T10:38:14.262 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-13T10:38:14.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-13T10:38:14.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-13T10:38:14.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-13T10:38:14.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-13T10:38:14.263 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-13T10:38:14.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-13T10:38:14.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-13T10:38:14.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-13T10:38:14.264 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-13T10:38:14.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-13T10:38:14.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-13T10:38:14.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-13T10:38:14.265 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-13T10:38:14.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-13T10:38:14.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-13T10:38:14.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-13T10:38:14.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-13T10:38:14.266 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-13T10:38:14.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-13T10:38:14.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-13T10:38:14.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-13T10:38:14.267 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-13T10:38:14.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-13T10:38:14.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-13T10:38:14.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-13T10:38:14.268 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-13T10:38:14.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-13T10:38:14.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-13T10:38:14.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-13T10:38:14.269 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-13T10:38:14.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-13T10:38:14.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-13T10:38:14.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-13T10:38:14.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-13T10:38:14.270 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-13T10:38:14.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-13T10:38:14.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-13T10:38:14.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-13T10:38:14.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-13T10:38:14.271 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-13T10:38:14.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-13T10:38:14.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-13T10:38:14.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-13T10:38:14.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-13T10:38:14.272 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-13T10:38:14.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-13T10:38:14.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-13T10:38:14.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-13T10:38:14.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-13T10:38:14.273 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-13T10:38:14.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-13T10:38:14.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-13T10:38:14.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-13T10:38:14.274 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-13T10:38:14.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-13T10:38:14.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-13T10:38:14.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-13T10:38:14.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-13T10:38:14.275 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-13T10:38:14.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-13T10:38:14.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-13T10:38:14.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-13T10:38:14.276 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-13T10:38:14.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-13T10:38:14.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-13T10:38:14.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-13T10:38:14.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-13T10:38:14.277 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-13T10:38:14.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-13T10:38:14.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-13T10:38:14.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-13T10:38:14.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-13T10:38:14.278 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-13T10:38:14.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-13T10:38:14.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-13T10:38:14.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-13T10:38:14.279 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-13T10:38:14.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-13T10:38:14.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-13T10:38:14.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-13T10:38:14.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-13T10:38:14.280 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-13T10:38:14.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-13T10:38:14.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-13T10:38:14.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-13T10:38:14.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-13T10:38:14.281 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-13T10:38:14.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-13T10:38:14.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-13T10:38:14.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-13T10:38:14.282 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-13T10:38:14.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-13T10:38:14.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-13T10:38:14.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-13T10:38:14.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-13T10:38:14.283 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-13T10:38:14.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-13T10:38:14.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-13T10:38:14.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-13T10:38:14.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-13T10:38:14.284 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-13T10:38:14.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-13T10:38:14.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-13T10:38:14.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-13T10:38:14.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-13T10:38:14.285 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-13T10:38:14.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-13T10:38:14.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-13T10:38:14.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-13T10:38:14.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-13T10:38:14.286 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-13T10:38:14.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-13T10:38:14.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-13T10:38:14.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-13T10:38:14.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-13T10:38:14.287 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-13T10:38:14.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-13T10:38:14.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-13T10:38:14.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-13T10:38:14.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-13T10:38:14.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-13T10:38:14.288 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-13T10:38:14.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-13T10:38:14.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-13T10:38:14.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-13T10:38:14.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-13T10:38:14.289 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-13T10:38:14.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-13T10:38:14.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-13T10:38:14.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-13T10:38:14.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-13T10:38:14.290 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-13T10:38:14.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-13T10:38:14.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-13T10:38:14.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-13T10:38:14.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-13T10:38:14.291 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-13T10:38:14.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-13T10:38:14.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-13T10:38:14.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-13T10:38:14.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-13T10:38:14.292 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-13T10:38:14.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-13T10:38:14.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-13T10:38:14.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-13T10:38:14.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-13T10:38:14.293 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-13T10:38:14.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-13T10:38:14.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-13T10:38:14.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-13T10:38:14.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-13T10:38:14.294 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-13T10:38:14.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-13T10:38:14.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-13T10:38:14.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-13T10:38:14.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-13T10:38:14.295 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-13T10:38:14.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-13T10:38:14.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-13T10:38:14.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-13T10:38:14.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-13T10:38:14.296 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-13T10:38:14.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-13T10:38:14.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-13T10:38:14.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-13T10:38:14.297 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-13T10:38:14.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-13T10:38:14.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-13T10:38:14.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-13T10:38:14.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-13T10:38:14.298 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-13T10:38:14.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-13T10:38:14.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-13T10:38:14.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-13T10:38:14.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-13T10:38:14.299 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-13T10:38:14.300 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-13T10:38:14.300 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-13T10:38:14.300 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-13T10:38:14.300 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-13T10:38:14.300 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-13T10:38:14.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-13T10:38:14.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-13T10:38:14.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-13T10:38:14.301 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-13T10:38:14.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-13T10:38:14.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-13T10:38:14.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-13T10:38:14.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-13T10:38:14.302 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-13T10:38:14.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-13T10:38:14.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-13T10:38:14.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-13T10:38:14.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-13T10:38:14.303 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-13T10:38:14.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-13T10:38:14.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-13T10:38:14.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-13T10:38:14.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-13T10:38:14.304 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-13T10:38:14.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-13T10:38:14.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-13T10:38:14.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-13T10:38:14.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-13T10:38:14.305 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-13T10:38:14.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-13T10:38:14.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-13T10:38:14.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-13T10:38:14.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-13T10:38:14.306 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-13T10:38:14.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-13T10:38:14.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-13T10:38:14.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-13T10:38:14.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-13T10:38:14.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-13T10:38:14.307 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-13T10:38:14.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-13T10:38:14.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-13T10:38:14.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-13T10:38:14.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-13T10:38:14.308 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-13T10:38:14.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-13T10:38:14.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-13T10:38:14.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-13T10:38:14.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-13T10:38:14.309 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-13T10:38:14.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-13T10:38:14.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-13T10:38:14.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-13T10:38:14.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-13T10:38:14.310 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-13T10:38:14.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-13T10:38:14.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-13T10:38:14.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-13T10:38:14.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-13T10:38:14.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-13T10:38:14.311 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-13T10:38:14.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-13T10:38:14.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-13T10:38:14.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-13T10:38:14.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-13T10:38:14.312 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-13T10:38:14.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-13T10:38:14.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-13T10:38:14.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-13T10:38:14.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-13T10:38:14.313 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-13T10:38:14.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-13T10:38:14.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-13T10:38:14.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-13T10:38:14.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-13T10:38:14.314 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-13T10:38:14.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-13T10:38:14.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-13T10:38:14.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-13T10:38:14.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-13T10:38:14.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-13T10:38:14.315 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-13T10:38:14.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-13T10:38:14.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-13T10:38:14.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-13T10:38:14.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-13T10:38:14.316 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-13T10:38:14.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-13T10:38:14.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-13T10:38:14.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-13T10:38:14.317 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-13T10:38:14.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-13T10:38:14.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-13T10:38:14.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-13T10:38:14.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-13T10:38:14.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-13T10:38:14.318 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-13T10:38:14.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-13T10:38:14.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-13T10:38:14.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-13T10:38:14.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-13T10:38:14.319 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-13T10:38:14.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-13T10:38:14.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-13T10:38:14.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-13T10:38:14.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-13T10:38:14.320 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-13T10:38:14.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-13T10:38:14.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-13T10:38:14.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-13T10:38:14.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-13T10:38:14.321 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-13T10:38:14.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-13T10:38:14.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-13T10:38:14.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-13T10:38:14.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-13T10:38:14.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-13T10:38:14.322 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-13T10:38:14.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-13T10:38:14.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-13T10:38:14.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-13T10:38:14.323 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-13T10:38:14.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-13T10:38:14.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-13T10:38:14.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-13T10:38:14.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-13T10:38:14.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-13T10:38:14.324 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-13T10:38:14.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-13T10:38:14.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-13T10:38:14.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-13T10:38:14.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-13T10:38:14.325 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-13T10:38:14.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-13T10:38:14.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-13T10:38:14.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-13T10:38:14.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-13T10:38:14.326 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-13T10:38:14.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-13T10:38:14.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-13T10:38:14.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-13T10:38:14.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-13T10:38:14.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-13T10:38:14.327 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-13T10:38:14.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-13T10:38:14.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-13T10:38:14.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-13T10:38:14.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-13T10:38:14.328 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-13T10:38:14.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-13T10:38:14.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-13T10:38:14.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-13T10:38:14.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-13T10:38:14.329 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-13T10:38:14.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-13T10:38:14.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-13T10:38:14.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-13T10:38:14.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-13T10:38:14.330 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-13T10:38:14.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-13T10:38:14.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-13T10:38:14.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-13T10:38:14.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-13T10:38:14.331 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-13T10:38:14.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-13T10:38:14.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-13T10:38:14.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-13T10:38:14.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-13T10:38:14.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-13T10:38:14.332 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-13T10:38:14.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-13T10:38:14.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-13T10:38:14.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-13T10:38:14.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-13T10:38:14.333 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-13T10:38:14.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-13T10:38:14.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-13T10:38:14.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-13T10:38:14.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-13T10:38:14.334 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-13T10:38:14.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-13T10:38:14.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-13T10:38:14.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-13T10:38:14.335 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-13T10:38:14.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-13T10:38:14.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-13T10:38:14.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-13T10:38:14.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-13T10:38:14.336 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-13T10:38:14.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-13T10:38:14.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-13T10:38:14.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-13T10:38:14.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-13T10:38:14.337 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-13T10:38:14.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-13T10:38:14.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-13T10:38:14.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-13T10:38:14.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-13T10:38:14.338 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-13T10:38:14.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-13T10:38:14.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-13T10:38:14.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-13T10:38:14.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-13T10:38:14.339 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-13T10:38:14.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-13T10:38:14.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-13T10:38:14.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-13T10:38:14.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-13T10:38:14.340 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-13T10:38:14.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-13T10:38:14.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-13T10:38:14.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-13T10:38:14.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-13T10:38:14.341 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-13T10:38:14.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-13T10:38:14.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-13T10:38:14.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-13T10:38:14.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-13T10:38:14.342 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-13T10:38:14.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-13T10:38:14.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-13T10:38:14.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-13T10:38:14.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-13T10:38:14.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-13T10:38:14.343 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-13T10:38:14.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-13T10:38:14.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-13T10:38:14.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-13T10:38:14.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-13T10:38:14.344 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-13T10:38:14.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-13T10:38:14.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-13T10:38:14.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-13T10:38:14.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-13T10:38:14.345 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-13T10:38:14.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-13T10:38:14.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-13T10:38:14.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-13T10:38:14.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-13T10:38:14.346 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-13T10:38:14.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-13T10:38:14.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-13T10:38:14.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-13T10:38:14.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-13T10:38:14.347 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-13T10:38:14.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-13T10:38:14.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-13T10:38:14.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-13T10:38:14.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-13T10:38:14.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-13T10:38:14.348 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-13T10:38:14.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-13T10:38:14.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-13T10:38:14.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-13T10:38:14.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-13T10:38:14.349 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-13T10:38:14.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-13T10:38:14.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-13T10:38:14.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-13T10:38:14.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-13T10:38:14.350 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-13T10:38:14.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-13T10:38:14.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-13T10:38:14.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-13T10:38:14.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-13T10:38:14.351 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-13T10:38:14.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-13T10:38:14.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-13T10:38:14.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-13T10:38:14.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-13T10:38:14.352 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-13T10:38:14.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-13T10:38:14.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-13T10:38:14.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-13T10:38:14.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-13T10:38:14.353 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-13T10:38:14.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-13T10:38:14.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-13T10:38:14.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-13T10:38:14.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-13T10:38:14.354 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-13T10:38:14.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-13T10:38:14.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-13T10:38:14.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-13T10:38:14.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-13T10:38:14.355 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-13T10:38:14.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-13T10:38:14.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-13T10:38:14.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-13T10:38:14.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-13T10:38:14.356 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-13T10:38:14.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-13T10:38:14.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-13T10:38:14.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-13T10:38:14.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-13T10:38:14.357 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-13T10:38:14.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-13T10:38:14.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-13T10:38:14.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-13T10:38:14.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-13T10:38:14.358 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-13T10:38:14.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-13T10:38:14.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-13T10:38:14.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-13T10:38:14.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-13T10:38:14.359 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-13T10:38:14.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-13T10:38:14.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-13T10:38:14.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-13T10:38:14.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-13T10:38:14.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-13T10:38:14.360 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-13T10:38:14.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-13T10:38:14.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-13T10:38:14.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-13T10:38:14.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-13T10:38:14.361 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-13T10:38:14.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-13T10:38:14.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-13T10:38:14.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-13T10:38:14.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-13T10:38:14.362 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-13T10:38:14.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-13T10:38:14.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-13T10:38:14.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-13T10:38:14.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-13T10:38:14.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-13T10:38:14.363 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-13T10:38:14.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-13T10:38:14.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-13T10:38:14.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-13T10:38:14.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-13T10:38:14.364 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-13T10:38:14.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-13T10:38:14.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-13T10:38:14.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-13T10:38:14.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-13T10:38:14.365 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-13T10:38:14.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-13T10:38:14.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-13T10:38:14.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-13T10:38:14.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-13T10:38:14.366 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-13T10:38:14.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-13T10:38:14.367 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-13T10:38:14.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-13T10:38:14.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-13T10:38:14.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-13T10:38:14.369 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-13T10:38:14.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-13T10:38:14.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-13T10:38:14.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-13T10:38:14.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-13T10:38:14.370 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-13T10:38:14.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-13T10:38:14.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-13T10:38:14.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-13T10:38:14.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-13T10:38:14.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-13T10:38:14.371 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-13T10:38:14.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-13T10:38:14.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-13T10:38:14.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-13T10:38:14.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-13T10:38:14.372 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-13T10:38:14.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-13T10:38:14.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-13T10:38:14.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-13T10:38:14.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-13T10:38:14.373 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-13T10:38:14.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-13T10:38:14.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-13T10:38:14.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-13T10:38:14.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-13T10:38:14.374 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-13T10:38:14.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-13T10:38:14.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-13T10:38:14.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-13T10:38:14.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-13T10:38:14.375 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-13T10:38:14.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-13T10:38:14.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-13T10:38:14.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-13T10:38:14.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-13T10:38:14.376 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-13T10:38:14.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-13T10:38:14.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-13T10:38:14.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-13T10:38:14.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-13T10:38:14.377 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-13T10:38:14.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-13T10:38:14.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-13T10:38:14.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-13T10:38:14.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-13T10:38:14.378 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-13T10:38:14.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-13T10:38:14.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-13T10:38:14.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-13T10:38:14.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-13T10:38:14.379 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-13T10:38:14.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-13T10:38:14.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-13T10:38:14.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-13T10:38:14.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-13T10:38:14.380 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-13T10:38:14.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-13T10:38:14.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-13T10:38:14.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-13T10:38:14.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-13T10:38:14.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-13T10:38:14.381 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-13T10:38:14.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-13T10:38:14.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-13T10:38:14.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-13T10:38:14.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-13T10:38:14.382 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-13T10:38:14.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-13T10:38:14.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-13T10:38:14.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-13T10:38:14.383 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-13T10:38:14.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-13T10:38:14.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-13T10:38:14.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-13T10:38:14.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-13T10:38:14.384 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-13T10:38:14.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-13T10:38:14.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-13T10:38:14.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-13T10:38:14.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-13T10:38:14.385 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-13T10:38:14.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-13T10:38:14.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-13T10:38:14.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-13T10:38:14.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-13T10:38:14.386 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-13T10:38:14.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-13T10:38:14.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-13T10:38:14.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-13T10:38:14.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-13T10:38:14.387 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-13T10:38:14.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-13T10:38:14.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-13T10:38:14.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-13T10:38:14.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-13T10:38:14.388 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-13T10:38:14.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-13T10:38:14.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-13T10:38:14.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-13T10:38:14.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-13T10:38:14.389 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-13T10:38:14.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-13T10:38:14.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-13T10:38:14.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-13T10:38:14.390 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-13T10:38:14.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-13T10:38:14.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-13T10:38:14.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-13T10:38:14.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-13T10:38:14.391 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-13T10:38:14.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-13T10:38:14.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-13T10:38:14.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-13T10:38:14.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-13T10:38:14.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-13T10:38:14.392 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-13T10:38:14.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-13T10:38:14.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-13T10:38:14.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-13T10:38:14.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-13T10:38:14.393 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-13T10:38:14.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-13T10:38:14.394 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-13T10:38:14.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-13T10:38:14.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-13T10:38:14.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-13T10:38:14.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-13T10:38:14.395 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-13T10:38:14.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-13T10:38:14.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-13T10:38:14.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-13T10:38:14.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-13T10:38:14.396 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-13T10:38:14.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-13T10:38:14.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-13T10:38:14.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-13T10:38:14.397 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-13T10:38:14.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-13T10:38:14.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-13T10:38:14.398 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-13T10:38:14.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-13T10:38:14.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-13T10:38:14.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-13T10:38:14.399 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-13T10:38:14.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-13T10:38:14.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-13T10:38:14.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-13T10:38:14.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-13T10:38:14.400 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-13T10:38:14.401 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-13T10:38:14.401 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-13T10:38:14.401 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-13T10:38:14.401 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-13T10:38:14.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-13T10:38:14.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-13T10:38:14.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-13T10:38:14.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-13T10:38:14.402 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-13T10:38:14.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-13T10:38:14.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-13T10:38:14.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-13T10:38:14.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-13T10:38:14.403 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-13T10:38:14.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-13T10:38:14.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-13T10:38:14.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-13T10:38:14.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-13T10:38:14.404 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-13T10:38:14.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-13T10:38:14.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-13T10:38:14.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-13T10:38:14.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-13T10:38:14.405 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-13T10:38:14.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-13T10:38:14.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-13T10:38:14.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-13T10:38:14.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-13T10:38:14.406 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-13T10:38:14.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-13T10:38:14.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-13T10:38:14.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-13T10:38:14.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-13T10:38:14.407 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-13T10:38:14.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-13T10:38:14.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-13T10:38:14.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-13T10:38:14.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-13T10:38:14.408 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-13T10:38:14.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-13T10:38:14.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-13T10:38:14.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-13T10:38:14.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-13T10:38:14.409 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-13T10:38:14.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-13T10:38:14.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-13T10:38:14.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-13T10:38:14.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-13T10:38:14.410 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-13T10:38:14.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-13T10:38:14.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-13T10:38:14.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-13T10:38:14.411 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-13T10:38:14.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-13T10:38:14.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-13T10:38:14.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-13T10:38:14.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-13T10:38:14.412 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-13T10:38:14.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-13T10:38:14.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-13T10:38:14.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-13T10:38:14.413 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-13T10:38:14.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-13T10:38:14.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-13T10:38:14.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-13T10:38:14.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-13T10:38:14.414 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-13T10:38:14.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-13T10:38:14.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-13T10:38:14.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-13T10:38:14.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-13T10:38:14.415 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-13T10:38:14.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-13T10:38:14.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-13T10:38:14.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-13T10:38:14.416 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-13T10:38:14.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-13T10:38:14.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-13T10:38:14.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-13T10:38:14.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-13T10:38:14.417 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-13T10:38:14.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-13T10:38:14.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-13T10:38:14.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-13T10:38:14.418 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-13T10:38:14.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-13T10:38:14.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-13T10:38:14.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-13T10:38:14.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-13T10:38:14.419 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-13T10:38:14.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-13T10:38:14.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-13T10:38:14.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-13T10:38:14.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-13T10:38:14.420 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-13T10:38:14.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-13T10:38:14.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-13T10:38:14.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-13T10:38:14.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-13T10:38:14.421 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-13T10:38:14.422 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-13T10:38:14.422 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-13T10:38:14.422 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-13T10:38:14.422 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-13T10:38:14.422 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-13T10:38:14.423 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-13T10:38:14.423 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-13T10:38:14.423 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-13T10:38:14.423 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-13T10:38:14.424 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-13T10:38:14.424 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-13T10:38:14.424 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-13T10:38:14.424 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-13T10:38:14.424 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-13T10:38:14.425 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-13T10:38:14.425 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-13T10:38:14.425 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-13T10:38:14.425 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-13T10:38:14.425 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-13T10:38:14.426 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-13T10:38:14.426 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-13T10:38:14.426 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-13T10:38:14.426 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-13T10:38:14.426 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-13T10:38:14.427 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-13T10:38:14.427 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-13T10:38:14.427 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-13T10:38:14.427 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-13T10:38:14.427 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-13T10:38:14.428 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-13T10:38:14.428 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-13T10:38:14.428 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-13T10:38:14.428 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-13T10:38:14.428 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-13T10:38:14.429 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-13T10:38:14.429 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-13T10:38:14.429 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-13T10:38:14.429 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-13T10:38:14.429 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-13T10:38:14.430 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-13T10:38:14.430 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-13T10:38:14.430 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-13T10:38:14.430 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-13T10:38:14.430 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-13T10:38:14.431 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-13T10:38:14.431 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-13T10:38:14.431 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-13T10:38:14.431 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-13T10:38:14.431 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-13T10:38:14.431 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-13T10:38:14.432 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-13T10:38:14.432 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-13T10:38:14.432 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-13T10:38:14.432 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-13T10:38:14.432 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-13T10:38:14.433 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-13T10:38:14.433 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-13T10:38:14.433 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-13T10:38:14.433 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-13T10:38:14.433 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-13T10:38:14.434 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-13T10:38:14.434 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-13T10:38:14.434 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-13T10:38:14.434 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-13T10:38:14.434 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-13T10:38:14.435 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-13T10:38:14.435 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-13T10:38:14.435 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-13T10:38:14.435 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-13T10:38:14.435 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-13T10:38:14.435 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-13T10:38:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-13T10:38:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-13T10:38:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-13T10:38:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-13T10:38:14.436 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-13T10:38:14.437 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-13T10:38:14.437 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-13T10:38:14.437 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-13T10:38:14.437 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-13T10:38:14.438 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-13T10:38:14.438 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-13T10:38:14.438 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-13T10:38:14.438 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-13T10:38:14.438 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-13T10:38:14.439 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-13T10:38:14.439 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-13T10:38:14.439 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-13T10:38:14.439 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-13T10:38:14.439 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-13T10:38:14.439 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-13T10:38:14.440 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-13T10:38:14.440 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-13T10:38:14.440 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-13T10:38:14.440 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-13T10:38:14.440 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-13T10:38:14.441 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-13T10:38:14.441 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-13T10:38:14.441 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-13T10:38:14.441 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-13T10:38:14.441 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-13T10:38:14.442 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-13T10:38:14.442 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-13T10:38:14.442 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-13T10:38:14.442 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-13T10:38:14.443 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-13T10:38:14.443 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-13T10:38:14.443 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-13T10:38:14.443 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-13T10:38:14.443 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-13T10:38:14.444 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-13T10:38:14.444 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-13T10:38:14.444 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-13T10:38:14.444 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-13T10:38:14.444 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-13T10:38:14.445 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-13T10:38:14.445 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-13T10:38:14.445 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-13T10:38:14.445 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-13T10:38:14.445 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-13T10:38:14.446 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-13T10:38:14.446 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-13T10:38:14.446 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-13T10:38:14.446 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-13T10:38:14.446 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-13T10:38:14.447 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-13T10:38:14.447 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-13T10:38:14.447 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-13T10:38:14.447 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-13T10:38:14.448 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-13T10:38:14.448 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-13T10:38:14.448 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-13T10:38:14.448 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-13T10:38:14.448 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-13T10:38:14.449 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-13T10:38:14.449 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-13T10:38:14.449 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-13T10:38:14.449 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-13T10:38:14.449 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-13T10:38:14.449 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-13T10:38:14.450 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-13T10:38:14.450 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-13T10:38:14.450 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-13T10:38:14.450 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-13T10:38:14.450 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-13T10:38:14.451 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-13T10:38:14.451 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-13T10:38:14.451 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-13T10:38:14.451 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-13T10:38:14.451 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-13T10:38:14.452 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-13T10:38:14.452 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-13T10:38:14.452 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-13T10:38:14.452 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-13T10:38:14.452 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-13T10:38:14.453 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-13T10:38:14.453 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-13T10:38:14.453 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-13T10:38:14.453 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-13T10:38:14.453 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-13T10:38:14.453 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-13T10:38:14.454 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-13T10:38:14.454 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-13T10:38:14.454 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-13T10:38:14.454 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-13T10:38:14.454 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-13T10:38:14.455 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-13T10:38:14.455 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-13T10:38:14.455 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-13T10:38:14.455 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-13T10:38:14.455 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-13T10:38:14.456 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-13T10:38:14.456 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-13T10:38:14.456 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-13T10:38:14.456 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-13T10:38:14.456 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-13T10:38:14.457 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-13T10:38:14.457 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-13T10:38:14.457 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-13T10:38:14.457 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-13T10:38:14.457 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-13T10:38:14.458 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-13T10:38:14.458 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-13T10:38:14.458 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-13T10:38:14.458 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-13T10:38:14.458 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-13T10:38:14.459 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-13T10:38:14.459 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-13T10:38:14.459 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-13T10:38:14.459 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-13T10:38:14.459 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-13T10:38:14.460 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-13T10:38:14.460 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-13T10:38:14.460 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-13T10:38:14.460 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-13T10:38:14.461 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-13T10:38:14.461 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-13T10:38:14.461 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-13T10:38:14.461 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-13T10:38:14.461 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-13T10:38:14.462 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-13T10:38:14.462 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-13T10:38:14.462 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-13T10:38:14.462 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-13T10:38:14.462 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-13T10:38:14.463 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-13T10:38:14.463 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-13T10:38:14.463 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-13T10:38:14.463 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-13T10:38:14.463 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-13T10:38:14.464 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-13T10:38:14.464 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-13T10:38:14.464 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-13T10:38:14.464 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-13T10:38:14.464 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-13T10:38:14.465 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-13T10:38:14.465 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-13T10:38:14.465 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-13T10:38:14.465 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-13T10:38:14.465 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-13T10:38:14.466 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-13T10:38:14.466 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-13T10:38:14.466 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-13T10:38:14.466 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-13T10:38:14.466 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-13T10:38:14.467 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-13T10:38:14.467 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-13T10:38:14.467 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-13T10:38:14.467 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-13T10:38:14.467 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-13T10:38:14.468 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-13T10:38:14.468 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-13T10:38:14.468 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-13T10:38:14.468 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-13T10:38:14.468 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-13T10:38:14.469 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-13T10:38:14.469 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-13T10:38:14.469 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-13T10:38:14.469 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-13T10:38:14.469 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-13T10:38:14.470 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-13T10:38:14.470 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-13T10:38:14.470 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-13T10:38:14.470 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-13T10:38:14.470 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-13T10:38:14.470 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-13T10:38:14.471 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-13T10:38:14.471 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-13T10:38:14.471 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-13T10:38:14.471 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-13T10:38:14.471 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-13T10:38:14.472 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-13T10:38:14.472 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-13T10:38:14.472 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-13T10:38:14.472 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-13T10:38:14.472 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-13T10:38:14.473 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-13T10:38:14.473 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-13T10:38:14.473 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-13T10:38:14.473 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-13T10:38:14.473 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-13T10:38:14.474 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-13T10:38:14.474 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-13T10:38:14.474 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-13T10:38:14.474 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-13T10:38:14.474 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-13T10:38:14.475 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-13T10:38:14.475 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-13T10:38:14.475 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-13T10:38:14.475 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-13T10:38:14.475 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-13T10:38:14.476 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-13T10:38:14.476 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-13T10:38:14.476 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-13T10:38:14.476 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-13T10:38:14.476 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-13T10:38:14.477 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-13T10:38:14.477 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-13T10:38:14.477 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-13T10:38:14.477 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-13T10:38:14.477 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-13T10:38:14.478 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-13T10:38:14.478 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-13T10:38:14.478 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-13T10:38:14.478 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-13T10:38:14.478 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-13T10:38:14.479 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-13T10:38:14.479 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-13T10:38:14.479 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-13T10:38:14.479 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-13T10:38:14.479 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-13T10:38:14.480 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-13T10:38:14.480 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-13T10:38:14.480 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-13T10:38:14.480 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-13T10:38:14.480 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-13T10:38:14.481 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-13T10:38:14.481 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-13T10:38:14.481 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-13T10:38:14.481 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-13T10:38:14.481 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-13T10:38:14.481 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-13T10:38:14.482 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-13T10:38:14.482 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-13T10:38:14.482 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-13T10:38:14.482 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-13T10:38:14.483 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-13T10:38:14.483 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-13T10:38:14.483 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-13T10:38:14.483 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-13T10:38:14.483 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-13T10:38:14.483 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-13T10:38:14.484 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-13T10:38:14.484 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-13T10:38:14.484 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-13T10:38:14.484 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-13T10:38:14.484 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-13T10:38:14.485 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-13T10:38:14.485 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-13T10:38:14.485 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-13T10:38:14.485 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-13T10:38:14.485 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-13T10:38:14.486 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-13T10:38:14.486 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-13T10:38:14.486 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-13T10:38:14.486 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-13T10:38:14.486 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-13T10:38:14.486 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-13T10:38:14.487 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-13T10:38:14.487 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-13T10:38:14.487 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-13T10:38:14.487 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-13T10:38:14.487 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-13T10:38:14.488 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-13T10:38:14.488 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-13T10:38:14.488 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-13T10:38:14.488 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-13T10:38:14.488 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-13T10:38:14.488 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-13T10:38:14.489 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-13T10:38:14.489 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-13T10:38:14.489 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-13T10:38:14.489 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-13T10:38:14.489 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-13T10:38:14.490 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-13T10:38:14.490 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-13T10:38:14.490 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-13T10:38:14.490 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-13T10:38:14.490 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-13T10:38:14.491 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-13T10:38:14.491 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-13T10:38:14.491 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-13T10:38:14.491 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-13T10:38:14.491 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-13T10:38:14.492 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-13T10:38:14.492 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-13T10:38:14.492 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-13T10:38:14.492 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-13T10:38:14.492 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-13T10:38:14.492 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-13T10:38:14.493 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-13T10:38:14.493 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-13T10:38:14.493 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-13T10:38:14.493 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-13T10:38:14.493 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-13T10:38:14.494 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-13T10:38:14.494 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-13T10:38:14.494 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-13T10:38:14.494 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-13T10:38:14.494 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-13T10:38:14.495 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-13T10:38:14.495 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-13T10:38:14.495 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-13T10:38:14.495 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-13T10:38:14.495 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-13T10:38:14.496 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-13T10:38:14.496 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-13T10:38:14.496 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-13T10:38:14.496 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-13T10:38:14.496 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-13T10:38:14.497 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-13T10:38:14.497 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-13T10:38:14.497 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-13T10:38:14.497 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-13T10:38:14.497 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-13T10:38:14.498 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-13T10:38:14.498 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-13T10:38:14.498 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-13T10:38:14.498 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-13T10:38:14.498 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-13T10:38:14.499 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-13T10:38:14.499 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-13T10:38:14.499 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-13T10:38:14.499 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-13T10:38:14.499 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-13T10:38:14.500 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-13T10:38:14.500 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-13T10:38:14.500 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-13T10:38:14.500 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-13T10:38:14.500 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-13T10:38:14.501 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-13T10:38:14.501 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-13T10:38:14.501 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-13T10:38:14.501 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-13T10:38:14.501 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-13T10:38:14.502 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-13T10:38:14.502 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-13T10:38:14.502 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-13T10:38:14.502 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-13T10:38:14.503 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-13T10:38:14.503 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-13T10:38:14.503 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-13T10:38:14.503 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-13T10:38:14.503 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-13T10:38:14.503 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-13T10:38:14.504 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-13T10:38:14.504 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-13T10:38:14.504 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-13T10:38:14.504 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-13T10:38:14.504 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-13T10:38:14.505 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-13T10:38:14.505 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-13T10:38:14.505 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-13T10:38:14.505 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-13T10:38:14.505 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-13T10:38:14.506 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-13T10:38:14.506 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-13T10:38:14.506 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-13T10:38:14.506 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-13T10:38:14.506 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-13T10:38:14.507 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-13T10:38:14.507 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-13T10:38:14.507 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-13T10:38:14.507 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-13T10:38:14.507 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-13T10:38:14.508 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-13T10:38:14.508 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-13T10:38:14.508 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-13T10:38:14.508 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-13T10:38:14.508 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-13T10:38:14.509 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-13T10:38:14.509 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-13T10:38:14.509 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-13T10:38:14.509 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-13T10:38:14.509 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-13T10:38:14.510 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-13T10:38:14.510 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-13T10:38:14.510 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-13T10:38:14.510 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-13T10:38:14.510 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-13T10:38:14.511 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-13T10:38:14.511 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-13T10:38:14.511 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-13T10:38:14.511 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-13T10:38:14.511 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-13T10:38:14.512 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-13T10:38:14.512 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-13T10:38:14.512 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-13T10:38:14.512 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-13T10:38:14.512 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-13T10:38:14.512 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-13T10:38:14.513 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-13T10:38:14.513 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-13T10:38:14.513 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-13T10:38:14.513 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-13T10:38:14.513 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-13T10:38:14.514 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-13T10:38:14.514 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-13T10:38:14.514 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-13T10:38:14.514 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-13T10:38:14.514 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-13T10:38:14.515 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-13T10:38:14.515 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-13T10:38:14.515 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-13T10:38:14.515 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-13T10:38:14.515 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-13T10:38:14.516 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-13T10:38:14.516 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-13T10:38:14.516 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-13T10:38:14.516 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-13T10:38:14.516 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-13T10:38:14.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-13T10:38:14.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-13T10:38:14.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-13T10:38:14.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-13T10:38:14.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-13T10:38:14.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-13T10:38:14.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-13T10:38:14.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-13T10:38:14.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-13T10:38:14.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-13T10:38:14.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-13T10:38:14.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-13T10:38:14.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-13T10:38:14.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-13T10:38:14.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-13T10:38:14.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-13T10:38:14.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-13T10:38:14.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-13T10:38:14.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-13T10:38:14.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-13T10:38:14.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-13T10:38:14.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-13T10:38:14.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-13T10:38:14.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-13T10:38:14.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-13T10:38:14.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-13T10:38:14.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-13T10:38:14.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-13T10:38:14.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-13T10:38:14.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-13T10:38:14.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-13T10:38:14.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-13T10:38:14.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-13T10:38:14.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-13T10:38:14.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-13T10:38:14.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-13T10:38:14.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-13T10:38:14.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-13T10:38:14.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-13T10:38:14.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-13T10:38:14.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-13T10:38:14.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-13T10:38:14.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-13T10:38:14.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-13T10:38:14.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-13T10:38:14.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-13T10:38:14.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-13T10:38:14.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-13T10:38:14.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-13T10:38:14.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-13T10:38:14.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.alias.bin 2023-11-13T10:38:14.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-13T10:38:14.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-13T10:38:14.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-13T10:38:14.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-13T10:38:14.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-13T10:38:14.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-13T10:38:14.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-13T10:38:14.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-13T10:38:14.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-13T10:38:14.529 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:14.529 DEBUG:teuthology.orchestra.run.smithi184:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-13T10:38:14.553 INFO:teuthology.orchestra.run.smithi184.stdout:postinstall scriptlet (using /bin/sh): 2023-11-13T10:38:14.554 INFO:teuthology.orchestra.run.smithi184.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-13T10:38:14.554 INFO:teuthology.orchestra.run.smithi184.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-13T10:38:14.554 INFO:teuthology.orchestra.run.smithi184.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-13T10:38:14.554 INFO:teuthology.orchestra.run.smithi184.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:14.554 INFO:teuthology.orchestra.run.smithi184.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-13T10:38:14.555 INFO:teuthology.orchestra.run.smithi184.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-13T10:38:14.555 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2023-11-13T10:38:14.555 INFO:teuthology.orchestra.run.smithi184.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-13T10:38:14.555 INFO:teuthology.orchestra.run.smithi184.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-13T10:38:14.555 INFO:teuthology.orchestra.run.smithi184.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-13T10:38:14.556 INFO:teuthology.orchestra.run.smithi184.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-13T10:38:14.556 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:14.556 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2023-11-13T10:38:14.556 INFO:teuthology.orchestra.run.smithi184.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-13T10:38:14.556 INFO:teuthology.orchestra.run.smithi184.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-13T10:38:14.557 INFO:teuthology.orchestra.run.smithi184.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:14.557 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2023-11-13T10:38:14.558 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2023-11-13T10:38:14.634 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2023-11-13T10:38:14.635 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2023-11-13T10:38:14.635 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.06 2023-11-13T10:38:14.635 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 68.el9 2023-11-13T10:38:14.636 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2023-11-13T10:38:14.636 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-13T10:38:14.636 INFO:teuthology.orchestra.run.smithi184.stdout:Group : Unspecified 2023-11-13T10:38:14.636 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 8264254 2023-11-13T10:38:14.637 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2023-11-13T10:38:14.637 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-13T10:38:14.637 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-13T10:38:14.637 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-13T10:38:14.637 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-13T10:38:14.638 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : builder@centos.org 2023-11-13T10:38:14.638 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : CentOS 2023-11-13T10:38:14.638 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-13T10:38:14.638 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2023-11-13T10:38:14.638 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2023-11-13T10:38:14.639 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:38:14.639 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-13T10:38:14.639 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-13T10:38:14.639 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-13T10:38:14.639 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2023-11-13T10:38:14.640 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:38:14.640 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2023-11-13T10:38:14.641 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-13T10:38:14.641 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:14.642 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-13T10:38:15.108 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying... ######################################## 2023-11-13T10:38:15.175 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing... ######################################## 2023-11-13T10:38:15.221 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2023-11-13T10:38:15.261 INFO:teuthology.orchestra.run.smithi016.stdout:Updating / installing... 2023-11-13T10:38:16.533 INFO:teuthology.orchestra.run.smithi184.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-13T10:38:16.565 INFO:teuthology.orchestra.run.smithi184.stderr:done 2023-11-13T10:38:16.567 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-11-13T10:38:16.583 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.VsoLoXQdv6 2023-11-13T10:38:16.583 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VsoLoXQdv6 2023-11-13T10:38:16.675 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.VsoLoXQdv6 2023-11-13T10:38:16.777 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.VsoLoXQdv6 is 6KB 2023-11-13T10:38:16.829 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.VsoLoXQdv6 2023-11-13T10:38:16.843 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2023-11-13T10:38:16.913 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-13T10:38:16.914 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-13T10:38:16.914 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-13T10:38:16.914 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-13T10:38:16.915 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:17.062 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2023-11-13T10:38:22.685 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-13T10:38:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-13T10:38:47.065 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-13T10:38:47.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-11-13T10:38:47.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:38:58.456 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-13T10:38:58.456 DEBUG:teuthology.orchestra.run.smithi016:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-13T10:38:58.535 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:58.536 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:58.536 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:58.536 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:58.536 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-13T10:38:58.537 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-13T10:38:58.537 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-13T10:38:58.537 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-13T10:38:58.537 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-13T10:38:58.537 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-13T10:38:58.538 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-13T10:38:58.538 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-13T10:38:58.538 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-13T10:38:58.538 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-13T10:38:58.539 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-13T10:38:58.539 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-13T10:38:58.539 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-13T10:38:58.539 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-13T10:38:58.539 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-13T10:38:58.540 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-13T10:38:58.540 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-13T10:38:58.540 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-13T10:38:58.540 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-13T10:38:58.540 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-13T10:38:58.541 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-13T10:38:58.541 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-13T10:38:58.541 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-13T10:38:58.541 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-13T10:38:58.541 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-13T10:38:58.542 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-13T10:38:58.542 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-13T10:38:58.542 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-13T10:38:58.542 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-13T10:38:58.543 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-13T10:38:58.543 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-13T10:38:58.543 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-13T10:38:58.543 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-13T10:38:58.543 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-13T10:38:58.544 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-13T10:38:58.544 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-13T10:38:58.544 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-13T10:38:58.544 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-13T10:38:58.544 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-13T10:38:58.545 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-13T10:38:58.545 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-13T10:38:58.545 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-13T10:38:58.545 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-13T10:38:58.545 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-13T10:38:58.546 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-13T10:38:58.546 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-13T10:38:58.546 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-13T10:38:58.547 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-13T10:38:58.547 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-13T10:38:58.547 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-13T10:38:58.547 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-13T10:38:58.547 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-13T10:38:58.548 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-13T10:38:58.548 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-13T10:38:58.548 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-13T10:38:58.548 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-13T10:38:58.549 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-13T10:38:58.549 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-13T10:38:58.549 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-13T10:38:58.549 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-13T10:38:58.550 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-13T10:38:58.550 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-13T10:38:58.550 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-13T10:38:58.550 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-13T10:38:58.550 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-13T10:38:58.551 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-13T10:38:58.551 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-13T10:38:58.551 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-13T10:38:58.551 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-13T10:38:58.551 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-13T10:38:58.552 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-13T10:38:58.552 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-13T10:38:58.552 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-13T10:38:58.552 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-13T10:38:58.552 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-13T10:38:58.553 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-13T10:38:58.553 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-13T10:38:58.553 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-13T10:38:58.553 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-13T10:38:58.553 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-13T10:38:58.554 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-13T10:38:58.554 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-13T10:38:58.554 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-13T10:38:58.554 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-13T10:38:58.554 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-13T10:38:58.555 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-13T10:38:58.555 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-13T10:38:58.555 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-13T10:38:58.555 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-13T10:38:58.555 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-13T10:38:58.556 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-13T10:38:58.556 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-13T10:38:58.556 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-13T10:38:58.556 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-13T10:38:58.557 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-13T10:38:58.557 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-13T10:38:58.557 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-13T10:38:58.557 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-13T10:38:58.558 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-13T10:38:58.558 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-13T10:38:58.558 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-13T10:38:58.558 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-13T10:38:58.559 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-13T10:38:58.559 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-13T10:38:58.559 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-13T10:38:58.560 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-13T10:38:58.560 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-13T10:38:58.560 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-13T10:38:58.560 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-13T10:38:58.560 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-13T10:38:58.561 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-13T10:38:58.561 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-13T10:38:58.561 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-13T10:38:58.561 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-13T10:38:58.561 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-13T10:38:58.561 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-13T10:38:58.562 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-13T10:38:58.562 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-13T10:38:58.562 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-13T10:38:58.563 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-13T10:38:58.563 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-13T10:38:58.563 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-13T10:38:58.564 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-13T10:38:58.564 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-13T10:38:58.564 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-13T10:38:58.564 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-13T10:38:58.565 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-13T10:38:58.565 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-13T10:38:58.566 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-13T10:38:58.566 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-13T10:38:58.566 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-13T10:38:58.567 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-13T10:38:58.567 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-13T10:38:58.567 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-13T10:38:58.568 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-13T10:38:58.568 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-13T10:38:58.568 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-13T10:38:58.569 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-13T10:38:58.569 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-13T10:38:58.569 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-13T10:38:58.570 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-13T10:38:58.570 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-13T10:38:58.570 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-13T10:38:58.571 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-13T10:38:58.571 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-13T10:38:58.571 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-13T10:38:58.571 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-13T10:38:58.572 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-13T10:38:58.572 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-13T10:38:58.572 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-13T10:38:58.572 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-13T10:38:58.572 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-13T10:38:58.572 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-13T10:38:58.573 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-13T10:38:58.573 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-13T10:38:58.573 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-13T10:38:58.573 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-13T10:38:58.573 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-13T10:38:58.574 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-13T10:38:58.574 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-13T10:38:58.574 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-13T10:38:58.574 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-13T10:38:58.574 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-13T10:38:58.574 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-13T10:38:58.575 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-13T10:38:58.575 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-13T10:38:58.575 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-13T10:38:58.575 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-13T10:38:58.575 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-13T10:38:58.576 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-13T10:38:58.576 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-13T10:38:58.576 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-13T10:38:58.576 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-13T10:38:58.576 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-13T10:38:58.576 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-13T10:38:58.577 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-13T10:38:58.577 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-13T10:38:58.577 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-13T10:38:58.577 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-13T10:38:58.577 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-13T10:38:58.578 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-13T10:38:58.578 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-13T10:38:58.578 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-13T10:38:58.578 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-13T10:38:58.578 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-13T10:38:58.579 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-13T10:38:58.579 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-13T10:38:58.579 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-13T10:38:58.579 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-13T10:38:58.579 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-13T10:38:58.580 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-13T10:38:58.580 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-13T10:38:58.580 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-13T10:38:58.580 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-13T10:38:58.580 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-13T10:38:58.580 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-13T10:38:58.581 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-13T10:38:58.581 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-13T10:38:58.581 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-13T10:38:58.581 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-13T10:38:58.581 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-13T10:38:58.581 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-13T10:38:58.582 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-13T10:38:58.582 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-13T10:38:58.582 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-13T10:38:58.582 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-13T10:38:58.583 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-13T10:38:58.583 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-13T10:38:58.583 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-13T10:38:58.583 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-13T10:38:58.583 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-13T10:38:58.584 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-13T10:38:58.584 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-13T10:38:58.584 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-13T10:38:58.584 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-13T10:38:58.584 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-13T10:38:58.584 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-13T10:38:58.585 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-13T10:38:58.585 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-13T10:38:58.585 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-13T10:38:58.585 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-13T10:38:58.585 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-13T10:38:58.586 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-13T10:38:58.586 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-13T10:38:58.586 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-13T10:38:58.586 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-13T10:38:58.586 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-13T10:38:58.587 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-13T10:38:58.587 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-13T10:38:58.587 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-13T10:38:58.587 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-13T10:38:58.587 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-13T10:38:58.588 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-13T10:38:58.588 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-13T10:38:58.588 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-13T10:38:58.588 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-13T10:38:58.588 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-13T10:38:58.589 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-13T10:38:58.589 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-13T10:38:58.589 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-13T10:38:58.589 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-13T10:38:58.589 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-13T10:38:58.590 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-13T10:38:58.590 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-13T10:38:58.590 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-13T10:38:58.591 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-13T10:38:58.591 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-13T10:38:58.591 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-13T10:38:58.591 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-13T10:38:58.591 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-13T10:38:58.592 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-13T10:38:58.592 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-13T10:38:58.592 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-13T10:38:58.592 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-13T10:38:58.592 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-13T10:38:58.592 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-13T10:38:58.593 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-13T10:38:58.593 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-13T10:38:58.593 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-13T10:38:58.593 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-13T10:38:58.593 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-13T10:38:58.594 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-13T10:38:58.594 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-13T10:38:58.594 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-13T10:38:58.594 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-13T10:38:58.594 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-13T10:38:58.595 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-13T10:38:58.595 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-13T10:38:58.595 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-13T10:38:58.595 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-13T10:38:58.595 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-13T10:38:58.596 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-13T10:38:58.596 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-13T10:38:58.596 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-13T10:38:58.596 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-13T10:38:58.596 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-13T10:38:58.596 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-13T10:38:58.597 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-13T10:38:58.597 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-13T10:38:58.597 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-13T10:38:58.597 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-13T10:38:58.597 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-13T10:38:58.598 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-13T10:38:58.598 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-13T10:38:58.598 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-13T10:38:58.598 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-13T10:38:58.599 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-13T10:38:58.599 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-13T10:38:58.599 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-13T10:38:58.599 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-13T10:38:58.600 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-13T10:38:58.600 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-13T10:38:58.600 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-13T10:38:58.601 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-13T10:38:58.601 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-13T10:38:58.601 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-13T10:38:58.601 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-13T10:38:58.601 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-13T10:38:58.602 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-13T10:38:58.602 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-13T10:38:58.602 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-13T10:38:58.602 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-13T10:38:58.602 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-13T10:38:58.603 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-13T10:38:58.603 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-13T10:38:58.603 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-13T10:38:58.603 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-13T10:38:58.603 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-13T10:38:58.604 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-13T10:38:58.604 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-13T10:38:58.604 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-13T10:38:58.604 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-13T10:38:58.604 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-13T10:38:58.604 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-13T10:38:58.605 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-13T10:38:58.605 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-13T10:38:58.605 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-13T10:38:58.605 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-13T10:38:58.605 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-13T10:38:58.606 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-13T10:38:58.606 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-13T10:38:58.606 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-13T10:38:58.606 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-13T10:38:58.606 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-13T10:38:58.607 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-13T10:38:58.607 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-13T10:38:58.607 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-13T10:38:58.607 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-13T10:38:58.607 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-13T10:38:58.608 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-13T10:38:58.608 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-13T10:38:58.608 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-13T10:38:58.608 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-13T10:38:58.609 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-13T10:38:58.609 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-13T10:38:58.609 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-13T10:38:58.609 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-13T10:38:58.609 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-13T10:38:58.609 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-13T10:38:58.610 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-13T10:38:58.610 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-13T10:38:58.610 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-13T10:38:58.610 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-13T10:38:58.611 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-13T10:38:58.611 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-13T10:38:58.611 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-13T10:38:58.611 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-13T10:38:58.611 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-13T10:38:58.612 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-13T10:38:58.612 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-13T10:38:58.612 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-13T10:38:58.612 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-13T10:38:58.612 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-13T10:38:58.613 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-13T10:38:58.613 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-13T10:38:58.613 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-13T10:38:58.613 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-13T10:38:58.613 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-13T10:38:58.614 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-13T10:38:58.614 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-13T10:38:58.614 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-13T10:38:58.614 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-13T10:38:58.614 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-13T10:38:58.615 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-13T10:38:58.615 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-13T10:38:58.615 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-13T10:38:58.615 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-13T10:38:58.615 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-13T10:38:58.616 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-13T10:38:58.616 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-13T10:38:58.616 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-13T10:38:58.616 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-13T10:38:58.616 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-13T10:38:58.617 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-13T10:38:58.617 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-13T10:38:58.617 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-13T10:38:58.617 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-13T10:38:58.618 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-13T10:38:58.618 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-13T10:38:58.618 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-13T10:38:58.618 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-13T10:38:58.619 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-13T10:38:58.619 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-13T10:38:58.619 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-13T10:38:58.619 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-13T10:38:58.620 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-13T10:38:58.620 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-13T10:38:58.620 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-13T10:38:58.620 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-13T10:38:58.620 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-13T10:38:58.621 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-13T10:38:58.621 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-13T10:38:58.621 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-13T10:38:58.621 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-13T10:38:58.621 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-13T10:38:58.622 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-13T10:38:58.622 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-13T10:38:58.622 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-13T10:38:58.622 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-13T10:38:58.622 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-13T10:38:58.623 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-13T10:38:58.623 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-13T10:38:58.623 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-13T10:38:58.623 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-13T10:38:58.623 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-13T10:38:58.623 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-13T10:38:58.624 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-13T10:38:58.624 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-13T10:38:58.624 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-13T10:38:58.624 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-13T10:38:58.624 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-13T10:38:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-13T10:38:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-13T10:38:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-13T10:38:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-13T10:38:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-13T10:38:58.625 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-13T10:38:58.626 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-13T10:38:58.626 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-13T10:38:58.626 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-13T10:38:58.627 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-13T10:38:58.627 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-13T10:38:58.627 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-13T10:38:58.627 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-13T10:38:58.627 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-13T10:38:58.628 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-13T10:38:58.628 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-13T10:38:58.628 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-13T10:38:58.629 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-13T10:38:58.629 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-13T10:38:58.629 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-13T10:38:58.630 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-13T10:38:58.630 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-13T10:38:58.630 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-13T10:38:58.631 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-13T10:38:58.631 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-13T10:38:58.632 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-13T10:38:58.632 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-13T10:38:58.632 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-13T10:38:58.633 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-13T10:38:58.633 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-13T10:38:58.633 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-13T10:38:58.634 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-13T10:38:58.634 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-13T10:38:58.634 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-13T10:38:58.635 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-13T10:38:58.635 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-13T10:38:58.635 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-13T10:38:58.636 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-13T10:38:58.636 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-13T10:38:58.637 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-13T10:38:58.637 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-13T10:38:58.637 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-13T10:38:58.637 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-13T10:38:58.638 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-13T10:38:58.638 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-13T10:38:58.638 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-13T10:38:58.638 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-13T10:38:58.639 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-13T10:38:58.639 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-13T10:38:58.639 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-13T10:38:58.639 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-13T10:38:58.640 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-13T10:38:58.640 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-13T10:38:58.640 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-13T10:38:58.640 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-13T10:38:58.640 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-13T10:38:58.641 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-13T10:38:58.641 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-13T10:38:58.641 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-13T10:38:58.641 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-13T10:38:58.642 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-13T10:38:58.642 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-13T10:38:58.643 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-13T10:38:58.643 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-13T10:38:58.643 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-13T10:38:58.644 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-13T10:38:58.644 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-13T10:38:58.644 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-13T10:38:58.645 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-13T10:38:58.645 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-13T10:38:58.646 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-13T10:38:58.646 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-13T10:38:58.646 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-13T10:38:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-13T10:38:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-13T10:38:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-13T10:38:58.648 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-13T10:38:58.648 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-13T10:38:58.649 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-13T10:38:58.649 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-13T10:38:58.649 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-13T10:38:58.650 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-13T10:38:58.650 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-13T10:38:58.650 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-13T10:38:58.651 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-13T10:38:58.651 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-13T10:38:58.651 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-13T10:38:58.652 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-13T10:38:58.652 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-13T10:38:58.653 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-13T10:38:58.653 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-13T10:38:58.653 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-13T10:38:58.654 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-13T10:38:58.654 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-13T10:38:58.654 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-13T10:38:58.655 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-13T10:38:58.655 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-13T10:38:58.656 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-13T10:38:58.656 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-13T10:38:58.656 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-13T10:38:58.657 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-13T10:38:58.657 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-13T10:38:58.657 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-13T10:38:58.658 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-13T10:38:58.658 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-13T10:38:58.658 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-13T10:38:58.659 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-13T10:38:58.659 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-13T10:38:58.659 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-13T10:38:58.660 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-13T10:38:58.660 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-13T10:38:58.661 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-13T10:38:58.661 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-13T10:38:58.661 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-13T10:38:58.662 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-13T10:38:58.662 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-13T10:38:58.662 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-13T10:38:58.663 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-13T10:38:58.663 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-13T10:38:58.664 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-13T10:38:58.664 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-13T10:38:58.664 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-13T10:38:58.665 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-13T10:38:58.665 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-13T10:38:58.665 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-13T10:38:58.666 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-13T10:38:58.666 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-13T10:38:58.666 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-13T10:38:58.667 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-13T10:38:58.667 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-13T10:38:58.667 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-13T10:38:58.668 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-13T10:38:58.668 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-13T10:38:58.668 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-13T10:38:58.668 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-13T10:38:58.669 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-13T10:38:58.669 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-13T10:38:58.669 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-13T10:38:58.670 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-13T10:38:58.670 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-13T10:38:58.670 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-13T10:38:58.670 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-13T10:38:58.671 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-13T10:38:58.671 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-13T10:38:58.671 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-13T10:38:58.671 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-13T10:38:58.672 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-13T10:38:58.672 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-13T10:38:58.672 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-13T10:38:58.672 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-13T10:38:58.672 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-13T10:38:58.673 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-13T10:38:58.673 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-13T10:38:58.673 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-13T10:38:58.673 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-13T10:38:58.673 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-13T10:38:58.673 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-13T10:38:58.674 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-13T10:38:58.674 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-13T10:38:58.674 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-13T10:38:58.674 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-13T10:38:58.675 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-13T10:38:58.675 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-13T10:38:58.675 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-13T10:38:58.675 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-13T10:38:58.677 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-13T10:38:58.677 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-13T10:38:58.678 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-13T10:38:58.678 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-13T10:38:58.678 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-13T10:38:58.678 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-13T10:38:58.678 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-13T10:38:58.679 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-13T10:38:58.679 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-13T10:38:58.679 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-13T10:38:58.679 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-13T10:38:58.679 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-13T10:38:58.680 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-13T10:38:58.680 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-13T10:38:58.680 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-13T10:38:58.680 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-13T10:38:58.680 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-13T10:38:58.681 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-13T10:38:58.681 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-13T10:38:58.681 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-13T10:38:58.681 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-13T10:38:58.681 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-13T10:38:58.682 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-13T10:38:58.682 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-13T10:38:58.682 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-13T10:38:58.682 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-13T10:38:58.682 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-13T10:38:58.683 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-13T10:38:58.683 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-13T10:38:58.683 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-13T10:38:58.683 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-13T10:38:58.683 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-13T10:38:58.684 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-13T10:38:58.684 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-13T10:38:58.684 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-13T10:38:58.684 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-13T10:38:58.684 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-13T10:38:58.685 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-13T10:38:58.685 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-13T10:38:58.685 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-13T10:38:58.685 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-13T10:38:58.685 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-13T10:38:58.686 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-13T10:38:58.686 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-13T10:38:58.686 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-13T10:38:58.686 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-13T10:38:58.686 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-13T10:38:58.686 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-13T10:38:58.687 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-13T10:38:58.687 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-13T10:38:58.687 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-13T10:38:58.687 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-13T10:38:58.687 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-13T10:38:58.688 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-13T10:38:58.688 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-13T10:38:58.688 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-13T10:38:58.688 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-13T10:38:58.688 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-13T10:38:58.689 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-13T10:38:58.689 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-13T10:38:58.689 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-13T10:38:58.689 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-13T10:38:58.689 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-13T10:38:58.690 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-13T10:38:58.690 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-13T10:38:58.690 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-13T10:38:58.690 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-13T10:38:58.690 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-13T10:38:58.690 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-13T10:38:58.691 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-13T10:38:58.691 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-13T10:38:58.691 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-13T10:38:58.691 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-13T10:38:58.692 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-13T10:38:58.692 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-13T10:38:58.692 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-13T10:38:58.692 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-13T10:38:58.692 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-13T10:38:58.692 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-13T10:38:58.693 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-13T10:38:58.693 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-13T10:38:58.693 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-13T10:38:58.693 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-13T10:38:58.693 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-13T10:38:58.694 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-13T10:38:58.694 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-13T10:38:58.694 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-13T10:38:58.694 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-13T10:38:58.694 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-13T10:38:58.695 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-13T10:38:58.695 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-13T10:38:58.695 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-13T10:38:58.695 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-13T10:38:58.696 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-13T10:38:58.696 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-13T10:38:58.696 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-13T10:38:58.696 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-13T10:38:58.696 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-13T10:38:58.696 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-13T10:38:58.697 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-13T10:38:58.697 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-13T10:38:58.697 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-13T10:38:58.697 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-13T10:38:58.697 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-13T10:38:58.698 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-13T10:38:58.698 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-13T10:38:58.698 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-13T10:38:58.698 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-13T10:38:58.698 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-13T10:38:58.699 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-13T10:38:58.699 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-13T10:38:58.699 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-13T10:38:58.699 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-13T10:38:58.700 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-13T10:38:58.700 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-13T10:38:58.700 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-13T10:38:58.700 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-13T10:38:58.700 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-13T10:38:58.700 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-13T10:38:58.701 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-13T10:38:58.701 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-13T10:38:58.701 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-13T10:38:58.701 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-13T10:38:58.701 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-13T10:38:58.702 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-13T10:38:58.702 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-13T10:38:58.702 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-13T10:38:58.702 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-13T10:38:58.702 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-13T10:38:58.703 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-13T10:38:58.703 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-13T10:38:58.703 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-13T10:38:58.703 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-13T10:38:58.703 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-13T10:38:58.704 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-13T10:38:58.704 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-13T10:38:58.704 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-13T10:38:58.704 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-13T10:38:58.704 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-13T10:38:58.705 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-13T10:38:58.705 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-13T10:38:58.705 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-13T10:38:58.705 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-13T10:38:58.705 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-13T10:38:58.706 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-13T10:38:58.706 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-13T10:38:58.706 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-13T10:38:58.706 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-13T10:38:58.706 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-13T10:38:58.707 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-13T10:38:58.707 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-13T10:38:58.707 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-13T10:38:58.707 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-13T10:38:58.707 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-13T10:38:58.707 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-13T10:38:58.708 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-13T10:38:58.708 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-13T10:38:58.708 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-13T10:38:58.708 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-13T10:38:58.708 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-13T10:38:58.709 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-13T10:38:58.709 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-13T10:38:58.709 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-13T10:38:58.709 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-13T10:38:58.709 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-13T10:38:58.710 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-13T10:38:58.710 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-13T10:38:58.710 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-13T10:38:58.710 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-13T10:38:58.710 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-13T10:38:58.710 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-13T10:38:58.711 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-13T10:38:58.711 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-13T10:38:58.711 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-13T10:38:58.711 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-13T10:38:58.711 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-13T10:38:58.712 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-13T10:38:58.712 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-13T10:38:58.712 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-13T10:38:58.712 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-13T10:38:58.712 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-13T10:38:58.713 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-13T10:38:58.713 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-13T10:38:58.713 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-13T10:38:58.713 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-13T10:38:58.713 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-13T10:38:58.713 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-13T10:38:58.714 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-13T10:38:58.714 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-13T10:38:58.714 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-13T10:38:58.714 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-13T10:38:58.714 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-13T10:38:58.715 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-13T10:38:58.715 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-13T10:38:58.715 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-13T10:38:58.715 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-13T10:38:58.715 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-13T10:38:58.716 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-13T10:38:58.716 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-13T10:38:58.716 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-13T10:38:58.716 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-13T10:38:58.716 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-13T10:38:58.717 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-13T10:38:58.717 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-13T10:38:58.717 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-13T10:38:58.717 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-13T10:38:58.717 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-13T10:38:58.717 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-13T10:38:58.718 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-13T10:38:58.718 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-13T10:38:58.718 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-13T10:38:58.718 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-13T10:38:58.718 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-13T10:38:58.719 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-13T10:38:58.719 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-13T10:38:58.719 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-13T10:38:58.719 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-13T10:38:58.719 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-13T10:38:58.720 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-13T10:38:58.720 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-13T10:38:58.720 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-13T10:38:58.720 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-13T10:38:58.720 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-13T10:38:58.721 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-13T10:38:58.721 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-13T10:38:58.721 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-13T10:38:58.721 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-13T10:38:58.721 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-13T10:38:58.722 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-13T10:38:58.722 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-13T10:38:58.722 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-13T10:38:58.722 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-13T10:38:58.722 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-13T10:38:58.723 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-13T10:38:58.723 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-13T10:38:58.723 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-13T10:38:58.723 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-13T10:38:58.723 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-13T10:38:58.723 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-13T10:38:58.724 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-13T10:38:58.724 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-13T10:38:58.724 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-13T10:38:58.724 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-13T10:38:58.724 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-13T10:38:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-13T10:38:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-13T10:38:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-13T10:38:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-13T10:38:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-13T10:38:58.726 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-13T10:38:58.726 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-13T10:38:58.726 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-13T10:38:58.726 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-13T10:38:58.726 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-13T10:38:58.727 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-13T10:38:58.727 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-13T10:38:58.727 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-13T10:38:58.727 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-13T10:38:58.727 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-13T10:38:58.728 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-13T10:38:58.728 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-13T10:38:58.728 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-13T10:38:58.728 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-13T10:38:58.728 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-13T10:38:58.728 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-13T10:38:58.729 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-13T10:38:58.729 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-13T10:38:58.729 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-13T10:38:58.729 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-13T10:38:58.729 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-13T10:38:58.730 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-13T10:38:58.730 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-13T10:38:58.730 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-13T10:38:58.730 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-13T10:38:58.730 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-13T10:38:58.731 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-13T10:38:58.731 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-13T10:38:58.731 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-13T10:38:58.731 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-13T10:38:58.731 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-13T10:38:58.732 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-13T10:38:58.732 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-13T10:38:58.732 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-13T10:38:58.732 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-13T10:38:58.732 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-13T10:38:58.732 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-13T10:38:58.733 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-13T10:38:58.733 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-13T10:38:58.733 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-13T10:38:58.733 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-13T10:38:58.733 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-13T10:38:58.734 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-13T10:38:58.734 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-13T10:38:58.734 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-13T10:38:58.734 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-13T10:38:58.734 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-13T10:38:58.735 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-13T10:38:58.735 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-13T10:38:58.735 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-13T10:38:58.735 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-13T10:38:58.735 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-13T10:38:58.736 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-13T10:38:58.736 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-13T10:38:58.736 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-13T10:38:58.736 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-13T10:38:58.736 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-13T10:38:58.737 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-13T10:38:58.737 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-13T10:38:58.737 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-13T10:38:58.737 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-13T10:38:58.737 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-13T10:38:58.738 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-13T10:38:58.738 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-13T10:38:58.738 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-13T10:38:58.738 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-13T10:38:58.738 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-13T10:38:58.739 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-13T10:38:58.739 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-13T10:38:58.739 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-13T10:38:58.739 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-13T10:38:58.739 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-13T10:38:58.740 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-13T10:38:58.740 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-13T10:38:58.740 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-13T10:38:58.740 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-13T10:38:58.740 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-13T10:38:58.741 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-13T10:38:58.741 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-13T10:38:58.741 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-13T10:38:58.741 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-13T10:38:58.741 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-13T10:38:58.742 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-13T10:38:58.742 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-13T10:38:58.742 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-13T10:38:58.742 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-13T10:38:58.742 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-13T10:38:58.743 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-13T10:38:58.743 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-13T10:38:58.743 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-13T10:38:58.743 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-13T10:38:58.743 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-13T10:38:58.744 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-13T10:38:58.744 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-13T10:38:58.744 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-13T10:38:58.744 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-13T10:38:58.744 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-13T10:38:58.745 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-13T10:38:58.745 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-13T10:38:58.745 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-13T10:38:58.745 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-13T10:38:58.745 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-13T10:38:58.746 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-13T10:38:58.746 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-13T10:38:58.746 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-13T10:38:58.746 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-13T10:38:58.746 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-13T10:38:58.747 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-13T10:38:58.747 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-13T10:38:58.747 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-13T10:38:58.747 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-13T10:38:58.747 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-13T10:38:58.748 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-13T10:38:58.748 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-13T10:38:58.748 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-13T10:38:58.748 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-13T10:38:58.748 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-13T10:38:58.749 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-13T10:38:58.749 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-13T10:38:58.749 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-13T10:38:58.749 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-13T10:38:58.749 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-13T10:38:58.750 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-13T10:38:58.750 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-13T10:38:58.750 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-13T10:38:58.750 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-13T10:38:58.750 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-13T10:38:58.751 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-13T10:38:58.751 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-13T10:38:58.751 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-13T10:38:58.751 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-13T10:38:58.751 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-13T10:38:58.752 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-13T10:38:58.752 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-13T10:38:58.752 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-13T10:38:58.752 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-13T10:38:58.752 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-13T10:38:58.753 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-13T10:38:58.753 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-13T10:38:58.753 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-13T10:38:58.753 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-13T10:38:58.753 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-13T10:38:58.754 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-13T10:38:58.754 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-13T10:38:58.754 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-13T10:38:58.754 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-13T10:38:58.754 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-13T10:38:58.755 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-13T10:38:58.755 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-13T10:38:58.755 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-13T10:38:58.755 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-13T10:38:58.755 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-13T10:38:58.756 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-13T10:38:58.756 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-13T10:38:58.756 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-13T10:38:58.756 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-13T10:38:58.757 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-13T10:38:58.757 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-13T10:38:58.757 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-13T10:38:58.757 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-13T10:38:58.757 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-13T10:38:58.757 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-13T10:38:58.758 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-13T10:38:58.758 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-13T10:38:58.758 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-13T10:38:58.758 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-13T10:38:58.758 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-13T10:38:58.759 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-13T10:38:58.759 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-13T10:38:58.759 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-13T10:38:58.759 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-13T10:38:58.760 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-13T10:38:58.760 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-13T10:38:58.760 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-13T10:38:58.760 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-13T10:38:58.761 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-13T10:38:58.761 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-13T10:38:58.761 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-13T10:38:58.761 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-13T10:38:58.761 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-13T10:38:58.761 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-13T10:38:58.762 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-13T10:38:58.762 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-13T10:38:58.762 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-13T10:38:58.762 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-13T10:38:58.763 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-13T10:38:58.763 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-13T10:38:58.763 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-13T10:38:58.763 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-13T10:38:58.763 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-13T10:38:58.764 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-13T10:38:58.764 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-13T10:38:58.764 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-13T10:38:58.764 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-13T10:38:58.764 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-13T10:38:58.765 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-13T10:38:58.765 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-13T10:38:58.765 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-13T10:38:58.765 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-13T10:38:58.765 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-13T10:38:58.765 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-13T10:38:58.766 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-13T10:38:58.766 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-13T10:38:58.766 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-13T10:38:58.766 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-13T10:38:58.766 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-13T10:38:58.767 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-13T10:38:58.767 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-13T10:38:58.767 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-13T10:38:58.767 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-13T10:38:58.767 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-13T10:38:58.768 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-13T10:38:58.768 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-13T10:38:58.768 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-13T10:38:58.768 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-13T10:38:58.768 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-13T10:38:58.769 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-13T10:38:58.769 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-13T10:38:58.769 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-13T10:38:58.769 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-13T10:38:58.769 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-13T10:38:58.769 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-13T10:38:58.770 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-13T10:38:58.770 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-13T10:38:58.770 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-13T10:38:58.770 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-13T10:38:58.770 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-13T10:38:58.771 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-13T10:38:58.771 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-13T10:38:58.771 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-13T10:38:58.771 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-13T10:38:58.771 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-13T10:38:58.772 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-13T10:38:58.772 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-13T10:38:58.772 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-13T10:38:58.772 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-13T10:38:58.772 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-13T10:38:58.773 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-13T10:38:58.773 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-13T10:38:58.773 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-13T10:38:58.773 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-13T10:38:58.773 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-13T10:38:58.773 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-13T10:38:58.774 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-13T10:38:58.774 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-13T10:38:58.774 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-13T10:38:58.774 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-13T10:38:58.774 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-13T10:38:58.775 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-13T10:38:58.775 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-13T10:38:58.775 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-13T10:38:58.775 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-13T10:38:58.775 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-13T10:38:58.776 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-13T10:38:58.776 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-13T10:38:58.776 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-13T10:38:58.776 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-13T10:38:58.776 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-13T10:38:58.777 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-13T10:38:58.777 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-13T10:38:58.777 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-13T10:38:58.777 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-13T10:38:58.777 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-13T10:38:58.778 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-13T10:38:58.778 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-13T10:38:58.778 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-13T10:38:58.778 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-13T10:38:58.779 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-13T10:38:58.779 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-13T10:38:58.779 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-13T10:38:58.779 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-13T10:38:58.779 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-13T10:38:58.780 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-13T10:38:58.780 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-13T10:38:58.780 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-13T10:38:58.780 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-13T10:38:58.780 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-13T10:38:58.781 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-13T10:38:58.781 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-13T10:38:58.781 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-13T10:38:58.781 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-13T10:38:58.781 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-13T10:38:58.782 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-13T10:38:58.782 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-13T10:38:58.782 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-13T10:38:58.782 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-13T10:38:58.782 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-13T10:38:58.783 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-13T10:38:58.783 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-13T10:38:58.783 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-13T10:38:58.783 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-13T10:38:58.783 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-13T10:38:58.784 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-13T10:38:58.784 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-13T10:38:58.784 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-13T10:38:58.784 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-13T10:38:58.784 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-13T10:38:58.784 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-13T10:38:58.785 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-13T10:38:58.785 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-13T10:38:58.785 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-13T10:38:58.785 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-13T10:38:58.786 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-13T10:38:58.786 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-13T10:38:58.786 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-13T10:38:58.786 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-13T10:38:58.786 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-13T10:38:58.786 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-13T10:38:58.787 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-13T10:38:58.787 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-13T10:38:58.787 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-13T10:38:58.787 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-13T10:38:58.787 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-13T10:38:58.788 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-13T10:38:58.788 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-13T10:38:58.788 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-13T10:38:58.788 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-13T10:38:58.788 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-13T10:38:58.789 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-13T10:38:58.789 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-13T10:38:58.789 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-13T10:38:58.789 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-13T10:38:58.789 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-13T10:38:58.790 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-13T10:38:58.790 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-13T10:38:58.790 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-13T10:38:58.790 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-13T10:38:58.790 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-13T10:38:58.791 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-13T10:38:58.791 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-13T10:38:58.791 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-13T10:38:58.791 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-13T10:38:58.791 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-13T10:38:58.792 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-13T10:38:58.792 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-13T10:38:58.792 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-13T10:38:58.792 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-13T10:38:58.792 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-13T10:38:58.792 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-13T10:38:58.793 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-13T10:38:58.793 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-13T10:38:58.793 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-13T10:38:58.793 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-13T10:38:58.793 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-13T10:38:58.794 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-13T10:38:58.794 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-13T10:38:58.794 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-13T10:38:58.794 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-13T10:38:58.794 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-13T10:38:58.795 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-13T10:38:58.795 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-13T10:38:58.795 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-13T10:38:58.795 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-13T10:38:58.795 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-13T10:38:58.795 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-13T10:38:58.796 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-13T10:38:58.796 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-13T10:38:58.796 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-13T10:38:58.796 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-13T10:38:58.796 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-13T10:38:58.797 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-13T10:38:58.797 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-13T10:38:58.797 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-13T10:38:58.797 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-13T10:38:58.797 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-13T10:38:58.797 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-13T10:38:58.798 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-13T10:38:58.798 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-13T10:38:58.798 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-13T10:38:58.798 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-13T10:38:58.798 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-13T10:38:58.799 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-13T10:38:58.799 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-13T10:38:58.799 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-13T10:38:58.799 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-13T10:38:58.799 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-13T10:38:58.800 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-13T10:38:58.800 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-13T10:38:58.800 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-13T10:38:58.800 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-13T10:38:58.800 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-13T10:38:58.801 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-13T10:38:58.801 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-13T10:38:58.801 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-13T10:38:58.801 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-13T10:38:58.801 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-13T10:38:58.801 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-13T10:38:58.802 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-13T10:38:58.802 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-13T10:38:58.802 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-13T10:38:58.802 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-13T10:38:58.802 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-13T10:38:58.803 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-13T10:38:58.803 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-13T10:38:58.803 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-13T10:38:58.803 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-13T10:38:58.803 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-13T10:38:58.804 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-13T10:38:58.804 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-13T10:38:58.804 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-13T10:38:58.804 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-13T10:38:58.804 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-13T10:38:58.804 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-13T10:38:58.805 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-13T10:38:58.805 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-13T10:38:58.805 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-13T10:38:58.805 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-13T10:38:58.805 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-13T10:38:58.806 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-13T10:38:58.806 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-13T10:38:58.806 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-13T10:38:58.806 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-13T10:38:58.806 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-13T10:38:58.807 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-13T10:38:58.807 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-13T10:38:58.807 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-13T10:38:58.807 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-13T10:38:58.807 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-13T10:38:58.808 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-13T10:38:58.808 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-13T10:38:58.808 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-13T10:38:58.808 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-13T10:38:58.808 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-13T10:38:58.809 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-13T10:38:58.809 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-13T10:38:58.809 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-13T10:38:58.809 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-13T10:38:58.809 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-13T10:38:58.810 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-13T10:38:58.810 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-13T10:38:58.810 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-13T10:38:58.810 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-13T10:38:58.810 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-13T10:38:58.810 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-13T10:38:58.811 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-13T10:38:58.811 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-13T10:38:58.811 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-13T10:38:58.811 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-13T10:38:58.811 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-13T10:38:58.812 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-13T10:38:58.812 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-13T10:38:58.812 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-13T10:38:58.812 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-13T10:38:58.812 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-13T10:38:58.813 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-13T10:38:58.813 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-13T10:38:58.813 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-13T10:38:58.813 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-13T10:38:58.813 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-13T10:38:58.813 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-13T10:38:58.814 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-13T10:38:58.814 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-13T10:38:58.814 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-13T10:38:58.814 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-13T10:38:58.814 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-13T10:38:58.815 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-13T10:38:58.815 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-13T10:38:58.815 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-13T10:38:58.815 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-13T10:38:58.815 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-13T10:38:58.816 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-13T10:38:58.816 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-13T10:38:58.816 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-13T10:38:58.816 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-13T10:38:58.816 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-13T10:38:58.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-13T10:38:58.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-13T10:38:58.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-13T10:38:58.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-13T10:38:58.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-13T10:38:58.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-13T10:38:58.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-13T10:38:58.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-13T10:38:58.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-13T10:38:58.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-13T10:38:58.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-13T10:38:58.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-13T10:38:58.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-13T10:38:58.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-13T10:38:58.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-13T10:38:58.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-13T10:38:58.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-13T10:38:58.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-13T10:38:58.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-13T10:38:58.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-13T10:38:58.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-13T10:38:58.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-13T10:38:58.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-13T10:38:58.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-13T10:38:58.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-13T10:38:58.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-13T10:38:58.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-13T10:38:58.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-13T10:38:58.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-13T10:38:58.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-13T10:38:58.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-13T10:38:58.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-13T10:38:58.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-13T10:38:58.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-13T10:38:58.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-13T10:38:58.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-13T10:38:58.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-13T10:38:58.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-13T10:38:58.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-13T10:38:58.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-13T10:38:58.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-13T10:38:58.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-13T10:38:58.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-13T10:38:58.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-13T10:38:58.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-13T10:38:58.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-13T10:38:58.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-13T10:38:58.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-13T10:38:58.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-13T10:38:58.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-13T10:38:58.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-13T10:38:58.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-13T10:38:58.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-13T10:38:58.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-13T10:38:58.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-13T10:38:58.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-13T10:38:58.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-13T10:38:58.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-13T10:38:58.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-13T10:38:58.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-13T10:38:58.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-13T10:38:58.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-13T10:38:58.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-13T10:38:58.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-13T10:38:58.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-13T10:38:58.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-13T10:38:58.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-13T10:38:58.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-13T10:38:58.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-13T10:38:58.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-13T10:38:58.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-13T10:38:58.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-13T10:38:58.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-13T10:38:58.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-13T10:38:58.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-13T10:38:58.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-13T10:38:58.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-13T10:38:58.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-13T10:38:58.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-13T10:38:58.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-13T10:38:58.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-13T10:38:58.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-13T10:38:58.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-13T10:38:58.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-13T10:38:58.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-13T10:38:58.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-13T10:38:58.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-13T10:38:58.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-13T10:38:58.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-13T10:38:58.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-13T10:38:58.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-13T10:38:58.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-13T10:38:58.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-13T10:38:58.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-13T10:38:58.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-13T10:38:58.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-13T10:38:58.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-13T10:38:58.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-13T10:38:58.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-13T10:38:58.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-13T10:38:58.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-13T10:38:58.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-13T10:38:58.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-13T10:38:58.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-13T10:38:58.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-13T10:38:58.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-13T10:38:58.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-13T10:38:58.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-13T10:38:58.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-13T10:38:58.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-13T10:38:58.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-13T10:38:58.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-13T10:38:58.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-13T10:38:58.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-13T10:38:58.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-13T10:38:58.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-13T10:38:58.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-13T10:38:58.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-13T10:38:58.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-13T10:38:58.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-13T10:38:58.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-13T10:38:58.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-13T10:38:58.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-13T10:38:58.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-13T10:38:58.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-13T10:38:58.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-13T10:38:58.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-13T10:38:58.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-13T10:38:58.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-13T10:38:58.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-13T10:38:58.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-13T10:38:58.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-13T10:38:58.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-13T10:38:58.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-13T10:38:58.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-13T10:38:58.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-13T10:38:58.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-13T10:38:58.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-13T10:38:58.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-13T10:38:58.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-13T10:38:58.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-13T10:38:58.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-13T10:38:58.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-13T10:38:58.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-13T10:38:58.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-13T10:38:58.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-13T10:38:58.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-13T10:38:58.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-13T10:38:58.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-13T10:38:58.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-13T10:38:58.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-13T10:38:58.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-13T10:38:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-13T10:38:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-13T10:38:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-13T10:38:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-13T10:38:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-13T10:38:58.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-13T10:38:58.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-13T10:38:58.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-13T10:38:58.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-13T10:38:58.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-13T10:38:58.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-13T10:38:58.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-13T10:38:58.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-13T10:38:58.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-13T10:38:58.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-13T10:38:58.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-13T10:38:58.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-13T10:38:58.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-13T10:38:58.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-13T10:38:58.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-13T10:38:58.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-13T10:38:58.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-13T10:38:58.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-13T10:38:58.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-13T10:38:58.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-13T10:38:58.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-13T10:38:58.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-13T10:38:58.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-13T10:38:58.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-13T10:38:58.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-13T10:38:58.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-13T10:38:58.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-13T10:38:58.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-13T10:38:58.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-13T10:38:58.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-13T10:38:58.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-13T10:38:58.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-13T10:38:58.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-13T10:38:58.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-13T10:38:58.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-13T10:38:58.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-13T10:38:58.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-13T10:38:58.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-13T10:38:58.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-13T10:38:58.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-13T10:38:58.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-13T10:38:58.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-13T10:38:58.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-13T10:38:58.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-13T10:38:58.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-13T10:38:58.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-13T10:38:58.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-13T10:38:58.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-13T10:38:58.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-13T10:38:58.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-13T10:38:58.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-13T10:38:58.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-13T10:38:58.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-13T10:38:58.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-13T10:38:58.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-13T10:38:58.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-13T10:38:58.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-13T10:38:58.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-13T10:38:58.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-13T10:38:58.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-13T10:38:58.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-13T10:38:58.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-13T10:38:58.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-13T10:38:58.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-13T10:38:58.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-13T10:38:58.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-13T10:38:58.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-13T10:38:58.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-13T10:38:58.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-13T10:38:58.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-13T10:38:58.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-13T10:38:58.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-13T10:38:58.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-13T10:38:58.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-13T10:38:58.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-13T10:38:58.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-13T10:38:58.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-13T10:38:58.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-13T10:38:58.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-13T10:38:58.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-13T10:38:58.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-13T10:38:58.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-13T10:38:58.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-13T10:38:58.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-13T10:38:58.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-13T10:38:58.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-13T10:38:58.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-13T10:38:58.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-13T10:38:58.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-13T10:38:58.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-13T10:38:58.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-13T10:38:58.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-13T10:38:58.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-13T10:38:58.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-13T10:38:58.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-13T10:38:58.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-13T10:38:58.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-13T10:38:58.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-13T10:38:58.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-13T10:38:58.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-13T10:38:58.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-13T10:38:58.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-13T10:38:58.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-13T10:38:58.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-13T10:38:58.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-13T10:38:58.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-13T10:38:58.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-13T10:38:58.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-13T10:38:58.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-13T10:38:58.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-13T10:38:58.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-13T10:38:58.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-13T10:38:58.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-13T10:38:58.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-13T10:38:58.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-13T10:38:58.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-13T10:38:58.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-13T10:38:58.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-13T10:38:58.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-13T10:38:58.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-13T10:38:58.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-13T10:38:58.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-13T10:38:58.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-13T10:38:58.872 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-13T10:38:58.872 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-13T10:38:58.872 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-13T10:38:58.872 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-13T10:38:58.872 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-13T10:38:58.873 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-13T10:38:58.873 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-13T10:38:58.873 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-13T10:38:58.873 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-13T10:38:58.873 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-13T10:38:58.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-13T10:38:58.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-13T10:38:58.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-13T10:38:58.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-13T10:38:58.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-13T10:38:58.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-13T10:38:58.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-13T10:38:58.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-13T10:38:58.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-13T10:38:58.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-13T10:38:58.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-13T10:38:58.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-13T10:38:58.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-13T10:38:58.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-13T10:38:58.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-13T10:38:58.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-13T10:38:58.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-13T10:38:58.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-13T10:38:58.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-13T10:38:58.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-13T10:38:58.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-13T10:38:58.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-13T10:38:58.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-13T10:38:58.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-13T10:38:58.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-13T10:38:58.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-13T10:38:58.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-13T10:38:58.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-13T10:38:58.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-13T10:38:58.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-13T10:38:58.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-13T10:38:58.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-13T10:38:58.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-13T10:38:58.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-13T10:38:58.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-13T10:38:58.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-13T10:38:58.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-13T10:38:58.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-13T10:38:58.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-13T10:38:58.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-13T10:38:58.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-13T10:38:58.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-13T10:38:58.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-13T10:38:58.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-13T10:38:58.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-13T10:38:58.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-13T10:38:58.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-13T10:38:58.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-13T10:38:58.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-13T10:38:58.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-13T10:38:58.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-13T10:38:58.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-13T10:38:58.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-13T10:38:58.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-13T10:38:58.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-13T10:38:58.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-13T10:38:58.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-13T10:38:58.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-13T10:38:58.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-13T10:38:58.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-13T10:38:58.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-13T10:38:58.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-13T10:38:58.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-13T10:38:58.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-13T10:38:58.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-13T10:38:58.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-13T10:38:58.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-13T10:38:58.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-13T10:38:58.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-13T10:38:58.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-13T10:38:58.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-13T10:38:58.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-13T10:38:58.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-13T10:38:58.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-13T10:38:58.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-13T10:38:58.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-13T10:38:58.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-13T10:38:58.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-13T10:38:58.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-13T10:38:58.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-13T10:38:58.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-13T10:38:58.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-13T10:38:58.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-13T10:38:58.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-13T10:38:58.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-13T10:38:58.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-13T10:38:58.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-13T10:38:58.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-13T10:38:58.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-13T10:38:58.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-13T10:38:58.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-13T10:38:58.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-13T10:38:58.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-13T10:38:58.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-13T10:38:58.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-13T10:38:58.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-13T10:38:58.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-13T10:38:58.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-13T10:38:58.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-13T10:38:58.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-13T10:38:58.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-13T10:38:58.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-13T10:38:58.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-13T10:38:58.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-13T10:38:58.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-13T10:38:58.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-13T10:38:58.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-13T10:38:58.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-13T10:38:58.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-13T10:38:58.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-13T10:38:58.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-13T10:38:58.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-13T10:38:58.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-13T10:38:58.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-13T10:38:58.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-13T10:38:58.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-13T10:38:58.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-13T10:38:58.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-13T10:38:58.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-13T10:38:58.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-13T10:38:58.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-13T10:38:58.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-13T10:38:58.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-13T10:38:58.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-13T10:38:58.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-13T10:38:58.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-13T10:38:58.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-13T10:38:58.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-13T10:38:58.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-13T10:38:58.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-13T10:38:58.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-13T10:38:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-13T10:38:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-13T10:38:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-13T10:38:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-13T10:38:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-13T10:38:58.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-13T10:38:58.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-13T10:38:58.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-13T10:38:58.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-13T10:38:58.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-13T10:38:58.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-13T10:38:58.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-13T10:38:58.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-13T10:38:58.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-13T10:38:58.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-13T10:38:58.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-13T10:38:58.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-13T10:38:58.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-13T10:38:58.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-13T10:38:58.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-13T10:38:58.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-13T10:38:58.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-13T10:38:58.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-13T10:38:58.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-13T10:38:58.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-13T10:38:58.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-13T10:38:58.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-13T10:38:58.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-13T10:38:58.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-13T10:38:58.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-13T10:38:58.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-13T10:38:58.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-13T10:38:58.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-13T10:38:58.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-13T10:38:58.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-13T10:38:58.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-13T10:38:58.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-13T10:38:58.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-13T10:38:58.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-13T10:38:58.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-13T10:38:58.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-13T10:38:58.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-13T10:38:58.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-13T10:38:58.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-13T10:38:58.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-13T10:38:58.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-13T10:38:58.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-13T10:38:58.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-13T10:38:58.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-13T10:38:58.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-13T10:38:58.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-13T10:38:58.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-13T10:38:58.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-13T10:38:58.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-13T10:38:58.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-13T10:38:58.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-13T10:38:58.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-13T10:38:58.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-13T10:38:58.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-13T10:38:58.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-13T10:38:58.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-13T10:38:58.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-13T10:38:58.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-13T10:38:58.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-13T10:38:58.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-13T10:38:58.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-13T10:38:58.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-13T10:38:58.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-13T10:38:58.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-13T10:38:58.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-13T10:38:58.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-13T10:38:58.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-13T10:38:58.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-13T10:38:58.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-13T10:38:58.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-13T10:38:58.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-13T10:38:58.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-13T10:38:58.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-13T10:38:58.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-13T10:38:58.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-13T10:38:58.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-13T10:38:58.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-13T10:38:58.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-13T10:38:58.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-13T10:38:58.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-13T10:38:58.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-13T10:38:58.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-13T10:38:58.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-13T10:38:58.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-13T10:38:58.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-13T10:38:58.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-13T10:38:58.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-13T10:38:58.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-13T10:38:58.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-13T10:38:58.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-13T10:38:58.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-13T10:38:58.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-13T10:38:58.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-13T10:38:58.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-13T10:38:58.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-13T10:38:58.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-13T10:38:58.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-13T10:38:58.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-13T10:38:58.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-13T10:38:58.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-13T10:38:58.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-13T10:38:58.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-13T10:38:58.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-13T10:38:58.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-13T10:38:58.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-13T10:38:58.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-13T10:38:58.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-13T10:38:58.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-13T10:38:58.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-13T10:38:58.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-13T10:38:58.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-13T10:38:58.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-13T10:38:58.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-13T10:38:58.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-13T10:38:58.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-13T10:38:58.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-13T10:38:58.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-13T10:38:58.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-13T10:38:58.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-13T10:38:58.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-13T10:38:58.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-13T10:38:58.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-13T10:38:58.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-13T10:38:58.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-13T10:38:58.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-13T10:38:58.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-13T10:38:58.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-13T10:38:58.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-13T10:38:58.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-13T10:38:58.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-13T10:38:58.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-13T10:38:58.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-13T10:38:58.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-13T10:38:58.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-13T10:38:58.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-13T10:38:58.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-13T10:38:58.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-13T10:38:58.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-13T10:38:58.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-13T10:38:58.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-13T10:38:58.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-13T10:38:58.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-13T10:38:58.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-13T10:38:58.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-13T10:38:58.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-13T10:38:58.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-13T10:38:58.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-13T10:38:58.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-13T10:38:58.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-13T10:38:58.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-13T10:38:58.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-13T10:38:58.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-13T10:38:58.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-13T10:38:58.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-13T10:38:58.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-13T10:38:58.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-13T10:38:58.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-13T10:38:58.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-13T10:38:58.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-13T10:38:58.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-13T10:38:58.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-13T10:38:58.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-13T10:38:58.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-13T10:38:58.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-13T10:38:58.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-13T10:38:58.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-13T10:38:58.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-13T10:38:58.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-13T10:38:58.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-13T10:38:58.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-13T10:38:58.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-13T10:38:58.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-13T10:38:58.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-13T10:38:58.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-13T10:38:58.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-13T10:38:58.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-13T10:38:58.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-13T10:38:58.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-13T10:38:58.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-13T10:38:58.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-13T10:38:58.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-13T10:38:58.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-13T10:38:58.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-13T10:38:58.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-13T10:38:58.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-13T10:38:58.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-13T10:38:58.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-13T10:38:58.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-13T10:38:58.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-13T10:38:58.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-13T10:38:58.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-13T10:38:58.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-13T10:38:58.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-13T10:38:58.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-13T10:38:58.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-13T10:38:58.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-13T10:38:58.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-13T10:38:58.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-13T10:38:58.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-13T10:38:58.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-13T10:38:58.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-13T10:38:58.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-13T10:38:58.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-13T10:38:58.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-13T10:38:58.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-13T10:38:58.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-13T10:38:58.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-13T10:38:58.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-13T10:38:58.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-13T10:38:58.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-13T10:38:58.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-13T10:38:58.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-13T10:38:58.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-13T10:38:58.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-13T10:38:58.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-13T10:38:58.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-13T10:38:58.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-13T10:38:58.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-13T10:38:58.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-13T10:38:58.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-13T10:38:58.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.alias.bin 2023-11-13T10:38:58.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-13T10:38:58.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-13T10:38:58.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-13T10:38:58.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-13T10:38:58.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-13T10:38:58.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-13T10:38:58.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-13T10:38:58.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-13T10:38:58.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-13T10:38:58.949 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:58.950 DEBUG:teuthology.orchestra.run.smithi016:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-13T10:38:58.976 INFO:teuthology.orchestra.run.smithi016.stdout:postinstall scriptlet (using /bin/sh): 2023-11-13T10:38:58.976 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-13T10:38:58.977 INFO:teuthology.orchestra.run.smithi016.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-13T10:38:58.977 INFO:teuthology.orchestra.run.smithi016.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-13T10:38:58.977 INFO:teuthology.orchestra.run.smithi016.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:58.977 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-13T10:38:58.977 INFO:teuthology.orchestra.run.smithi016.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-13T10:38:58.978 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2023-11-13T10:38:58.978 INFO:teuthology.orchestra.run.smithi016.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-13T10:38:58.978 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-13T10:38:58.978 INFO:teuthology.orchestra.run.smithi016.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-13T10:38:58.979 INFO:teuthology.orchestra.run.smithi016.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-13T10:38:58.979 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:58.979 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2023-11-13T10:38:58.979 INFO:teuthology.orchestra.run.smithi016.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-13T10:38:58.979 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-13T10:38:58.980 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:58.980 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2023-11-13T10:38:58.981 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2023-11-13T10:38:59.065 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2023-11-13T10:38:59.065 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2023-11-13T10:38:59.065 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2023-11-13T10:38:59.066 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 68.el9 2023-11-13T10:38:59.066 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2023-11-13T10:38:59.066 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-13T10:38:59.066 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2023-11-13T10:38:59.066 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8264254 2023-11-13T10:38:59.067 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2023-11-13T10:38:59.067 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-13T10:38:59.067 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-13T10:38:59.067 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-13T10:38:59.067 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-13T10:38:59.068 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2023-11-13T10:38:59.068 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2023-11-13T10:38:59.068 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-13T10:38:59.068 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2023-11-13T10:38:59.068 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2023-11-13T10:38:59.069 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:38:59.069 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-13T10:38:59.069 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-13T10:38:59.069 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-13T10:38:59.069 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2023-11-13T10:38:59.070 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:38:59.070 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2023-11-13T10:38:59.071 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-13T10:38:59.071 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:38:59.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-13T10:38:59.753 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2023-11-13T10:39:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-13T10:39:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:done 2023-11-13T10:39:01.220 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-11-13T10:39:01.275 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.dVhFbQE7sS 2023-11-13T10:39:01.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dVhFbQE7sS 2023-11-13T10:39:01.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.dVhFbQE7sS 2023-11-13T10:39:01.496 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.dVhFbQE7sS is 6KB 2023-11-13T10:39:01.545 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.dVhFbQE7sS 2023-11-13T10:39:01.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2023-11-13T10:39:01.630 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-13T10:39:01.630 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-13T10:39:01.630 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-13T10:39:01.631 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-13T10:39:01.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:39:01.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2023-11-13T10:39:05.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-11-13T10:39:14.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-11-13T10:39:14.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:39:17.658 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-11-13T10:39:29.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-11-13T10:39:29.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:39:31.793 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-13T10:39:31.794 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-13T10:39:31.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:39:48.091 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-11-13T10:39:50.138 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-11-13T10:39:59.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-13T10:39:59.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:40:03.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-11-13T10:40:03.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:40:03.533 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-11-13T10:40:04.142 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-11-13T10:40:04.143 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-13T10:40:04.143 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2023-11-13T10:40:04.183 INFO:teuthology.orchestra.run.smithi184.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:40:04.184 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:40:04.184 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-13T10:40:04.184 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-13T10:40:05.185 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-13T10:40:05.186 DEBUG:teuthology.orchestra.run.smithi184:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-13T10:40:05.303 INFO:teuthology.orchestra.run.smithi184.stdout:ttyS1 2023-11-13T10:40:05.342 DEBUG:teuthology.parallel:result is None 2023-11-13T10:40:59.151 DEBUG:teuthology.orchestra.remote:timed out 2023-11-13T10:41:11.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-13T10:41:11.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:41:11.571 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-11-13T10:41:12.230 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-13T10:41:12.231 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-13T10:41:12.231 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-11-13T10:41:12.286 INFO:teuthology.orchestra.run.smithi016.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:41:12.286 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-13T10:41:12.287 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-13T10:41:12.287 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-13T10:41:13.288 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-13T10:41:13.288 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-13T10:41:13.431 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2023-11-13T10:41:13.476 DEBUG:teuthology.parallel:result is None 2023-11-13T10:41:13.476 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-13T10:41:13.484 INFO:teuthology.task.internal:Creating test directory... 2023-11-13T10:41:13.484 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-13T10:41:13.494 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-13T10:41:13.558 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-13T10:41:13.568 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-13T10:41:13.574 INFO:teuthology.task.internal:Creating archive directory... 2023-11-13T10:41:13.575 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-13T10:41:13.584 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-13T10:41:13.659 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-13T10:41:13.666 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-13T10:41:13.666 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-13T10:41:13.672 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-13T10:41:13.793 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-13T10:41:13.805 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-13T10:41:13.842 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-13T10:41:13.848 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-13T10:41:13.852 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-13T10:41:13.860 INFO:teuthology.task.internal:Configuring sudo... 2023-11-13T10:41:13.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-13T10:41:13.869 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-13T10:41:13.971 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-13T10:41:13.980 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-13T10:41:13.980 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-13T10:41:13.987 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-13T10:41:14.045 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-13T10:41:14.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-13T10:41:14.252 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-13T10:41:14.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-13T10:41:14.451 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:41:14.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-13T10:41:14.557 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-13T10:41:14.650 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-13T10:41:14.755 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-13T10:41:14.841 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-13T10:41:14.944 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:41:14.944 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-13T10:41:15.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2023-11-13T10:41:15.054 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2023-11-13T10:41:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-13T10:41:15.231 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-13T10:41:15.589 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-13T10:41:15.597 INFO:teuthology.task.internal:Starting timer... 2023-11-13T10:41:15.598 INFO:teuthology.run_tasks:Running task pcp... 2023-11-13T10:41:15.647 INFO:teuthology.run_tasks:Running task selinux... 2023-11-13T10:41:15.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2023-11-13T10:41:15.790 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2023-11-13T10:41:15.795 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2023-11-13T10:41:15.953 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2023-11-13T10:41:15.959 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-13T10:41:15.959 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2023-11-13T10:41:16.023 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2023-11-13T10:41:16.025 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2023-11-13T10:41:16.095 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2023-11-13T10:41:16.096 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2023-11-13T10:41:16.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-13T10:41:16.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T10:41:16.198 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-13T10:41:16.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T10:41:16.297 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-13T10:41:16.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2023-11-13T10:41:16.420 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2023-11-13T10:41:16.526 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-13T10:41:16.537 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-13T10:41:16.537 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-13T10:41:16.556 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-13T10:41:16.557 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-13T10:52:20.325 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2023-11-13T10:52:20.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-13T10:52:20.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:52:20.479 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-11-13T10:52:20.640 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-13T10:52:20.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-11-13T10:52:20.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T10:52:20.784 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-11-13T10:52:20.941 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-11-13T10:52:20.942 INFO:teuthology.run_tasks:Running task clock... 2023-11-13T10:52:20.979 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-13T10:52:20.980 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-13T10:52:20.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-13T10:52:20.999 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-13T10:52:21.000 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-13T10:52:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-13T10:52:21.106 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-13T10:52:21.146 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-13T10:52:21.158 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-13T10:52:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2023-11-13T10:52:21.251 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2023-11-13T10:52:21.290 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2023-11-13T10:52:21.291 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2023-11-13T10:52:21.340 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-13T10:52:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-13T10:52:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-13T10:52:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-13T10:52:21.519 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2023-11-13T10:52:21.525 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-13T10:52:21.525 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2023-11-13T10:52:21.526 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T10:52:21.526 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T10:52:21.526 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T10:52:21.526 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T10:52:21.537 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2023-11-13T10:52:21.545 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-13T10:52:21.546 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-11-13T10:52:21.546 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T10:52:21.546 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T10:52:21.546 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T10:52:21.546 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T10:52:21.548 INFO:teuthology.run_tasks:Running task install... 2023-11-13T10:52:21.558 DEBUG:teuthology.task.install:project ceph 2023-11-13T10:52:21.559 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b'}} 2023-11-13T10:52:21.559 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b'} 2023-11-13T10:52:21.565 INFO:teuthology.task.install:Using flavor: default 2023-11-13T10:52:21.589 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-13T10:52:21.590 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-13T10:52:21.591 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b', 'tag': None, 'wait_for_package': False} 2023-11-13T10:52:21.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=35f0debb3cc28955b7132ef4c509cd138a09d56b 2023-11-13T10:52:21.594 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b', 'tag': None, 'wait_for_package': False} 2023-11-13T10:52:21.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=35f0debb3cc28955b7132ef4c509cd138a09d56b 2023-11-13T10:52:21.748 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231110.052303/35f0debb3cc28955b7132ef4c509cd138a09d56b/centos/9/flavors/default/ 2023-11-13T10:52:21.748 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7288.g35f0debb 2023-11-13T10:52:21.834 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231110.052303/35f0debb3cc28955b7132ef4c509cd138a09d56b/centos/9/flavors/default/ 2023-11-13T10:52:21.834 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7288.g35f0debb 2023-11-13T10:52:21.994 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231110.052303/35f0debb3cc28955b7132ef4c509cd138a09d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231110.052303/35f0debb3cc28955b7132ef4c509cd138a09d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231110.052303/35f0debb3cc28955b7132ef4c509cd138a09d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-13T10:52:21.995 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:52:21.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-13T10:52:22.087 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231110.052303/35f0debb3cc28955b7132ef4c509cd138a09d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231110.052303/35f0debb3cc28955b7132ef4c509cd138a09d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231110.052303/35f0debb3cc28955b7132ef4c509cd138a09d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-13T10:52:22.087 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:52:22.087 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-13T10:52:22.114 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-13T10:52:22.115 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/35f0debb3cc28955b7132ef4c509cd138a09d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-13T10:52:22.189 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-13T10:52:22.190 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/35f0debb3cc28955b7132ef4c509cd138a09d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-13T10:52:22.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-13T10:52:22.331 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-13T10:52:22.414 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-13T10:52:22.467 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-13T10:52:22.513 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2023-11-13T10:52:22.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2023-11-13T10:52:22.554 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2023-11-13T10:52:22.560 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2023-11-13T10:52:22.889 INFO:teuthology.orchestra.run.smithi016.stdout:72 files removed 2023-11-13T10:52:22.908 INFO:teuthology.orchestra.run.smithi184.stdout:72 files removed 2023-11-13T10:52:22.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-13T10:52:22.949 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-13T10:52:23.877 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 187 kB/s | 78 kB 00:00 2023-11-13T10:52:23.925 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 169 kB/s | 78 kB 00:00 2023-11-13T10:52:24.428 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 21 kB/s | 9.5 kB 00:00 2023-11-13T10:52:24.542 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 18 kB/s | 9.5 kB 00:00 2023-11-13T10:52:24.879 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2023-11-13T10:52:25.068 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2023-11-13T10:52:25.567 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 10 MB/s | 6.3 MB 00:00 2023-11-13T10:52:25.622 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 13 MB/s | 6.3 MB 00:00 2023-11-13T10:52:27.124 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2023-11-13T10:52:27.172 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2023-11-13T10:52:28.512 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 7.9 MB 00:00 2023-11-13T10:52:29.386 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 7.9 MB 00:01 2023-11-13T10:52:31.101 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 14 MB/s | 18 MB 00:01 2023-11-13T10:52:31.381 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 23 MB/s | 18 MB 00:00 2023-11-13T10:52:35.370 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.8 MB 00:00 2023-11-13T10:52:36.221 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 5.8 MB 00:01 2023-11-13T10:52:37.003 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 15 kB 00:00 2023-11-13T10:52:37.680 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 15 kB 00:00 2023-11-13T10:52:39.722 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 19 MB 00:02 2023-11-13T10:52:40.496 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 19 MB 00:02 2023-11-13T10:52:45.483 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2023-11-13T10:52:46.173 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2023-11-13T10:52:48.430 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-13T10:52:48.432 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-13T10:52:48.457 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-13T10:52:48.458 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-13T10:52:48.458 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-13T10:52:48.459 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-13T10:52:48.466 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-13T10:52:48.468 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-13T10:52:48.468 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-13T10:52:48.553 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-11-13T10:52:48.576 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-11-13T10:52:48.576 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-11-13T10:52:48.576 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-11-13T10:52:48.577 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-11-13T10:52:48.577 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-13T10:52:48.577 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 6.5 k 2023-11-13T10:52:48.577 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 5.2 M 2023-11-13T10:52:48.577 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 860 k 2023-11-13T10:52:48.578 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 141 k 2023-11-13T10:52:48.578 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 1.5 M 2023-11-13T10:52:48.578 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 132 k 2023-11-13T10:52:48.578 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 1.8 M 2023-11-13T10:52:48.579 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 7.4 M 2023-11-13T10:52:48.579 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 50 k 2023-11-13T10:52:48.579 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 8.6 M 2023-11-13T10:52:48.579 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 41 M 2023-11-13T10:52:48.579 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 262 k 2023-11-13T10:52:48.580 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 766 k 2023-11-13T10:52:48.580 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 217 k 2023-11-13T10:52:48.580 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 18 k 2023-11-13T10:52:48.581 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-11-13T10:52:48.581 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-13T10:52:48.581 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-13T10:52:48.581 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 31 k 2023-11-13T10:52:48.581 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 719 k 2023-11-13T10:52:48.582 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 126 k 2023-11-13T10:52:48.582 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-13T10:52:48.582 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-11-13T10:52:48.583 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-11-13T10:52:48.583 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-13T10:52:48.583 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-13T10:52:48.584 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-13T10:52:48.584 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 162 k 2023-11-13T10:52:48.584 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 320 k 2023-11-13T10:52:48.584 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 299 k 2023-11-13T10:52:48.584 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 99 k 2023-11-13T10:52:48.585 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 85 k 2023-11-13T10:52:48.585 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 3.0 M 2023-11-13T10:52:48.585 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 170 k 2023-11-13T10:52:48.586 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-11-13T10:52:48.586 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-11-13T10:52:48.586 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-13T10:52:48.586 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-13T10:52:48.586 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-13T10:52:48.587 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-13T10:52:48.587 INFO:teuthology.orchestra.run.smithi016.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-13T10:52:48.587 INFO:teuthology.orchestra.run.smithi016.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-13T10:52:48.587 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-13T10:52:48.587 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 3.3 M 2023-11-13T10:52:48.588 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 3.0 M 2023-11-13T10:52:48.588 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-13T10:52:48.588 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-11-13T10:52:48.588 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-11-13T10:52:48.588 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-13T10:52:48.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 19 M 2023-11-13T10:52:48.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 23 k 2023-11-13T10:52:48.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 2.1 M 2023-11-13T10:52:48.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 243 k 2023-11-13T10:52:48.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 4.4 M 2023-11-13T10:52:48.590 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 18 M 2023-11-13T10:52:48.590 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 15 k 2023-11-13T10:52:48.590 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 24 k 2023-11-13T10:52:48.590 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-13T10:52:48.590 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-13T10:52:48.591 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-13T10:52:48.591 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-13T10:52:48.591 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-13T10:52:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-13T10:52:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-13T10:52:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-13T10:52:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 164 k 2023-11-13T10:52:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-13T10:52:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-13T10:52:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-13T10:52:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-13T10:52:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-13T10:52:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 475 k 2023-11-13T10:52:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-13T10:52:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 4.8 M 2023-11-13T10:52:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-13T10:52:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-13T10:52:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-13T10:52:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-13T10:52:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-13T10:52:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-13T10:52:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-13T10:52:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-13T10:52:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-13T10:52:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-13T10:52:48.619 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-13T10:52:48.619 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-13T10:52:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 45 k 2023-11-13T10:52:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 127 k 2023-11-13T10:52:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-13T10:52:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-13T10:52:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-13T10:52:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-13T10:52:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-13T10:52:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-13T10:52:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-13T10:52:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-13T10:52:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-13T10:52:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-13T10:52:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-13T10:52:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-13T10:52:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-13T10:52:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-13T10:52:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-13T10:52:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-13T10:52:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-13T10:52:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-13T10:52:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-13T10:52:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-13T10:52:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-13T10:52:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-13T10:52:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-13T10:52:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-13T10:52:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-13T10:52:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-13T10:52:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-13T10:52:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-13T10:52:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-13T10:52:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-13T10:52:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-13T10:52:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-13T10:52:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-13T10:52:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-13T10:52:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-13T10:52:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-13T10:52:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-13T10:52:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-13T10:52:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-13T10:52:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-13T10:52:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-13T10:52:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-13T10:52:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-13T10:52:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-13T10:52:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-13T10:52:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-13T10:52:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-13T10:52:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-13T10:52:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-13T10:52:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-13T10:52:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-13T10:52:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-13T10:52:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-13T10:52:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-11-13T10:52:48.632 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2023-11-13T10:52:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-13T10:52:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:52:48.633 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-11-13T10:52:48.633 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-11-13T10:52:48.633 INFO:teuthology.orchestra.run.smithi016.stdout:Install 132 Packages 2023-11-13T10:52:48.633 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 10 Packages 2023-11-13T10:52:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:52:48.635 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 200 M 2023-11-13T10:52:48.635 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-11-13T10:52:48.992 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-13T10:52:48.992 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-13T10:52:49.011 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-13T10:52:49.013 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-13T10:52:49.014 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-13T10:52:49.016 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-13T10:52:49.018 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-13T10:52:49.022 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-13T10:52:49.024 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-13T10:52:49.100 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2023-11-13T10:52:49.117 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2023-11-13T10:52:49.117 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2023-11-13T10:52:49.117 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2023-11-13T10:52:49.118 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2023-11-13T10:52:49.118 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-13T10:52:49.118 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 6.5 k 2023-11-13T10:52:49.119 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 5.2 M 2023-11-13T10:52:49.119 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 860 k 2023-11-13T10:52:49.119 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 141 k 2023-11-13T10:52:49.119 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 1.5 M 2023-11-13T10:52:49.119 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 132 k 2023-11-13T10:52:49.120 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 1.8 M 2023-11-13T10:52:49.120 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 7.4 M 2023-11-13T10:52:49.120 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 50 k 2023-11-13T10:52:49.121 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 8.6 M 2023-11-13T10:52:49.121 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 41 M 2023-11-13T10:52:49.121 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 262 k 2023-11-13T10:52:49.121 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 766 k 2023-11-13T10:52:49.122 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 217 k 2023-11-13T10:52:49.122 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 18 k 2023-11-13T10:52:49.122 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-11-13T10:52:49.122 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-13T10:52:49.122 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-13T10:52:49.123 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 31 k 2023-11-13T10:52:49.123 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 719 k 2023-11-13T10:52:49.123 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 126 k 2023-11-13T10:52:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-13T10:52:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-11-13T10:52:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-11-13T10:52:49.124 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-13T10:52:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-13T10:52:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-13T10:52:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 162 k 2023-11-13T10:52:49.125 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 320 k 2023-11-13T10:52:49.126 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 299 k 2023-11-13T10:52:49.126 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 99 k 2023-11-13T10:52:49.126 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 85 k 2023-11-13T10:52:49.126 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 3.0 M 2023-11-13T10:52:49.127 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 170 k 2023-11-13T10:52:49.127 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-11-13T10:52:49.127 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2023-11-13T10:52:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-13T10:52:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-13T10:52:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-13T10:52:49.128 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-13T10:52:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-13T10:52:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-13T10:52:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-13T10:52:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 3.3 M 2023-11-13T10:52:49.129 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 3.0 M 2023-11-13T10:52:49.130 INFO:teuthology.orchestra.run.smithi184.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-13T10:52:49.139 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2023-11-13T10:52:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-11-13T10:52:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-13T10:52:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 19 M 2023-11-13T10:52:49.140 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 23 k 2023-11-13T10:52:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 2.1 M 2023-11-13T10:52:49.141 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 243 k 2023-11-13T10:52:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 4.4 M 2023-11-13T10:52:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 18 M 2023-11-13T10:52:49.149 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7288.g35f0debb.el9 ceph-noarch 15 k 2023-11-13T10:52:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 24 k 2023-11-13T10:52:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-13T10:52:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-13T10:52:49.150 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-13T10:52:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-13T10:52:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-13T10:52:49.151 INFO:teuthology.orchestra.run.smithi184.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-13T10:52:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-13T10:52:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-13T10:52:49.155 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 164 k 2023-11-13T10:52:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-13T10:52:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-13T10:52:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-13T10:52:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-13T10:52:49.156 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-13T10:52:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 475 k 2023-11-13T10:52:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-13T10:52:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 4.8 M 2023-11-13T10:52:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-13T10:52:49.157 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-13T10:52:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-13T10:52:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-13T10:52:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-13T10:52:49.158 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-13T10:52:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-13T10:52:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-13T10:52:49.159 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-13T10:52:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-13T10:52:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-13T10:52:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-13T10:52:49.161 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 45 k 2023-11-13T10:52:49.162 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:18.0.0-7288.g35f0debb.el9 ceph 127 k 2023-11-13T10:52:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-13T10:52:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-13T10:52:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-13T10:52:49.163 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-13T10:52:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-13T10:52:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-13T10:52:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-13T10:52:49.164 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-13T10:52:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-13T10:52:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-13T10:52:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-13T10:52:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-13T10:52:49.165 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-13T10:52:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-13T10:52:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-13T10:52:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-13T10:52:49.166 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-13T10:52:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-13T10:52:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-13T10:52:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-13T10:52:49.167 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-13T10:52:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-13T10:52:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-13T10:52:49.180 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-13T10:52:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-13T10:52:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-13T10:52:49.181 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-13T10:52:49.183 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-13T10:52:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-13T10:52:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-13T10:52:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-13T10:52:49.184 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-13T10:52:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-13T10:52:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-13T10:52:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-13T10:52:49.185 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-13T10:52:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-13T10:52:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-13T10:52:49.186 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-13T10:52:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-13T10:52:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-13T10:52:49.187 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-13T10:52:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-13T10:52:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-13T10:52:49.188 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-13T10:52:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-13T10:52:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-13T10:52:49.189 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-13T10:52:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-13T10:52:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-13T10:52:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-13T10:52:49.190 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-13T10:52:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-13T10:52:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-11-13T10:52:49.191 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2023-11-13T10:52:49.191 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-13T10:52:49.192 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:52:49.192 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2023-11-13T10:52:49.192 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2023-11-13T10:52:49.193 INFO:teuthology.orchestra.run.smithi184.stdout:Install 132 Packages 2023-11-13T10:52:49.193 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 10 Packages 2023-11-13T10:52:49.193 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:52:49.194 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 200 M 2023-11-13T10:52:49.194 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2023-11-13T10:52:49.490 INFO:teuthology.orchestra.run.smithi016.stdout:(1/142): ceph-18.0.0-7288.g35f0debb.el9.x86_64. 25 kB/s | 6.5 kB 00:00 2023-11-13T10:52:49.715 INFO:teuthology.orchestra.run.smithi016.stdout:(2/142): ceph-fuse-18.0.0-7288.g35f0debb.el9.x8 3.7 MB/s | 860 kB 00:00 2023-11-13T10:52:49.790 INFO:teuthology.orchestra.run.smithi016.stdout:(3/142): ceph-immutable-object-cache-18.0.0-728 1.9 MB/s | 141 kB 00:00 2023-11-13T10:52:50.082 INFO:teuthology.orchestra.run.smithi016.stdout:(4/142): ceph-base-18.0.0-7288.g35f0debb.el9.x8 6.1 MB/s | 5.2 MB 00:00 2023-11-13T10:52:50.116 INFO:teuthology.orchestra.run.smithi184.stdout:(1/142): ceph-18.0.0-7288.g35f0debb.el9.x86_64. 14 kB/s | 6.5 kB 00:00 2023-11-13T10:52:50.166 INFO:teuthology.orchestra.run.smithi016.stdout:(5/142): ceph-mds-18.0.0-7288.g35f0debb.el9.x86 5.6 MB/s | 2.1 MB 00:00 2023-11-13T10:52:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:(6/142): ceph-mgr-18.0.0-7288.g35f0debb.el9.x86 8.4 MB/s | 1.5 MB 00:00 2023-11-13T10:52:50.833 INFO:teuthology.orchestra.run.smithi016.stdout:(7/142): ceph-mon-18.0.0-7288.g35f0debb.el9.x86 6.6 MB/s | 4.4 MB 00:00 2023-11-13T10:52:51.325 INFO:teuthology.orchestra.run.smithi016.stdout:(8/142): ceph-common-18.0.0-7288.g35f0debb.el9. 9.1 MB/s | 19 MB 00:02 2023-11-13T10:52:51.408 INFO:teuthology.orchestra.run.smithi016.stdout:(9/142): ceph-selinux-18.0.0-7288.g35f0debb.el9 293 kB/s | 24 kB 00:00 2023-11-13T10:52:52.050 INFO:teuthology.orchestra.run.smithi016.stdout:(10/142): ceph-radosgw-18.0.0-7288.g35f0debb.el 7.1 MB/s | 8.6 MB 00:01 2023-11-13T10:52:52.191 INFO:teuthology.orchestra.run.smithi184.stdout:(2/142): ceph-fuse-18.0.0-7288.g35f0debb.el9.x8 415 kB/s | 860 kB 00:02 2023-11-13T10:52:52.320 INFO:teuthology.orchestra.run.smithi016.stdout:(11/142): ceph-osd-18.0.0-7288.g35f0debb.el9.x8 8.6 MB/s | 18 MB 00:02 2023-11-13T10:52:52.351 INFO:teuthology.orchestra.run.smithi016.stdout:(12/142): cephfs-mirror-18.0.0-7288.g35f0debb.e 725 kB/s | 217 kB 00:00 2023-11-13T10:52:52.376 INFO:teuthology.orchestra.run.smithi016.stdout:(13/142): libcephfs-devel-18.0.0-7288.g35f0debb 543 kB/s | 31 kB 00:00 2023-11-13T10:52:52.435 INFO:teuthology.orchestra.run.smithi016.stdout:(14/142): libcephsqlite-18.0.0-7288.g35f0debb.e 2.8 MB/s | 164 kB 00:00 2023-11-13T10:52:52.475 INFO:teuthology.orchestra.run.smithi184.stdout:(3/142): ceph-base-18.0.0-7288.g35f0debb.el9.x8 1.8 MB/s | 5.2 MB 00:02 2023-11-13T10:52:52.476 INFO:teuthology.orchestra.run.smithi016.stdout:(15/142): libcephfs2-18.0.0-7288.g35f0debb.el9. 5.6 MB/s | 719 kB 00:00 2023-11-13T10:52:52.508 INFO:teuthology.orchestra.run.smithi184.stdout:(4/142): ceph-immutable-object-cache-18.0.0-728 446 kB/s | 141 kB 00:00 2023-11-13T10:52:52.510 INFO:teuthology.orchestra.run.smithi016.stdout:(16/142): librados-devel-18.0.0-7288.g35f0debb. 1.7 MB/s | 126 kB 00:00 2023-11-13T10:52:52.561 INFO:teuthology.orchestra.run.smithi016.stdout:(17/142): libradosstriper1-18.0.0-7288.g35f0deb 5.6 MB/s | 475 kB 00:00 2023-11-13T10:52:52.638 INFO:teuthology.orchestra.run.smithi016.stdout:(18/142): python3-ceph-argparse-18.0.0-7288.g35 608 kB/s | 45 kB 00:00 2023-11-13T10:52:52.720 INFO:teuthology.orchestra.run.smithi016.stdout:(19/142): python3-ceph-common-18.0.0-7288.g35f0 1.5 MB/s | 127 kB 00:00 2023-11-13T10:52:52.811 INFO:teuthology.orchestra.run.smithi016.stdout:(20/142): python3-cephfs-18.0.0-7288.g35f0debb. 1.7 MB/s | 162 kB 00:00 2023-11-13T10:52:52.912 INFO:teuthology.orchestra.run.smithi016.stdout:(21/142): python3-rados-18.0.0-7288.g35f0debb.e 3.2 MB/s | 320 kB 00:00 2023-11-13T10:52:53.020 INFO:teuthology.orchestra.run.smithi016.stdout:(22/142): librgw2-18.0.0-7288.g35f0debb.el9.x86 9.4 MB/s | 4.8 MB 00:00 2023-11-13T10:52:53.062 INFO:teuthology.orchestra.run.smithi016.stdout:(23/142): python3-rbd-18.0.0-7288.g35f0debb.el9 2.0 MB/s | 299 kB 00:00 2023-11-13T10:52:53.067 INFO:teuthology.orchestra.run.smithi184.stdout:(5/142): ceph-mgr-18.0.0-7288.g35f0debb.el9.x86 2.6 MB/s | 1.5 MB 00:00 2023-11-13T10:52:53.097 INFO:teuthology.orchestra.run.smithi016.stdout:(24/142): python3-rgw-18.0.0-7288.g35f0debb.el9 1.3 MB/s | 99 kB 00:00 2023-11-13T10:52:53.121 INFO:teuthology.orchestra.run.smithi016.stdout:(25/142): rbd-fuse-18.0.0-7288.g35f0debb.el9.x8 1.4 MB/s | 85 kB 00:00 2023-11-13T10:52:53.205 INFO:teuthology.orchestra.run.smithi016.stdout:(26/142): rbd-nbd-18.0.0-7288.g35f0debb.el9.x86 2.0 MB/s | 170 kB 00:00 2023-11-13T10:52:53.298 INFO:teuthology.orchestra.run.smithi016.stdout:(27/142): ceph-grafana-dashboards-18.0.0-7288.g 252 kB/s | 23 kB 00:00 2023-11-13T10:52:53.432 INFO:teuthology.orchestra.run.smithi016.stdout:(28/142): rbd-mirror-18.0.0-7288.g35f0debb.el9. 9.0 MB/s | 3.0 MB 00:00 2023-11-13T10:52:53.457 INFO:teuthology.orchestra.run.smithi016.stdout:(29/142): ceph-mgr-cephadm-18.0.0-7288.g35f0deb 840 kB/s | 132 kB 00:00 2023-11-13T10:52:53.657 INFO:teuthology.orchestra.run.smithi016.stdout:(30/142): ceph-mgr-dashboard-18.0.0-7288.g35f0d 7.8 MB/s | 1.8 MB 00:00 2023-11-13T10:52:53.741 INFO:teuthology.orchestra.run.smithi016.stdout:(31/142): ceph-mgr-modules-core-18.0.0-7288.g35 2.9 MB/s | 243 kB 00:00 2023-11-13T10:52:53.817 INFO:teuthology.orchestra.run.smithi016.stdout:(32/142): ceph-mgr-rook-18.0.0-7288.g35f0debb.e 678 kB/s | 50 kB 00:00 2023-11-13T10:52:53.892 INFO:teuthology.orchestra.run.smithi016.stdout:(33/142): ceph-prometheus-alerts-18.0.0-7288.g3 204 kB/s | 15 kB 00:00 2023-11-13T10:52:53.984 INFO:teuthology.orchestra.run.smithi016.stdout:(34/142): ceph-volume-18.0.0-7288.g35f0debb.el9 2.8 MB/s | 262 kB 00:00 2023-11-13T10:52:54.101 INFO:teuthology.orchestra.run.smithi016.stdout:(35/142): cephadm-18.0.0-7288.g35f0debb.el9.noa 6.5 MB/s | 766 kB 00:00 2023-11-13T10:52:54.176 INFO:teuthology.orchestra.run.smithi016.stdout:(36/142): cephfs-top-18.0.0-7288.g35f0debb.el9. 246 kB/s | 18 kB 00:00 2023-11-13T10:52:54.360 INFO:teuthology.orchestra.run.smithi016.stdout:(37/142): bison-3.7.4-5.el9.x86_64.rpm 4.9 MB/s | 920 kB 00:00 2023-11-13T10:52:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:(38/142): boost-program-options-1.75.0-8.el9.x8 3.2 MB/s | 106 kB 00:00 2023-11-13T10:52:54.542 INFO:teuthology.orchestra.run.smithi184.stdout:(6/142): ceph-mon-18.0.0-7288.g35f0debb.el9.x86 3.0 MB/s | 4.4 MB 00:01 2023-11-13T10:52:54.552 INFO:teuthology.orchestra.run.smithi016.stdout:(39/142): ceph-mgr-diskprediction-local-18.0.0- 6.8 MB/s | 7.4 MB 00:01 2023-11-13T10:52:54.585 INFO:teuthology.orchestra.run.smithi016.stdout:(40/142): flex-2.6.4-9.el9.x86_64.rpm 1.6 MB/s | 307 kB 00:00 2023-11-13T10:52:54.611 INFO:teuthology.orchestra.run.smithi016.stdout:(41/142): flexiblas-3.0.4-8.el9.x86_64.rpm 579 kB/s | 32 kB 00:00 2023-11-13T10:52:54.711 INFO:teuthology.orchestra.run.smithi016.stdout:(42/142): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2023-11-13T10:52:54.728 INFO:teuthology.orchestra.run.smithi016.stdout:(43/142): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 17 kB 00:00 2023-11-13T10:52:54.753 INFO:teuthology.orchestra.run.smithi016.stdout:(44/142): libaio-devel-0.3.111-13.el9.x86_64.rp 426 kB/s | 10 kB 00:00 2023-11-13T10:52:54.787 INFO:teuthology.orchestra.run.smithi016.stdout:(45/142): jq-1.6-16.el9.x86_64.rpm 3.2 MB/s | 186 kB 00:00 2023-11-13T10:52:54.813 INFO:teuthology.orchestra.run.smithi016.stdout:(46/142): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.8 MB/s | 159 kB 00:00 2023-11-13T10:52:54.843 INFO:teuthology.orchestra.run.smithi184.stdout:(7/142): ceph-mds-18.0.0-7288.g35f0debb.el9.x86 903 kB/s | 2.1 MB 00:02 2023-11-13T10:52:54.871 INFO:teuthology.orchestra.run.smithi016.stdout:(47/142): librdkafka-1.6.1-102.el9.x86_64.rpm 7.9 MB/s | 661 kB 00:00 2023-11-13T10:52:54.913 INFO:teuthology.orchestra.run.smithi016.stdout:(48/142): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 578 kB 00:00 2023-11-13T10:52:54.938 INFO:teuthology.orchestra.run.smithi016.stdout:(49/142): libzstd-devel-1.5.1-2.el9.x86_64.rpm 761 kB/s | 46 kB 00:00 2023-11-13T10:52:54.971 INFO:teuthology.orchestra.run.smithi016.stdout:(50/142): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-11-13T10:52:55.005 INFO:teuthology.orchestra.run.smithi016.stdout:(51/142): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-11-13T10:52:55.031 INFO:teuthology.orchestra.run.smithi016.stdout:(52/142): openblas-0.3.21-2.el9.x86_64.rpm 607 kB/s | 34 kB 00:00 2023-11-13T10:52:55.175 INFO:teuthology.orchestra.run.smithi016.stdout:(53/142): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2023-11-13T10:52:55.214 INFO:teuthology.orchestra.run.smithi016.stdout:(54/142): postgresql-13.11-1.el9.x86_64.rpm 9.0 MB/s | 1.5 MB 00:00 2023-11-13T10:52:55.257 INFO:teuthology.orchestra.run.smithi016.stdout:(55/142): postgresql-private-libs-13.11-1.el9.x 3.2 MB/s | 134 kB 00:00 2023-11-13T10:52:55.299 INFO:teuthology.orchestra.run.smithi016.stdout:(56/142): postgresql-contrib-13.11-1.el9.x86_64 9.5 MB/s | 811 kB 00:00 2023-11-13T10:52:55.457 INFO:teuthology.orchestra.run.smithi016.stdout:(57/142): postgresql-server-13.11-1.el9.x86_64. 29 MB/s | 5.7 MB 00:00 2023-11-13T10:52:55.591 INFO:teuthology.orchestra.run.smithi016.stdout:(58/142): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2023-11-13T10:52:55.616 INFO:teuthology.orchestra.run.smithi016.stdout:(59/142): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 205 kB 00:00 2023-11-13T10:52:55.667 INFO:teuthology.orchestra.run.smithi016.stdout:(60/142): python3-jsonpatch-1.21-16.el9.noarch. 502 kB/s | 24 kB 00:00 2023-11-13T10:52:55.695 INFO:teuthology.orchestra.run.smithi016.stdout:(61/142): python3-jinja2-2.11.3-4.el9.noarch.rp 3.0 MB/s | 228 kB 00:00 2023-11-13T10:52:55.717 INFO:teuthology.orchestra.run.smithi016.stdout:(62/142): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 18 kB 00:00 2023-11-13T10:52:55.742 INFO:teuthology.orchestra.run.smithi016.stdout:(63/142): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-13T10:52:55.768 INFO:teuthology.orchestra.run.smithi016.stdout:(64/142): python3-markupsafe-1.1.1-12.el9.x86_6 658 kB/s | 32 kB 00:00 2023-11-13T10:52:55.901 INFO:teuthology.orchestra.run.smithi016.stdout:(65/142): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2023-11-13T10:52:55.929 INFO:teuthology.orchestra.run.smithi016.stdout:(66/142): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2023-11-13T10:52:55.968 INFO:teuthology.orchestra.run.smithi016.stdout:(67/142): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2023-11-13T10:52:56.006 INFO:teuthology.orchestra.run.smithi016.stdout:(68/142): python3-oauthlib-3.1.1-5.el9.noarch.r 2.3 MB/s | 174 kB 00:00 2023-11-13T10:52:56.027 INFO:teuthology.orchestra.run.smithi016.stdout:(69/142): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2023-11-13T10:52:56.061 INFO:teuthology.orchestra.run.smithi016.stdout:(70/142): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 211 kB 00:00 2023-11-13T10:52:56.086 INFO:teuthology.orchestra.run.smithi016.stdout:(71/142): python3-requests-oauthlib-1.3.0-12.el 767 kB/s | 43 kB 00:00 2023-11-13T10:52:56.353 INFO:teuthology.orchestra.run.smithi016.stdout:(72/142): python3-scipy-1.6.2-8.el9.x86_64.rpm 54 MB/s | 16 MB 00:00 2023-11-13T10:52:56.378 INFO:teuthology.orchestra.run.smithi016.stdout:(73/142): python3-toml-0.10.2-6.el9.noarch.rpm 140 kB/s | 39 kB 00:00 2023-11-13T10:52:56.412 INFO:teuthology.orchestra.run.smithi016.stdout:(74/142): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2023-11-13T10:52:56.445 INFO:teuthology.orchestra.run.smithi016.stdout:(75/142): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 300 kB 00:00 2023-11-13T10:52:56.471 INFO:teuthology.orchestra.run.smithi016.stdout:(76/142): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2023-11-13T10:52:56.497 INFO:teuthology.orchestra.run.smithi016.stdout:(77/142): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2023-11-13T10:52:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:(78/142): libgfortran-11.4.1-2.3.el9.x86_64.rpm 12 MB/s | 803 kB 00:00 2023-11-13T10:52:56.572 INFO:teuthology.orchestra.run.smithi016.stdout:(79/142): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.8 MB/s | 195 kB 00:00 2023-11-13T10:52:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:(80/142): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 241 kB 00:00 2023-11-13T10:52:56.639 INFO:teuthology.orchestra.run.smithi016.stdout:(81/142): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2023-11-13T10:52:56.656 INFO:teuthology.orchestra.run.smithi016.stdout:(82/142): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2023-11-13T10:52:56.673 INFO:teuthology.orchestra.run.smithi016.stdout:(83/142): python3-requests-2.25.1-7.el9.noarch. 3.5 MB/s | 113 kB 00:00 2023-11-13T10:52:56.708 INFO:teuthology.orchestra.run.smithi016.stdout:(84/142): python3-urllib3-1.26.5-4.el9.noarch.r 3.8 MB/s | 187 kB 00:00 2023-11-13T10:52:56.899 INFO:teuthology.orchestra.run.smithi184.stdout:(8/142): ceph-osd-18.0.0-7288.g35f0debb.el9.x86 7.5 MB/s | 18 MB 00:02 2023-11-13T10:52:56.962 INFO:teuthology.orchestra.run.smithi184.stdout:(9/142): ceph-selinux-18.0.0-7288.g35f0debb.el9 410 kB/s | 24 kB 00:00 2023-11-13T10:52:57.173 INFO:teuthology.orchestra.run.smithi016.stdout:(85/142): ceph-test-18.0.0-7288.g35f0debb.el9.x 7.2 MB/s | 41 MB 00:05 2023-11-13T10:52:57.324 INFO:teuthology.orchestra.run.smithi016.stdout:(86/142): python3-cryptography-36.0.1-4.el9.x86 1.9 MB/s | 1.2 MB 00:00 2023-11-13T10:52:57.386 INFO:teuthology.orchestra.run.smithi016.stdout:(87/142): elfutils-libelf-devel-0.189-3.el9.x86 106 kB/s | 22 kB 00:00 2023-11-13T10:52:57.408 INFO:teuthology.orchestra.run.smithi016.stdout:(88/142): librabbitmq-0.11.0-7.el9.x86_64.rpm 545 kB/s | 45 kB 00:00 2023-11-13T10:52:57.458 INFO:teuthology.orchestra.run.smithi016.stdout:(89/142): automake-1.16.2-8.el9.noarch.rpm 918 kB/s | 689 kB 00:00 2023-11-13T10:52:57.520 INFO:teuthology.orchestra.run.smithi016.stdout:(90/142): python3-pytz-2021.1-5.el9.noarch.rpm 888 kB/s | 51 kB 00:00 2023-11-13T10:52:57.542 INFO:teuthology.orchestra.run.smithi016.stdout:(91/142): libuuid-devel-2.37.4-15.el9.x86_64.rp 165 kB/s | 26 kB 00:00 2023-11-13T10:52:57.576 INFO:teuthology.orchestra.run.smithi016.stdout:(92/142): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.4 MB/s | 86 kB 00:00 2023-11-13T10:52:57.645 INFO:teuthology.orchestra.run.smithi016.stdout:(93/142): zlib-devel-1.2.11-40.el9.x86_64.rpm 453 kB/s | 45 kB 00:00 2023-11-13T10:52:57.793 INFO:teuthology.orchestra.run.smithi016.stdout:(94/142): openssl-devel-3.0.7-24.el9.x86_64.rpm 11 MB/s | 4.1 MB 00:00 2023-11-13T10:52:57.956 INFO:teuthology.orchestra.run.smithi016.stdout:(95/142): libarrow-doc-9.0.0-9.el9.noarch.rpm 150 kB/s | 24 kB 00:00 2023-11-13T10:52:57.996 INFO:teuthology.orchestra.run.smithi016.stdout:(96/142): gperftools-libs-2.9.1-2.el9.x86_64.rp 742 kB/s | 309 kB 00:00 2023-11-13T10:52:58.069 INFO:teuthology.orchestra.run.smithi016.stdout:(97/142): liboath-2.6.7-2.el9.x86_64.rpm 435 kB/s | 49 kB 00:00 2023-11-13T10:52:58.094 INFO:teuthology.orchestra.run.smithi016.stdout:(98/142): libunwind-1.6.2-1.el9.x86_64.rpm 677 kB/s | 67 kB 00:00 2023-11-13T10:52:58.211 INFO:teuthology.orchestra.run.smithi016.stdout:(99/142): libarrow-9.0.0-9.el9.x86_64.rpm 7.8 MB/s | 4.4 MB 00:00 2023-11-13T10:52:58.278 INFO:teuthology.orchestra.run.smithi016.stdout:(100/142): python3-bcrypt-3.2.2-1.el9.x86_64.rp 662 kB/s | 43 kB 00:00 2023-11-13T10:52:58.336 INFO:teuthology.orchestra.run.smithi016.stdout:(101/142): python3-asyncssh-2.13.2-3.el9.noarch 2.2 MB/s | 548 kB 00:00 2023-11-13T10:52:58.362 INFO:teuthology.orchestra.run.smithi016.stdout:(102/142): python3-cachetools-4.2.4-1.el9.noarc 389 kB/s | 32 kB 00:00 2023-11-13T10:52:58.393 INFO:teuthology.orchestra.run.smithi184.stdout:(10/142): ceph-radosgw-18.0.0-7288.g35f0debb.el 2.4 MB/s | 8.6 MB 00:03 2023-11-13T10:52:58.395 INFO:teuthology.orchestra.run.smithi016.stdout:(103/142): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2023-11-13T10:52:58.430 INFO:teuthology.orchestra.run.smithi016.stdout:(104/142): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2023-11-13T10:52:58.466 INFO:teuthology.orchestra.run.smithi016.stdout:(105/142): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.1 MB/s | 837 kB 00:00 2023-11-13T10:52:58.496 INFO:teuthology.orchestra.run.smithi016.stdout:(106/142): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2023-11-13T10:52:58.555 INFO:teuthology.orchestra.run.smithi016.stdout:(107/142): python3-google-auth-2.23.4-1.el9.noa 1.5 MB/s | 197 kB 00:00 2023-11-13T10:52:58.580 INFO:teuthology.orchestra.run.smithi016.stdout:(108/142): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2023-11-13T10:52:58.605 INFO:teuthology.orchestra.run.smithi016.stdout:(109/142): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2023-11-13T10:52:58.631 INFO:teuthology.orchestra.run.smithi016.stdout:(110/142): python3-jaraco-collections-3.0.0-8.e 310 kB/s | 23 kB 00:00 2023-11-13T10:52:58.656 INFO:teuthology.orchestra.run.smithi016.stdout:(111/142): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2023-11-13T10:52:58.681 INFO:teuthology.orchestra.run.smithi016.stdout:(112/142): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2023-11-13T10:52:58.706 INFO:teuthology.orchestra.run.smithi016.stdout:(113/142): python3-jwt+crypto-2.4.0-1.el9.noarc 121 kB/s | 9.0 kB 00:00 2023-11-13T10:52:58.718 INFO:teuthology.orchestra.run.smithi184.stdout:(11/142): ceph-common-18.0.0-7288.g35f0debb.el9 2.1 MB/s | 19 MB 00:09 2023-11-13T10:52:58.732 INFO:teuthology.orchestra.run.smithi016.stdout:(114/142): python3-jwt-2.4.0-1.el9.noarch.rpm 547 kB/s | 41 kB 00:00 2023-11-13T10:52:58.751 INFO:teuthology.orchestra.run.smithi184.stdout:(12/142): cephfs-mirror-18.0.0-7288.g35f0debb.e 607 kB/s | 217 kB 00:00 2023-11-13T10:52:58.773 INFO:teuthology.orchestra.run.smithi016.stdout:(115/142): python3-logutils-0.3.5-21.el9.noarch 701 kB/s | 46 kB 00:00 2023-11-13T10:52:58.785 INFO:teuthology.orchestra.run.smithi184.stdout:(13/142): libcephfs-devel-18.0.0-7288.g35f0debb 472 kB/s | 31 kB 00:00 2023-11-13T10:52:58.801 INFO:teuthology.orchestra.run.smithi016.stdout:(116/142): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2023-11-13T10:52:58.832 INFO:teuthology.orchestra.run.smithi016.stdout:(117/142): python3-natsort-7.1.1-5.el9.noarch.r 999 kB/s | 58 kB 00:00 2023-11-13T10:52:58.875 INFO:teuthology.orchestra.run.smithi016.stdout:(118/142): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2023-11-13T10:52:58.894 INFO:teuthology.orchestra.run.smithi184.stdout:(14/142): libcephsqlite-18.0.0-7288.g35f0debb.e 1.5 MB/s | 164 kB 00:00 2023-11-13T10:52:58.900 INFO:teuthology.orchestra.run.smithi016.stdout:(119/142): python3-portend-3.1.0-2.el9.noarch.r 247 kB/s | 16 kB 00:00 2023-11-13T10:52:58.933 INFO:teuthology.orchestra.run.smithi016.stdout:(120/142): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2023-11-13T10:52:58.966 INFO:teuthology.orchestra.run.smithi184.stdout:(15/142): librados-devel-18.0.0-7288.g35f0debb. 1.8 MB/s | 126 kB 00:00 2023-11-13T10:52:58.967 INFO:teuthology.orchestra.run.smithi016.stdout:(121/142): python3-repoze-lru-0.7-16.el9.noarch 464 kB/s | 31 kB 00:00 2023-11-13T10:52:59.017 INFO:teuthology.orchestra.run.smithi016.stdout:(122/142): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2023-11-13T10:52:59.049 INFO:teuthology.orchestra.run.smithi184.stdout:(16/142): libcephfs2-18.0.0-7288.g35f0debb.el9. 2.4 MB/s | 719 kB 00:00 2023-11-13T10:52:59.059 INFO:teuthology.orchestra.run.smithi016.stdout:(123/142): python3-kubernetes-26.1.0-2.el9.noar 2.7 MB/s | 1.0 MB 00:00 2023-11-13T10:52:59.106 INFO:teuthology.orchestra.run.smithi016.stdout:(124/142): python3-rsa-4.9-2.el9.noarch.rpm 428 kB/s | 59 kB 00:00 2023-11-13T10:52:59.124 INFO:teuthology.orchestra.run.smithi184.stdout:(17/142): libradosstriper1-18.0.0-7288.g35f0deb 2.9 MB/s | 475 kB 00:00 2023-11-13T10:52:59.135 INFO:teuthology.orchestra.run.smithi016.stdout:(125/142): python3-tempora-5.0.0-2.el9.noarch.r 307 kB/s | 36 kB 00:00 2023-11-13T10:52:59.160 INFO:teuthology.orchestra.run.smithi016.stdout:(126/142): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2023-11-13T10:52:59.191 INFO:teuthology.orchestra.run.smithi184.stdout:(18/142): python3-ceph-argparse-18.0.0-7288.g35 683 kB/s | 45 kB 00:00 2023-11-13T10:52:59.194 INFO:teuthology.orchestra.run.smithi016.stdout:(127/142): python3-webob-1.8.7-6.el9.noarch.rpm 2.6 MB/s | 230 kB 00:00 2023-11-13T10:52:59.219 INFO:teuthology.orchestra.run.smithi016.stdout:(128/142): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2023-11-13T10:52:59.254 INFO:teuthology.orchestra.run.smithi016.stdout:(129/142): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2023-11-13T10:52:59.258 INFO:teuthology.orchestra.run.smithi184.stdout:(19/142): python3-ceph-common-18.0.0-7288.g35f0 1.9 MB/s | 127 kB 00:00 2023-11-13T10:52:59.270 INFO:teuthology.orchestra.run.smithi016.stdout:(130/142): python3-zc-lockfile-2.0-10.el9.noarc 267 kB/s | 20 kB 00:00 2023-11-13T10:52:59.333 INFO:teuthology.orchestra.run.smithi184.stdout:(20/142): python3-cephfs-18.0.0-7288.g35f0debb. 2.1 MB/s | 162 kB 00:00 2023-11-13T10:52:59.345 INFO:teuthology.orchestra.run.smithi016.stdout:(131/142): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2023-11-13T10:52:59.412 INFO:teuthology.orchestra.run.smithi016.stdout:(132/142): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-11-13T10:52:59.446 INFO:teuthology.orchestra.run.smithi016.stdout:(133/142): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-11-13T10:52:59.450 INFO:teuthology.orchestra.run.smithi184.stdout:(21/142): python3-rados-18.0.0-7288.g35f0debb.e 2.7 MB/s | 320 kB 00:00 2023-11-13T10:52:59.529 INFO:teuthology.orchestra.run.smithi016.stdout:(134/142): NetworkManager-1.45.6-1.el9.x86_64.r 27 MB/s | 2.2 MB 00:00 2023-11-13T10:52:59.609 INFO:teuthology.orchestra.run.smithi184.stdout:(22/142): python3-rbd-18.0.0-7288.g35f0debb.el9 1.9 MB/s | 299 kB 00:00 2023-11-13T10:52:59.613 INFO:teuthology.orchestra.run.smithi016.stdout:(135/142): NetworkManager-libnm-1.45.6-1.el9.x8 21 MB/s | 1.8 MB 00:00 2023-11-13T10:52:59.646 INFO:teuthology.orchestra.run.smithi016.stdout:(136/142): NetworkManager-team-1.45.6-1.el9.x86 1.3 MB/s | 42 kB 00:00 2023-11-13T10:52:59.701 INFO:teuthology.orchestra.run.smithi184.stdout:(23/142): python3-rgw-18.0.0-7288.g35f0debb.el9 1.1 MB/s | 99 kB 00:00 2023-11-13T10:52:59.705 INFO:teuthology.orchestra.run.smithi016.stdout:(137/142): NetworkManager-tui-1.45.6-1.el9.x86_ 4.2 MB/s | 245 kB 00:00 2023-11-13T10:52:59.739 INFO:teuthology.orchestra.run.smithi016.stdout:(138/142): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 69 kB 00:00 2023-11-13T10:52:59.772 INFO:teuthology.orchestra.run.smithi016.stdout:(139/142): libacl-2.3.1-4.el9.x86_64.rpm 677 kB/s | 22 kB 00:00 2023-11-13T10:52:59.793 INFO:teuthology.orchestra.run.smithi184.stdout:(24/142): rbd-fuse-18.0.0-7288.g35f0debb.el9.x8 939 kB/s | 85 kB 00:00 2023-11-13T10:52:59.914 INFO:teuthology.orchestra.run.smithi016.stdout:(140/142): xfsdump-3.1.12-4.el9.x86_64.rpm 2.2 MB/s | 321 kB 00:00 2023-11-13T10:53:00.146 INFO:teuthology.orchestra.run.smithi016.stdout:(141/142): librados2-18.0.0-7288.g35f0debb.el9. 3.8 MB/s | 3.3 MB 00:00 2023-11-13T10:53:00.223 INFO:teuthology.orchestra.run.smithi016.stdout:(142/142): librbd1-18.0.0-7288.g35f0debb.el9.x8 3.4 MB/s | 3.0 MB 00:00 2023-11-13T10:53:00.237 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-11-13T10:53:00.238 INFO:teuthology.orchestra.run.smithi016.stdout:Total 17 MB/s | 200 MB 00:11 2023-11-13T10:53:00.543 INFO:teuthology.orchestra.run.smithi184.stdout:(25/142): rbd-mirror-18.0.0-7288.g35f0debb.el9. 4.0 MB/s | 3.0 MB 00:00 2023-11-13T10:53:00.643 INFO:teuthology.orchestra.run.smithi184.stdout:(26/142): librgw2-18.0.0-7288.g35f0debb.el9.x86 3.0 MB/s | 4.8 MB 00:01 2023-11-13T10:53:00.644 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-11-13T10:53:00.679 INFO:teuthology.orchestra.run.smithi184.stdout:(27/142): rbd-nbd-18.0.0-7288.g35f0debb.el9.x86 1.3 MB/s | 170 kB 00:00 2023-11-13T10:53:00.719 INFO:teuthology.orchestra.run.smithi184.stdout:(28/142): ceph-grafana-dashboards-18.0.0-7288.g 313 kB/s | 23 kB 00:00 2023-11-13T10:53:00.753 INFO:teuthology.orchestra.run.smithi184.stdout:(29/142): ceph-mgr-cephadm-18.0.0-7288.g35f0deb 1.7 MB/s | 132 kB 00:00 2023-11-13T10:53:00.863 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-11-13T10:53:00.863 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-11-13T10:53:01.503 INFO:teuthology.orchestra.run.smithi184.stdout:(30/142): ceph-test-18.0.0-7288.g35f0debb.el9.x 9.1 MB/s | 41 MB 00:04 2023-11-13T10:53:01.569 INFO:teuthology.orchestra.run.smithi184.stdout:(31/142): ceph-mgr-dashboard-18.0.0-7288.g35f0d 2.1 MB/s | 1.8 MB 00:00 2023-11-13T10:53:01.603 INFO:teuthology.orchestra.run.smithi184.stdout:(32/142): ceph-mgr-modules-core-18.0.0-7288.g35 2.4 MB/s | 243 kB 00:00 2023-11-13T10:53:01.637 INFO:teuthology.orchestra.run.smithi184.stdout:(33/142): ceph-mgr-rook-18.0.0-7288.g35f0debb.e 759 kB/s | 50 kB 00:00 2023-11-13T10:53:01.662 INFO:teuthology.orchestra.run.smithi184.stdout:(34/142): ceph-prometheus-alerts-18.0.0-7288.g3 261 kB/s | 15 kB 00:00 2023-11-13T10:53:01.746 INFO:teuthology.orchestra.run.smithi184.stdout:(35/142): ceph-volume-18.0.0-7288.g35f0debb.el9 2.4 MB/s | 262 kB 00:00 2023-11-13T10:53:01.791 INFO:teuthology.orchestra.run.smithi184.stdout:(36/142): cephadm-18.0.0-7288.g35f0debb.el9.noa 6.0 MB/s | 766 kB 00:00 2023-11-13T10:53:01.813 INFO:teuthology.orchestra.run.smithi184.stdout:(37/142): cephfs-top-18.0.0-7288.g35f0debb.el9. 275 kB/s | 18 kB 00:00 2023-11-13T10:53:01.988 INFO:teuthology.orchestra.run.smithi184.stdout:(38/142): boost-program-options-1.75.0-8.el9.x8 609 kB/s | 106 kB 00:00 2023-11-13T10:53:02.114 INFO:teuthology.orchestra.run.smithi184.stdout:(39/142): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 920 kB 00:00 2023-11-13T10:53:02.147 INFO:teuthology.orchestra.run.smithi184.stdout:(40/142): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 307 kB 00:00 2023-11-13T10:53:02.173 INFO:teuthology.orchestra.run.smithi184.stdout:(41/142): flexiblas-3.0.4-8.el9.x86_64.rpm 572 kB/s | 32 kB 00:00 2023-11-13T10:53:02.331 INFO:teuthology.orchestra.run.smithi184.stdout:(42/142): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2023-11-13T10:53:02.356 INFO:teuthology.orchestra.run.smithi184.stdout:(43/142): flexiblas-openblas-openmp-3.0.4-8.el9 96 kB/s | 17 kB 00:00 2023-11-13T10:53:02.415 INFO:teuthology.orchestra.run.smithi184.stdout:(44/142): libaio-devel-0.3.111-13.el9.x86_64.rp 176 kB/s | 10 kB 00:00 2023-11-13T10:53:02.448 INFO:teuthology.orchestra.run.smithi184.stdout:(45/142): jq-1.6-16.el9.x86_64.rpm 2.0 MB/s | 186 kB 00:00 2023-11-13T10:53:02.482 INFO:teuthology.orchestra.run.smithi184.stdout:(46/142): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 159 kB 00:00 2023-11-13T10:53:02.524 INFO:teuthology.orchestra.run.smithi184.stdout:(47/142): librdkafka-1.6.1-102.el9.x86_64.rpm 8.8 MB/s | 661 kB 00:00 2023-11-13T10:53:02.566 INFO:teuthology.orchestra.run.smithi184.stdout:(48/142): libtool-2.4.6-45.el9.x86_64.rpm 7.3 MB/s | 578 kB 00:00 2023-11-13T10:53:02.591 INFO:teuthology.orchestra.run.smithi184.stdout:(49/142): libzstd-devel-1.5.1-2.el9.x86_64.rpm 758 kB/s | 46 kB 00:00 2023-11-13T10:53:02.625 INFO:teuthology.orchestra.run.smithi184.stdout:(50/142): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-11-13T10:53:02.658 INFO:teuthology.orchestra.run.smithi184.stdout:(51/142): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-11-13T10:53:02.684 INFO:teuthology.orchestra.run.smithi184.stdout:(52/142): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 34 kB 00:00 2023-11-13T10:53:02.817 INFO:teuthology.orchestra.run.smithi184.stdout:(53/142): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2023-11-13T10:53:02.884 INFO:teuthology.orchestra.run.smithi184.stdout:(54/142): postgresql-13.11-1.el9.x86_64.rpm 8.3 MB/s | 1.5 MB 00:00 2023-11-13T10:53:02.926 INFO:teuthology.orchestra.run.smithi184.stdout:(55/142): postgresql-contrib-13.11-1.el9.x86_64 8.3 MB/s | 811 kB 00:00 2023-11-13T10:53:02.952 INFO:teuthology.orchestra.run.smithi184.stdout:(56/142): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2023-11-13T10:53:03.032 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-11-13T10:53:03.033 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-11-13T10:53:03.102 INFO:teuthology.orchestra.run.smithi184.stdout:(57/142): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2023-11-13T10:53:03.260 INFO:teuthology.orchestra.run.smithi184.stdout:(58/142): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2023-11-13T10:53:03.319 INFO:teuthology.orchestra.run.smithi184.stdout:(59/142): python3-devel-3.9.17-2.el9.x86_64.rpm 1.0 MB/s | 205 kB 00:00 2023-11-13T10:53:03.444 INFO:teuthology.orchestra.run.smithi184.stdout:(60/142): ceph-mgr-diskprediction-local-18.0.0- 2.7 MB/s | 7.4 MB 00:02 2023-11-13T10:53:03.470 INFO:teuthology.orchestra.run.smithi184.stdout:(61/142): python3-jsonpatch-1.21-16.el9.noarch. 164 kB/s | 24 kB 00:00 2023-11-13T10:53:03.503 INFO:teuthology.orchestra.run.smithi184.stdout:(62/142): python3-jinja2-2.11.3-4.el9.noarch.rp 1.2 MB/s | 228 kB 00:00 2023-11-13T10:53:03.528 INFO:teuthology.orchestra.run.smithi184.stdout:(63/142): python3-jsonpointer-2.0-4.el9.noarch. 218 kB/s | 18 kB 00:00 2023-11-13T10:53:03.554 INFO:teuthology.orchestra.run.smithi184.stdout:(64/142): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2023-11-13T10:53:03.595 INFO:teuthology.orchestra.run.smithi184.stdout:(65/142): python3-markupsafe-1.1.1-12.el9.x86_6 489 kB/s | 32 kB 00:00 2023-11-13T10:53:03.721 INFO:teuthology.orchestra.run.smithi184.stdout:(66/142): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-11-13T10:53:03.747 INFO:teuthology.orchestra.run.smithi184.stdout:(67/142): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2023-11-13T10:53:03.771 INFO:teuthology.orchestra.run.smithi184.stdout:(68/142): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2023-11-13T10:53:03.813 INFO:teuthology.orchestra.run.smithi184.stdout:(69/142): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2023-11-13T10:53:03.838 INFO:teuthology.orchestra.run.smithi184.stdout:(70/142): python3-prettytable-0.7.2-27.el9.noar 626 kB/s | 41 kB 00:00 2023-11-13T10:53:03.865 INFO:teuthology.orchestra.run.smithi184.stdout:(71/142): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2023-11-13T10:53:03.889 INFO:teuthology.orchestra.run.smithi184.stdout:(72/142): python3-requests-oauthlib-1.3.0-12.el 584 kB/s | 43 kB 00:00 2023-11-13T10:53:03.922 INFO:teuthology.orchestra.run.smithi184.stdout:(73/142): python3-toml-0.10.2-6.el9.noarch.rpm 679 kB/s | 39 kB 00:00 2023-11-13T10:53:04.039 INFO:teuthology.orchestra.run.smithi184.stdout:(74/142): socat-1.7.4.1-5.el9.x86_64.rpm 2.0 MB/s | 300 kB 00:00 2023-11-13T10:53:04.248 INFO:teuthology.orchestra.run.smithi184.stdout:(75/142): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2023-11-13T10:53:04.273 INFO:teuthology.orchestra.run.smithi184.stdout:(76/142): uuid-1.6.2-55.el9.x86_64.rpm 168 kB/s | 56 kB 00:00 2023-11-13T10:53:04.298 INFO:teuthology.orchestra.run.smithi184.stdout:(77/142): xmlstarlet-1.6.1-20.el9.x86_64.rpm 249 kB/s | 63 kB 00:00 2023-11-13T10:53:04.324 INFO:teuthology.orchestra.run.smithi184.stdout:(78/142): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2023-11-13T10:53:04.358 INFO:teuthology.orchestra.run.smithi184.stdout:(79/142): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.4 MB/s | 195 kB 00:00 2023-11-13T10:53:04.399 INFO:teuthology.orchestra.run.smithi184.stdout:(80/142): libgfortran-11.4.1-2.3.el9.x86_64.rpm 7.9 MB/s | 803 kB 00:00 2023-11-13T10:53:04.424 INFO:teuthology.orchestra.run.smithi184.stdout:(81/142): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.4 MB/s | 241 kB 00:00 2023-11-13T10:53:04.449 INFO:teuthology.orchestra.run.smithi184.stdout:(82/142): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2023-11-13T10:53:04.483 INFO:teuthology.orchestra.run.smithi184.stdout:(83/142): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2023-11-13T10:53:04.510 INFO:teuthology.orchestra.run.smithi184.stdout:(84/142): python3-requests-2.25.1-7.el9.noarch. 1.3 MB/s | 113 kB 00:00 2023-11-13T10:53:04.550 INFO:teuthology.orchestra.run.smithi184.stdout:(85/142): python3-urllib3-1.26.5-4.el9.noarch.r 1.8 MB/s | 187 kB 00:00 2023-11-13T10:53:04.684 INFO:teuthology.orchestra.run.smithi184.stdout:(86/142): elfutils-libelf-devel-0.189-3.el9.x86 168 kB/s | 22 kB 00:00 2023-11-13T10:53:04.784 INFO:teuthology.orchestra.run.smithi184.stdout:(87/142): librabbitmq-0.11.0-7.el9.x86_64.rpm 457 kB/s | 45 kB 00:00 2023-11-13T10:53:04.843 INFO:teuthology.orchestra.run.smithi184.stdout:(88/142): libuuid-devel-2.37.4-15.el9.x86_64.rp 444 kB/s | 26 kB 00:00 2023-11-13T10:53:04.876 INFO:teuthology.orchestra.run.smithi184.stdout:(89/142): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2023-11-13T10:53:04.936 INFO:teuthology.orchestra.run.smithi184.stdout:(90/142): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2023-11-13T10:53:04.977 INFO:teuthology.orchestra.run.smithi184.stdout:(91/142): python3-pytz-2021.1-5.el9.noarch.rpm 521 kB/s | 51 kB 00:00 2023-11-13T10:53:05.003 INFO:teuthology.orchestra.run.smithi184.stdout:(92/142): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.3 MB/s | 86 kB 00:00 2023-11-13T10:53:05.042 INFO:teuthology.orchestra.run.smithi184.stdout:(93/142): zlib-devel-1.2.11-40.el9.x86_64.rpm 782 kB/s | 45 kB 00:00 2023-11-13T10:53:05.370 INFO:teuthology.orchestra.run.smithi184.stdout:(94/142): openssl-devel-3.0.7-24.el9.x86_64.rpm 7.8 MB/s | 4.1 MB 00:00 2023-11-13T10:53:05.853 INFO:teuthology.orchestra.run.smithi184.stdout:(95/142): gperftools-libs-2.9.1-2.el9.x86_64.rp 363 kB/s | 309 kB 00:00 2023-11-13T10:53:05.878 INFO:teuthology.orchestra.run.smithi184.stdout:(96/142): libarrow-doc-9.0.0-9.el9.noarch.rpm 48 kB/s | 24 kB 00:00 2023-11-13T10:53:05.987 INFO:teuthology.orchestra.run.smithi184.stdout:(97/142): liboath-2.6.7-2.el9.x86_64.rpm 367 kB/s | 49 kB 00:00 2023-11-13T10:53:06.154 INFO:teuthology.orchestra.run.smithi184.stdout:(98/142): libunwind-1.6.2-1.el9.x86_64.rpm 245 kB/s | 67 kB 00:00 2023-11-13T10:53:06.329 INFO:teuthology.orchestra.run.smithi184.stdout:(99/142): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.4 MB/s | 837 kB 00:00 2023-11-13T10:53:06.421 INFO:teuthology.orchestra.run.smithi184.stdout:(100/142): libarrow-9.0.0-9.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2023-11-13T10:53:06.467 INFO:teuthology.orchestra.run.smithi184.stdout:(101/142): python3-bcrypt-3.2.2-1.el9.x86_64.rp 327 kB/s | 43 kB 00:00 2023-11-13T10:53:06.538 INFO:teuthology.orchestra.run.smithi184.stdout:(102/142): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2023-11-13T10:53:06.580 INFO:teuthology.orchestra.run.smithi184.stdout:(103/142): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2023-11-13T10:53:06.622 INFO:teuthology.orchestra.run.smithi184.stdout:(104/142): python3-asyncssh-2.13.2-3.el9.noarch 1.1 MB/s | 548 kB 00:00 2023-11-13T10:53:06.731 INFO:teuthology.orchestra.run.smithi184.stdout:(105/142): python3-cheroot-8.6.0-4.el9.noarch.r 897 kB/s | 172 kB 00:00 2023-11-13T10:53:06.772 INFO:teuthology.orchestra.run.smithi184.stdout:(106/142): python3-google-auth-2.23.4-1.el9.noa 1.3 MB/s | 197 kB 00:00 2023-11-13T10:53:06.815 INFO:teuthology.orchestra.run.smithi184.stdout:(107/142): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2023-11-13T10:53:06.873 INFO:teuthology.orchestra.run.smithi184.stdout:(108/142): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2023-11-13T10:53:06.907 INFO:teuthology.orchestra.run.smithi184.stdout:(109/142): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2023-11-13T10:53:06.965 INFO:teuthology.orchestra.run.smithi184.stdout:(110/142): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2023-11-13T10:53:07.001 INFO:teuthology.orchestra.run.smithi184.stdout:(111/142): python3-jaraco-functools-3.5.0-2.el9 156 kB/s | 19 kB 00:00 2023-11-13T10:53:07.075 INFO:teuthology.orchestra.run.smithi184.stdout:(112/142): python3-jaraco-text-3.2.0-6.el9.noar 118 kB/s | 20 kB 00:00 2023-11-13T10:53:07.100 INFO:teuthology.orchestra.run.smithi184.stdout:(113/142): python3-jwt+crypto-2.4.0-1.el9.noarc 68 kB/s | 9.0 kB 00:00 2023-11-13T10:53:07.125 INFO:teuthology.orchestra.run.smithi184.stdout:(114/142): python3-jwt-2.4.0-1.el9.noarch.rpm 327 kB/s | 41 kB 00:00 2023-11-13T10:53:07.300 INFO:teuthology.orchestra.run.smithi184.stdout:(115/142): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2023-11-13T10:53:07.342 INFO:teuthology.orchestra.run.smithi184.stdout:(116/142): python3-logutils-0.3.5-21.el9.noarch 192 kB/s | 46 kB 00:00 2023-11-13T10:53:07.367 INFO:teuthology.orchestra.run.smithi184.stdout:(117/142): python3-more-itertools-8.12.0-2.el9. 326 kB/s | 79 kB 00:00 2023-11-13T10:53:07.468 INFO:teuthology.orchestra.run.smithi184.stdout:(118/142): python3-natsort-7.1.1-5.el9.noarch.r 347 kB/s | 58 kB 00:00 2023-11-13T10:53:07.526 INFO:teuthology.orchestra.run.smithi184.stdout:(119/142): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2023-11-13T10:53:07.570 INFO:teuthology.orchestra.run.smithi184.stdout:(120/142): python3-portend-3.1.0-2.el9.noarch.r 82 kB/s | 16 kB 00:00 2023-11-13T10:53:07.593 INFO:teuthology.orchestra.run.smithi184.stdout:(121/142): python3-pyOpenSSL-21.0.0-1.el9.noarc 720 kB/s | 90 kB 00:00 2023-11-13T10:53:07.677 INFO:teuthology.orchestra.run.smithi184.stdout:(122/142): python3-repoze-lru-0.7-16.el9.noarch 205 kB/s | 31 kB 00:00 2023-11-13T10:53:07.713 INFO:teuthology.orchestra.run.smithi184.stdout:(123/142): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2023-11-13T10:53:07.765 INFO:teuthology.orchestra.run.smithi184.stdout:(124/142): python3-rsa-4.9-2.el9.noarch.rpm 355 kB/s | 59 kB 00:00 2023-11-13T10:53:07.877 INFO:teuthology.orchestra.run.smithi184.stdout:(125/142): python3-tempora-5.0.0-2.el9.noarch.r 180 kB/s | 36 kB 00:00 2023-11-13T10:53:07.938 INFO:teuthology.orchestra.run.smithi184.stdout:(126/142): python3-typing-extensions-4.4.0-2.el 227 kB/s | 51 kB 00:00 2023-11-13T10:53:07.970 INFO:teuthology.orchestra.run.smithi184.stdout:(127/142): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2023-11-13T10:53:08.012 INFO:teuthology.orchestra.run.smithi184.stdout:(128/142): python3-websocket-client-1.2.3-2.el9 668 kB/s | 90 kB 00:00 2023-11-13T10:53:08.095 INFO:teuthology.orchestra.run.smithi184.stdout:(129/142): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2023-11-13T10:53:08.151 INFO:teuthology.orchestra.run.smithi184.stdout:(130/142): python3-zc-lockfile-2.0-10.el9.noarc 114 kB/s | 20 kB 00:00 2023-11-13T10:53:08.179 INFO:teuthology.orchestra.run.smithi184.stdout:(131/142): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2023-11-13T10:53:08.280 INFO:teuthology.orchestra.run.smithi184.stdout:(132/142): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2023-11-13T10:53:08.319 INFO:teuthology.orchestra.run.smithi184.stdout:(133/142): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-11-13T10:53:08.406 INFO:teuthology.orchestra.run.smithi184.stdout:(134/142): NetworkManager-1.45.6-1.el9.x86_64.r 25 MB/s | 2.2 MB 00:00 2023-11-13T10:53:08.481 INFO:teuthology.orchestra.run.smithi184.stdout:(135/142): NetworkManager-libnm-1.45.6-1.el9.x8 24 MB/s | 1.8 MB 00:00 2023-11-13T10:53:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:(136/142): NetworkManager-team-1.45.6-1.el9.x86 857 kB/s | 42 kB 00:00 2023-11-13T10:53:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:(137/142): NetworkManager-tui-1.45.6-1.el9.x86_ 4.9 MB/s | 245 kB 00:00 2023-11-13T10:53:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:(138/142): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 69 kB 00:00 2023-11-13T10:53:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:(139/142): libacl-2.3.1-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2023-11-13T10:53:08.707 INFO:teuthology.orchestra.run.smithi184.stdout:(140/142): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2023-11-13T10:53:08.804 INFO:teuthology.orchestra.run.smithi184.stdout:(141/142): librbd1-18.0.0-7288.g35f0debb.el9.x8 4.8 MB/s | 3.0 MB 00:00 2023-11-13T10:53:09.108 INFO:teuthology.orchestra.run.smithi184.stdout:(142/142): librados2-18.0.0-7288.g35f0debb.el9. 3.5 MB/s | 3.3 MB 00:00 2023-11-13T10:53:09.123 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2023-11-13T10:53:09.123 INFO:teuthology.orchestra.run.smithi184.stdout:Total 10 MB/s | 200 MB 00:19 2023-11-13T10:53:09.319 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-11-13T10:53:09.393 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2023-11-13T10:53:09.429 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2023-11-13T10:53:09.482 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2023-11-13T10:53:09.520 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2023-11-13T10:53:09.582 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/152 2023-11-13T10:53:09.720 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2023-11-13T10:53:09.720 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2023-11-13T10:53:09.994 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/152 2023-11-13T10:53:10.071 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:18.0.0-7288.g35f0debb.el9.x86_64 6/152 2023-11-13T10:53:10.219 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:18.0.0-7288.g35f0debb.el9.x86_64 6/152 2023-11-13T10:53:10.414 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:18.0.0-7288.g35f0debb.el9.x86_64 7/152 2023-11-13T10:53:10.568 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:18.0.0-7288.g35f0debb.el9.x86_64 7/152 2023-11-13T10:53:10.678 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:18.0.0-7288.g35f0debb.el9.x86_64 8/152 2023-11-13T10:53:10.726 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/152 2023-11-13T10:53:11.007 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/152 2023-11-13T10:53:11.085 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/152 2023-11-13T10:53:11.111 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:18.0.0-7288.g35f0debb.el9.x86_64 12/152 2023-11-13T10:53:11.310 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7288.g35f0debb.el9.x86_64 12/152 2023-11-13T10:53:11.353 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:18.0.0-7288.g35f0debb.el9.x86 13/152 2023-11-13T10:53:11.448 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7288.g35f0debb.el9.x86 13/152 2023-11-13T10:53:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/152 2023-11-13T10:53:11.862 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2023-11-13T10:53:11.863 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2023-11-13T10:53:11.942 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-13T10:53:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-13T10:53:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-13T10:53:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-13T10:53:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-13T10:53:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/152 2023-11-13T10:53:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2023-11-13T10:53:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2023-11-13T10:53:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/152 2023-11-13T10:53:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/152 2023-11-13T10:53:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/152 2023-11-13T10:53:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/152 2023-11-13T10:53:13.391 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/152 2023-11-13T10:53:13.425 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/152 2023-11-13T10:53:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2023-11-13T10:53:13.589 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:18.0.0-7288.g35f0debb.el9. 27/152 2023-11-13T10:53:13.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:18.0.0-7288.g35f0debb.el 28/152 2023-11-13T10:53:13.737 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:18.0.0-7288.g35f0debb.el9.x86_6 29/152 2023-11-13T10:53:13.927 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/152 2023-11-13T10:53:14.000 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/152 2023-11-13T10:53:14.729 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/152 2023-11-13T10:53:14.835 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/152 2023-11-13T10:53:15.037 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/152 2023-11-13T10:53:15.080 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:18.0.0-7288.g35f0debb.el9.x86_6 35/152 2023-11-13T10:53:15.125 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/152 2023-11-13T10:53:15.304 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/152 2023-11-13T10:53:15.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/152 2023-11-13T10:53:15.431 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/152 2023-11-13T10:53:15.535 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/152 2023-11-13T10:53:15.577 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/152 2023-11-13T10:53:15.615 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/152 2023-11-13T10:53:15.693 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/152 2023-11-13T10:53:15.758 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/152 2023-11-13T10:53:15.818 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/152 2023-11-13T10:53:16.030 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/152 2023-11-13T10:53:16.082 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/152 2023-11-13T10:53:16.148 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/152 2023-11-13T10:53:16.295 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/152 2023-11-13T10:53:16.333 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 50/152 2023-11-13T10:53:16.392 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/152 2023-11-13T10:53:16.447 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/152 2023-11-13T10:53:16.478 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/152 2023-11-13T10:53:16.534 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/152 2023-11-13T10:53:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/152 2023-11-13T10:53:16.749 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 56/152 2023-11-13T10:53:16.823 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/152 2023-11-13T10:53:16.921 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/152 2023-11-13T10:53:17.216 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/152 2023-11-13T10:53:17.279 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/152 2023-11-13T10:53:17.459 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/152 2023-11-13T10:53:17.516 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/152 2023-11-13T10:53:17.567 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/152 2023-11-13T10:53:17.618 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/152 2023-11-13T10:53:17.667 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/152 2023-11-13T10:53:17.710 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/152 2023-11-13T10:53:17.769 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/152 2023-11-13T10:53:17.848 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2023-11-13T10:53:17.856 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/152 2023-11-13T10:53:17.908 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/152 2023-11-13T10:53:17.969 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/152 2023-11-13T10:53:17.969 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2023-11-13T10:53:17.995 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2023-11-13T10:53:18.102 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/152 2023-11-13T10:53:18.147 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/152 2023-11-13T10:53:18.189 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/152 2023-11-13T10:53:18.194 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2023-11-13T10:53:18.256 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/152 2023-11-13T10:53:18.303 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/152 2023-11-13T10:53:18.304 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/152 2023-11-13T10:53:18.456 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/152 2023-11-13T10:53:18.499 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/152 2023-11-13T10:53:18.653 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/152 2023-11-13T10:53:18.724 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/152 2023-11-13T10:53:18.749 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2023-11-13T10:53:18.771 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:18.0.0-7288.g35f0debb.el9.x86_64 6/152 2023-11-13T10:53:18.909 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:18.0.0-7288.g35f0debb.el9.x86_64 6/152 2023-11-13T10:53:18.946 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:18.0.0-7288.g35f0debb.el9.x86_64 7/152 2023-11-13T10:53:19.054 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:18.0.0-7288.g35f0debb.el9.x86_64 7/152 2023-11-13T10:53:19.116 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:18.0.0-7288.g35f0debb.el9.x86_64 8/152 2023-11-13T10:53:19.162 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/152 2023-11-13T10:53:19.418 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/152 2023-11-13T10:53:19.484 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/152 2023-11-13T10:53:19.512 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:18.0.0-7288.g35f0debb.el9.x86_64 12/152 2023-11-13T10:53:19.669 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7288.g35f0debb.el9.x86_64 12/152 2023-11-13T10:53:19.708 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:18.0.0-7288.g35f0debb.el9.x86 13/152 2023-11-13T10:53:19.786 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2023-11-13T10:53:19.794 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7288.g35f0debb.el9.x86 13/152 2023-11-13T10:53:19.873 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2023-11-13T10:53:19.906 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2023-11-13T10:53:20.052 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2023-11-13T10:53:20.097 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/152 2023-11-13T10:53:20.147 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/152 2023-11-13T10:53:20.323 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-13T10:53:20.358 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/152 2023-11-13T10:53:20.394 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/152 2023-11-13T10:53:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-13T10:53:20.537 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/152 2023-11-13T10:53:20.572 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-13T10:53:20.677 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-13T10:53:21.089 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-13T10:53:21.131 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/152 2023-11-13T10:53:21.184 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2023-11-13T10:53:21.278 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2023-11-13T10:53:21.346 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/152 2023-11-13T10:53:21.394 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/152 2023-11-13T10:53:21.448 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/152 2023-11-13T10:53:21.457 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/152 2023-11-13T10:53:21.549 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/152 2023-11-13T10:53:21.595 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/152 2023-11-13T10:53:21.605 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/152 2023-11-13T10:53:21.629 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/152 2023-11-13T10:53:21.739 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2023-11-13T10:53:21.776 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:18.0.0-7288.g35f0debb.el9. 27/152 2023-11-13T10:53:21.825 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:18.0.0-7288.g35f0debb.el 28/152 2023-11-13T10:53:21.938 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:18.0.0-7288.g35f0debb.el9.x86_6 29/152 2023-11-13T10:53:22.124 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/152 2023-11-13T10:53:22.186 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/152 2023-11-13T10:53:22.879 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/152 2023-11-13T10:53:22.969 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/152 2023-11-13T10:53:23.013 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/152 2023-11-13T10:53:23.063 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:18.0.0-7288.g35f0debb.el9.x86_6 35/152 2023-11-13T10:53:23.099 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/152 2023-11-13T10:53:23.263 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/152 2023-11-13T10:53:23.287 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/152 2023-11-13T10:53:23.327 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/152 2023-11-13T10:53:23.384 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/152 2023-11-13T10:53:23.394 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/152 2023-11-13T10:53:23.430 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/152 2023-11-13T10:53:23.469 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-16.el9.x86_64 93/152 2023-11-13T10:53:23.488 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/152 2023-11-13T10:53:23.511 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2023-11-13T10:53:23.527 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/152 2023-11-13T10:53:23.566 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/152 2023-11-13T10:53:23.631 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/152 2023-11-13T10:53:23.710 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/152 2023-11-13T10:53:23.770 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/152 2023-11-13T10:53:23.899 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/152 2023-11-13T10:53:23.940 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:18.0.0-7288.g35f0debb.el9.x86_64 96/152 2023-11-13T10:53:23.964 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/152 2023-11-13T10:53:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/152 2023-11-13T10:53:24.061 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:18.0.0-7288.g35f0debb.el9.x86_64 96/152 2023-11-13T10:53:24.079 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/152 2023-11-13T10:53:24.095 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:18.0.0-7288.g35f0debb.el9.x86_64 97/152 2023-11-13T10:53:24.151 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/152 2023-11-13T10:53:24.209 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/152 2023-11-13T10:53:24.243 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 50/152 2023-11-13T10:53:24.289 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/152 2023-11-13T10:53:24.332 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/152 2023-11-13T10:53:24.380 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/152 2023-11-13T10:53:24.434 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/152 2023-11-13T10:53:24.588 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/152 2023-11-13T10:53:24.645 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.2-8.el9.noarch 56/152 2023-11-13T10:53:24.679 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/152 2023-11-13T10:53:24.716 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:18.0.0-7288.g35f0debb.el9.x86_64 100/152 2023-11-13T10:53:24.717 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/152 2023-11-13T10:53:24.794 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/152 2023-11-13T10:53:24.818 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:18.0.0-7288.g35f0debb.el9.x86_64 100/152 2023-11-13T10:53:24.843 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:18.0.0-7288.g35f0debb.el9.x86_64 101/152 2023-11-13T10:53:25.066 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/152 2023-11-13T10:53:25.127 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/152 2023-11-13T10:53:25.302 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/152 2023-11-13T10:53:25.355 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/152 2023-11-13T10:53:25.391 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/152 2023-11-13T10:53:25.425 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/152 2023-11-13T10:53:25.477 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/152 2023-11-13T10:53:25.530 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/152 2023-11-13T10:53:25.578 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/152 2023-11-13T10:53:25.659 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/152 2023-11-13T10:53:25.714 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/152 2023-11-13T10:53:25.754 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/152 2023-11-13T10:53:25.880 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/152 2023-11-13T10:53:25.919 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/152 2023-11-13T10:53:25.953 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/152 2023-11-13T10:53:26.007 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/152 2023-11-13T10:53:26.049 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/152 2023-11-13T10:53:26.204 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/152 2023-11-13T10:53:26.248 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/152 2023-11-13T10:53:26.409 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/152 2023-11-13T10:53:26.479 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2023-11-13T10:53:27.430 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 102/152 2023-11-13T10:53:27.470 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2023-11-13T10:53:27.482 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 102/152 2023-11-13T10:53:27.520 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2023-11-13T10:53:27.554 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2023-11-13T10:53:27.677 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2023-11-13T10:53:27.720 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/152 2023-11-13T10:53:27.950 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/152 2023-11-13T10:53:27.987 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/152 2023-11-13T10:53:28.075 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/152 2023-11-13T10:53:28.154 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 102/152 2023-11-13T10:53:28.194 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:18.0.0-7288.g35f0debb.el9.x86_64 103/152 2023-11-13T10:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:18.0.0-7288.g35f0debb.el9.x86_64 103/152 2023-11-13T10:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-13T10:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-13T10:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:28.403 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:18.0.0-7288.g35f0debb.el9.x86_64 104/152 2023-11-13T10:53:28.897 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/152 2023-11-13T10:53:29.029 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/152 2023-11-13T10:53:30.606 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/152 2023-11-13T10:53:30.704 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/152 2023-11-13T10:53:30.741 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/152 2023-11-13T10:53:30.772 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : jq-1.6-16.el9.x86_64 93/152 2023-11-13T10:53:30.814 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2023-11-13T10:53:31.183 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/152 2023-11-13T10:53:31.235 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:18.0.0-7288.g35f0debb.el9.x86_64 96/152 2023-11-13T10:53:31.383 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:18.0.0-7288.g35f0debb.el9.x86_64 96/152 2023-11-13T10:53:31.416 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:18.0.0-7288.g35f0debb.el9.x86_64 97/152 2023-11-13T10:53:31.472 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/152 2023-11-13T10:53:31.956 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/152 2023-11-13T10:53:31.989 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:18.0.0-7288.g35f0debb.el9.x86_64 100/152 2023-11-13T10:53:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:18.0.0-7288.g35f0debb.el9.x86_64 100/152 2023-11-13T10:53:32.110 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:18.0.0-7288.g35f0debb.el9.x86_64 101/152 2023-11-13T10:53:34.560 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 102/152 2023-11-13T10:53:34.595 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 102/152 2023-11-13T10:53:35.215 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 102/152 2023-11-13T10:53:35.273 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:18.0.0-7288.g35f0debb.el9.x86_64 103/152 2023-11-13T10:53:35.489 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:18.0.0-7288.g35f0debb.el9.x86_64 103/152 2023-11-13T10:53:35.489 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-13T10:53:35.489 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-13T10:53:35.490 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:35.532 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:18.0.0-7288.g35f0debb.el9.x86_64 104/152 2023-11-13T10:53:41.644 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7288.g35f0debb.el9.x86_64 104/152 2023-11-13T10:53:41.644 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2023-11-13T10:53:41.645 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2023-11-13T10:53:41.645 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2023-11-13T10:53:41.645 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2023-11-13T10:53:41.645 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2023-11-13T10:53:41.645 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2023-11-13T10:53:41.646 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2023-11-13T10:53:41.646 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:41.868 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:18.0.0-7288.g35f0debb.el9.noarch 105/152 2023-11-13T10:53:41.917 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:18.0.0-7288.g35f0debb.el9.noarch 105/152 2023-11-13T10:53:41.918 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:41.918 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-13T10:53:41.918 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:43.670 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:18.0.0-7288.g35f0debb.el9.x86_64 106/152 2023-11-13T10:53:43.771 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:18.0.0-7288.g35f0debb.el9.x86_64 106/152 2023-11-13T10:53:43.771 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:43.771 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-13T10:53:43.771 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-13T10:53:43.772 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-13T10:53:43.772 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:44.012 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:18.0.0-7288.g35f0debb.el9.x86_64 107/152 2023-11-13T10:53:44.103 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:18.0.0-7288.g35f0debb.el9.x86_64 107/152 2023-11-13T10:53:44.103 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:44.103 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-13T10:53:44.103 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-13T10:53:44.104 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-13T10:53:44.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:44.632 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:18.0.0-7288.g35f0debb.el9.x86_64 108/152 2023-11-13T10:53:44.720 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:18.0.0-7288.g35f0debb.el9.x86_64 108/152 2023-11-13T10:53:44.721 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:44.721 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-13T10:53:44.721 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-13T10:53:44.721 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-13T10:53:44.722 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 109/152 2023-11-13T10:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 109/152 2023-11-13T10:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7288.g35f0debb.e 110/152 2023-11-13T10:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7288.g35f0debb. 111/152 2023-11-13T10:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7288.g35f0debb.el9.noa 112/152 2023-11-13T10:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7288.g35f0debb.el9.noa 112/152 2023-11-13T10:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7288.g35f0debb.el9.n 113/152 2023-11-13T10:53:47.283 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7288.g35f0debb.el9.n 113/152 2023-11-13T10:53:47.312 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7288.g35f 114/152 2023-11-13T10:53:47.461 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7288.g35f 114/152 2023-11-13T10:53:47.638 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7288.g35f0debb.el 115/152 2023-11-13T10:53:47.664 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:18.0.0-7288.g35f0debb.el9.x86_64 116/152 2023-11-13T10:53:47.755 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7288.g35f0debb.el9.x86_64 116/152 2023-11-13T10:53:47.755 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:47.755 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-13T10:53:47.755 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-13T10:53:47.756 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-13T10:53:47.756 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:47.815 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:18.0.0-7288.g35f0debb.el9.noarch 117/152 2023-11-13T10:53:47.857 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7288.g35f0debb.el9.noarch 117/152 2023-11-13T10:53:48.270 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7288.g35f0debb.el9.x86_64 104/152 2023-11-13T10:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2023-11-13T10:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2023-11-13T10:53:48.271 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2023-11-13T10:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2023-11-13T10:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2023-11-13T10:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2023-11-13T10:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2023-11-13T10:53:48.272 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:48.494 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:18.0.0-7288.g35f0debb.el9.noarch 105/152 2023-11-13T10:53:48.541 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:18.0.0-7288.g35f0debb.el9.noarch 105/152 2023-11-13T10:53:48.541 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:48.541 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-13T10:53:48.546 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:18.0.0-7288.g35f0debb.el9.x86_64 118/152 2023-11-13T10:53:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:18.0.0-7288.g35f0debb.el9.x86_64 119/152 2023-11-13T10:53:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7288.g35f0debb.el9.x86_64 119/152 2023-11-13T10:53:48.900 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:48.900 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-13T10:53:48.900 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-13T10:53:48.900 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-13T10:53:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7288.g35f0d 120/152 2023-11-13T10:53:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7288.g35f0d 120/152 2023-11-13T10:53:49.049 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:49.049 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-13T10:53:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 121/152 2023-11-13T10:53:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 121/152 2023-11-13T10:53:49.238 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:49.238 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-13T10:53:49.238 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-13T10:53:49.239 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-13T10:53:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 122/152 2023-11-13T10:53:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 122/152 2023-11-13T10:53:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:49.699 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-13T10:53:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-13T10:53:49.700 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-13T10:53:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:50.213 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:18.0.0-7288.g35f0debb.el9.x86_64 106/152 2023-11-13T10:53:50.306 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:18.0.0-7288.g35f0debb.el9.x86_64 106/152 2023-11-13T10:53:50.306 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:50.306 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-13T10:53:50.306 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-13T10:53:50.307 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-13T10:53:50.307 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:50.530 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:18.0.0-7288.g35f0debb.el9.x86_64 107/152 2023-11-13T10:53:50.609 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:18.0.0-7288.g35f0debb.el9.x86_64 107/152 2023-11-13T10:53:50.609 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:50.609 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-13T10:53:50.610 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-13T10:53:50.610 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-13T10:53:50.610 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:51.097 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:18.0.0-7288.g35f0debb.el9.x86_64 108/152 2023-11-13T10:53:51.177 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:18.0.0-7288.g35f0debb.el9.x86_64 108/152 2023-11-13T10:53:51.177 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:51.177 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-13T10:53:51.177 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-13T10:53:51.178 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-13T10:53:51.178 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:51.734 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 109/152 2023-11-13T10:53:51.774 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 109/152 2023-11-13T10:53:51.819 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7288.g35f0debb.e 110/152 2023-11-13T10:53:51.920 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7288.g35f0debb. 111/152 2023-11-13T10:53:51.956 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7288.g35f0debb.el9.noa 112/152 2023-11-13T10:53:52.270 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7288.g35f0debb.el9.noa 112/152 2023-11-13T10:53:52.311 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7288.g35f0debb.el9.n 113/152 2023-11-13T10:53:53.446 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7288.g35f0debb.el9.n 113/152 2023-11-13T10:53:53.481 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7288.g35f 114/152 2023-11-13T10:53:53.620 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7288.g35f 114/152 2023-11-13T10:53:53.786 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7288.g35f0debb.el 115/152 2023-11-13T10:53:53.831 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:18.0.0-7288.g35f0debb.el9.x86_64 116/152 2023-11-13T10:53:53.905 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7288.g35f0debb.el9.x86_64 116/152 2023-11-13T10:53:53.905 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:53.905 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-13T10:53:53.906 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-13T10:53:53.906 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-13T10:53:53.906 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:53.965 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:18.0.0-7288.g35f0debb.el9.noarch 117/152 2023-11-13T10:53:54.004 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7288.g35f0debb.el9.noarch 117/152 2023-11-13T10:53:54.095 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:18.0.0-7288.g35f0debb.el9.x86_64 123/152 2023-11-13T10:53:54.152 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 124/152 2023-11-13T10:53:54.200 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:18.0.0-7288.g35f0debb.el9.x86_64 125/152 2023-11-13T10:53:54.492 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/152 2023-11-13T10:53:54.628 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/152 2023-11-13T10:53:54.687 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 128/152 2023-11-13T10:53:54.734 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/152 2023-11-13T10:53:54.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/152 2023-11-13T10:53:54.793 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:18.0.0-7288.g35f0debb.el9.x86_ 131/152 2023-11-13T10:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 132/152 2023-11-13T10:53:54.871 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:18.0.0-7288.g35f0debb.el9.x86_64 118/152 2023-11-13T10:53:54.908 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:18.0.0-7288.g35f0debb.el9.x86_64 119/152 2023-11-13T10:53:54.934 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 133/152 2023-11-13T10:53:54.990 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7288.g35f0debb.el9.x86_64 119/152 2023-11-13T10:53:54.990 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:54.990 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-13T10:53:54.991 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-13T10:53:54.991 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-13T10:53:54.991 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:55.010 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 134/152 2023-11-13T10:53:55.041 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7288.g35f0d 120/152 2023-11-13T10:53:55.046 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 135/152 2023-11-13T10:53:55.135 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7288.g35f0d 120/152 2023-11-13T10:53:55.135 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:55.136 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-13T10:53:55.136 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:55.201 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 121/152 2023-11-13T10:53:55.279 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 121/152 2023-11-13T10:53:55.280 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:55.280 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-13T10:53:55.280 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-13T10:53:55.280 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-13T10:53:55.281 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:55.660 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 122/152 2023-11-13T10:53:55.732 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 122/152 2023-11-13T10:53:55.733 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-13T10:53:55.733 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-13T10:53:55.733 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-13T10:53:55.733 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-13T10:53:55.734 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:53:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-13T10:53:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-13T10:53:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-13T10:53:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:18.0.0-7288.g35f0debb.el9.noarch 137/152 2023-11-13T10:53:57.666 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 138/152 2023-11-13T10:53:57.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 139/152 2023-11-13T10:53:57.846 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 140/152 2023-11-13T10:53:57.990 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 141/152 2023-11-13T10:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 142/152 2023-11-13T10:53:58.070 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 143/152 2023-11-13T10:53:58.070 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-13T10:53:58.206 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-13T10:53:58.238 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 145/152 2023-11-13T10:53:58.268 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 146/152 2023-11-13T10:53:58.301 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/152 2023-11-13T10:53:58.344 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-13T10:53:58.345 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-13T10:53:58.693 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-13T10:53:58.693 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-13T10:53:58.832 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-13T10:53:58.867 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-13T10:53:58.867 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-13T10:53:59.034 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-13T10:53:59.164 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-13T10:53:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-13T10:53:59.423 INFO:teuthology.orchestra.run.smithi016.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-13T10:53:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:53:59.912 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:18.0.0-7288.g35f0debb.el9.x86_64 123/152 2023-11-13T10:53:59.969 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 124/152 2023-11-13T10:54:00.017 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:18.0.0-7288.g35f0debb.el9.x86_64 125/152 2023-11-13T10:54:00.300 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/152 2023-11-13T10:54:00.423 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/152 2023-11-13T10:54:00.485 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 128/152 2023-11-13T10:54:00.533 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/152 2023-11-13T10:54:00.563 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/152 2023-11-13T10:54:00.599 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:18.0.0-7288.g35f0debb.el9.x86_ 131/152 2023-11-13T10:54:00.629 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 132/152 2023-11-13T10:54:00.732 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 133/152 2023-11-13T10:54:00.810 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 134/152 2023-11-13T10:54:00.855 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 135/152 2023-11-13T10:54:01.917 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-13T10:54:01.985 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-13T10:54:02.042 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-13T10:54:02.113 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:18.0.0-7288.g35f0debb.el9.noarch 137/152 2023-11-13T10:54:03.504 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 138/152 2023-11-13T10:54:03.620 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 139/152 2023-11-13T10:54:03.682 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 140/152 2023-11-13T10:54:03.822 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.4-9.el9.x86_64 141/152 2023-11-13T10:54:03.862 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.7.4-5.el9.x86_64 142/152 2023-11-13T10:54:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 143/152 2023-11-13T10:54:03.899 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-13T10:54:04.011 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-13T10:54:04.040 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 145/152 2023-11-13T10:54:04.073 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 146/152 2023-11-13T10:54:04.114 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/152 2023-11-13T10:54:04.156 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-13T10:54:04.156 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-13T10:54:04.357 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:18.0.0-7288.g35f0debb.el9.x86_64 1/152 2023-11-13T10:54:04.357 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:18.0.0-7288.g35f0debb.el9.x86_64 2/152 2023-11-13T10:54:04.357 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 3/152 2023-11-13T10:54:04.357 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 4/152 2023-11-13T10:54:04.358 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7288.g35f0d 5/152 2023-11-13T10:54:04.358 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:18.0.0-7288.g35f0debb.el9.x86_64 6/152 2023-11-13T10:54:04.358 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:18.0.0-7288.g35f0debb.el9.x86_64 7/152 2023-11-13T10:54:04.358 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:18.0.0-7288.g35f0debb.el9.x86_64 8/152 2023-11-13T10:54:04.359 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:18.0.0-7288.g35f0debb.el9.x86_64 9/152 2023-11-13T10:54:04.359 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:18.0.0-7288.g35f0debb.el9.x86_64 10/152 2023-11-13T10:54:04.359 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:18.0.0-7288.g35f0debb.el9.x86_64 11/152 2023-11-13T10:54:04.359 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:18.0.0-7288.g35f0debb.el9.x86_64 12/152 2023-11-13T10:54:04.359 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 13/152 2023-11-13T10:54:04.359 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:18.0.0-7288.g35f0debb.el9.x86_ 14/152 2023-11-13T10:54:04.360 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:18.0.0-7288.g35f0debb.el9.x86_64 15/152 2023-11-13T10:54:04.360 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:18.0.0-7288.g35f0debb.el9.x86_64 16/152 2023-11-13T10:54:04.360 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:18.0.0-7288.g35f0debb.el9.x86_6 17/152 2023-11-13T10:54:04.360 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:18.0.0-7288.g35f0debb.el9.x86 18/152 2023-11-13T10:54:04.361 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:18.0.0-7288.g35f0debb.el9.x86_64 19/152 2023-11-13T10:54:04.361 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7288.g35f0debb.el 20/152 2023-11-13T10:54:04.361 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:18.0.0-7288.g35f0debb.el9. 21/152 2023-11-13T10:54:04.362 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:18.0.0-7288.g35f0debb.el9.x86_6 22/152 2023-11-13T10:54:04.362 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:18.0.0-7288.g35f0debb.el9.x86_64 23/152 2023-11-13T10:54:04.363 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:18.0.0-7288.g35f0debb.el9.x86_64 24/152 2023-11-13T10:54:04.363 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:18.0.0-7288.g35f0debb.el9.x86_64 25/152 2023-11-13T10:54:04.363 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 26/152 2023-11-13T10:54:04.363 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 27/152 2023-11-13T10:54:04.410 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:18.0.0-7288.g35f0debb.el9.x86_64 28/152 2023-11-13T10:54:04.413 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7288.g35f0debb. 29/152 2023-11-13T10:54:04.414 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7288.g35f0debb.el9.noa 30/152 2023-11-13T10:54:04.414 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7288.g35f0debb.el9.n 31/152 2023-11-13T10:54:04.414 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7288.g35f 32/152 2023-11-13T10:54:04.414 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7288.g35f0debb.el 33/152 2023-11-13T10:54:04.414 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7288.g35f0debb.el9.noarch 34/152 2023-11-13T10:54:04.419 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7288.g35f0debb.e 35/152 2023-11-13T10:54:04.420 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:18.0.0-7288.g35f0debb.el9.noarch 36/152 2023-11-13T10:54:04.420 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 37/152 2023-11-13T10:54:04.420 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:18.0.0-7288.g35f0debb.el9.noarch 38/152 2023-11-13T10:54:04.420 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/152 2023-11-13T10:54:04.420 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/152 2023-11-13T10:54:04.421 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/152 2023-11-13T10:54:04.421 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/152 2023-11-13T10:54:04.421 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/152 2023-11-13T10:54:04.421 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/152 2023-11-13T10:54:04.422 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-16.el9.x86_64 45/152 2023-11-13T10:54:04.423 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/152 2023-11-13T10:54:04.423 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/152 2023-11-13T10:54:04.423 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 48/152 2023-11-13T10:54:04.423 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 49/152 2023-11-13T10:54:04.424 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 50/152 2023-11-13T10:54:04.424 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 51/152 2023-11-13T10:54:04.424 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 52/152 2023-11-13T10:54:04.424 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 53/152 2023-11-13T10:54:04.424 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 54/152 2023-11-13T10:54:04.425 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.11-1.el9.x86_64 55/152 2023-11-13T10:54:04.425 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 56/152 2023-11-13T10:54:04.425 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 57/152 2023-11-13T10:54:04.425 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 58/152 2023-11-13T10:54:04.425 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/152 2023-11-13T10:54:04.426 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 60/152 2023-11-13T10:54:04.426 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 61/152 2023-11-13T10:54:04.426 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/152 2023-11-13T10:54:04.426 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/152 2023-11-13T10:54:04.426 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/152 2023-11-13T10:54:04.427 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/152 2023-11-13T10:54:04.427 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/152 2023-11-13T10:54:04.427 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/152 2023-11-13T10:54:04.427 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/152 2023-11-13T10:54:04.427 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/152 2023-11-13T10:54:04.437 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/152 2023-11-13T10:54:04.437 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/152 2023-11-13T10:54:04.437 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 72/152 2023-11-13T10:54:04.438 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 73/152 2023-11-13T10:54:04.438 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 74/152 2023-11-13T10:54:04.438 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/152 2023-11-13T10:54:04.438 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 76/152 2023-11-13T10:54:04.438 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 77/152 2023-11-13T10:54:04.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/152 2023-11-13T10:54:04.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/152 2023-11-13T10:54:04.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/152 2023-11-13T10:54:04.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/152 2023-11-13T10:54:04.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 82/152 2023-11-13T10:54:04.440 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 83/152 2023-11-13T10:54:04.440 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 84/152 2023-11-13T10:54:04.440 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 85/152 2023-11-13T10:54:04.440 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 86/152 2023-11-13T10:54:04.445 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 87/152 2023-11-13T10:54:04.445 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 88/152 2023-11-13T10:54:04.445 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/152 2023-11-13T10:54:04.445 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/152 2023-11-13T10:54:04.446 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/152 2023-11-13T10:54:04.446 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/152 2023-11-13T10:54:04.446 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/152 2023-11-13T10:54:04.446 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/152 2023-11-13T10:54:04.446 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/152 2023-11-13T10:54:04.447 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/152 2023-11-13T10:54:04.447 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/152 2023-11-13T10:54:04.447 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/152 2023-11-13T10:54:04.447 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/152 2023-11-13T10:54:04.447 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/152 2023-11-13T10:54:04.448 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/152 2023-11-13T10:54:04.448 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/152 2023-11-13T10:54:04.448 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/152 2023-11-13T10:54:04.448 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/152 2023-11-13T10:54:04.448 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/152 2023-11-13T10:54:04.448 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/152 2023-11-13T10:54:04.449 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/152 2023-11-13T10:54:04.449 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/152 2023-11-13T10:54:04.449 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/152 2023-11-13T10:54:04.449 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/152 2023-11-13T10:54:04.449 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/152 2023-11-13T10:54:04.450 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/152 2023-11-13T10:54:04.450 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/152 2023-11-13T10:54:04.450 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/152 2023-11-13T10:54:04.450 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/152 2023-11-13T10:54:04.450 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/152 2023-11-13T10:54:04.452 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/152 2023-11-13T10:54:04.452 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/152 2023-11-13T10:54:04.452 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/152 2023-11-13T10:54:04.452 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/152 2023-11-13T10:54:04.452 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/152 2023-11-13T10:54:04.453 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/152 2023-11-13T10:54:04.453 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/152 2023-11-13T10:54:04.453 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/152 2023-11-13T10:54:04.453 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/152 2023-11-13T10:54:04.454 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/152 2023-11-13T10:54:04.454 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/152 2023-11-13T10:54:04.454 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/152 2023-11-13T10:54:04.454 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/152 2023-11-13T10:54:04.454 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/152 2023-11-13T10:54:04.455 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/152 2023-11-13T10:54:04.455 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/152 2023-11-13T10:54:04.455 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:18.0.0-7288.g35f0debb.el9.x86_64 133/152 2023-11-13T10:54:04.455 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/152 2023-11-13T10:54:04.455 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:18.0.0-7288.g35f0debb.el9.x86_64 135/152 2023-11-13T10:54:04.455 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/152 2023-11-13T10:54:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/152 2023-11-13T10:54:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/152 2023-11-13T10:54:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/152 2023-11-13T10:54:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/152 2023-11-13T10:54:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/152 2023-11-13T10:54:04.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/152 2023-11-13T10:54:04.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/152 2023-11-13T10:54:04.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/152 2023-11-13T10:54:04.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/152 2023-11-13T10:54:04.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/152 2023-11-13T10:54:04.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/152 2023-11-13T10:54:04.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/152 2023-11-13T10:54:04.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/152 2023-11-13T10:54:04.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-13T10:54:04.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/152 2023-11-13T10:54:04.497 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-13T10:54:04.498 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-13T10:54:04.644 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-13T10:54:04.704 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-13T10:54:04.704 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-13T10:54:04.842 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-13T10:54:04.965 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-13T10:54:05.288 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-13T10:54:05.288 INFO:teuthology.orchestra.run.smithi184.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-13T10:54:05.288 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:06.563 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-13T10:54:06.564 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:06.564 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-11-13T10:54:06.564 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-13T10:54:06.564 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-13T10:54:06.565 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-13T10:54:06.565 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-13T10:54:06.565 INFO:teuthology.orchestra.run.smithi016.stdout: acl-2.3.1-4.el9.x86_64 2023-11-13T10:54:06.565 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-13T10:54:06.565 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-13T10:54:06.566 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.566 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.567 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-13T10:54:06.567 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-11-13T10:54:06.567 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2023-11-13T10:54:06.567 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2023-11-13T10:54:06.567 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-13T10:54:06.584 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.584 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.586 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.586 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.586 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.586 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.586 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.587 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.587 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.587 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.587 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.587 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.588 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.588 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.588 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:06.589 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-11-13T10:54:06.589 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2023-11-13T10:54:06.589 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-13T10:54:06.589 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-13T10:54:06.589 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-13T10:54:06.590 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-13T10:54:06.590 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-13T10:54:06.590 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-16.el9.x86_64 2023-11-13T10:54:06.590 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-13T10:54:06.612 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-13T10:54:06.612 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-13T10:54:06.612 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.612 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.613 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.613 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-13T10:54:06.613 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-13T10:54:06.619 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-13T10:54:06.619 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-13T10:54:06.619 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-13T10:54:06.620 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.620 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.620 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-13T10:54:06.620 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.620 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-13T10:54:06.621 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-13T10:54:06.621 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-13T10:54:06.625 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-13T10:54:06.626 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-13T10:54:06.626 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-13T10:54:06.626 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-13T10:54:06.626 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-13T10:54:06.627 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-13T10:54:06.627 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-13T10:54:06.627 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.11-1.el9.x86_64 2023-11-13T10:54:06.627 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-13T10:54:06.627 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-13T10:54:06.628 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-13T10:54:06.628 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-13T10:54:06.628 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-13T10:54:06.628 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-13T10:54:06.628 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-13T10:54:06.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-13T10:54:06.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-13T10:54:06.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-13T10:54:06.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-13T10:54:06.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-13T10:54:06.651 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-13T10:54:06.651 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-13T10:54:06.651 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-13T10:54:06.651 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-13T10:54:06.652 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-13T10:54:06.652 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-13T10:54:06.652 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-13T10:54:06.652 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-13T10:54:06.652 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-13T10:54:06.653 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-13T10:54:06.653 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-13T10:54:06.653 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-13T10:54:06.653 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-13T10:54:06.653 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-13T10:54:06.653 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-13T10:54:06.654 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-13T10:54:06.654 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-13T10:54:06.654 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-13T10:54:06.654 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-13T10:54:06.655 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-13T10:54:06.655 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-13T10:54:06.656 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-13T10:54:06.656 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2023-11-13T10:54:06.657 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-13T10:54:06.657 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-13T10:54:06.657 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-13T10:54:06.658 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-13T10:54:06.658 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-13T10:54:06.658 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-13T10:54:06.658 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-13T10:54:06.659 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.659 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.659 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-13T10:54:06.659 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-13T10:54:06.660 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-13T10:54:06.660 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.660 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-13T10:54:06.660 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-13T10:54:06.660 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-13T10:54:06.661 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-13T10:54:06.661 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-13T10:54:06.661 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-13T10:54:06.661 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-13T10:54:06.661 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-13T10:54:06.662 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-13T10:54:06.662 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-13T10:54:06.662 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-13T10:54:06.662 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.662 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.663 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:06.663 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2023-11-13T10:54:06.663 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-13T10:54:06.663 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-13T10:54:06.664 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-13T10:54:06.664 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-13T10:54:06.664 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-13T10:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-13T10:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-11-13T10:54:07.095 DEBUG:teuthology.parallel:result is None 2023-11-13T10:54:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:18.0.0-7288.g35f0debb.el9.x86_64 1/152 2023-11-13T10:54:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:18.0.0-7288.g35f0debb.el9.x86_64 2/152 2023-11-13T10:54:10.093 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 3/152 2023-11-13T10:54:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 4/152 2023-11-13T10:54:10.094 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7288.g35f0d 5/152 2023-11-13T10:54:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:18.0.0-7288.g35f0debb.el9.x86_64 6/152 2023-11-13T10:54:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:18.0.0-7288.g35f0debb.el9.x86_64 7/152 2023-11-13T10:54:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:18.0.0-7288.g35f0debb.el9.x86_64 8/152 2023-11-13T10:54:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:18.0.0-7288.g35f0debb.el9.x86_64 9/152 2023-11-13T10:54:10.095 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:18.0.0-7288.g35f0debb.el9.x86_64 10/152 2023-11-13T10:54:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:18.0.0-7288.g35f0debb.el9.x86_64 11/152 2023-11-13T10:54:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:18.0.0-7288.g35f0debb.el9.x86_64 12/152 2023-11-13T10:54:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 13/152 2023-11-13T10:54:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:18.0.0-7288.g35f0debb.el9.x86_ 14/152 2023-11-13T10:54:10.096 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:18.0.0-7288.g35f0debb.el9.x86_64 15/152 2023-11-13T10:54:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:18.0.0-7288.g35f0debb.el9.x86_64 16/152 2023-11-13T10:54:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:18.0.0-7288.g35f0debb.el9.x86_6 17/152 2023-11-13T10:54:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:18.0.0-7288.g35f0debb.el9.x86 18/152 2023-11-13T10:54:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:18.0.0-7288.g35f0debb.el9.x86_64 19/152 2023-11-13T10:54:10.097 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7288.g35f0debb.el 20/152 2023-11-13T10:54:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:18.0.0-7288.g35f0debb.el9. 21/152 2023-11-13T10:54:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:18.0.0-7288.g35f0debb.el9.x86_6 22/152 2023-11-13T10:54:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:18.0.0-7288.g35f0debb.el9.x86_64 23/152 2023-11-13T10:54:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:18.0.0-7288.g35f0debb.el9.x86_64 24/152 2023-11-13T10:54:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:18.0.0-7288.g35f0debb.el9.x86_64 25/152 2023-11-13T10:54:10.098 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 26/152 2023-11-13T10:54:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 27/152 2023-11-13T10:54:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:18.0.0-7288.g35f0debb.el9.x86_64 28/152 2023-11-13T10:54:10.099 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7288.g35f0debb. 29/152 2023-11-13T10:54:10.152 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7288.g35f0debb.el9.noa 30/152 2023-11-13T10:54:10.152 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7288.g35f0debb.el9.n 31/152 2023-11-13T10:54:10.152 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7288.g35f 32/152 2023-11-13T10:54:10.153 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7288.g35f0debb.el 33/152 2023-11-13T10:54:10.153 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7288.g35f0debb.el9.noarch 34/152 2023-11-13T10:54:10.153 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7288.g35f0debb.e 35/152 2023-11-13T10:54:10.154 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:18.0.0-7288.g35f0debb.el9.noarch 36/152 2023-11-13T10:54:10.154 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 37/152 2023-11-13T10:54:10.154 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:18.0.0-7288.g35f0debb.el9.noarch 38/152 2023-11-13T10:54:10.172 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/152 2023-11-13T10:54:10.173 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/152 2023-11-13T10:54:10.175 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/152 2023-11-13T10:54:10.175 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/152 2023-11-13T10:54:10.175 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/152 2023-11-13T10:54:10.176 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/152 2023-11-13T10:54:10.176 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : jq-1.6-16.el9.x86_64 45/152 2023-11-13T10:54:10.176 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/152 2023-11-13T10:54:10.176 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/152 2023-11-13T10:54:10.177 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 48/152 2023-11-13T10:54:10.177 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 49/152 2023-11-13T10:54:10.177 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 50/152 2023-11-13T10:54:10.177 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 51/152 2023-11-13T10:54:10.177 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 52/152 2023-11-13T10:54:10.178 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 53/152 2023-11-13T10:54:10.178 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 54/152 2023-11-13T10:54:10.178 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-13.11-1.el9.x86_64 55/152 2023-11-13T10:54:10.178 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 56/152 2023-11-13T10:54:10.179 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 57/152 2023-11-13T10:54:10.179 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 58/152 2023-11-13T10:54:10.179 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/152 2023-11-13T10:54:10.179 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 60/152 2023-11-13T10:54:10.179 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 61/152 2023-11-13T10:54:10.180 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/152 2023-11-13T10:54:10.180 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/152 2023-11-13T10:54:10.180 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/152 2023-11-13T10:54:10.188 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/152 2023-11-13T10:54:10.189 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/152 2023-11-13T10:54:10.189 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/152 2023-11-13T10:54:10.189 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/152 2023-11-13T10:54:10.190 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/152 2023-11-13T10:54:10.190 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/152 2023-11-13T10:54:10.190 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/152 2023-11-13T10:54:10.190 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 72/152 2023-11-13T10:54:10.191 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 73/152 2023-11-13T10:54:10.191 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 74/152 2023-11-13T10:54:10.191 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/152 2023-11-13T10:54:10.191 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 76/152 2023-11-13T10:54:10.192 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 77/152 2023-11-13T10:54:10.192 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/152 2023-11-13T10:54:10.192 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/152 2023-11-13T10:54:10.192 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/152 2023-11-13T10:54:10.192 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/152 2023-11-13T10:54:10.192 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 82/152 2023-11-13T10:54:10.193 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 83/152 2023-11-13T10:54:10.193 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 84/152 2023-11-13T10:54:10.193 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 85/152 2023-11-13T10:54:10.193 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 86/152 2023-11-13T10:54:10.194 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.2-8.el9.noarch 87/152 2023-11-13T10:54:10.194 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 88/152 2023-11-13T10:54:10.194 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/152 2023-11-13T10:54:10.194 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/152 2023-11-13T10:54:10.194 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/152 2023-11-13T10:54:10.195 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/152 2023-11-13T10:54:10.195 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/152 2023-11-13T10:54:10.195 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/152 2023-11-13T10:54:10.195 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/152 2023-11-13T10:54:10.195 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/152 2023-11-13T10:54:10.196 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/152 2023-11-13T10:54:10.196 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/152 2023-11-13T10:54:10.196 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/152 2023-11-13T10:54:10.197 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/152 2023-11-13T10:54:10.197 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/152 2023-11-13T10:54:10.197 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/152 2023-11-13T10:54:10.197 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/152 2023-11-13T10:54:10.198 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/152 2023-11-13T10:54:10.198 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/152 2023-11-13T10:54:10.198 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/152 2023-11-13T10:54:10.198 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/152 2023-11-13T10:54:10.199 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/152 2023-11-13T10:54:10.199 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/152 2023-11-13T10:54:10.200 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/152 2023-11-13T10:54:10.200 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/152 2023-11-13T10:54:10.200 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/152 2023-11-13T10:54:10.200 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/152 2023-11-13T10:54:10.200 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/152 2023-11-13T10:54:10.201 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/152 2023-11-13T10:54:10.201 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/152 2023-11-13T10:54:10.201 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/152 2023-11-13T10:54:10.201 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/152 2023-11-13T10:54:10.201 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/152 2023-11-13T10:54:10.202 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/152 2023-11-13T10:54:10.202 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/152 2023-11-13T10:54:10.202 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/152 2023-11-13T10:54:10.202 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/152 2023-11-13T10:54:10.202 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/152 2023-11-13T10:54:10.203 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/152 2023-11-13T10:54:10.203 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/152 2023-11-13T10:54:10.203 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/152 2023-11-13T10:54:10.203 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/152 2023-11-13T10:54:10.203 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/152 2023-11-13T10:54:10.204 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/152 2023-11-13T10:54:10.204 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/152 2023-11-13T10:54:10.204 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/152 2023-11-13T10:54:10.204 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:18.0.0-7288.g35f0debb.el9.x86_64 133/152 2023-11-13T10:54:10.204 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/152 2023-11-13T10:54:10.205 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:18.0.0-7288.g35f0debb.el9.x86_64 135/152 2023-11-13T10:54:10.205 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/152 2023-11-13T10:54:10.206 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/152 2023-11-13T10:54:10.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/152 2023-11-13T10:54:10.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/152 2023-11-13T10:54:10.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/152 2023-11-13T10:54:10.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/152 2023-11-13T10:54:10.207 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/152 2023-11-13T10:54:10.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/152 2023-11-13T10:54:10.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/152 2023-11-13T10:54:10.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/152 2023-11-13T10:54:10.208 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/152 2023-11-13T10:54:10.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/152 2023-11-13T10:54:10.255 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/152 2023-11-13T10:54:10.255 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/152 2023-11-13T10:54:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-13T10:54:10.257 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/152 2023-11-13T10:54:12.195 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-13T10:54:12.196 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:12.196 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2023-11-13T10:54:12.196 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-13T10:54:12.196 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-13T10:54:12.197 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-13T10:54:12.197 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-13T10:54:12.197 INFO:teuthology.orchestra.run.smithi184.stdout: acl-2.3.1-4.el9.x86_64 2023-11-13T10:54:12.197 INFO:teuthology.orchestra.run.smithi184.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-13T10:54:12.197 INFO:teuthology.orchestra.run.smithi184.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-13T10:54:12.197 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.198 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.198 INFO:teuthology.orchestra.run.smithi184.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-13T10:54:12.198 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2023-11-13T10:54:12.198 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.2-8.el9.noarch 2023-11-13T10:54:12.198 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.7.4-5.el9.x86_64 2023-11-13T10:54:12.199 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-13T10:54:12.199 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.199 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.199 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.199 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.200 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.218 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.218 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.218 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.219 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.219 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.219 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.219 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.219 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.220 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.221 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.221 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.221 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.221 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:18.0.0-7288.g35f0debb.el9.noarch 2023-11-13T10:54:12.221 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-11-13T10:54:12.222 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.4-9.el9.x86_64 2023-11-13T10:54:12.222 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-13T10:54:12.222 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-13T10:54:12.222 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-13T10:54:12.222 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-13T10:54:12.224 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-13T10:54:12.224 INFO:teuthology.orchestra.run.smithi184.stdout: jq-1.6-16.el9.x86_64 2023-11-13T10:54:12.224 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-13T10:54:12.224 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-13T10:54:12.226 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-13T10:54:12.226 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.226 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.226 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.226 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-13T10:54:12.230 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-13T10:54:12.230 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-13T10:54:12.230 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-13T10:54:12.231 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-13T10:54:12.231 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.231 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.231 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-13T10:54:12.231 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.232 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-13T10:54:12.232 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-13T10:54:12.232 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-13T10:54:12.232 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-13T10:54:12.232 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-13T10:54:12.233 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-13T10:54:12.240 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-13T10:54:12.241 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-13T10:54:12.241 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-13T10:54:12.242 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-13T10:54:12.242 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-13.11-1.el9.x86_64 2023-11-13T10:54:12.245 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-13T10:54:12.245 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-13T10:54:12.245 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-13T10:54:12.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-13T10:54:12.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-13T10:54:12.248 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-13T10:54:12.250 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-13T10:54:12.250 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.250 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.250 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.251 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-13T10:54:12.251 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-13T10:54:12.251 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-13T10:54:12.251 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-13T10:54:12.251 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-13T10:54:12.252 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-13T10:54:12.252 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-13T10:54:12.252 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-13T10:54:12.252 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-13T10:54:12.252 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-13T10:54:12.253 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-13T10:54:12.253 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-13T10:54:12.253 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-13T10:54:12.253 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-13T10:54:12.253 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-13T10:54:12.253 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-13T10:54:12.254 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-13T10:54:12.254 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-13T10:54:12.254 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-13T10:54:12.254 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-13T10:54:12.254 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-13T10:54:12.255 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-13T10:54:12.255 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-13T10:54:12.255 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-13T10:54:12.255 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-13T10:54:12.255 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-13T10:54:12.256 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-13T10:54:12.256 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2023-11-13T10:54:12.256 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-13T10:54:12.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-13T10:54:12.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-13T10:54:12.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-13T10:54:12.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-13T10:54:12.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-13T10:54:12.258 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-13T10:54:12.258 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.258 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.258 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-13T10:54:12.258 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-13T10:54:12.259 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-13T10:54:12.259 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.259 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-13T10:54:12.259 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-13T10:54:12.259 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-13T10:54:12.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-13T10:54:12.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-13T10:54:12.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-13T10:54:12.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-13T10:54:12.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-13T10:54:12.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-13T10:54:12.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-13T10:54:12.262 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-13T10:54:12.262 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.262 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.262 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:18.0.0-7288.g35f0debb.el9.x86_64 2023-11-13T10:54:12.262 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-3.el9.x86_64 2023-11-13T10:54:12.262 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-13T10:54:12.263 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-13T10:54:12.263 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-13T10:54:12.263 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-13T10:54:12.263 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-13T10:54:12.264 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-13T10:54:12.264 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:12.264 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2023-11-13T10:54:12.755 DEBUG:teuthology.parallel:result is None 2023-11-13T10:54:12.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=35f0debb3cc28955b7132ef4c509cd138a09d56b 2023-11-13T10:54:12.934 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-13T10:54:13.008 INFO:teuthology.orchestra.run.smithi016.stdout:18.0.0-7288.g35f0debb.el9 2023-11-13T10:54:13.009 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7288.g35f0debb.el9 2023-11-13T10:54:13.009 INFO:teuthology.task.install:The correct ceph version 18.0.0-7288.g35f0debb is installed. 2023-11-13T10:54:13.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=35f0debb3cc28955b7132ef4c509cd138a09d56b 2023-11-13T10:54:13.193 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-13T10:54:13.252 INFO:teuthology.orchestra.run.smithi184.stdout:18.0.0-7288.g35f0debb.el9 2023-11-13T10:54:13.253 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7288.g35f0debb.el9 2023-11-13T10:54:13.253 INFO:teuthology.task.install:The correct ceph version 18.0.0-7288.g35f0debb is installed. 2023-11-13T10:54:13.254 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-13T10:54:13.255 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:13.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-13T10:54:13.366 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:13.367 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-13T10:54:13.454 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-13T10:54:13.454 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:13.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2023-11-13T10:54:13.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-13T10:54:13.693 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:13.694 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2023-11-13T10:54:13.779 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-13T10:54:13.909 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-13T10:54:13.909 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:13.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-13T10:54:14.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-13T10:54:14.152 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:14.153 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-13T10:54:14.240 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-13T10:54:14.370 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-13T10:54:14.371 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:14.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2023-11-13T10:54:14.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-13T10:54:14.609 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:14.609 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2023-11-13T10:54:14.703 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-13T10:54:14.829 INFO:teuthology.run_tasks:Running task ceph... 2023-11-13T10:54:14.957 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-13T10:54:14.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2023-11-13T10:54:14.969 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2023-11-13T10:54:15.068 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-13T10:54:15.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-13T10:54:15.078 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-13T10:54:15.179 INFO:tasks.ceph:Creating extra log directories... 2023-11-13T10:54:15.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-13T10:54:15.189 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-13T10:54:15.310 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-13T10:54:15.311 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-13T10:54:15.311 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-13T10:54:15.312 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:15.312 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-13T10:54:15.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-13T10:54:15.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-13T10:54:15.760 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:15.760 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-13T10:54:15.813 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-13T10:54:16.042 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-13T10:54:16.144 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-13T10:54:16.144 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-13T10:54:16.145 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203', 'branch': 'wip-mchangir-testing-20231110.052303', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7456203', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing-20231110.052303'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing-20231110.052303', 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b'}}, 'owner': 'mchangir', 'priority': 1000, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9746, 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b', 'sleep_before_teardown': 0, 'subset': '768/1024', 'suite': 'fs', 'suite_branch': 'wip-mchangir-testing-20231110.052303', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_35f0debb3cc28955b7132ef4c509cd138a09d56b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmDbJx+U9GmeFR4wwiA9x34wyOtfCZEFig5NaMF0vaUMTTjcj/SfrLykbTjtRWCfPw3oIGybvtHApcHSzVhUZg=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJpsKYQqUYDjhRd5uNMPwBwJ509eIRR2VHJ+32ddcRdjFkfLmbJY/0edFsI/2jlUR0AIUdrZhPsuw93QGjPHEZY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '35f0debb3cc28955b7132ef4c509cd138a09d56b', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-13_10:27:15', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714'} 2023-11-13T10:54:16.149 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-13T10:54:16.170 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-13T10:54:16.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2023-11-13T10:54:16.231 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2023-11-13T10:54:16.332 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:16.333 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2023-11-13T10:54:16.395 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-13T10:54:16.396 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2023-11-13T10:54:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-13T10:54:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-13T10:54:16.453 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-11-13T10:54:16.453 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-13T10:54:16.453 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-11-13T10:54:16.453 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-11-13 10:53:59.880179423 +0000 2023-11-13T10:54:16.453 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-11-13 10:51:26.773879787 +0000 2023-11-13T10:54:16.454 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-11-13 10:51:26.773879787 +0000 2023-11-13T10:54:16.454 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-11-13 10:51:26.773879787 +0000 2023-11-13T10:54:16.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-13T10:54:16.546 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-13T10:54:16.547 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-13T10:54:16.547 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000243164 s, 2.1 MB/s 2023-11-13T10:54:16.557 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-13T10:54:16.623 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2023-11-13T10:54:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-13T10:54:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-13T10:54:16.682 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 651 Links: 1 2023-11-13T10:54:16.682 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-13T10:54:16.682 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-11-13T10:54:16.683 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-11-13 10:53:59.881179407 +0000 2023-11-13T10:54:16.683 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-11-13 10:51:28.043860361 +0000 2023-11-13T10:54:16.683 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-11-13 10:51:28.043860361 +0000 2023-11-13T10:54:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-11-13 10:51:28.043860361 +0000 2023-11-13T10:54:16.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-13T10:54:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-13T10:54:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-13T10:54:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000233797 s, 2.2 MB/s 2023-11-13T10:54:16.792 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-13T10:54:16.859 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2023-11-13T10:54:16.916 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-13T10:54:16.916 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-13T10:54:16.916 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-11-13T10:54:16.917 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-13T10:54:16.917 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-11-13T10:54:16.917 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-11-13 10:53:59.882179392 +0000 2023-11-13T10:54:16.917 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-11-13 10:51:29.318840858 +0000 2023-11-13T10:54:16.917 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-11-13 10:51:29.318840858 +0000 2023-11-13T10:54:16.918 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-11-13 10:51:29.318840858 +0000 2023-11-13T10:54:16.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-13T10:54:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-13T10:54:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-13T10:54:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000241313 s, 2.1 MB/s 2023-11-13T10:54:17.022 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-13T10:54:17.084 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2023-11-13T10:54:17.140 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-13T10:54:17.140 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-13T10:54:17.140 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-11-13T10:54:17.141 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-13T10:54:17.141 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-11-13T10:54:17.141 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-11-13 10:53:59.883179377 +0000 2023-11-13T10:54:17.141 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-11-13 10:51:30.548822043 +0000 2023-11-13T10:54:17.141 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-11-13 10:51:30.548822043 +0000 2023-11-13T10:54:17.142 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-11-13 10:51:30.548822043 +0000 2023-11-13T10:54:17.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-13T10:54:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-13T10:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-13T10:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000249048 s, 2.1 MB/s 2023-11-13T10:54:17.254 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-13T10:54:17.325 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:17.326 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:17.326 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2023-11-13T10:54:17.388 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-13T10:54:17.389 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2023-11-13T10:54:17.440 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-13T10:54:17.441 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-13T10:54:17.441 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-11-13T10:54:17.441 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-13T10:54:17.442 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-11-13T10:54:17.442 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-11-13 10:54:05.734293990 +0000 2023-11-13T10:54:17.442 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-11-13 10:50:57.254332450 +0000 2023-11-13T10:54:17.442 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-11-13 10:50:57.254332450 +0000 2023-11-13T10:54:17.442 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2023-11-13 10:50:57.254332450 +0000 2023-11-13T10:54:17.443 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-13T10:54:17.566 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-11-13T10:54:17.567 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-11-13T10:54:17.567 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.0003439 s, 1.5 MB/s 2023-11-13T10:54:17.569 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-13T10:54:17.625 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2023-11-13T10:54:17.675 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-13T10:54:17.675 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-13T10:54:17.676 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-11-13T10:54:17.676 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-13T10:54:17.676 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-11-13T10:54:17.676 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-11-13 10:54:05.735293970 +0000 2023-11-13T10:54:17.676 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-11-13 10:50:58.429308242 +0000 2023-11-13T10:54:17.677 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-11-13 10:50:58.429308242 +0000 2023-11-13T10:54:17.677 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2023-11-13 10:50:58.429308242 +0000 2023-11-13T10:54:17.677 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-13T10:54:17.769 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-11-13T10:54:17.769 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-11-13T10:54:17.769 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000348953 s, 1.5 MB/s 2023-11-13T10:54:17.775 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-13T10:54:17.838 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2023-11-13T10:54:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-13T10:54:17.891 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-13T10:54:17.892 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-11-13T10:54:17.892 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-13T10:54:17.892 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-11-13T10:54:17.892 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-11-13 10:54:05.736293949 +0000 2023-11-13T10:54:17.892 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-11-13 10:50:59.695282159 +0000 2023-11-13T10:54:17.892 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-11-13 10:50:59.695282159 +0000 2023-11-13T10:54:17.893 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2023-11-13 10:50:59.695282159 +0000 2023-11-13T10:54:17.893 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-13T10:54:17.989 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-11-13T10:54:17.990 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-11-13T10:54:17.990 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000326745 s, 1.6 MB/s 2023-11-13T10:54:17.991 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-13T10:54:18.055 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2023-11-13T10:54:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-13T10:54:18.105 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-13T10:54:18.105 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-11-13T10:54:18.106 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-13T10:54:18.106 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-11-13T10:54:18.106 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-11-13 10:54:05.737293928 +0000 2023-11-13T10:54:18.106 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-11-13 10:51:00.890257539 +0000 2023-11-13T10:54:18.106 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-11-13 10:51:00.890257539 +0000 2023-11-13T10:54:18.107 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2023-11-13 10:51:00.890257539 +0000 2023-11-13T10:54:18.107 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-13T10:54:18.196 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-11-13T10:54:18.196 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-11-13T10:54:18.197 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00038383 s, 1.3 MB/s 2023-11-13T10:54:18.199 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-13T10:54:18.258 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:18.258 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-13T10:54:18.258 INFO:tasks.ceph:Generating config... 2023-11-13T10:54:18.260 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-13T10:54:18.260 INFO:tasks.ceph:[client] debug client = 20 2023-11-13T10:54:18.260 INFO:tasks.ceph:[client] debug ms = 1 2023-11-13T10:54:18.261 INFO:tasks.ceph:[client] pid file = 2023-11-13T10:54:18.261 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-13T10:54:18.261 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-13T10:54:18.262 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-13T10:54:18.262 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-13T10:54:18.263 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-13T10:54:18.263 INFO:tasks.ceph:[mds] mds cache max size = 1000 2023-11-13T10:54:18.263 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-13T10:54:18.264 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-13T10:54:18.264 INFO:tasks.ceph:[mds] mds log max segments = 1 2023-11-13T10:54:18.275 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-13T10:54:18.276 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-13T10:54:18.277 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-13T10:54:18.277 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-13T10:54:18.277 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-13T10:54:18.278 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-13T10:54:18.278 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-13T10:54:18.278 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-13T10:54:18.279 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-13T10:54:18.279 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-13T10:54:18.279 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-13T10:54:18.279 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-13T10:54:18.280 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-13T10:54:18.280 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-13T10:54:18.280 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-13T10:54:18.281 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-13T10:54:18.281 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-13T10:54:18.281 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-13T10:54:18.282 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-13T10:54:18.282 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-13T10:54:18.282 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-13T10:54:18.283 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-13T10:54:18.283 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-13T10:54:18.283 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-13T10:54:18.284 INFO:tasks.ceph:Setting up mon.a... 2023-11-13T10:54:18.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-13T10:54:18.466 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2023-11-13T10:54:18.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-13T10:54:18.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-13T10:54:18.728 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.184'), ('mon.c', '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]')] 2023-11-13T10:54:18.729 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-13T10:54:18.730 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:18.730 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-13T10:54:18.792 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.184 --addv c '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:18.956 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:18.956 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid 130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9 2023-11-13T10:54:18.957 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = pacific 2023-11-13T10:54:18.957 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2023-11-13T10:54:18.957 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9 2023-11-13T10:54:18.957 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2023-11-13T10:54:18.953260+0000 2023-11-13T10:54:18.958 INFO:teuthology.orchestra.run.smithi016.stdout:created 2023-11-13T10:54:18.953260+0000 2023-11-13T10:54:18.958 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 16 (pacific) 2023-11-13T10:54:18.958 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2023-11-13T10:54:18.958 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2023-11-13T10:54:18.958 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.b 2023-11-13T10:54:18.959 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] mon.c 2023-11-13T10:54:18.959 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-13T10:54:18.960 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-13T10:54:19.020 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9... 2023-11-13T10:54:19.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-13T10:54:19.036 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-13T10:54:19.191 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-11-13T10:54:19.191 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-11-13T10:54:19.191 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-13T10:54:19.191 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-11-13T10:54:19.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.192 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-11-13T10:54:19.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.192 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-11-13T10:54:19.193 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.193 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-11-13T10:54:19.193 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-11-13T10:54:19.193 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.193 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-11-13T10:54:19.193 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-11-13T10:54:19.194 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.194 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-13T10:54:19.195 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-11-13T10:54:19.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.195 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-11-13T10:54:19.195 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-13T10:54:19.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.196 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-11-13T10:54:19.196 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.196 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-11-13T10:54:19.196 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-11-13T10:54:19.196 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-11-13T10:54:19.197 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-11-13T10:54:19.197 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-11-13T10:54:19.198 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-11-13T10:54:19.198 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-11-13T10:54:19.198 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-13T10:54:19.198 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-13T10:54:19.199 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-11-13T10:54:19.199 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.199 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-13T10:54:19.199 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.199 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-11-13T10:54:19.200 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.200 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-11-13T10:54:19.200 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.200 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-11-13T10:54:19.200 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-11-13T10:54:19.201 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-11-13T10:54:19.201 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-11-13T10:54:19.201 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9 2023-11-13T10:54:19.201 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.201 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-11-13T10:54:19.202 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-11-13T10:54:19.202 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.202 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-11-13T10:54:19.202 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-11-13T10:54:19.202 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-11-13T10:54:19.202 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.203 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-11-13T10:54:19.203 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-11-13T10:54:19.203 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.204 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-11-13T10:54:19.204 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-11-13T10:54:19.204 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-11-13T10:54:19.204 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2023-11-13T10:54:19.208 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.208 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-11-13T10:54:19.209 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-11-13T10:54:19.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.209 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-11-13T10:54:19.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.209 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-11-13T10:54:19.210 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.210 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-11-13T10:54:19.210 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.210 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-11-13T10:54:19.210 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-11-13T10:54:19.211 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.211 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-11-13T10:54:19.211 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-11-13T10:54:19.211 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-11-13T10:54:19.217 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-11-13T10:54:19.217 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-11-13T10:54:19.217 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-11-13T10:54:19.218 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-13T10:54:19.226 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-11-13T10:54:19.226 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-11-13T10:54:19.226 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-11-13T10:54:19.226 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-11-13T10:54:19.227 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-11-13T10:54:19.227 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-11-13T10:54:19.227 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-11-13T10:54:19.227 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-11-13T10:54:19.227 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.228 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-11-13T10:54:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-13T10:54:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-11-13T10:54:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-11-13T10:54:19.230 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-11-13T10:54:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-11-13T10:54:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-11-13T10:54:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-11-13T10:54:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.232 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-11-13T10:54:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-13T10:54:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-11-13T10:54:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-11-13T10:54:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-11-13T10:54:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-11-13T10:54:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-11-13T10:54:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-11-13T10:54:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-11-13T10:54:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-11-13T10:54:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-11-13T10:54:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-13T10:54:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-11-13T10:54:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-11-13T10:54:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-11-13T10:54:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-13T10:54:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-13T10:54:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-11-13T10:54:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-11-13T10:54:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-11-13T10:54:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-11-13T10:54:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:54:19.237 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-11-13T10:54:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-11-13T10:54:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-11-13T10:54:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-11-13T10:54:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-13T10:54:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-13T10:54:19.239 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-11-13T10:54:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-11-13T10:54:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-13T10:54:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-11-13T10:54:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-11-13T10:54:19.242 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-11-13T10:54:19.242 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-11-13T10:54:19.242 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-11-13T10:54:19.242 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-11-13T10:54:19.243 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-11-13T10:54:19.243 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-11-13T10:54:19.243 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-11-13T10:54:19.243 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-11-13T10:54:19.243 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-11-13T10:54:19.243 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-11-13T10:54:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-11-13T10:54:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-13T10:54:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: mds cache max size = 1000 2023-11-13T10:54:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-11-13T10:54:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2023-11-13T10:54:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: mds log max segments = 1 2023-11-13T10:54:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-11-13T10:54:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2023-11-13T10:54:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-11-13T10:54:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-11-13T10:54:19.246 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-11-13T10:54:19.247 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2023-11-13T10:54:19.248 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2023-11-13T10:54:19.248 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-13T10:54:19.248 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2023-11-13T10:54:19.248 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.249 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2023-11-13T10:54:19.249 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.249 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2023-11-13T10:54:19.249 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.250 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2023-11-13T10:54:19.251 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2023-11-13T10:54:19.252 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.252 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2023-11-13T10:54:19.253 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2023-11-13T10:54:19.253 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.253 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-13T10:54:19.253 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2023-11-13T10:54:19.253 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.254 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2023-11-13T10:54:19.254 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-13T10:54:19.254 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.254 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2023-11-13T10:54:19.254 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.255 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2023-11-13T10:54:19.255 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2023-11-13T10:54:19.255 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2023-11-13T10:54:19.255 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2023-11-13T10:54:19.256 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2023-11-13T10:54:19.256 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2023-11-13T10:54:19.256 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2023-11-13T10:54:19.257 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-13T10:54:19.257 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-13T10:54:19.257 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2023-11-13T10:54:19.257 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.258 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-13T10:54:19.258 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.258 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2023-11-13T10:54:19.258 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.258 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2023-11-13T10:54:19.259 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.259 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2023-11-13T10:54:19.259 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2023-11-13T10:54:19.259 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2023-11-13T10:54:19.259 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.16,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-11-13T10:54:19.260 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = 130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9 2023-11-13T10:54:19.261 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.262 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2023-11-13T10:54:19.263 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2023-11-13T10:54:19.263 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.263 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2023-11-13T10:54:19.263 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2023-11-13T10:54:19.263 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2023-11-13T10:54:19.264 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.264 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2023-11-13T10:54:19.264 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2023-11-13T10:54:19.264 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.265 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2023-11-13T10:54:19.265 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2023-11-13T10:54:19.265 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2023-11-13T10:54:19.265 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug trim objects = true 2023-11-13T10:54:19.265 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.266 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2023-11-13T10:54:19.266 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2023-11-13T10:54:19.266 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.266 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2023-11-13T10:54:19.266 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.267 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2023-11-13T10:54:19.268 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.268 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2023-11-13T10:54:19.268 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.268 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2023-11-13T10:54:19.268 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2023-11-13T10:54:19.269 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.269 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2023-11-13T10:54:19.269 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2023-11-13T10:54:19.269 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2023-11-13T10:54:19.269 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2023-11-13T10:54:19.270 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 20 2023-11-13T10:54:19.270 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 20 2023-11-13T10:54:19.270 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-11-13T10:54:19.272 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2023-11-13T10:54:19.273 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 10 2023-11-13T10:54:19.273 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2023-11-13T10:54:19.274 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2023-11-13T10:54:19.274 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2023-11-13T10:54:19.274 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2023-11-13T10:54:19.274 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2023-11-13T10:54:19.274 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-11-13T10:54:19.275 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.275 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2023-11-13T10:54:19.275 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-11-13T10:54:19.275 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2023-11-13T10:54:19.275 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-11-13T10:54:19.276 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-11-13T10:54:19.276 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-11-13T10:54:19.276 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-11-13T10:54:19.276 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2023-11-13T10:54:19.276 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.276 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2023-11-13T10:54:19.277 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-11-13T10:54:19.277 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-11-13T10:54:19.277 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2023-11-13T10:54:19.277 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-11-13T10:54:19.277 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2023-11-13T10:54:19.278 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2023-11-13T10:54:19.279 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-11-13T10:54:19.279 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2023-11-13T10:54:19.279 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2023-11-13T10:54:19.279 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-11-13T10:54:19.279 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.280 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-13T10:54:19.280 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2023-11-13T10:54:19.280 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2023-11-13T10:54:19.280 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.280 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2023-11-13T10:54:19.280 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-13T10:54:19.281 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-13T10:54:19.282 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.282 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2023-11-13T10:54:19.283 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2023-11-13T10:54:19.284 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.284 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2023-11-13T10:54:19.284 INFO:teuthology.orchestra.run.smithi184.stdout: mon op complaint time = 120 2023-11-13T10:54:19.284 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-11-13T10:54:19.284 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2023-11-13T10:54:19.285 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2023-11-13T10:54:19.285 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2023-11-13T10:54:19.285 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2023-11-13T10:54:19.285 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-13T10:54:19.285 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-13T10:54:19.286 INFO:teuthology.orchestra.run.smithi184.stdout: client mount timeout = 600 2023-11-13T10:54:19.286 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2023-11-13T10:54:19.286 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-11-13T10:54:19.286 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = "" 2023-11-13T10:54:19.286 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-11-13T10:54:19.287 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-11-13T10:54:19.287 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2023-11-13T10:54:19.288 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2023-11-13T10:54:19.288 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2023-11-13T10:54:19.288 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2023-11-13T10:54:19.289 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2023-11-13T10:54:19.289 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2023-11-13T10:54:19.289 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2023-11-13T10:54:19.289 INFO:teuthology.orchestra.run.smithi184.stdout:[mds] 2023-11-13T10:54:19.290 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds = 20 2023-11-13T10:54:19.290 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds balancer = 20 2023-11-13T10:54:19.290 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-11-13T10:54:19.290 INFO:teuthology.orchestra.run.smithi184.stdout: mds cache max size = 1000 2023-11-13T10:54:19.290 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug frag = True 2023-11-13T10:54:19.291 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug scatterstat = True 2023-11-13T10:54:19.291 INFO:teuthology.orchestra.run.smithi184.stdout: mds log max segments = 1 2023-11-13T10:54:19.291 INFO:teuthology.orchestra.run.smithi184.stdout: mds op complaint time = 180 2023-11-13T10:54:19.299 INFO:teuthology.orchestra.run.smithi184.stdout: mds verify scatter = True 2023-11-13T10:54:19.299 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-11-13T10:54:19.299 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-11-13T10:54:19.300 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-11-13T10:54:19.306 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-13T10:54:19.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-13T10:54:19.471 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-13T10:54:19.471 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:19.472 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-13T10:54:19.533 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:19.533 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-13T10:54:19.609 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T10:54:19.610 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:19.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-13T10:54:19.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-13T10:54:19.768 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:19.768 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:19.870 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T10:54:19.871 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:19.871 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-13T10:54:19.872 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-13T10:54:20.010 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:20.010 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:20.100 INFO:tasks.ceph:Setting up mon nodes... 2023-11-13T10:54:20.100 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-13T10:54:20.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-13T10:54:20.302 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-13T10:54:20.313 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-13T10:54:20.475 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-13T10:54:20.483 INFO:tasks.ceph:Setting up mds nodes... 2023-11-13T10:54:20.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-13T10:54:20.689 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-13T10:54:20.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-13T10:54:20.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-13T10:54:21.006 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-13T10:54:21.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-13T10:54:21.126 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-13T10:54:21.291 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-13T10:54:21.296 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-13T10:54:21.382 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-13T10:54:21.524 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-13T10:54:21.532 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-13T10:54:21.625 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-13T10:54:21.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-13T10:54:21.770 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-13T10:54:21.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-13T10:54:21.965 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-13T10:54:22.027 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-13T10:54:22.163 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-13T10:54:22.214 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-13T10:54:22.347 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-13T10:54:22.396 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-13T10:54:22.397 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-13T10:54:22.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-13T10:54:22.509 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:22.510 INFO:tasks.ceph:role: osd.0 2023-11-13T10:54:22.510 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T10:54:22.511 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-13T10:54:22.614 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-13T10:54:22.615 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-13T10:54:22.615 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-13T10:54:22.615 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-13T10:54:22.615 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-13T10:54:22.616 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-11-13T10:54:22.616 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-13T10:54:22.616 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-13T10:54:22.616 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-13T10:54:22.616 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-13T10:54:22.651 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-11-13T10:54:22.656 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-11-13T10:54:22.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-13T10:54:22.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-13T10:54:22.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-13T10:54:22.967 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:22.967 INFO:tasks.ceph:role: osd.1 2023-11-13T10:54:22.967 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T10:54:22.968 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-13T10:54:23.060 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-13T10:54:23.060 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-13T10:54:23.060 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-13T10:54:23.061 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-13T10:54:23.061 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-13T10:54:23.061 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-11-13T10:54:23.061 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-13T10:54:23.062 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-13T10:54:23.062 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-13T10:54:23.062 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-13T10:54:23.097 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-11-13T10:54:23.107 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-11-13T10:54:23.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-13T10:54:23.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-13T10:54:23.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-13T10:54:23.412 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:23.413 INFO:tasks.ceph:role: osd.2 2023-11-13T10:54:23.413 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T10:54:23.413 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-13T10:54:23.516 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-13T10:54:23.517 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-13T10:54:23.517 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-13T10:54:23.517 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-13T10:54:23.518 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-13T10:54:23.518 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-11-13T10:54:23.518 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-13T10:54:23.518 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-13T10:54:23.518 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-13T10:54:23.519 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-13T10:54:23.552 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-11-13T10:54:23.560 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-11-13T10:54:23.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-13T10:54:23.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-13T10:54:23.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-13T10:54:23.856 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:23.856 INFO:tasks.ceph:role: osd.3 2023-11-13T10:54:23.857 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T10:54:23.857 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-13T10:54:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-13T10:54:23.955 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-13T10:54:23.955 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-13T10:54:23.955 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-13T10:54:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-13T10:54:23.955 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-11-13T10:54:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-13T10:54:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-13T10:54:23.956 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-13T10:54:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-13T10:54:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-11-13T10:54:23.997 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-11-13T10:54:23.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-13T10:54:24.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-13T10:54:24.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:24.366+0000 7f765c307500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-13T10:54:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:24.366+0000 7f765c307500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-13T10:54:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:24.366+0000 7f765c307500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-13T10:54:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:24.366+0000 7f765c307500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-13T10:54:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:24.366+0000 7f765c307500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-13T10:54:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:24.370+0000 7f765c307500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-13T10:54:27.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-13T10:54:27.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:27.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:27.558+0000 7f06a068a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-13T10:54:27.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:27.559+0000 7f06a068a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-13T10:54:27.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:27.559+0000 7f06a068a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-13T10:54:27.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:27.559+0000 7f06a068a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-13T10:54:27.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:27.559+0000 7f06a068a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-13T10:54:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:27.561+0000 7f06a068a500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-13T10:54:30.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-13T10:54:30.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:30.753+0000 7fa086414500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-13T10:54:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:30.754+0000 7fa086414500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-13T10:54:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:30.754+0000 7fa086414500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-13T10:54:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:30.754+0000 7fa086414500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-13T10:54:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:30.754+0000 7fa086414500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-13T10:54:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:30.756+0000 7fa086414500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-13T10:54:33.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-13T10:54:34.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:34.203+0000 7f6a4ee21500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-13T10:54:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:34.203+0000 7f6a4ee21500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-13T10:54:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:34.203+0000 7f6a4ee21500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-13T10:54:34.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:34.203+0000 7f6a4ee21500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-13T10:54:34.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:34.203+0000 7f6a4ee21500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-13T10:54:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:54:34.205+0000 7f6a4ee21500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-13T10:54:37.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-13T10:54:37.516 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-13T10:54:37.619 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:37.620 INFO:tasks.ceph:role: osd.4 2023-11-13T10:54:37.620 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T10:54:37.620 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-13T10:54:37.726 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-13T10:54:37.727 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-13T10:54:37.727 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-13T10:54:37.727 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-13T10:54:37.727 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-13T10:54:37.728 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-11-13T10:54:37.728 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-13T10:54:37.728 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-13T10:54:37.728 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-13T10:54:37.729 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-13T10:54:38.243 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-11-13T10:54:38.249 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-11-13T10:54:38.250 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-13T10:54:38.439 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-13T10:54:38.553 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-13T10:54:38.651 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:38.652 INFO:tasks.ceph:role: osd.5 2023-11-13T10:54:38.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T10:54:38.653 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-13T10:54:38.750 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-13T10:54:38.750 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-13T10:54:38.751 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-13T10:54:38.751 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-13T10:54:38.751 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-13T10:54:38.754 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-11-13T10:54:38.755 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-13T10:54:38.755 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-13T10:54:38.755 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-13T10:54:38.755 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-13T10:54:39.244 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-11-13T10:54:39.250 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-11-13T10:54:39.251 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-13T10:54:39.385 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-13T10:54:39.484 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-13T10:54:39.571 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:39.571 INFO:tasks.ceph:role: osd.6 2023-11-13T10:54:39.572 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T10:54:39.572 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-13T10:54:39.668 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-13T10:54:39.668 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-13T10:54:39.669 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-13T10:54:39.669 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-13T10:54:39.669 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-13T10:54:39.669 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-11-13T10:54:39.670 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-13T10:54:39.670 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-13T10:54:39.670 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-13T10:54:39.670 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-13T10:54:40.169 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-11-13T10:54:40.176 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-11-13T10:54:40.176 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-13T10:54:40.361 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-13T10:54:40.467 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-13T10:54:40.557 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-13T10:54:40.557 INFO:tasks.ceph:role: osd.7 2023-11-13T10:54:40.558 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T10:54:40.558 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-13T10:54:40.661 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-13T10:54:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-13T10:54:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-13T10:54:40.662 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-13T10:54:40.663 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-13T10:54:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-11-13T10:54:40.663 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-13T10:54:40.663 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-13T10:54:40.663 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-13T10:54:40.664 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-13T10:54:41.122 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-11-13T10:54:41.128 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-11-13T10:54:41.129 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-13T10:54:41.267 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-13T10:54:41.382 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:41.538 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:41.534+0000 7f628a20e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-13T10:54:41.541 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:41.537+0000 7f628a20e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-13T10:54:41.542 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:41.537+0000 7f628a20e500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-13T10:54:41.542 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:41.538+0000 7f628a20e500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-13T10:54:41.542 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:41.538+0000 7f628a20e500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-13T10:54:41.547 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:41.543+0000 7f628a20e500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-13T10:54:44.488 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-13T10:54:44.591 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:44.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:44.725+0000 7f697980e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-13T10:54:44.730 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:44.726+0000 7f697980e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-13T10:54:44.730 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:44.726+0000 7f697980e500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-13T10:54:44.730 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:44.726+0000 7f697980e500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-13T10:54:44.731 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:44.726+0000 7f697980e500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-13T10:54:44.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:44.728+0000 7f697980e500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-13T10:54:46.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:54:46.161 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:54:47.920 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-13T10:54:48.017 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:48.153 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:48.149+0000 7f5d94810500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-13T10:54:48.154 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:48.150+0000 7f5d94810500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-13T10:54:48.155 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:48.150+0000 7f5d94810500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-13T10:54:48.155 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:48.150+0000 7f5d94810500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-13T10:54:48.155 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:48.150+0000 7f5d94810500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-13T10:54:48.156 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:48.152+0000 7f5d94810500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-13T10:54:51.338 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-13T10:54:51.437 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:54:51.578 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:51.574+0000 7f56d9c18500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-13T10:54:51.579 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:51.575+0000 7f56d9c18500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-13T10:54:51.580 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:51.575+0000 7f56d9c18500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-13T10:54:51.580 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:51.575+0000 7f56d9c18500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-13T10:54:51.580 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:51.575+0000 7f56d9c18500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-13T10:54:51.581 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:54:51.577+0000 7f56d9c18500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-13T10:54:54.508 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-13T10:54:54.612 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-13T10:54:54.612 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:54.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-13T10:54:54.727 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:54.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-13T10:54:54.837 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:54.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-13T10:54:54.948 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:54.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-13T10:54:55.052 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:55.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-13T10:54:55.156 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:55.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-13T10:54:55.254 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:55.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-13T10:54:55.351 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:55.351 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-13T10:54:55.460 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:55.460 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-13T10:54:55.541 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:55.541 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-13T10:54:55.681 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:55.681 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-13T10:54:55.779 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:55.779 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-13T10:54:55.886 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:55.886 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-13T10:54:56.035 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:56.035 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-13T10:54:56.135 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:56.136 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-13T10:54:56.204 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:54:56.204 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-13T10:54:56.258 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:56.259 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-13T10:54:56.332 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:54:56.332 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-13T10:54:56.437 INFO:tasks.ceph:Adding keys to all mons... 2023-11-13T10:54:56.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-13T10:54:56.446 DEBUG:teuthology.orchestra.run.smithi184:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-13T10:54:56.543 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2023-11-13T10:54:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBcAFJluuvTERAAn6zxosZEaeBkgENpZDAHqg== 2023-11-13T10:54:56.543 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-11-13T10:54:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBcAFJlFf7FKBAAzl2CzFIHt1vexIHicDcMOQ== 2023-11-13T10:54:56.544 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-11-13T10:54:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBdAFJl3f8tABAAn77Y5rtLX6KXfDt6O64jHw== 2023-11-13T10:54:56.544 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2023-11-13T10:54:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBgAFJl6OfsFRAA/pxN6JyHj0g/ZTvQTTBvHg== 2023-11-13T10:54:56.545 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2023-11-13T10:54:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBjAFJlCcthIRAAD9rCdKqLeqILvx+RcVu3aA== 2023-11-13T10:54:56.545 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2023-11-13T10:54:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBmAFJldeAFLRAA6BDzhw1Sml5AWptU6G7MEA== 2023-11-13T10:54:56.546 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2023-11-13T10:54:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBqAFJlhz82DBAAmRhdLjHKXpAyNNqUkXk5qw== 2023-11-13T10:54:56.551 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2023-11-13T10:54:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBcAFJlvFoIHBAAIXc5UQJIKTtHmeVznGZPSA== 2023-11-13T10:54:56.553 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-11-13T10:54:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBdAFJlxksXERAApz8MD99vmb9+TVqjK4yoHQ== 2023-11-13T10:54:56.553 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-11-13T10:54:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBdAFJliC8UHxAAp4RpLcBhknvkfcD6+u0cWg== 2023-11-13T10:54:56.554 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2023-11-13T10:54:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBxAFJlgXLmHxAAHpM0gY+C/Ionsp0sG9KoKQ== 2023-11-13T10:54:56.555 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2023-11-13T10:54:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQB0AFJlfsdPKxAA5Ba0gTAcUVQ6PueL9Ul6qQ== 2023-11-13T10:54:56.555 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2023-11-13T10:54:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQB4AFJlD6j5CBAAXuiAkwIbpfUSJ1j1j3QP4w== 2023-11-13T10:54:56.556 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2023-11-13T10:54:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQB7AFJlkX9TIhAAPjuDlChUGXjej7ot5x6vDg== 2023-11-13T10:54:56.556 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-13T10:54:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBdAFJl9p66LRAAYRThccLplmNpsr+W647dOA== 2023-11-13T10:54:56.557 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2023-11-13T10:54:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBdAFJlrk4zORAA7tCfwi0UlPhDcL1GZ6UIIg== 2023-11-13T10:54:56.557 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2023-11-13T10:54:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBeAFJlkymOCRAATnUAZgjg1er9vu82gnrxtw== 2023-11-13T10:54:56.557 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2023-11-13T10:54:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBeAFJlrh6DFBAAtrGSamisl/Jexs9ewdiAkQ== 2023-11-13T10:54:56.560 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.y] 2023-11-13T10:54:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBcAFJluuvTERAAn6zxosZEaeBkgENpZDAHqg== 2023-11-13T10:54:56.560 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2023-11-13T10:54:56.560 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBcAFJlFf7FKBAAzl2CzFIHt1vexIHicDcMOQ== 2023-11-13T10:54:56.560 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2023-11-13T10:54:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBdAFJl3f8tABAAn77Y5rtLX6KXfDt6O64jHw== 2023-11-13T10:54:56.561 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.0] 2023-11-13T10:54:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBgAFJl6OfsFRAA/pxN6JyHj0g/ZTvQTTBvHg== 2023-11-13T10:54:56.561 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.1] 2023-11-13T10:54:56.561 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBjAFJlCcthIRAAD9rCdKqLeqILvx+RcVu3aA== 2023-11-13T10:54:56.562 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.2] 2023-11-13T10:54:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBmAFJldeAFLRAA6BDzhw1Sml5AWptU6G7MEA== 2023-11-13T10:54:56.563 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.3] 2023-11-13T10:54:56.563 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBqAFJlhz82DBAAmRhdLjHKXpAyNNqUkXk5qw== 2023-11-13T10:54:56.564 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.x] 2023-11-13T10:54:56.564 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBcAFJlvFoIHBAAIXc5UQJIKTtHmeVznGZPSA== 2023-11-13T10:54:56.565 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2023-11-13T10:54:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBdAFJlxksXERAApz8MD99vmb9+TVqjK4yoHQ== 2023-11-13T10:54:56.566 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2023-11-13T10:54:56.568 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBdAFJliC8UHxAAp4RpLcBhknvkfcD6+u0cWg== 2023-11-13T10:54:56.569 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.4] 2023-11-13T10:54:56.579 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBxAFJlgXLmHxAAHpM0gY+C/Ionsp0sG9KoKQ== 2023-11-13T10:54:56.579 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.5] 2023-11-13T10:54:56.580 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQB0AFJlfsdPKxAA5Ba0gTAcUVQ6PueL9Ul6qQ== 2023-11-13T10:54:56.590 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.6] 2023-11-13T10:54:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQB4AFJlD6j5CBAAXuiAkwIbpfUSJ1j1j3QP4w== 2023-11-13T10:54:56.590 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.7] 2023-11-13T10:54:56.590 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQB7AFJlkX9TIhAAPjuDlChUGXjej7ot5x6vDg== 2023-11-13T10:54:56.590 INFO:teuthology.orchestra.run.smithi184.stdout:[client.0] 2023-11-13T10:54:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBdAFJl9p66LRAAYRThccLplmNpsr+W647dOA== 2023-11-13T10:54:56.592 INFO:teuthology.orchestra.run.smithi184.stdout:[client.2] 2023-11-13T10:54:56.592 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBdAFJlrk4zORAA7tCfwi0UlPhDcL1GZ6UIIg== 2023-11-13T10:54:56.592 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2023-11-13T10:54:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBeAFJlkymOCRAATnUAZgjg1er9vu82gnrxtw== 2023-11-13T10:54:56.593 INFO:teuthology.orchestra.run.smithi184.stdout:[client.3] 2023-11-13T10:54:56.593 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBeAFJlrh6DFBAAtrGSamisl/Jexs9ewdiAkQ== 2023-11-13T10:54:56.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-13T10:54:56.614 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-13T10:54:56.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-13T10:54:56.778 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-13T10:54:56.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-13T10:54:56.938 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-13T10:54:57.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:57.097 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:57.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:57.271 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:57.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:57.430 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:57.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:57.587 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:57.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-13T10:54:57.734 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-13T10:54:57.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-13T10:54:57.887 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-13T10:54:58.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-13T10:54:58.041 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-13T10:54:58.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:58.195 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:58.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:58.349 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:58.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:58.515 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:58.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:58.674 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-13T10:54:58.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-13T10:54:58.838 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-13T10:54:58.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-13T10:54:58.999 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-13T10:54:59.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-13T10:54:59.151 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-13T10:54:59.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-13T10:54:59.310 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-13T10:54:59.461 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-13T10:54:59.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-13T10:54:59.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-13T10:54:59.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-13T10:54:59.831 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-13T10:54:59.934 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-13T10:55:00.096 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-13T10:55:00.177 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-13T10:55:00.268 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-13T10:55:00.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-13T10:55:00.503 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:55:00.513 DEBUG:teuthology.orchestra.run.smithi184:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-13T10:55:00.567 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-13T10:55:00.568 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-13T10:55:00.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-13T10:55:00.576 INFO:tasks.ceph.mon.a:Started 2023-11-13T10:55:00.576 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-13T10:55:00.577 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-13T10:55:00.585 INFO:tasks.ceph.mon.b:Started 2023-11-13T10:55:00.585 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-13T10:55:00.585 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-13T10:55:00.616 INFO:tasks.ceph.mon.c:Started 2023-11-13T10:55:00.616 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-13T10:55:00.618 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-13T10:55:00.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-13T10:55:00.646 INFO:tasks.ceph.mgr.y:Started 2023-11-13T10:55:00.658 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-13T10:55:00.659 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-13T10:55:00.666 INFO:tasks.ceph.mgr.x:Started 2023-11-13T10:55:00.666 INFO:tasks.ceph:Setting crush tunables to default 2023-11-13T10:55:00.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2023-11-13T10:55:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.856+0000 7f7288982640 1 Processor -- start 2023-11-13T10:55:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.856+0000 7f7288982640 1 -- start start 2023-11-13T10:55:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f7288982640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 0x7f72840ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f7288982640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72840ace20 0x7f728406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f7288982640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72840ad240 con 0x7f72840715b0 2023-11-13T10:55:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f7288982640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728406c920 con 0x7f72840ace20 2023-11-13T10:55:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f7288982640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728406bca0 con 0x7f728406b6c0 2023-11-13T10:55:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f7282d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f72840715b0 0x7f72840b3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46294/0 (socket says 172.21.15.16:46294) 2023-11-13T10:55:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f7282d9d640 1 -- 172.21.15.16:0/3553837101 learned_addr learned my addr 172.21.15.16:0/3553837101 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f728359e640 1 --2- 172.21.15.16:0/3553837101 >> v2:172.21.15.16:3300/0 conn(0x7f72840ace20 0x7f728406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:00.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f728359e640 1 -- 172.21.15.16:0/3553837101 >> v2:172.21.15.16:3300/0 conn(0x7f72840ace20 msgr2=0x7f728406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-13T10:55:00.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f728359e640 1 -- 172.21.15.16:0/3553837101 >> v2:172.21.15.16:3300/0 conn(0x7f72840ace20 msgr2=0x7f728406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-13T10:55:00.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f728359e640 1 --2- 172.21.15.16:0/3553837101 >> v2:172.21.15.16:3300/0 conn(0x7f72840ace20 0x7f728406fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-13T10:55:00.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.858+0000 7f728359e640 1 --2- 172.21.15.16:0/3553837101 >> v2:172.21.15.16:3300/0 conn(0x7f72840ace20 0x7f728406fb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-13T10:55:00.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.859+0000 7f727bfff640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 0x7f72840ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:00.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.859+0000 7f727bfff640 1 -- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 msgr2=0x7f72840ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-13T10:55:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.859+0000 7f727bfff640 1 -- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 msgr2=0x7f72840ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-13T10:55:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.859+0000 7f727bfff640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 0x7f72840ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-13T10:55:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:00.859+0000 7f727bfff640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 0x7f72840ac8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-13T10:55:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.033+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126583618 0 0) 0x7f72840ad240 con 0x7f72840715b0 2023-11-13T10:55:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.033+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7270003520 con 0x7f72840715b0 2023-11-13T10:55:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3378381261 0 0) 0x7f7270003520 con 0x7f72840715b0 2023-11-13T10:55:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 msgr2=0x7f72840ac8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-13T10:55:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 0x7f72840ac8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 >> v2:172.21.15.16:3300/0 conn(0x7f72840ace20 msgr2=0x7f728406fb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-13T10:55:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 --2- 172.21.15.16:0/3553837101 >> v2:172.21.15.16:3300/0 conn(0x7f72840ace20 0x7f728406fb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728406bf80 con 0x7f72840715b0 2023-11-13T10:55:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f7274001670 con 0x7f72840715b0 2023-11-13T10:55:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 >> v1:172.21.15.16:6789/0 conn(0x7f72840715b0 legacy=0x7f72840b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 0x7f7270003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7270003fa0 0x7f72700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270004d00 0x7f7270005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728406c920 con 0x7f7270004d00 2023-11-13T10:55:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728406bca0 con 0x7f7270003fa0 2023-11-13T10:55:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.034+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727000bb40 con 0x7f728406b6c0 2023-11-13T10:55:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.035+0000 7f727bfff640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7270003fa0 0x7f72700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.035+0000 7f728359e640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270004d00 0x7f7270005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.036+0000 7f728359e640 1 -- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 msgr2=0x7f7270003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.036+0000 7f728359e640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f728406b6c0 0x7f7270003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.036+0000 7f728359e640 1 -- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7270003fa0 msgr2=0x7f72700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.036+0000 7f728359e640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7270003fa0 0x7f72700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.036+0000 7f728359e640 1 -- 172.21.15.16:0/3553837101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7270003520 con 0x7f7270004d00 2023-11-13T10:55:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.039+0000 7f728359e640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270004d00 0x7f7270005180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f727c009fd0 tx=0x7f72700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a754b609b7a63413 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.039+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c02ccb0 con 0x7f7270004d00 2023-11-13T10:55:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.039+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c02ce10 con 0x7f7270004d00 2023-11-13T10:55:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.039+0000 7f728259c640 1 -- 172.21.15.16:0/3553837101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c034680 con 0x7f7270004d00 2023-11-13T10:55:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.041+0000 7f7288982640 1 -- 172.21.15.16:0/3553837101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270004d00 msgr2=0x7f7270005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.041+0000 7f7288982640 1 --2- 172.21.15.16:0/3553837101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270004d00 0x7f7270005180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f727c009fd0 tx=0x7f72700057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.041+0000 7f728359e640 1 -- 172.21.15.16:0/3553837101 reap_dead start 2023-11-13T10:55:01.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.041+0000 7f7288982640 1 -- 172.21.15.16:0/3553837101 shutdown_connections 2023-11-13T10:55:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.041+0000 7f7288982640 1 -- 172.21.15.16:0/3553837101 >> 172.21.15.16:0/3553837101 conn(0x7f72840b0970 msgr2=0x7f72840b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.041+0000 7f7288982640 1 -- 172.21.15.16:0/3553837101 shutdown_connections 2023-11-13T10:55:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.041+0000 7f7288982640 1 -- 172.21.15.16:0/3553837101 wait complete. 2023-11-13T10:55:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.044+0000 7f7288982640 1 Processor -- start 2023-11-13T10:55:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.044+0000 7f7288982640 1 -- start start 2023-11-13T10:55:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.046+0000 7f7288982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72841f5720 0x7f728413eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.046+0000 7f7288982640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f728413f5e0 0x7f7284070120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.046+0000 7f7288982640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7284070660 0x7f728413cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.046+0000 7f7288982640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7284070af0 con 0x7f72841f5720 2023-11-13T10:55:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.046+0000 7f7288982640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72841f3220 con 0x7f728413f5e0 2023-11-13T10:55:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.046+0000 7f7288982640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72841f28f0 con 0x7f7284070660 2023-11-13T10:55:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.046+0000 7f727bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f728413f5e0 0x7f7284070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f728359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7284070660 0x7f728413cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f727bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f728413f5e0 0x7f7284070120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:35680/0 (socket says 172.21.15.16:35680) 2023-11-13T10:55:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f728359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7284070660 0x7f728413cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50470/0 (socket says 172.21.15.16:50470) 2023-11-13T10:55:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f727bfff640 1 -- 172.21.15.16:0/3141665236 learned_addr learned my addr 172.21.15.16:0/3141665236 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f7282d9d640 1 --2- 172.21.15.16:0/3141665236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72841f5720 0x7f728413eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f727bfff640 1 -- 172.21.15.16:0/3141665236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7284070660 msgr2=0x7f728413cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f727bfff640 1 --2- 172.21.15.16:0/3141665236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7284070660 0x7f728413cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f727bfff640 1 -- 172.21.15.16:0/3141665236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72841f5720 msgr2=0x7f728413eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f727bfff640 1 --2- 172.21.15.16:0/3141665236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72841f5720 0x7f728413eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.047+0000 7f727bfff640 1 -- 172.21.15.16:0/3141665236 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72841f2bd0 con 0x7f728413f5e0 2023-11-13T10:55:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.048+0000 7f727bfff640 1 --2- 172.21.15.16:0/3141665236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f728413f5e0 0x7f7284070120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f726c010900 tx=0x7f726c010dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.048+0000 7f727b7fe640 1 -- 172.21.15.16:0/3141665236 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c00b5b0 con 0x7f728413f5e0 2023-11-13T10:55:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.048+0000 7f727b7fe640 1 -- 172.21.15.16:0/3141665236 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c00ea80 con 0x7f728413f5e0 2023-11-13T10:55:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.048+0000 7f7288982640 1 -- 172.21.15.16:0/3141665236 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72841f2eb0 con 0x7f728413f5e0 2023-11-13T10:55:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.048+0000 7f727b7fe640 1 -- 172.21.15.16:0/3141665236 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c00ee10 con 0x7f728413f5e0 2023-11-13T10:55:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.049+0000 7f7288982640 1 -- 172.21.15.16:0/3141665236 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728413f030 con 0x7f728413f5e0 2023-11-13T10:55:01.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.049+0000 7f727b7fe640 1 -- 172.21.15.16:0/3141665236 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f726c016780 con 0x7f728413f5e0 2023-11-13T10:55:01.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.049+0000 7f727b7fe640 1 -- 172.21.15.16:0/3141665236 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f726c014950 con 0x7f728413f5e0 2023-11-13T10:55:01.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.050+0000 7f7288982640 1 -- 172.21.15.16:0/3141665236 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72841f33b0 con 0x7f728413f5e0 2023-11-13T10:55:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.053+0000 7f727b7fe640 1 -- 172.21.15.16:0/3141665236 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f726c03ece0 con 0x7f728413f5e0 2023-11-13T10:55:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.129+0000 7f7288982640 1 -- 172.21.15.16:0/3141665236 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f728404e4d0 con 0x7f728413f5e0 2023-11-13T10:55:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.137+0000 7f727b7fe640 1 -- 172.21.15.16:0/3141665236 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f726c02d020 con 0x7f728413f5e0 2023-11-13T10:55:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2023-11-13T10:55:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.138+0000 7f7288982640 1 -- 172.21.15.16:0/3141665236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f728413f5e0 msgr2=0x7f7284070120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:01.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.138+0000 7f7288982640 1 --2- 172.21.15.16:0/3141665236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f728413f5e0 0x7f7284070120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f726c010900 tx=0x7f726c010dc0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.139+0000 7f7288982640 1 -- 172.21.15.16:0/3141665236 shutdown_connections 2023-11-13T10:55:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.139+0000 7f7288982640 1 --2- 172.21.15.16:0/3141665236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7284070660 0x7f728413cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.139+0000 7f7288982640 1 --2- 172.21.15.16:0/3141665236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f728413f5e0 0x7f7284070120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.139+0000 7f7288982640 1 --2- 172.21.15.16:0/3141665236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72841f5720 0x7f728413eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.139+0000 7f7288982640 1 -- 172.21.15.16:0/3141665236 >> 172.21.15.16:0/3141665236 conn(0x7f72840b0970 msgr2=0x7f72840ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.139+0000 7f7288982640 1 -- 172.21.15.16:0/3141665236 shutdown_connections 2023-11-13T10:55:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:01.139+0000 7f7288982640 1 -- 172.21.15.16:0/3141665236 wait complete. 2023-11-13T10:55:01.193 INFO:tasks.ceph:check_enable_crimson: False 2023-11-13T10:55:01.194 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-13T10:55:01.195 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:55:01.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-13T10:55:01.310 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:55:01.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-13T10:55:01.459 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:55:01.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-13T10:55:01.548 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:55:01.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-13T10:55:01.603 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:01.590+0000 7fd75c632040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-13T10:55:01.654 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:01.649+0000 7fd75c632040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-13T10:55:01.655 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:01.647+0000 7fc01b616040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-13T10:55:01.691 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:55:01.692 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-13T10:55:01.719 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:01.715+0000 7fc01b616040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-13T10:55:01.783 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:55:01.783 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-13T10:55:01.910 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:55:01.910 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-13T10:55:02.045 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:55:02.045 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-13T10:55:02.178 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 24c47695-0622-4dd1-a2a3-8337282f0c0d 0 2023-11-13T10:55:02.337 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.333+0000 7fd75c632040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-13T10:55:02.355 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.352+0000 7fc01b616040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-13T10:55:02.371 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.366+0000 7fba91bff640 1 Processor -- start 2023-11-13T10:55:02.371 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.366+0000 7fba91bff640 1 -- start start 2023-11-13T10:55:02.373 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.367+0000 7fba91bff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba8c15ce70 0x7fba8c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.373 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.367+0000 7fba91bff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fba8c0cdbb0 0x7fba8c0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.373 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.367+0000 7fba91bff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8c0cac40 con 0x7fba8c0d1850 2023-11-13T10:55:02.382 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.367+0000 7fba91bff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8c0ce0e0 con 0x7fba8c15ce70 2023-11-13T10:55:02.386 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.367+0000 7fba91bff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8c0ce3c0 con 0x7fba8c0cdbb0 2023-11-13T10:55:02.386 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.368+0000 7fba8bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fba8c0cdbb0 0x7fba8c0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.389 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.368+0000 7fba8bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fba8c0cdbb0 0x7fba8c0d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41806/0 (socket says 172.21.15.184:41806) 2023-11-13T10:55:02.389 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.368+0000 7fba8bfff640 1 -- 172.21.15.184:0/3397662131 learned_addr learned my addr 172.21.15.184:0/3397662131 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:02.389 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.368+0000 7fba90bfd640 1 --2- 172.21.15.184:0/3397662131 >> v2:172.21.15.16:3300/0 conn(0x7fba8c15ce70 0x7fba8c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.389 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.368+0000 7fba8bfff640 1 -- 172.21.15.184:0/3397662131 >> v2:172.21.15.16:3300/0 conn(0x7fba8c15ce70 msgr2=0x7fba8c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:02.390 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.368+0000 7fba8bfff640 1 --2- 172.21.15.184:0/3397662131 >> v2:172.21.15.16:3300/0 conn(0x7fba8c15ce70 0x7fba8c0cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.390 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.368+0000 7fba8bfff640 1 -- 172.21.15.184:0/3397662131 >> v1:172.21.15.16:6789/0 conn(0x7fba8c0d1850 legacy=0x7fba8c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:02.390 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.368+0000 7fba8bfff640 1 -- 172.21.15.184:0/3397662131 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c0ce6a0 con 0x7fba8c0cdbb0 2023-11-13T10:55:02.391 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.369+0000 7fba8bfff640 1 --2- 172.21.15.184:0/3397662131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0cdbb0 0x7fba8c0d27c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0d0680 tx=0x7fba7802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6d07f0ac95b309 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:02.409 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.369+0000 7fba8b7fe640 1 -- 172.21.15.184:0/3397662131 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78031870 con 0x7fba8c0cdbb0 2023-11-13T10:55:02.410 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba8b7fe640 1 -- 172.21.15.184:0/3397662131 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba780319d0 con 0x7fba8c0cdbb0 2023-11-13T10:55:02.410 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba8b7fe640 1 -- 172.21.15.184:0/3397662131 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78031d60 con 0x7fba8c0cdbb0 2023-11-13T10:55:02.410 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba91bff640 1 -- 172.21.15.184:0/3397662131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0cdbb0 msgr2=0x7fba8c0d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.411 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba91bff640 1 --2- 172.21.15.184:0/3397662131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0cdbb0 0x7fba8c0d27c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0d0680 tx=0x7fba7802b910 comp rx=0 tx=0).stop 2023-11-13T10:55:02.411 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba91bff640 1 -- 172.21.15.184:0/3397662131 shutdown_connections 2023-11-13T10:55:02.411 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba91bff640 1 --2- 172.21.15.184:0/3397662131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0cdbb0 0x7fba8c0d27c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.411 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba91bff640 1 --2- 172.21.15.184:0/3397662131 >> v2:172.21.15.16:3300/0 conn(0x7fba8c15ce70 0x7fba8c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.411 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba91bff640 1 -- 172.21.15.184:0/3397662131 >> 172.21.15.184:0/3397662131 conn(0x7fba8c08c580 msgr2=0x7fba8c09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:02.412 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba91bff640 1 -- 172.21.15.184:0/3397662131 shutdown_connections 2023-11-13T10:55:02.412 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.370+0000 7fba91bff640 1 -- 172.21.15.184:0/3397662131 wait complete. 2023-11-13T10:55:02.412 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.372+0000 7fba91bff640 1 Processor -- start 2023-11-13T10:55:02.412 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.372+0000 7fba91bff640 1 -- start start 2023-11-13T10:55:02.413 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba91bff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba8c0cdbb0 0x7fba8c14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.413 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba91bff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0d1850 0x7fba8c0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.414 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba91bff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8c15ce70 0x7fba8c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.414 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba91bff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c14d570 con 0x7fba8c15ce70 2023-11-13T10:55:02.414 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba91bff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c1f6df0 con 0x7fba8c0d1850 2023-11-13T10:55:02.414 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba91bff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c1f61e0 con 0x7fba8c0cdbb0 2023-11-13T10:55:02.415 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba8bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0d1850 0x7fba8c0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.415 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba90bfd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba8c0cdbb0 0x7fba8c14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.415 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba8bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0d1850 0x7fba8c0cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41812/0 (socket says 172.21.15.184:41812) 2023-11-13T10:55:02.415 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.373+0000 7fba8bfff640 1 -- 172.21.15.184:0/293892428 learned_addr learned my addr 172.21.15.184:0/293892428 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:02.416 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba913fe640 1 --2- 172.21.15.184:0/293892428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8c15ce70 0x7fba8c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.416 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba8bfff640 1 -- 172.21.15.184:0/293892428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba8c0cdbb0 msgr2=0x7fba8c14d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.416 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba8bfff640 1 --2- 172.21.15.184:0/293892428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba8c0cdbb0 0x7fba8c14d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.416 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba8bfff640 1 -- 172.21.15.184:0/293892428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8c15ce70 msgr2=0x7fba8c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba8bfff640 1 --2- 172.21.15.184:0/293892428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8c15ce70 0x7fba8c0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba8bfff640 1 -- 172.21.15.184:0/293892428 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780094d0 con 0x7fba8c0d1850 2023-11-13T10:55:02.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba8bfff640 1 --2- 172.21.15.184:0/293892428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0d1850 0x7fba8c0cf6d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fba7802b950 tx=0x7fba780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:02.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba897fa640 1 -- 172.21.15.184:0/293892428 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78031860 con 0x7fba8c0d1850 2023-11-13T10:55:02.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba897fa640 1 -- 172.21.15.184:0/293892428 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba780443f0 con 0x7fba8c0d1850 2023-11-13T10:55:02.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.374+0000 7fba897fa640 1 -- 172.21.15.184:0/293892428 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780447a0 con 0x7fba8c0d1850 2023-11-13T10:55:02.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.375+0000 7fba91bff640 1 -- 172.21.15.184:0/293892428 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8c1f64c0 con 0x7fba8c0d1850 2023-11-13T10:55:02.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.375+0000 7fba91bff640 1 -- 172.21.15.184:0/293892428 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8c1f6a80 con 0x7fba8c0d1850 2023-11-13T10:55:02.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.375+0000 7fba897fa640 1 -- 172.21.15.184:0/293892428 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba78044c00 con 0x7fba8c0d1850 2023-11-13T10:55:02.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.375+0000 7fba897fa640 1 -- 172.21.15.184:0/293892428 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba7802d040 con 0x7fba8c0d1850 2023-11-13T10:55:02.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.376+0000 7fba91bff640 1 -- 172.21.15.184:0/293892428 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c0c9920 con 0x7fba8c0d1850 2023-11-13T10:55:02.457 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.378+0000 7fba897fa640 1 -- 172.21.15.184:0/293892428 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fba78035020 con 0x7fba8c0d1850 2023-11-13T10:55:02.460 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.397+0000 7fd75c632040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-13T10:55:02.468 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.450+0000 7fd75c632040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-13T10:55:02.469 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.456+0000 7fba91bff640 1 -- 172.21.15.184:0/293892428 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "24c47695-0622-4dd1-a2a3-8337282f0c0d", "id": 0} v 0) v1 -- 0x7fba8c1f6f60 con 0x7fba8c0d1850 2023-11-13T10:55:02.471 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.462+0000 7fba897fa640 1 -- 172.21.15.184:0/293892428 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "24c47695-0622-4dd1-a2a3-8337282f0c0d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fba7804d450 con 0x7fba8c0d1850 2023-11-13T10:55:02.472 INFO:teuthology.orchestra.run.smithi184.stdout:0 2023-11-13T10:55:02.473 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.463+0000 7fba91bff640 1 -- 172.21.15.184:0/293892428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0d1850 msgr2=0x7fba8c0cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.473 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.463+0000 7fba91bff640 1 --2- 172.21.15.184:0/293892428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0d1850 0x7fba8c0cf6d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fba7802b950 tx=0x7fba780046c0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.473 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.463+0000 7fba91bff640 1 -- 172.21.15.184:0/293892428 shutdown_connections 2023-11-13T10:55:02.473 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.463+0000 7fba91bff640 1 --2- 172.21.15.184:0/293892428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8c15ce70 0x7fba8c0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.473 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.463+0000 7fba91bff640 1 --2- 172.21.15.184:0/293892428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba8c0d1850 0x7fba8c0cf6d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.474 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.463+0000 7fba91bff640 1 --2- 172.21.15.184:0/293892428 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba8c0cdbb0 0x7fba8c14d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.474 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.463+0000 7fba91bff640 1 -- 172.21.15.184:0/293892428 >> 172.21.15.184:0/293892428 conn(0x7fba8c08c580 msgr2=0x7fba8c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:02.474 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.463+0000 7fba91bff640 1 -- 172.21.15.184:0/293892428 shutdown_connections 2023-11-13T10:55:02.474 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.464+0000 7fba91bff640 1 -- 172.21.15.184:0/293892428 wait complete. 2023-11-13T10:55:02.475 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.416+0000 7fc01b616040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-13T10:55:02.475 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.470+0000 7fc01b616040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-13T10:55:02.486 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 83b02569-c028-44b6-9662-38d2cf5099b6 1 2023-11-13T10:55:02.523 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.517+0000 7fd75c632040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-13T10:55:02.539 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.536+0000 7fc01b616040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-13T10:55:02.574 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.570+0000 7fd75c632040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-13T10:55:02.591 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.587+0000 7fc01b616040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-13T10:55:02.679 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.675+0000 7f44e3fff640 1 Processor -- start 2023-11-13T10:55:02.680 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.675+0000 7f44e3fff640 1 -- start start 2023-11-13T10:55:02.681 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.677+0000 7f44e3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f44e415ce70 0x7f44e40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.681 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.677+0000 7f44e3fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e40ce0e0 con 0x7f44e40d1850 2023-11-13T10:55:02.681 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.677+0000 7f44e3fff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e40ce3c0 con 0x7f44e40cdbb0 2023-11-13T10:55:02.682 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.677+0000 7f44e3fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e40ce6a0 con 0x7f44e415ce70 2023-11-13T10:55:02.682 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.677+0000 7f44dbfff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f44e40cdbb0 0x7f44e40d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:36452/0 (socket says 172.21.15.184:36452) 2023-11-13T10:55:02.682 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.677+0000 7f44dbfff640 1 -- 172.21.15.184:0/2488563196 learned_addr learned my addr 172.21.15.184:0/2488563196 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-11-13T10:55:02.686 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e2ffd640 1 --2- 172.21.15.184:0/2488563196 >> v2:172.21.15.16:3300/0 conn(0x7f44e415ce70 0x7f44e40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.686 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612394567 0 0) 0x7f44e40ce3c0 con 0x7f44e40cdbb0 2023-11-13T10:55:02.686 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d0003520 con 0x7f44e40cdbb0 2023-11-13T10:55:02.686 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4204732712 0 0) 0x7f44d0003520 con 0x7f44e40cdbb0 2023-11-13T10:55:02.687 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 >> v2:172.21.15.16:3300/0 conn(0x7f44e415ce70 msgr2=0x7f44e40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:02.687 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 --2- 172.21.15.184:0/2488563196 >> v2:172.21.15.16:3300/0 conn(0x7f44e415ce70 0x7f44e40cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.687 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 >> v1:172.21.15.16:6789/0 conn(0x7f44e40d1850 legacy=0x7f44e40c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.687 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40ce980 con 0x7f44e40cdbb0 2023-11-13T10:55:02.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f44cc001670 con 0x7f44e40cdbb0 2023-11-13T10:55:02.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 >> v1:172.21.15.184:6789/0 conn(0x7f44e40cdbb0 legacy=0x7f44e40d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0003970 0x7f44d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d00047c0 0x7f44d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d0005180 0x7f44d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.689 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40ce6a0 con 0x7f44d0005180 2023-11-13T10:55:02.689 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d000c020 con 0x7f44d0003970 2023-11-13T10:55:02.689 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d000c1e0 con 0x7f44d00047c0 2023-11-13T10:55:02.689 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.678+0000 7f44e2ffd640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0003970 0x7f44d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.690 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.679+0000 7f44dbfff640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d00047c0 0x7f44d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.690 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.679+0000 7f44e37fe640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d0005180 0x7f44d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.690 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.679+0000 7f44e2ffd640 1 -- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d00047c0 msgr2=0x7f44d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:02.691 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.679+0000 7f44e2ffd640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d00047c0 0x7f44d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.691 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.679+0000 7f44e2ffd640 1 -- 172.21.15.184:0/2488563196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d0005180 msgr2=0x7f44d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:02.691 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.679+0000 7f44e2ffd640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d0005180 0x7f44d000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.691 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.679+0000 7f44e2ffd640 1 -- 172.21.15.184:0/2488563196 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0003520 con 0x7f44d0003970 2023-11-13T10:55:02.692 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.679+0000 7f44e37fe640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d0005180 0x7f44d000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:02.692 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.679+0000 7f44e37fe640 1 -- 172.21.15.184:0/2488563196 reap_dead start 2023-11-13T10:55:02.692 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.680+0000 7f44e2ffd640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0003970 0x7f44d0003d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009ac0 tx=0x7f44d00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2ab66413ae185d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:02.692 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.680+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d402c6b0 con 0x7f44d0003970 2023-11-13T10:55:02.693 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.680+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d402c810 con 0x7f44d0003970 2023-11-13T10:55:02.694 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.680+0000 7f44e27fc640 1 -- 172.21.15.184:0/2488563196 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d402cba0 con 0x7f44d0003970 2023-11-13T10:55:02.695 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.681+0000 7f44e3fff640 1 -- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0003970 msgr2=0x7f44d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.695 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.681+0000 7f44e3fff640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0003970 0x7f44d0003d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009ac0 tx=0x7f44d00057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.695 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.681+0000 7f44e3fff640 1 -- 172.21.15.184:0/2488563196 shutdown_connections 2023-11-13T10:55:02.696 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.681+0000 7f44e3fff640 1 --2- 172.21.15.184:0/2488563196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0003970 0x7f44d0003d90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.696 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.681+0000 7f44e3fff640 1 -- 172.21.15.184:0/2488563196 >> 172.21.15.184:0/2488563196 conn(0x7f44e408c580 msgr2=0x7f44e409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:02.696 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.681+0000 7f44e3fff640 1 -- 172.21.15.184:0/2488563196 shutdown_connections 2023-11-13T10:55:02.696 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.681+0000 7f44e3fff640 1 -- 172.21.15.184:0/2488563196 wait complete. 2023-11-13T10:55:02.698 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.683+0000 7f44e3fff640 1 Processor -- start 2023-11-13T10:55:02.699 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.683+0000 7f44e3fff640 1 -- start start 2023-11-13T10:55:02.699 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.684+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0003970 0x7f44e40b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.699 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.684+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44e41554e0 0x7f44e40c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.699 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.684+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e415dd80 0x7f44e415e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:02.700 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.684+0000 7f44e3fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e415e770 con 0x7f44e415dd80 2023-11-13T10:55:02.700 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.684+0000 7f44e3fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40b7440 con 0x7f44e41554e0 2023-11-13T10:55:02.700 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.684+0000 7f44e3fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40b7720 con 0x7f44d0003970 2023-11-13T10:55:02.700 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.684+0000 7f44dbfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44e41554e0 0x7f44e40c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.700 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.684+0000 7f44e2ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0003970 0x7f44e40b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.701 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44e2ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0003970 0x7f44e40b6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:37932/0 (socket says 172.21.15.184:37932) 2023-11-13T10:55:02.701 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44dbfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44e41554e0 0x7f44e40c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41838/0 (socket says 172.21.15.184:41838) 2023-11-13T10:55:02.725 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44e2ffd640 1 -- 172.21.15.184:0/2115266695 learned_addr learned my addr 172.21.15.184:0/2115266695 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:02.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44e37fe640 1 --2- 172.21.15.184:0/2115266695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e415dd80 0x7f44e415e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:02.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44dbfff640 1 -- 172.21.15.184:0/2115266695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0003970 msgr2=0x7f44e40b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44dbfff640 1 --2- 172.21.15.184:0/2115266695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0003970 0x7f44e40b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44dbfff640 1 -- 172.21.15.184:0/2115266695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e415dd80 msgr2=0x7f44e415e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44dbfff640 1 --2- 172.21.15.184:0/2115266695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e415dd80 0x7f44e415e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44dbfff640 1 -- 172.21.15.184:0/2115266695 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40b7a00 con 0x7f44e41554e0 2023-11-13T10:55:02.730 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44dbfff640 1 --2- 172.21.15.184:0/2115266695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44e41554e0 0x7f44e40c8fd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f44e40ce010 tx=0x7f44cc007720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:02.730 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44e0ff9640 1 -- 172.21.15.184:0/2115266695 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc004ea0 con 0x7f44e41554e0 2023-11-13T10:55:02.730 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44e0ff9640 1 -- 172.21.15.184:0/2115266695 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc00f9c0 con 0x7f44e41554e0 2023-11-13T10:55:02.731 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.685+0000 7f44e0ff9640 1 -- 172.21.15.184:0/2115266695 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00fd50 con 0x7f44e41554e0 2023-11-13T10:55:02.731 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.686+0000 7f44e3fff640 1 -- 172.21.15.184:0/2115266695 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e40b7ce0 con 0x7f44e41554e0 2023-11-13T10:55:02.731 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.686+0000 7f44e3fff640 1 -- 172.21.15.184:0/2115266695 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e40b7130 con 0x7f44e41554e0 2023-11-13T10:55:02.731 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.686+0000 7f44e0ff9640 1 -- 172.21.15.184:0/2115266695 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f44cc01d760 con 0x7f44e41554e0 2023-11-13T10:55:02.731 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.686+0000 7f44e0ff9640 1 -- 172.21.15.184:0/2115266695 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f44cc013aa0 con 0x7f44e41554e0 2023-11-13T10:55:02.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.687+0000 7f44e3fff640 1 -- 172.21.15.184:0/2115266695 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e40675b0 con 0x7f44e41554e0 2023-11-13T10:55:02.742 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.689+0000 7f44e0ff9640 1 -- 172.21.15.184:0/2115266695 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f44cc013cb0 con 0x7f44e41554e0 2023-11-13T10:55:02.743 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.690+0000 7fd75c632040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-13T10:55:02.744 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.706+0000 7fc01b616040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-13T10:55:02.778 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.767+0000 7f44e3fff640 1 -- 172.21.15.184:0/2115266695 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "83b02569-c028-44b6-9662-38d2cf5099b6", "id": 1} v 0) v1 -- 0x7f44e40b8200 con 0x7f44e41554e0 2023-11-13T10:55:02.779 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.773+0000 7f44e0ff9640 1 -- 172.21.15.184:0/2115266695 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "83b02569-c028-44b6-9662-38d2cf5099b6", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f44e40b8200 con 0x7f44e41554e0 2023-11-13T10:55:02.779 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-11-13T10:55:02.780 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.774+0000 7f44e3fff640 1 -- 172.21.15.184:0/2115266695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44e41554e0 msgr2=0x7f44e40c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:02.780 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.774+0000 7f44e3fff640 1 --2- 172.21.15.184:0/2115266695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44e41554e0 0x7f44e40c8fd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f44e40ce010 tx=0x7f44cc007720 comp rx=0 tx=0).stop 2023-11-13T10:55:02.780 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.774+0000 7f44e3fff640 1 -- 172.21.15.184:0/2115266695 shutdown_connections 2023-11-13T10:55:02.781 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.774+0000 7f44e3fff640 1 --2- 172.21.15.184:0/2115266695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e415dd80 0x7f44e415e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.781 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.774+0000 7f44e3fff640 1 --2- 172.21.15.184:0/2115266695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44e41554e0 0x7f44e40c8fd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.774+0000 7f44e3fff640 1 --2- 172.21.15.184:0/2115266695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0003970 0x7f44e40b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:02.805 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.774+0000 7f44e3fff640 1 -- 172.21.15.184:0/2115266695 >> 172.21.15.184:0/2115266695 conn(0x7f44e408c580 msgr2=0x7f44e40cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:02.805 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.775+0000 7f44e3fff640 1 -- 172.21.15.184:0/2115266695 shutdown_connections 2023-11-13T10:55:02.805 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:02.775+0000 7f44e3fff640 1 -- 172.21.15.184:0/2115266695 wait complete. 2023-11-13T10:55:02.807 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.791+0000 7fc01b616040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-13T10:55:02.807 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.777+0000 7fd75c632040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-13T10:55:02.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 3fb0a6a5-4391-4978-a71d-a08a66c60637 2 2023-11-13T10:55:02.849 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.833+0000 7fd75c632040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-13T10:55:02.850 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.847+0000 7fc01b616040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-13T10:55:02.903 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.897+0000 7fd75c632040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-13T10:55:02.913 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.910+0000 7fc01b616040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-13T10:55:02.958 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:02.954+0000 7fd75c632040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-13T10:55:02.971 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:02.967+0000 7fc01b616040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-13T10:55:03.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.037+0000 7fd010900640 1 Processor -- start 2023-11-13T10:55:03.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.037+0000 7fd010900640 1 -- start start 2023-11-13T10:55:03.043 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.038+0000 7fd010900640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd00c15acc0 0x7fd00c0ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.043 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.039+0000 7fd010900640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c070960 con 0x7fd00c0cc220 2023-11-13T10:55:03.043 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.039+0000 7fd010900640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c0cc690 con 0x7fd00c06a8a0 2023-11-13T10:55:03.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.039+0000 7fd010900640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c15b0a0 con 0x7fd00c15acc0 2023-11-13T10:55:03.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.039+0000 7fd00b59e640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fd00c06a8a0 0x7fd00c164720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:36456/0 (socket says 172.21.15.184:36456) 2023-11-13T10:55:03.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.039+0000 7fd00b59e640 1 -- 172.21.15.184:0/3992445266 learned_addr learned my addr 172.21.15.184:0/3992445266 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-11-13T10:55:03.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.039+0000 7fd00ad9d640 1 --2- 172.21.15.184:0/3992445266 >> v2:172.21.15.16:3300/0 conn(0x7fd00c15acc0 0x7fd00c0ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.039+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076678130 0 0) 0x7fd00c0cc690 con 0x7fd00c06a8a0 2023-11-13T10:55:03.045 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.039+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcff8003520 con 0x7fd00c06a8a0 2023-11-13T10:55:03.049 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3042776881 0 0) 0x7fcff8003520 con 0x7fd00c06a8a0 2023-11-13T10:55:03.049 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 >> v2:172.21.15.16:3300/0 conn(0x7fd00c15acc0 msgr2=0x7fd00c0ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:03.049 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 --2- 172.21.15.184:0/3992445266 >> v2:172.21.15.16:3300/0 conn(0x7fd00c15acc0 0x7fd00c0ae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.049 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 >> v1:172.21.15.16:6789/0 conn(0x7fd00c0cc220 legacy=0x7fd00c06a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:03.050 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c070bb0 con 0x7fd00c06a8a0 2023-11-13T10:55:03.050 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fd0040019f0 con 0x7fd00c06a8a0 2023-11-13T10:55:03.050 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 >> v1:172.21.15.184:6789/0 conn(0x7fd00c06a8a0 legacy=0x7fd00c164720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.051 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff80039c0 0x7fcff8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.051 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcff80047c0 0x7fcff8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.051 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff8005180 0x7fcff800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.052 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c15b0a0 con 0x7fcff8005180 2023-11-13T10:55:03.052 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff800c030 con 0x7fcff80047c0 2023-11-13T10:55:03.052 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff800c1f0 con 0x7fcff80039c0 2023-11-13T10:55:03.053 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00ad9d640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff80039c0 0x7fcff8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.054 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd003fff640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcff80047c0 0x7fcff8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.054 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.040+0000 7fd00b59e640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff8005180 0x7fcff800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.054 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.041+0000 7fd00ad9d640 1 -- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcff80047c0 msgr2=0x7fcff8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:03.054 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.041+0000 7fd00ad9d640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcff80047c0 0x7fcff8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.055 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.041+0000 7fd00ad9d640 1 -- 172.21.15.184:0/3992445266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff8005180 msgr2=0x7fcff800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:03.055 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.041+0000 7fd00ad9d640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff8005180 0x7fcff800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.055 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.041+0000 7fd00ad9d640 1 -- 172.21.15.184:0/3992445266 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff8003520 con 0x7fcff80039c0 2023-11-13T10:55:03.055 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.041+0000 7fd00b59e640 1 -- 172.21.15.184:0/3992445266 reap_dead start 2023-11-13T10:55:03.055 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.042+0000 7fd00ad9d640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff80039c0 0x7fcff8003dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009ac0 tx=0x7fcff80057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9106681a33596b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:03.056 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.042+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc02c6b0 con 0x7fcff80039c0 2023-11-13T10:55:03.056 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.042+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc02c810 con 0x7fcff80039c0 2023-11-13T10:55:03.056 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.042+0000 7fd00a59c640 1 -- 172.21.15.184:0/3992445266 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc02cba0 con 0x7fcff80039c0 2023-11-13T10:55:03.058 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.042+0000 7fd010900640 1 -- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff80039c0 msgr2=0x7fcff8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.058 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.042+0000 7fd010900640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff80039c0 0x7fcff8003dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009ac0 tx=0x7fcff80057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.058 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.043+0000 7fd010900640 1 -- 172.21.15.184:0/3992445266 shutdown_connections 2023-11-13T10:55:03.058 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.043+0000 7fd010900640 1 --2- 172.21.15.184:0/3992445266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff80039c0 0x7fcff8003dc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.061 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.043+0000 7fd010900640 1 -- 172.21.15.184:0/3992445266 >> 172.21.15.184:0/3992445266 conn(0x7fd00c0b1420 msgr2=0x7fd00c0b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:03.062 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.043+0000 7fd010900640 1 -- 172.21.15.184:0/3992445266 shutdown_connections 2023-11-13T10:55:03.062 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.043+0000 7fd010900640 1 -- 172.21.15.184:0/3992445266 wait complete. 2023-11-13T10:55:03.071 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.045+0000 7fd010900640 1 Processor -- start 2023-11-13T10:55:03.072 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.045+0000 7fd010900640 1 -- start start 2023-11-13T10:55:03.084 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd010900640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff80039c0 0x7fd00c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.085 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd010900640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd00c153020 0x7fd00c1534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.085 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd010900640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c158840 0x7fd00c158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.085 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd010900640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c156b30 con 0x7fcff80039c0 2023-11-13T10:55:03.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd010900640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1539e0 con 0x7fd00c158840 2023-11-13T10:55:03.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd010900640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c159200 con 0x7fd00c153020 2023-11-13T10:55:03.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00b59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c158840 0x7fd00c158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.090 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00b59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c158840 0x7fd00c158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41858/0 (socket says 172.21.15.184:41858) 2023-11-13T10:55:03.090 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00b59e640 1 -- 172.21.15.184:0/3205991755 learned_addr learned my addr 172.21.15.184:0/3205991755 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:03.090 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd003fff640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd00c153020 0x7fd00c1534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00ad9d640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff80039c0 0x7fd00c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.092 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00b59e640 1 -- 172.21.15.184:0/3205991755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd00c153020 msgr2=0x7fd00c1534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.096 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00b59e640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd00c153020 0x7fd00c1534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.096 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00b59e640 1 -- 172.21.15.184:0/3205991755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff80039c0 msgr2=0x7fd00c1502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.096 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00b59e640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff80039c0 0x7fd00c1502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.096 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00b59e640 1 -- 172.21.15.184:0/3205991755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c157640 con 0x7fd00c158840 2023-11-13T10:55:03.096 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.046+0000 7fd00ad9d640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff80039c0 0x7fd00c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:03.097 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.047+0000 7fd00b59e640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c158840 0x7fd00c158cc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd00c06e3c0 tx=0x7fd00400e530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:03.097 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.047+0000 7fd0037fe640 1 -- 172.21.15.184:0/3205991755 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00400ec40 con 0x7fd00c158840 2023-11-13T10:55:03.097 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.047+0000 7fd0037fe640 1 -- 172.21.15.184:0/3205991755 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00400eda0 con 0x7fd00c158840 2023-11-13T10:55:03.097 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.047+0000 7fd0037fe640 1 -- 172.21.15.184:0/3205991755 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004014650 con 0x7fd00c158840 2023-11-13T10:55:03.097 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.047+0000 7fd010900640 1 -- 172.21.15.184:0/3205991755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c159420 con 0x7fd00c158840 2023-11-13T10:55:03.098 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.047+0000 7fd010900640 1 -- 172.21.15.184:0/3205991755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c156d10 con 0x7fd00c158840 2023-11-13T10:55:03.098 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.048+0000 7fd0037fe640 1 -- 172.21.15.184:0/3205991755 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd004014ab0 con 0x7fd00c158840 2023-11-13T10:55:03.098 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.048+0000 7fd0037fe640 1 -- 172.21.15.184:0/3205991755 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fd00401de00 con 0x7fd00c158840 2023-11-13T10:55:03.098 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.048+0000 7fd010900640 1 -- 172.21.15.184:0/3205991755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c04e4d0 con 0x7fd00c158840 2023-11-13T10:55:03.098 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.051+0000 7fd0037fe640 1 -- 172.21.15.184:0/3205991755 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd004022060 con 0x7fd00c158840 2023-11-13T10:55:03.133 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.129+0000 7fd010900640 1 -- 172.21.15.184:0/3205991755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3fb0a6a5-4391-4978-a71d-a08a66c60637", "id": 2} v 0) v1 -- 0x7fd00c06ef70 con 0x7fd00c158840 2023-11-13T10:55:03.141 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.135+0000 7fd0037fe640 1 -- 172.21.15.184:0/3205991755 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3fb0a6a5-4391-4978-a71d-a08a66c60637", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd00401d3f0 con 0x7fd00c158840 2023-11-13T10:55:03.142 INFO:teuthology.orchestra.run.smithi184.stdout:2 2023-11-13T10:55:03.143 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.136+0000 7fd010900640 1 -- 172.21.15.184:0/3205991755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c158840 msgr2=0x7fd00c158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.143 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.136+0000 7fd010900640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c158840 0x7fd00c158cc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd00c06e3c0 tx=0x7fd00400e530 comp rx=0 tx=0).stop 2023-11-13T10:55:03.143 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.137+0000 7fd010900640 1 -- 172.21.15.184:0/3205991755 shutdown_connections 2023-11-13T10:55:03.144 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.137+0000 7fd010900640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c158840 0x7fd00c158cc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.144 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.137+0000 7fd010900640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd00c153020 0x7fd00c1534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.144 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.137+0000 7fd010900640 1 --2- 172.21.15.184:0/3205991755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff80039c0 0x7fd00c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.144 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.137+0000 7fd010900640 1 -- 172.21.15.184:0/3205991755 >> 172.21.15.184:0/3205991755 conn(0x7fd00c0b1420 msgr2=0x7fd00c0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:03.144 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.137+0000 7fd010900640 1 -- 172.21.15.184:0/3205991755 shutdown_connections 2023-11-13T10:55:03.145 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.137+0000 7fd010900640 1 -- 172.21.15.184:0/3205991755 wait complete. 2023-11-13T10:55:03.166 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd 3 2023-11-13T10:55:03.364 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.360+0000 7f7863609640 1 Processor -- start 2023-11-13T10:55:03.364 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.360+0000 7f7863609640 1 -- start start 2023-11-13T10:55:03.366 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.361+0000 7f7863609640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f785c1604b0 0x7f785c0ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.366 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.361+0000 7f7863609640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.366 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.362+0000 7f7863609640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f785c0cd910 con 0x7f785c0d0b00 2023-11-13T10:55:03.367 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.362+0000 7f7863609640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785c0cdb40 con 0x7f785c1604b0 2023-11-13T10:55:03.367 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.362+0000 7f7863609640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0cde20 con 0x7f785c160890 2023-11-13T10:55:03.367 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.362+0000 7f7861ba6640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f785c0d0b00 0x7f785c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:36464/0 (socket says 172.21.15.184:36464) 2023-11-13T10:55:03.367 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.362+0000 7f7861ba6640 1 -- 172.21.15.184:0/3685059683 learned_addr learned my addr 172.21.15.184:0/3685059683 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-11-13T10:55:03.367 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.362+0000 7f7860ba4640 1 --2- 172.21.15.184:0/3685059683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.368 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.362+0000 7f78613a5640 1 --2- 172.21.15.184:0/3685059683 >> v2:172.21.15.16:3300/0 conn(0x7f785c1604b0 0x7f785c0ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.369 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.362+0000 7f7853fff640 1 -- 172.21.15.184:0/3685059683 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272266158 0 0) 0x7f785c0cd910 con 0x7f785c0d0b00 2023-11-13T10:55:03.369 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.363+0000 7f7853fff640 1 -- 172.21.15.184:0/3685059683 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7848003520 con 0x7f785c0d0b00 2023-11-13T10:55:03.369 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.363+0000 7f7860ba4640 1 -- 172.21.15.184:0/3685059683 >> v2:172.21.15.16:3300/0 conn(0x7f785c1604b0 msgr2=0x7f785c0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:03.369 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.363+0000 7f7860ba4640 1 --2- 172.21.15.184:0/3685059683 >> v2:172.21.15.16:3300/0 conn(0x7f785c1604b0 0x7f785c0ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.370 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.363+0000 7f7860ba4640 1 -- 172.21.15.184:0/3685059683 >> v1:172.21.15.184:6789/0 conn(0x7f785c0d0b00 legacy=0x7f785c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.371 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.363+0000 7f7860ba4640 1 -- 172.21.15.184:0/3685059683 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c0ce100 con 0x7f785c160890 2023-11-13T10:55:03.371 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.364+0000 7f7860ba4640 1 --2- 172.21.15.184:0/3685059683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0d1300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f784c0097a0 tx=0x7f784c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=180c65a74b9a733d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:03.371 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.365+0000 7f7853fff640 1 -- 172.21.15.184:0/3685059683 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c030800 con 0x7f785c160890 2023-11-13T10:55:03.371 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.365+0000 7f7853fff640 1 -- 172.21.15.184:0/3685059683 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c030960 con 0x7f785c160890 2023-11-13T10:55:03.371 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.365+0000 7f7853fff640 1 -- 172.21.15.184:0/3685059683 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c030cf0 con 0x7f785c160890 2023-11-13T10:55:03.374 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.366+0000 7f7863609640 1 -- 172.21.15.184:0/3685059683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 msgr2=0x7f785c0d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.375 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.366+0000 7f7863609640 1 --2- 172.21.15.184:0/3685059683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0d1300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f784c0097a0 tx=0x7f784c02b940 comp rx=0 tx=0).stop 2023-11-13T10:55:03.375 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.366+0000 7f7863609640 1 -- 172.21.15.184:0/3685059683 shutdown_connections 2023-11-13T10:55:03.375 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.366+0000 7f7863609640 1 --2- 172.21.15.184:0/3685059683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0d1300 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.377 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.366+0000 7f7863609640 1 --2- 172.21.15.184:0/3685059683 >> v2:172.21.15.16:3300/0 conn(0x7f785c1604b0 0x7f785c0ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.377 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.366+0000 7f7863609640 1 -- 172.21.15.184:0/3685059683 >> 172.21.15.184:0/3685059683 conn(0x7f785c0d2d20 msgr2=0x7f785c0d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:03.377 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.366+0000 7f7863609640 1 -- 172.21.15.184:0/3685059683 shutdown_connections 2023-11-13T10:55:03.378 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.366+0000 7f7863609640 1 -- 172.21.15.184:0/3685059683 wait complete. 2023-11-13T10:55:03.378 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.368+0000 7f7863609640 1 Processor -- start 2023-11-13T10:55:03.391 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.368+0000 7f7863609640 1 -- start start 2023-11-13T10:55:03.391 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f7863609640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f785c0d0b00 0x7f785c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.392 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f7863609640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f785c1604b0 0x7f785c148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.392 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f7863609640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.392 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f7863609640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c15d030 con 0x7f785c1604b0 2023-11-13T10:55:03.392 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f7863609640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0b7340 con 0x7f785c0d0b00 2023-11-13T10:55:03.393 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f7863609640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785c149310 con 0x7f785c160890 2023-11-13T10:55:03.393 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f78613a5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f785c0d0b00 0x7f785c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.393 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f7861ba6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.394 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f78613a5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f785c0d0b00 0x7f785c0c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41868/0 (socket says 172.21.15.184:41868) 2023-11-13T10:55:03.394 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f78613a5640 1 -- 172.21.15.184:0/323055498 learned_addr learned my addr 172.21.15.184:0/323055498 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:03.394 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f7861ba6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:37956/0 (socket says 172.21.15.184:37956) 2023-11-13T10:55:03.394 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f7860ba4640 1 --2- 172.21.15.184:0/323055498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f785c1604b0 0x7f785c148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.394 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f78613a5640 1 -- 172.21.15.184:0/323055498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 msgr2=0x7f785c0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.395 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.369+0000 7f78613a5640 1 --2- 172.21.15.184:0/323055498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.395 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f78613a5640 1 -- 172.21.15.184:0/323055498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f785c1604b0 msgr2=0x7f785c148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.395 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f78613a5640 1 --2- 172.21.15.184:0/323055498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f785c1604b0 0x7f785c148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.395 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f78613a5640 1 -- 172.21.15.184:0/323055498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c009400 con 0x7f785c0d0b00 2023-11-13T10:55:03.395 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f78613a5640 1 --2- 172.21.15.184:0/323055498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f785c0d0b00 0x7f785c0c9df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f785800e9b0 tx=0x7f785800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:03.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f7851ffb640 1 -- 172.21.15.184:0/323055498 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800c9a0 con 0x7f785c0d0b00 2023-11-13T10:55:03.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f7851ffb640 1 -- 172.21.15.184:0/323055498 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785800cb00 con 0x7f785c0d0b00 2023-11-13T10:55:03.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f7863609640 1 -- 172.21.15.184:0/323055498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c0b6730 con 0x7f785c0d0b00 2023-11-13T10:55:03.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f7851ffb640 1 -- 172.21.15.184:0/323055498 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858013470 con 0x7f785c0d0b00 2023-11-13T10:55:03.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f7863609640 1 -- 172.21.15.184:0/323055498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c0b6cf0 con 0x7f785c0d0b00 2023-11-13T10:55:03.397 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.370+0000 7f7851ffb640 1 -- 172.21.15.184:0/323055498 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7858005c90 con 0x7f785c0d0b00 2023-11-13T10:55:03.397 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.371+0000 7f7851ffb640 1 -- 172.21.15.184:0/323055498 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f7858006840 con 0x7f785c0d0b00 2023-11-13T10:55:03.399 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.371+0000 7f7863609640 1 -- 172.21.15.184:0/323055498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c0a0ca0 con 0x7f785c0d0b00 2023-11-13T10:55:03.399 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.374+0000 7f7851ffb640 1 -- 172.21.15.184:0/323055498 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f7858021060 con 0x7f785c0d0b00 2023-11-13T10:55:03.451 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:03.440+0000 7fd75c632040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-13T10:55:03.451 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:03.444+0000 7fc01b616040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-13T10:55:03.457 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.453+0000 7f7863609640 1 -- 172.21.15.184:0/323055498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd", "id": 3} v 0) v1 -- 0x7f785c0d0610 con 0x7f785c0d0b00 2023-11-13T10:55:03.464 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.460+0000 7f7851ffb640 1 -- 172.21.15.184:0/323055498 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7858029020 con 0x7f785c0d0b00 2023-11-13T10:55:03.465 INFO:teuthology.orchestra.run.smithi184.stdout:3 2023-11-13T10:55:03.466 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.461+0000 7f7863609640 1 -- 172.21.15.184:0/323055498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f785c0d0b00 msgr2=0x7f785c0c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.466 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.461+0000 7f7863609640 1 --2- 172.21.15.184:0/323055498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f785c0d0b00 0x7f785c0c9df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f785800e9b0 tx=0x7f785800ee70 comp rx=0 tx=0).stop 2023-11-13T10:55:03.466 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.461+0000 7f7863609640 1 -- 172.21.15.184:0/323055498 shutdown_connections 2023-11-13T10:55:03.466 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.461+0000 7f7863609640 1 --2- 172.21.15.184:0/323055498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f785c160890 0x7f785c0cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.467 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.461+0000 7f7863609640 1 --2- 172.21.15.184:0/323055498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f785c1604b0 0x7f785c148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.467 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.461+0000 7f7863609640 1 --2- 172.21.15.184:0/323055498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f785c0d0b00 0x7f785c0c9df0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.467 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.462+0000 7f7863609640 1 -- 172.21.15.184:0/323055498 >> 172.21.15.184:0/323055498 conn(0x7f785c0d2d20 msgr2=0x7f785c15f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:03.467 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.462+0000 7f7863609640 1 -- 172.21.15.184:0/323055498 shutdown_connections 2023-11-13T10:55:03.467 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.462+0000 7f7863609640 1 -- 172.21.15.184:0/323055498 wait complete. 2023-11-13T10:55:03.487 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new c52a2136-acd5-4f9c-90da-d67149500b73 4 2023-11-13T10:55:03.615 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:03.611+0000 7fd75c632040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-13T10:55:03.617 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:03.614+0000 7fc01b616040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-13T10:55:03.687 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.680+0000 7f493f3dc640 1 Processor -- start 2023-11-13T10:55:03.687 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.680+0000 7f493f3dc640 1 -- start start 2023-11-13T10:55:03.687 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.682+0000 7f493f3dc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f49380d1a10 0x7f49380ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.682+0000 7f493f3dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.682+0000 7f493f3dc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49380cd910 con 0x7f49381604b0 2023-11-13T10:55:03.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.682+0000 7f493f3dc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49380cdb40 con 0x7f49380d1a10 2023-11-13T10:55:03.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.682+0000 7f493f3dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49380cde20 con 0x7f4938160890 2023-11-13T10:55:03.688 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.682+0000 7f493d979640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.689 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.682+0000 7f493c977640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f49380d1a10 0x7f49380ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.689 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.682+0000 7f493d979640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:37964/0 (socket says 172.21.15.184:37964) 2023-11-13T10:55:03.689 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.682+0000 7f493d979640 1 -- 172.21.15.184:0/2621394020 learned_addr learned my addr 172.21.15.184:0/2621394020 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:03.689 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.683+0000 7f493c977640 1 -- 172.21.15.184:0/2621394020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 msgr2=0x7f49380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:03.689 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.683+0000 7f493c977640 1 --2- 172.21.15.184:0/2621394020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.690 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.683+0000 7f493c977640 1 -- 172.21.15.184:0/2621394020 >> v1:172.21.15.184:6789/0 conn(0x7f49381604b0 legacy=0x7f49380d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.690 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.683+0000 7f493c977640 1 -- 172.21.15.184:0/2621394020 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49380ce100 con 0x7f49380d1a10 2023-11-13T10:55:03.700 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.683+0000 7f492ffff640 1 -- 172.21.15.184:0/2621394020 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356315126 0 0) 0x7f49380cd910 con 0x7f49381604b0 2023-11-13T10:55:03.711 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.684+0000 7f493c977640 1 --2- 172.21.15.184:0/2621394020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 0x7f49380ce900 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49280097a0 tx=0x7f492802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64785a93212516cb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:03.714 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.684+0000 7f492ffff640 1 -- 172.21.15.184:0/2621394020 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928030800 con 0x7f49380d1a10 2023-11-13T10:55:03.715 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.684+0000 7f492ffff640 1 -- 172.21.15.184:0/2621394020 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928030960 con 0x7f49380d1a10 2023-11-13T10:55:03.720 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.684+0000 7f492ffff640 1 -- 172.21.15.184:0/2621394020 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928030cf0 con 0x7f49380d1a10 2023-11-13T10:55:03.720 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.685+0000 7f493f3dc640 1 -- 172.21.15.184:0/2621394020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 msgr2=0x7f49380ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.774 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.685+0000 7f493f3dc640 1 --2- 172.21.15.184:0/2621394020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 0x7f49380ce900 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49280097a0 tx=0x7f492802b940 comp rx=0 tx=0).stop 2023-11-13T10:55:03.782 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.685+0000 7f493f3dc640 1 -- 172.21.15.184:0/2621394020 shutdown_connections 2023-11-13T10:55:03.783 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.685+0000 7f493f3dc640 1 --2- 172.21.15.184:0/2621394020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.783 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.685+0000 7f493f3dc640 1 --2- 172.21.15.184:0/2621394020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 0x7f49380ce900 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.783 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.685+0000 7f493f3dc640 1 -- 172.21.15.184:0/2621394020 >> 172.21.15.184:0/2621394020 conn(0x7f49380d2d20 msgr2=0x7f49380d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:03.785 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.685+0000 7f493f3dc640 1 -- 172.21.15.184:0/2621394020 shutdown_connections 2023-11-13T10:55:03.785 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.685+0000 7f493f3dc640 1 -- 172.21.15.184:0/2621394020 wait complete. 2023-11-13T10:55:03.785 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.686+0000 7f493f3dc640 1 Processor -- start 2023-11-13T10:55:03.786 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.687+0000 7f493f3dc640 1 -- start start 2023-11-13T10:55:03.801 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.687+0000 7f493f3dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 0x7f49380ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.802 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.687+0000 7f493f3dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49381604b0 0x7f49380b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.802 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.687+0000 7f493f3dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:03.802 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493f3dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49380b4960 con 0x7f49381604b0 2023-11-13T10:55:03.802 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493f3dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493814c630 con 0x7f49380d1a10 2023-11-13T10:55:03.802 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493f3dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49380b3d50 con 0x7f4938160890 2023-11-13T10:55:03.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d178640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 0x7f49380ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d979640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d178640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 0x7f49380ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41896/0 (socket says 172.21.15.184:41896) 2023-11-13T10:55:03.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d178640 1 -- 172.21.15.184:0/1654796860 learned_addr learned my addr 172.21.15.184:0/1654796860 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:03.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493c977640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49381604b0 0x7f49380b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:03.818 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d178640 1 -- 172.21.15.184:0/1654796860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 msgr2=0x7f49380b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.820 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d178640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.820 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d178640 1 -- 172.21.15.184:0/1654796860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49381604b0 msgr2=0x7f49380b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.821 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d178640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49381604b0 0x7f49380b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.821 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d178640 1 -- 172.21.15.184:0/1654796860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928009400 con 0x7f49380d1a10 2023-11-13T10:55:03.821 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493d979640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:55:03.821 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.688+0000 7f493c977640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49381604b0 0x7f49380b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:03.821 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.689+0000 7f493d178640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 0x7f49380ccbc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f493400c4b0 tx=0x7f493400c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:03.822 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.689+0000 7f492dffb640 1 -- 172.21.15.184:0/1654796860 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934005480 con 0x7f49380d1a10 2023-11-13T10:55:03.822 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.689+0000 7f492dffb640 1 -- 172.21.15.184:0/1654796860 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49340035f0 con 0x7f49380d1a10 2023-11-13T10:55:03.822 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.689+0000 7f492dffb640 1 -- 172.21.15.184:0/1654796860 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340039e0 con 0x7f49380d1a10 2023-11-13T10:55:03.822 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.689+0000 7f493f3dc640 1 -- 172.21.15.184:0/1654796860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49380b4030 con 0x7f49380d1a10 2023-11-13T10:55:03.822 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.689+0000 7f493f3dc640 1 -- 172.21.15.184:0/1654796860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49380b45f0 con 0x7f49380d1a10 2023-11-13T10:55:03.823 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.689+0000 7f492dffb640 1 -- 172.21.15.184:0/1654796860 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f493401c440 con 0x7f49380d1a10 2023-11-13T10:55:03.823 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.689+0000 7f492dffb640 1 -- 172.21.15.184:0/1654796860 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f4934012910 con 0x7f49380d1a10 2023-11-13T10:55:03.823 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.690+0000 7f493f3dc640 1 -- 172.21.15.184:0/1654796860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49380a0ca0 con 0x7f49380d1a10 2023-11-13T10:55:03.823 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.692+0000 7f492dffb640 1 -- 172.21.15.184:0/1654796860 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4934017070 con 0x7f49380d1a10 2023-11-13T10:55:03.840 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:03.756+0000 7fd75c632040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-13T10:55:03.840 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:03.760+0000 7fc01b616040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-13T10:55:03.841 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:03.817+0000 7fc01b616040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-13T10:55:03.841 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.770+0000 7f493f3dc640 1 -- 172.21.15.184:0/1654796860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c52a2136-acd5-4f9c-90da-d67149500b73", "id": 4} v 0) v1 -- 0x7f49380d0610 con 0x7f49380d1a10 2023-11-13T10:55:03.841 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.776+0000 7f492dffb640 1 -- 172.21.15.184:0/1654796860 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c52a2136-acd5-4f9c-90da-d67149500b73", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4934012c30 con 0x7f49380d1a10 2023-11-13T10:55:03.842 INFO:teuthology.orchestra.run.smithi184.stdout:4 2023-11-13T10:55:03.844 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.777+0000 7f493f3dc640 1 -- 172.21.15.184:0/1654796860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 msgr2=0x7f49380ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:03.844 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.777+0000 7f493f3dc640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 0x7f49380ccbc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f493400c4b0 tx=0x7f493400c970 comp rx=0 tx=0).stop 2023-11-13T10:55:03.844 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.777+0000 7f493f3dc640 1 -- 172.21.15.184:0/1654796860 shutdown_connections 2023-11-13T10:55:03.845 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.777+0000 7f493f3dc640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4938160890 0x7f49380b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.845 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.777+0000 7f493f3dc640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49381604b0 0x7f49380b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.845 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.777+0000 7f493f3dc640 1 --2- 172.21.15.184:0/1654796860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49380d1a10 0x7f49380ccbc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:03.845 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.777+0000 7f493f3dc640 1 -- 172.21.15.184:0/1654796860 >> 172.21.15.184:0/1654796860 conn(0x7f49380d2d20 msgr2=0x7f493815f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:03.845 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.777+0000 7f493f3dc640 1 -- 172.21.15.184:0/1654796860 shutdown_connections 2023-11-13T10:55:03.846 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:03.777+0000 7f493f3dc640 1 -- 172.21.15.184:0/1654796860 wait complete. 2023-11-13T10:55:03.847 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new a9764441-fde9-439e-8bbd-76f0f53f31d8 5 2023-11-13T10:55:03.847 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:03.813+0000 7fd75c632040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-13T10:55:03.933 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:03.929+0000 7fd75c632040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-13T10:55:03.935 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:03.931+0000 7fc01b616040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-13T10:55:04.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.036+0000 7f62da66d640 1 Processor -- start 2023-11-13T10:55:04.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.037+0000 7f62da66d640 1 -- start start 2023-11-13T10:55:04.043 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62da66d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f62d415ce70 0x7f62d40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.043 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62da66d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d40d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.043 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62da66d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62d40d1850 0x7f62d40d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62da66d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d40d3240 con 0x7f62d40d1850 2023-11-13T10:55:04.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62da66d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d40ce0e0 con 0x7f62d415ce70 2023-11-13T10:55:04.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62da66d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d40ce3c0 con 0x7f62d40cdbb0 2023-11-13T10:55:04.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62d8e6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d40d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62d966b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f62d415ce70 0x7f62d40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.045 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62d8e6a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d40d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:37984/0 (socket says 172.21.15.184:37984) 2023-11-13T10:55:04.045 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.038+0000 7f62d8e6a640 1 -- 172.21.15.184:0/2010455552 learned_addr learned my addr 172.21.15.184:0/2010455552 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:04.045 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.039+0000 7f62d9e6c640 1 --2- 172.21.15.184:0/2010455552 >> v2:172.21.15.16:3300/0 conn(0x7f62d40d1850 0x7f62d40d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.045 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.039+0000 7f62d8e6a640 1 -- 172.21.15.184:0/2010455552 >> v2:172.21.15.184:3300/0 conn(0x7f62d415ce70 msgr2=0x7f62d40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:04.045 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.039+0000 7f62d8e6a640 1 --2- 172.21.15.184:0/2010455552 >> v2:172.21.15.184:3300/0 conn(0x7f62d415ce70 0x7f62d40cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.046 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.039+0000 7f62d8e6a640 1 -- 172.21.15.184:0/2010455552 >> v2:172.21.15.16:3300/0 conn(0x7f62d40d1850 msgr2=0x7f62d40d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:04.046 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.039+0000 7f62d8e6a640 1 --2- 172.21.15.184:0/2010455552 >> v2:172.21.15.16:3300/0 conn(0x7f62d40d1850 0x7f62d40d2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.047 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.039+0000 7f62d8e6a640 1 -- 172.21.15.184:0/2010455552 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d40ce6a0 con 0x7f62d40cdbb0 2023-11-13T10:55:04.056 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:04.048+0000 7fc01b616040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-13T10:55:04.057 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.040+0000 7f62d8e6a640 1 --2- 172.21.15.184:0/2010455552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d40d27c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009870 tx=0x7f62c402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d510af98a0b1369f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:04.057 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.040+0000 7f62cbfff640 1 -- 172.21.15.184:0/2010455552 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4031640 con 0x7f62d40cdbb0 2023-11-13T10:55:04.058 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.040+0000 7f62cbfff640 1 -- 172.21.15.184:0/2010455552 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c40317a0 con 0x7f62d40cdbb0 2023-11-13T10:55:04.058 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.040+0000 7f62cbfff640 1 -- 172.21.15.184:0/2010455552 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4031b30 con 0x7f62d40cdbb0 2023-11-13T10:55:04.058 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.041+0000 7f62da66d640 1 -- 172.21.15.184:0/2010455552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 msgr2=0x7f62d40d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.058 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.041+0000 7f62da66d640 1 --2- 172.21.15.184:0/2010455552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d40d27c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009870 tx=0x7f62c402b8f0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.058 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.041+0000 7f62da66d640 1 -- 172.21.15.184:0/2010455552 shutdown_connections 2023-11-13T10:55:04.059 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.041+0000 7f62da66d640 1 --2- 172.21.15.184:0/2010455552 >> v2:172.21.15.16:3300/0 conn(0x7f62d40d1850 0x7f62d40d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.059 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.041+0000 7f62da66d640 1 --2- 172.21.15.184:0/2010455552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d40d27c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.059 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.041+0000 7f62da66d640 1 --2- 172.21.15.184:0/2010455552 >> v2:172.21.15.184:3300/0 conn(0x7f62d415ce70 0x7f62d40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.059 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.041+0000 7f62da66d640 1 -- 172.21.15.184:0/2010455552 >> 172.21.15.184:0/2010455552 conn(0x7f62d408c580 msgr2=0x7f62d409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:04.059 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.041+0000 7f62da66d640 1 -- 172.21.15.184:0/2010455552 shutdown_connections 2023-11-13T10:55:04.060 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.041+0000 7f62da66d640 1 -- 172.21.15.184:0/2010455552 wait complete. 2023-11-13T10:55:04.060 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.043+0000 7f62da66d640 1 Processor -- start 2023-11-13T10:55:04.061 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.043+0000 7f62da66d640 1 -- start start 2023-11-13T10:55:04.061 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62da66d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d41f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.061 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62da66d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d40d1850 0x7f62d41f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.061 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62da66d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d415ce70 0x7f62d409d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.061 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62da66d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d409ea20 con 0x7f62d415ce70 2023-11-13T10:55:04.062 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62da66d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d409dda0 con 0x7f62d40d1850 2023-11-13T10:55:04.062 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62da66d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d409e080 con 0x7f62d40cdbb0 2023-11-13T10:55:04.062 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d8e6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d40d1850 0x7f62d41f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.062 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d966b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d41f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.072 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d8e6a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d40d1850 0x7f62d41f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41914/0 (socket says 172.21.15.184:41914) 2023-11-13T10:55:04.073 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d8e6a640 1 -- 172.21.15.184:0/3375200394 learned_addr learned my addr 172.21.15.184:0/3375200394 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:04.073 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d9e6c640 1 --2- 172.21.15.184:0/3375200394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d415ce70 0x7f62d409d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.077 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d966b640 1 -- 172.21.15.184:0/3375200394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d40d1850 msgr2=0x7f62d41f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.077 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d966b640 1 --2- 172.21.15.184:0/3375200394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d40d1850 0x7f62d41f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.077 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d966b640 1 -- 172.21.15.184:0/3375200394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d415ce70 msgr2=0x7f62d409d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.081 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d966b640 1 --2- 172.21.15.184:0/3375200394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d415ce70 0x7f62d409d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.082 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.044+0000 7f62d966b640 1 -- 172.21.15.184:0/3375200394 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c40094d0 con 0x7f62d40cdbb0 2023-11-13T10:55:04.082 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.045+0000 7f62d966b640 1 --2- 172.21.15.184:0/3375200394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d41f2490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f62d000e870 tx=0x7f62d000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:04.082 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.045+0000 7f62c9ffb640 1 -- 172.21.15.184:0/3375200394 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000ee60 con 0x7f62d40cdbb0 2023-11-13T10:55:04.082 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.045+0000 7f62c9ffb640 1 -- 172.21.15.184:0/3375200394 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d000c940 con 0x7f62d40cdbb0 2023-11-13T10:55:04.084 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.045+0000 7f62c9ffb640 1 -- 172.21.15.184:0/3375200394 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000ccd0 con 0x7f62d40cdbb0 2023-11-13T10:55:04.084 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.045+0000 7f62da66d640 1 -- 172.21.15.184:0/3375200394 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d409e360 con 0x7f62d40cdbb0 2023-11-13T10:55:04.087 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.045+0000 7f62da66d640 1 -- 172.21.15.184:0/3375200394 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d40a0750 con 0x7f62d40cdbb0 2023-11-13T10:55:04.088 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.045+0000 7f62c9ffb640 1 -- 172.21.15.184:0/3375200394 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f62d0005c90 con 0x7f62d40cdbb0 2023-11-13T10:55:04.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.045+0000 7f62c9ffb640 1 -- 172.21.15.184:0/3375200394 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f62d0006700 con 0x7f62d40cdbb0 2023-11-13T10:55:04.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.046+0000 7f62da66d640 1 -- 172.21.15.184:0/3375200394 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d404e4d0 con 0x7f62d40cdbb0 2023-11-13T10:55:04.090 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:04.048+0000 7fd75c632040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-13T10:55:04.090 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.049+0000 7f62c9ffb640 1 -- 172.21.15.184:0/3375200394 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f62d0021060 con 0x7f62d40cdbb0 2023-11-13T10:55:04.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.127+0000 7f62da66d640 1 -- 172.21.15.184:0/3375200394 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a9764441-fde9-439e-8bbd-76f0f53f31d8", "id": 5} v 0) v1 -- 0x7f62d409c040 con 0x7f62d40cdbb0 2023-11-13T10:55:04.171 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.133+0000 7f62c9ffb640 1 -- 172.21.15.184:0/3375200394 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9764441-fde9-439e-8bbd-76f0f53f31d8", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f62d0017070 con 0x7f62d40cdbb0 2023-11-13T10:55:04.172 INFO:teuthology.orchestra.run.smithi184.stdout:5 2023-11-13T10:55:04.174 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.134+0000 7f62da66d640 1 -- 172.21.15.184:0/3375200394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 msgr2=0x7f62d41f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.174 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.134+0000 7f62da66d640 1 --2- 172.21.15.184:0/3375200394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d41f2490 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f62d000e870 tx=0x7f62d000ed30 comp rx=0 tx=0).stop 2023-11-13T10:55:04.174 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.134+0000 7f62da66d640 1 -- 172.21.15.184:0/3375200394 shutdown_connections 2023-11-13T10:55:04.174 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.134+0000 7f62da66d640 1 --2- 172.21.15.184:0/3375200394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62d415ce70 0x7f62d409d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.175 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.134+0000 7f62da66d640 1 --2- 172.21.15.184:0/3375200394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62d40d1850 0x7f62d41f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.175 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.134+0000 7f62da66d640 1 --2- 172.21.15.184:0/3375200394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62d40cdbb0 0x7f62d41f2490 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.175 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.134+0000 7f62da66d640 1 -- 172.21.15.184:0/3375200394 >> 172.21.15.184:0/3375200394 conn(0x7f62d408c580 msgr2=0x7f62d406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:04.175 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.134+0000 7f62da66d640 1 -- 172.21.15.184:0/3375200394 shutdown_connections 2023-11-13T10:55:04.176 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.134+0000 7f62da66d640 1 -- 172.21.15.184:0/3375200394 wait complete. 2023-11-13T10:55:04.176 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c 6 2023-11-13T10:55:04.219 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:04.216+0000 7fc01b616040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-13T10:55:04.231 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:04.216+0000 7fd75c632040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-13T10:55:04.274 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:04.268+0000 7fd75c632040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-13T10:55:04.275 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:04.267+0000 7fc01b616040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-13T10:55:04.394 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.390+0000 7f7b4a290640 1 Processor -- start 2023-11-13T10:55:04.394 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.390+0000 7f7b4a290640 1 -- start start 2023-11-13T10:55:04.395 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.391+0000 7f7b4a290640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7b440d1a10 0x7f7b440ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.395 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.391+0000 7f7b4a290640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b44160890 0x7f7b440b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.391+0000 7f7b4a290640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b440cd910 con 0x7f7b441604b0 2023-11-13T10:55:04.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.391+0000 7f7b4a290640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b440cdb40 con 0x7f7b440d1a10 2023-11-13T10:55:04.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.391+0000 7f7b4a290640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440cde20 con 0x7f7b44160890 2023-11-13T10:55:04.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b3bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7b440d1a10 0x7f7b440ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.396 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b4882d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b44160890 0x7f7b440b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.397 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b43fff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f7b441604b0 0x7f7b440d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:36476/0 (socket says 172.21.15.184:36476) 2023-11-13T10:55:04.397 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b43fff640 1 -- 172.21.15.184:0/1285292531 learned_addr learned my addr 172.21.15.184:0/1285292531 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-11-13T10:55:04.397 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b3bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7b440d1a10 0x7f7b440ce900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41930/0 (socket says 172.21.15.184:41930) 2023-11-13T10:55:04.399 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b4882d640 1 -- 172.21.15.184:0/1285292531 >> v2:172.21.15.184:3300/0 conn(0x7f7b440d1a10 msgr2=0x7f7b440ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:04.399 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b4882d640 1 --2- 172.21.15.184:0/1285292531 >> v2:172.21.15.184:3300/0 conn(0x7f7b440d1a10 0x7f7b440ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.399 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b4882d640 1 -- 172.21.15.184:0/1285292531 >> v1:172.21.15.184:6789/0 conn(0x7f7b441604b0 legacy=0x7f7b440d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.399 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b4882d640 1 -- 172.21.15.184:0/1285292531 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440ce100 con 0x7f7b44160890 2023-11-13T10:55:04.399 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b437fe640 1 -- 172.21.15.184:0/1285292531 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345866945 0 0) 0x7f7b440cd910 con 0x7f7b441604b0 2023-11-13T10:55:04.400 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.392+0000 7f7b3bfff640 1 --2- 172.21.15.184:0/1285292531 >> v2:172.21.15.184:3300/0 conn(0x7f7b440d1a10 0x7f7b440ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:55:04.400 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.393+0000 7f7b4882d640 1 --2- 172.21.15.184:0/1285292531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b44160890 0x7f7b440b2270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009d90 tx=0x7f7b3c02bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce48d0e8e3f70270 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:04.400 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.393+0000 7f7b437fe640 1 -- 172.21.15.184:0/1285292531 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c02cce0 con 0x7f7b44160890 2023-11-13T10:55:04.400 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.393+0000 7f7b437fe640 1 -- 172.21.15.184:0/1285292531 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c02ce40 con 0x7f7b44160890 2023-11-13T10:55:04.400 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.393+0000 7f7b437fe640 1 -- 172.21.15.184:0/1285292531 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c034660 con 0x7f7b44160890 2023-11-13T10:55:04.401 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.394+0000 7f7b4a290640 1 -- 172.21.15.184:0/1285292531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b44160890 msgr2=0x7f7b440b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.401 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.394+0000 7f7b4a290640 1 --2- 172.21.15.184:0/1285292531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b44160890 0x7f7b440b2270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009d90 tx=0x7f7b3c02bf30 comp rx=0 tx=0).stop 2023-11-13T10:55:04.401 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.394+0000 7f7b4a290640 1 -- 172.21.15.184:0/1285292531 shutdown_connections 2023-11-13T10:55:04.401 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.394+0000 7f7b4a290640 1 --2- 172.21.15.184:0/1285292531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b44160890 0x7f7b440b2270 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.401 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.394+0000 7f7b4a290640 1 --2- 172.21.15.184:0/1285292531 >> v2:172.21.15.184:3300/0 conn(0x7f7b440d1a10 0x7f7b440ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.402 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.394+0000 7f7b4a290640 1 -- 172.21.15.184:0/1285292531 >> 172.21.15.184:0/1285292531 conn(0x7f7b440d2d20 msgr2=0x7f7b440d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:04.402 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.394+0000 7f7b4a290640 1 -- 172.21.15.184:0/1285292531 shutdown_connections 2023-11-13T10:55:04.402 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.394+0000 7f7b4a290640 1 -- 172.21.15.184:0/1285292531 wait complete. 2023-11-13T10:55:04.404 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.396+0000 7f7b4a290640 1 Processor -- start 2023-11-13T10:55:04.404 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.396+0000 7f7b4a290640 1 -- start start 2023-11-13T10:55:04.405 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.397+0000 7f7b4a290640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b440d1a10 0x7f7b441f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.405 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.397+0000 7f7b4a290640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b441604b0 0x7f7b440c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.405 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.397+0000 7f7b4a290640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44160890 0x7f7b440b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.405 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.397+0000 7f7b4a290640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440c8750 con 0x7f7b44160890 2023-11-13T10:55:04.406 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.397+0000 7f7b4a290640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440c7880 con 0x7f7b441604b0 2023-11-13T10:55:04.406 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.397+0000 7f7b4a290640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441f7320 con 0x7f7b440d1a10 2023-11-13T10:55:04.406 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.397+0000 7f7b43fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b440d1a10 0x7f7b441f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.407 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.397+0000 7f7b43fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b440d1a10 0x7f7b441f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:38006/0 (socket says 172.21.15.184:38006) 2023-11-13T10:55:04.407 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.397+0000 7f7b43fff640 1 -- 172.21.15.184:0/2626583226 learned_addr learned my addr 172.21.15.184:0/2626583226 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:04.407 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b3bfff640 1 --2- 172.21.15.184:0/2626583226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b441604b0 0x7f7b440c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.407 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b43fff640 1 -- 172.21.15.184:0/2626583226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b441604b0 msgr2=0x7f7b440c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.407 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b43fff640 1 --2- 172.21.15.184:0/2626583226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b441604b0 0x7f7b440c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.407 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b43fff640 1 -- 172.21.15.184:0/2626583226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44160890 msgr2=0x7f7b440b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b43fff640 1 --2- 172.21.15.184:0/2626583226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44160890 0x7f7b440b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b43fff640 1 -- 172.21.15.184:0/2626583226 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c009400 con 0x7f7b440d1a10 2023-11-13T10:55:04.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b43fff640 1 --2- 172.21.15.184:0/2626583226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b440d1a10 0x7f7b441f6de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400c4d0 tx=0x7f7b3400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:04.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b417fa640 1 -- 172.21.15.184:0/2626583226 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b340074e0 con 0x7f7b440d1a10 2023-11-13T10:55:04.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b417fa640 1 -- 172.21.15.184:0/2626583226 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b34007660 con 0x7f7b440d1a10 2023-11-13T10:55:04.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b417fa640 1 -- 172.21.15.184:0/2626583226 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34004a20 con 0x7f7b440d1a10 2023-11-13T10:55:04.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.398+0000 7f7b4a290640 1 -- 172.21.15.184:0/2626583226 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c009a70 con 0x7f7b440d1a10 2023-11-13T10:55:04.418 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.399+0000 7f7b4a290640 1 -- 172.21.15.184:0/2626583226 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b440c8090 con 0x7f7b440d1a10 2023-11-13T10:55:04.419 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.399+0000 7f7b417fa640 1 -- 172.21.15.184:0/2626583226 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7b34012400 con 0x7f7b440d1a10 2023-11-13T10:55:04.450 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.399+0000 7f7b417fa640 1 -- 172.21.15.184:0/2626583226 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f7b34012ca0 con 0x7f7b440d1a10 2023-11-13T10:55:04.450 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.400+0000 7f7b4a290640 1 -- 172.21.15.184:0/2626583226 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b440c8890 con 0x7f7b440d1a10 2023-11-13T10:55:04.451 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:04.432+0000 7fc01b616040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-13T10:55:04.452 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.402+0000 7f7b417fa640 1 -- 172.21.15.184:0/2626583226 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f7b440c8890 con 0x7f7b440d1a10 2023-11-13T10:55:04.452 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:04.434+0000 7fd75c632040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-13T10:55:04.525 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.479+0000 7f7b4a290640 1 -- 172.21.15.184:0/2626583226 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c", "id": 6} v 0) v1 -- 0x7f7b440a0ca0 con 0x7f7b440d1a10 2023-11-13T10:55:04.526 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.485+0000 7f7b417fa640 1 -- 172.21.15.184:0/2626583226 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7b34018070 con 0x7f7b440d1a10 2023-11-13T10:55:04.527 INFO:teuthology.orchestra.run.smithi184.stdout:6 2023-11-13T10:55:04.542 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.487+0000 7f7b4a290640 1 -- 172.21.15.184:0/2626583226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b440d1a10 msgr2=0x7f7b441f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.542 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.487+0000 7f7b4a290640 1 --2- 172.21.15.184:0/2626583226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b440d1a10 0x7f7b441f6de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400c4d0 tx=0x7f7b3400c990 comp rx=0 tx=0).stop 2023-11-13T10:55:04.542 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.487+0000 7f7b4a290640 1 -- 172.21.15.184:0/2626583226 shutdown_connections 2023-11-13T10:55:04.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.487+0000 7f7b4a290640 1 --2- 172.21.15.184:0/2626583226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44160890 0x7f7b440b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.487+0000 7f7b4a290640 1 --2- 172.21.15.184:0/2626583226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b441604b0 0x7f7b440c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.487+0000 7f7b4a290640 1 --2- 172.21.15.184:0/2626583226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b440d1a10 0x7f7b441f6de0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.487+0000 7f7b4a290640 1 -- 172.21.15.184:0/2626583226 >> 172.21.15.184:0/2626583226 conn(0x7f7b440d2d20 msgr2=0x7f7b4415f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:04.543 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.487+0000 7f7b4a290640 1 -- 172.21.15.184:0/2626583226 shutdown_connections 2023-11-13T10:55:04.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.487+0000 7f7b4a290640 1 -- 172.21.15.184:0/2626583226 wait complete. 2023-11-13T10:55:04.544 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 1ba7a631-0655-4a4a-b869-fbf4747f9ca8 7 2023-11-13T10:55:04.784 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.780+0000 7fc104b33640 1 Processor -- start 2023-11-13T10:55:04.784 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.780+0000 7fc104b33640 1 -- start start 2023-11-13T10:55:04.789 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc104b33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1000cf0e0 0x7fc10015a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.790 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc104b33640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10015d8d0 con 0x7fc1000ce5b0 2023-11-13T10:55:04.790 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc104b33640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10015af00 con 0x7fc1000ce1d0 2023-11-13T10:55:04.790 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc104b33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10015db90 con 0x7fc1000cf0e0 2023-11-13T10:55:04.790 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc0fe59c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fc1000ce1d0 0x7fc1000ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:36484/0 (socket says 172.21.15.184:36484) 2023-11-13T10:55:04.791 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc0fe59c640 1 -- 172.21.15.184:0/3016876600 learned_addr learned my addr 172.21.15.184:0/3016876600 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-11-13T10:55:04.791 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc0fed9d640 1 --2- 172.21.15.184:0/3016876600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1000cf0e0 0x7fc10015a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.791 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc0fdd9b640 1 -- 172.21.15.184:0/3016876600 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962535266 0 0) 0x7fc10015af00 con 0x7fc1000ce1d0 2023-11-13T10:55:04.791 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc0fdd9b640 1 -- 172.21.15.184:0/3016876600 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0ec005730 con 0x7fc1000ce1d0 2023-11-13T10:55:04.791 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc0fed9d640 1 -- 172.21.15.184:0/3016876600 >> v1:172.21.15.184:6789/0 conn(0x7fc1000ce1d0 legacy=0x7fc1000ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.792 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc0fed9d640 1 -- 172.21.15.184:0/3016876600 >> v1:172.21.15.16:6789/0 conn(0x7fc1000ce5b0 legacy=0x7fc10015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:04.792 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.782+0000 7fc0fed9d640 1 -- 172.21.15.184:0/3016876600 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10015de70 con 0x7fc1000cf0e0 2023-11-13T10:55:04.792 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.784+0000 7fc0fed9d640 1 --2- 172.21.15.184:0/3016876600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1000cf0e0 0x7fc10015a840 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009dc0 tx=0x7fc0f802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34158d7b15a18f47 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:04.792 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.784+0000 7fc0fdd9b640 1 -- 172.21.15.184:0/3016876600 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f802ccd0 con 0x7fc1000cf0e0 2023-11-13T10:55:04.792 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.784+0000 7fc0fdd9b640 1 -- 172.21.15.184:0/3016876600 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f802ce30 con 0x7fc1000cf0e0 2023-11-13T10:55:04.793 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.784+0000 7fc0fdd9b640 1 -- 172.21.15.184:0/3016876600 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8034660 con 0x7fc1000cf0e0 2023-11-13T10:55:04.793 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.785+0000 7fc104b33640 1 -- 172.21.15.184:0/3016876600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1000cf0e0 msgr2=0x7fc10015a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.793 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.785+0000 7fc104b33640 1 --2- 172.21.15.184:0/3016876600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1000cf0e0 0x7fc10015a840 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009dc0 tx=0x7fc0f802bf20 comp rx=0 tx=0).stop 2023-11-13T10:55:04.793 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.785+0000 7fc104b33640 1 -- 172.21.15.184:0/3016876600 shutdown_connections 2023-11-13T10:55:04.794 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.785+0000 7fc104b33640 1 --2- 172.21.15.184:0/3016876600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1000cf0e0 0x7fc10015a840 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.794 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.785+0000 7fc104b33640 1 -- 172.21.15.184:0/3016876600 >> 172.21.15.184:0/3016876600 conn(0x7fc10006a190 msgr2=0x7fc10006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:04.798 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.785+0000 7fc104b33640 1 -- 172.21.15.184:0/3016876600 shutdown_connections 2023-11-13T10:55:04.798 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.785+0000 7fc104b33640 1 -- 172.21.15.184:0/3016876600 wait complete. 2023-11-13T10:55:04.801 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.787+0000 7fc104b33640 1 Processor -- start 2023-11-13T10:55:04.801 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.787+0000 7fc104b33640 1 -- start start 2023-11-13T10:55:04.802 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc104b33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000ce1d0 0x7fc10013bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.802 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc104b33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1000cf0e0 0x7fc1000cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.802 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc104b33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1001e92c0 0x7fc1000ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:04.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc104b33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10013f520 con 0x7fc1000ce1d0 2023-11-13T10:55:04.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc104b33640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10013c240 con 0x7fc1000cf0e0 2023-11-13T10:55:04.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc104b33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1001e6e40 con 0x7fc1001e92c0 2023-11-13T10:55:04.803 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc0f7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1000cf0e0 0x7fc1000cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.804 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc0fed9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1001e92c0 0x7fc1000ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.804 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc0f7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1000cf0e0 0x7fc1000cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41952/0 (socket says 172.21.15.184:41952) 2023-11-13T10:55:04.804 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc0f7fff640 1 -- 172.21.15.184:0/2359741820 learned_addr learned my addr 172.21.15.184:0/2359741820 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-11-13T10:55:04.804 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc0fe59c640 1 --2- 172.21.15.184:0/2359741820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000ce1d0 0x7fc10013bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:04.804 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.788+0000 7fc0fed9d640 1 -- 172.21.15.184:0/2359741820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1000cf0e0 msgr2=0x7fc1000cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.805 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc0fed9d640 1 --2- 172.21.15.184:0/2359741820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1000cf0e0 0x7fc1000cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.805 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc0fed9d640 1 -- 172.21.15.184:0/2359741820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000ce1d0 msgr2=0x7fc10013bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.805 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc0fed9d640 1 --2- 172.21.15.184:0/2359741820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000ce1d0 0x7fc10013bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.805 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc0fed9d640 1 -- 172.21.15.184:0/2359741820 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8009a70 con 0x7fc1001e92c0 2023-11-13T10:55:04.806 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc0f7fff640 1 --2- 172.21.15.184:0/2359741820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1000cf0e0 0x7fc1000cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:55:04.806 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc0fed9d640 1 --2- 172.21.15.184:0/2359741820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1001e92c0 0x7fc1000ccd10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8033000 tx=0x7fc0f8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:04.806 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc0f6ffd640 1 -- 172.21.15.184:0/2359741820 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f803d600 con 0x7fc1001e92c0 2023-11-13T10:55:04.806 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc0f6ffd640 1 -- 172.21.15.184:0/2359741820 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f803d760 con 0x7fc1001e92c0 2023-11-13T10:55:04.806 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc104b33640 1 -- 172.21.15.184:0/2359741820 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1001e8260 con 0x7fc1001e92c0 2023-11-13T10:55:04.807 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc0f6ffd640 1 -- 172.21.15.184:0/2359741820 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f803db10 con 0x7fc1001e92c0 2023-11-13T10:55:04.807 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.789+0000 7fc104b33640 1 -- 172.21.15.184:0/2359741820 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1001e87e0 con 0x7fc1001e92c0 2023-11-13T10:55:04.807 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.790+0000 7fc0f6ffd640 1 -- 172.21.15.184:0/2359741820 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc0f803b040 con 0x7fc1001e92c0 2023-11-13T10:55:04.807 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.790+0000 7fc0f6ffd640 1 -- 172.21.15.184:0/2359741820 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fc0f8049400 con 0x7fc1001e92c0 2023-11-13T10:55:04.807 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.790+0000 7fc104b33640 1 -- 172.21.15.184:0/2359741820 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc100072520 con 0x7fc1001e92c0 2023-11-13T10:55:04.808 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.793+0000 7fc0f6ffd640 1 -- 172.21.15.184:0/2359741820 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc0f8044020 con 0x7fc1001e92c0 2023-11-13T10:55:04.875 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.871+0000 7fc104b33640 1 -- 172.21.15.184:0/2359741820 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1ba7a631-0655-4a4a-b869-fbf4747f9ca8", "id": 7} v 0) v1 -- 0x7fc100071370 con 0x7fc1001e92c0 2023-11-13T10:55:04.883 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.878+0000 7fc0f6ffd640 1 -- 172.21.15.184:0/2359741820 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ba7a631-0655-4a4a-b869-fbf4747f9ca8", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc0f8039070 con 0x7fc1001e92c0 2023-11-13T10:55:04.883 INFO:teuthology.orchestra.run.smithi184.stdout:7 2023-11-13T10:55:04.884 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.879+0000 7fc104b33640 1 -- 172.21.15.184:0/2359741820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1001e92c0 msgr2=0x7fc1000ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:04.885 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.879+0000 7fc104b33640 1 --2- 172.21.15.184:0/2359741820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1001e92c0 0x7fc1000ccd10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8033000 tx=0x7fc0f8004a40 comp rx=0 tx=0).stop 2023-11-13T10:55:04.885 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.880+0000 7fc104b33640 1 -- 172.21.15.184:0/2359741820 shutdown_connections 2023-11-13T10:55:04.885 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.880+0000 7fc104b33640 1 --2- 172.21.15.184:0/2359741820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1001e92c0 0x7fc1000ccd10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.885 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.880+0000 7fc104b33640 1 --2- 172.21.15.184:0/2359741820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1000cf0e0 0x7fc1000cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.885 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.880+0000 7fc104b33640 1 --2- 172.21.15.184:0/2359741820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000ce1d0 0x7fc10013bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:04.886 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.880+0000 7fc104b33640 1 -- 172.21.15.184:0/2359741820 >> 172.21.15.184:0/2359741820 conn(0x7fc10006a190 msgr2=0x7fc10006a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:04.886 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.880+0000 7fc104b33640 1 -- 172.21.15.184:0/2359741820 shutdown_connections 2023-11-13T10:55:04.886 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:55:04.880+0000 7fc104b33640 1 -- 172.21.15.184:0/2359741820 wait complete. 2023-11-13T10:55:04.900 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-13T10:55:04.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-13T10:55:04.910 INFO:tasks.ceph.osd.0:Started 2023-11-13T10:55:04.911 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-13T10:55:04.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-13T10:55:04.922 INFO:tasks.ceph.osd.1:Started 2023-11-13T10:55:04.922 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-13T10:55:04.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-13T10:55:04.932 INFO:tasks.ceph.osd.2:Started 2023-11-13T10:55:04.933 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-13T10:55:04.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-13T10:55:04.953 INFO:tasks.ceph.osd.3:Started 2023-11-13T10:55:04.953 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-13T10:55:04.954 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-13T10:55:04.963 INFO:tasks.ceph.osd.4:Started 2023-11-13T10:55:04.964 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-13T10:55:04.965 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-13T10:55:04.973 INFO:tasks.ceph.osd.5:Started 2023-11-13T10:55:04.974 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-13T10:55:04.974 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-13T10:55:04.983 INFO:tasks.ceph.osd.6:Started 2023-11-13T10:55:04.983 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-13T10:55:04.987 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-13T10:55:04.995 INFO:tasks.ceph.osd.7:Started 2023-11-13T10:55:04.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:05.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.194+0000 7f139bbc9640 1 Processor -- start 2023-11-13T10:55:05.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.194+0000 7f139bbc9640 1 -- start start 2023-11-13T10:55:05.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.196+0000 7f139bbc9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13940ccc60 0x7f1394071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:05.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.196+0000 7f139bbc9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394071980 0x7f139406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:05.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.196+0000 7f139bbc9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13940cd110 con 0x7f1394155410 2023-11-13T10:55:05.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.196+0000 7f139bbc9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1394071e10 con 0x7f13940ccc60 2023-11-13T10:55:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.196+0000 7f139bbc9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139415b500 con 0x7f1394071980 2023-11-13T10:55:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.196+0000 7f1399965640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1394155410 0x7f13940ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45196/0 (socket says 172.21.15.16:45196) 2023-11-13T10:55:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.196+0000 7f1399965640 1 -- 172.21.15.16:0/2324325092 learned_addr learned my addr 172.21.15.16:0/2324325092 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.196+0000 7f139a166640 1 --2- 172.21.15.16:0/2324325092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394071980 0x7f139406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.196+0000 7f1399164640 1 --2- 172.21.15.16:0/2324325092 >> v2:172.21.15.184:3300/0 conn(0x7f13940ccc60 0x7f1394071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:05.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.197+0000 7f1398963640 1 -- 172.21.15.16:0/2324325092 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586568066 0 0) 0x7f13940cd110 con 0x7f1394155410 2023-11-13T10:55:05.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.197+0000 7f1398963640 1 -- 172.21.15.16:0/2324325092 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1380003520 con 0x7f1394155410 2023-11-13T10:55:05.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.197+0000 7f1399164640 1 -- 172.21.15.16:0/2324325092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394071980 msgr2=0x7f139406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:05.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.197+0000 7f1399164640 1 --2- 172.21.15.16:0/2324325092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394071980 0x7f139406fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.197+0000 7f1399164640 1 -- 172.21.15.16:0/2324325092 >> v1:172.21.15.16:6789/0 conn(0x7f1394155410 legacy=0x7f13940ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:05.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.197+0000 7f1399164640 1 -- 172.21.15.16:0/2324325092 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139415b7e0 con 0x7f13940ccc60 2023-11-13T10:55:05.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.197+0000 7f139a166640 1 --2- 172.21.15.16:0/2324325092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394071980 0x7f139406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:55:05.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.198+0000 7f1399164640 1 --2- 172.21.15.16:0/2324325092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13940ccc60 0x7f1394071440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13840097a0 tx=0x7f138402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983e85b480806b62 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:05.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.198+0000 7f1398963640 1 -- 172.21.15.16:0/2324325092 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138402d040 con 0x7f13940ccc60 2023-11-13T10:55:05.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.198+0000 7f1398963640 1 -- 172.21.15.16:0/2324325092 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1384031810 con 0x7f13940ccc60 2023-11-13T10:55:05.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.198+0000 7f1398963640 1 -- 172.21.15.16:0/2324325092 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384031b80 con 0x7f13940ccc60 2023-11-13T10:55:05.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.199+0000 7f139bbc9640 1 -- 172.21.15.16:0/2324325092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13940ccc60 msgr2=0x7f1394071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:05.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.199+0000 7f139bbc9640 1 --2- 172.21.15.16:0/2324325092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13940ccc60 0x7f1394071440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13840097a0 tx=0x7f138402b8b0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.199+0000 7f139bbc9640 1 -- 172.21.15.16:0/2324325092 shutdown_connections 2023-11-13T10:55:05.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.199+0000 7f139bbc9640 1 --2- 172.21.15.16:0/2324325092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394071980 0x7f139406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.199+0000 7f139bbc9640 1 --2- 172.21.15.16:0/2324325092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13940ccc60 0x7f1394071440 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.199+0000 7f139bbc9640 1 -- 172.21.15.16:0/2324325092 >> 172.21.15.16:0/2324325092 conn(0x7f139406a190 msgr2=0x7f139406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.199+0000 7f139bbc9640 1 -- 172.21.15.16:0/2324325092 shutdown_connections 2023-11-13T10:55:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.199+0000 7f139bbc9640 1 -- 172.21.15.16:0/2324325092 wait complete. 2023-11-13T10:55:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.200+0000 7f139bbc9640 1 Processor -- start 2023-11-13T10:55:05.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.201+0000 7f139bbc9640 1 -- start start 2023-11-13T10:55:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.201+0000 7f139bbc9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394071980 0x7f139415de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.201+0000 7f139bbc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ccc60 0x7f13941442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.201+0000 7f139bbc9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394155410 0x7f13940b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:05.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.201+0000 7f139bbc9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139406c0a0 con 0x7f13940ccc60 2023-11-13T10:55:05.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.201+0000 7f139bbc9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139415c3f0 con 0x7f1394071980 2023-11-13T10:55:05.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.201+0000 7f139bbc9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139415c6d0 con 0x7f1394155410 2023-11-13T10:55:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399164640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ccc60 0x7f13941442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399164640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ccc60 0x7f13941442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54182/0 (socket says 172.21.15.16:54182) 2023-11-13T10:55:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399164640 1 -- 172.21.15.16:0/962471547 learned_addr learned my addr 172.21.15.16:0/962471547 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399965640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394071980 0x7f139415de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f139a166640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394155410 0x7f13940b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:05.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399164640 1 -- 172.21.15.16:0/962471547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394155410 msgr2=0x7f13940b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:05.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399164640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394155410 0x7f13940b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399164640 1 -- 172.21.15.16:0/962471547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394071980 msgr2=0x7f139415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:05.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399164640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394071980 0x7f139415de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399164640 1 -- 172.21.15.16:0/962471547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c009a70 con 0x7f13940ccc60 2023-11-13T10:55:05.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.202+0000 7f1399965640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394071980 0x7f139415de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:55:05.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.203+0000 7f1399164640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ccc60 0x7f13941442e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f138402b8f0 tx=0x7f13840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:05.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.203+0000 7f138a7fc640 1 -- 172.21.15.16:0/962471547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384006e10 con 0x7f13940ccc60 2023-11-13T10:55:05.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.203+0000 7f138a7fc640 1 -- 172.21.15.16:0/962471547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138403ad60 con 0x7f13940ccc60 2023-11-13T10:55:05.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.203+0000 7f138a7fc640 1 -- 172.21.15.16:0/962471547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840443f0 con 0x7f13940ccc60 2023-11-13T10:55:05.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.203+0000 7f139bbc9640 1 -- 172.21.15.16:0/962471547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1384009400 con 0x7f13940ccc60 2023-11-13T10:55:05.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.203+0000 7f139bbc9640 1 -- 172.21.15.16:0/962471547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139415c9b0 con 0x7f13940ccc60 2023-11-13T10:55:05.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.203+0000 7f138a7fc640 1 -- 172.21.15.16:0/962471547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1384044850 con 0x7f13940ccc60 2023-11-13T10:55:05.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.203+0000 7f138a7fc640 1 -- 172.21.15.16:0/962471547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f138404d450 con 0x7f13940ccc60 2023-11-13T10:55:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.204+0000 7f139bbc9640 1 -- 172.21.15.16:0/962471547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139406c1e0 con 0x7f13940ccc60 2023-11-13T10:55:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.207+0000 7f138a7fc640 1 -- 172.21.15.16:0/962471547 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f139406c1e0 con 0x7f13940ccc60 2023-11-13T10:55:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.273+0000 7f139bbc9640 1 -- 172.21.15.16:0/962471547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1394067090 con 0x7f13940ccc60 2023-11-13T10:55:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.274+0000 7f138a7fc640 1 -- 172.21.15.16:0/962471547 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1384035020 con 0x7f13940ccc60 2023-11-13T10:55:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:05.309 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:04.873821+0000","last_up_change":"0.000000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:05.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.275+0000 7f139bbc9640 1 -- 172.21.15.16:0/962471547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ccc60 msgr2=0x7f13941442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:05.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.275+0000 7f139bbc9640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ccc60 0x7f13941442e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f138402b8f0 tx=0x7f13840046c0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.275+0000 7f139bbc9640 1 -- 172.21.15.16:0/962471547 shutdown_connections 2023-11-13T10:55:05.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.275+0000 7f139bbc9640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1394155410 0x7f13940b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.275+0000 7f139bbc9640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ccc60 0x7f13941442e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.275+0000 7f139bbc9640 1 --2- 172.21.15.16:0/962471547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394071980 0x7f139415de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:05.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.275+0000 7f139bbc9640 1 -- 172.21.15.16:0/962471547 >> 172.21.15.16:0/962471547 conn(0x7f139406a190 msgr2=0x7f13941531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:05.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.275+0000 7f139bbc9640 1 -- 172.21.15.16:0/962471547 shutdown_connections 2023-11-13T10:55:05.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:05.275+0000 7f139bbc9640 1 -- 172.21.15.16:0/962471547 wait complete. 2023-11-13T10:55:05.315 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-13T10:55:05.316 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-13T10:55:05.463 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:05.459+0000 7fd75c632040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-13T10:55:05.525 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:05.522+0000 7fc01b616040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-13T10:55:05.610 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-11-13T10:55:05.606+0000 7ff731c1f500 -1 Falling back to public interface 2023-11-13T10:55:05.625 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-11-13T10:55:05.621+0000 7fb857d07500 -1 Falling back to public interface 2023-11-13T10:55:05.626 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-11-13T10:55:05.621+0000 7fdc729fb500 -1 Falling back to public interface 2023-11-13T10:55:05.649 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-11-13T10:55:05.643+0000 7f799b467500 -1 Falling back to public interface 2023-11-13T10:55:05.659 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-11-13T10:55:05.654+0000 7fc4f2939500 -1 Falling back to public interface 2023-11-13T10:55:05.662 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-11-13T10:55:05.659+0000 7f0e11e0c500 -1 Falling back to public interface 2023-11-13T10:55:05.675 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-11-13T10:55:05.671+0000 7f60ec40e500 -1 Falling back to public interface 2023-11-13T10:55:05.700 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-11-13T10:55:05.696+0000 7f1fe900c500 -1 Falling back to public interface 2023-11-13T10:55:05.770 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:05.766+0000 7fd75c632040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-13T10:55:05.833 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:05.825+0000 7fd75c632040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-13T10:55:05.844 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:05.841+0000 7fc01b616040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-13T10:55:05.906 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:05.903+0000 7fc01b616040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-13T10:55:05.941 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:05.937+0000 7fd75c632040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-13T10:55:06.031 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-13T10:55:06.028+0000 7fc01b616040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-13T10:55:06.127 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:06.305 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.302+0000 7fa49a4bb640 1 Processor -- start 2023-11-13T10:55:06.306 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.302+0000 7fa49a4bb640 1 -- start start 2023-11-13T10:55:06.308 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.303+0000 7fa49a4bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa49415aa90 0x7fa49406a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:06.308 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.303+0000 7fa49a4bb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa49406a6d0 0x7fa494070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:06.308 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa49a4bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49406d680 con 0x7fa4940708a0 2023-11-13T10:55:06.309 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa49a4bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49406ab50 con 0x7fa49415aa90 2023-11-13T10:55:06.309 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa49a4bb640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa494070d30 con 0x7fa49406a6d0 2023-11-13T10:55:06.309 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa4994b9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa49415aa90 0x7fa49406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:06.309 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa499cba640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa4940708a0 0x7fa4940b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45204/0 (socket says 172.21.15.16:45204) 2023-11-13T10:55:06.309 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa499cba640 1 -- 172.21.15.16:0/2210842607 learned_addr learned my addr 172.21.15.16:0/2210842607 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:06.310 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa4994b9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa49415aa90 0x7fa49406a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54184/0 (socket says 172.21.15.16:54184) 2023-11-13T10:55:06.321 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa4994b9640 1 -- 172.21.15.16:0/2210842607 >> v2:172.21.15.184:3300/0 conn(0x7fa49406a6d0 msgr2=0x7fa494070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:06.332 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa498cb8640 1 --2- 172.21.15.16:0/2210842607 >> v2:172.21.15.184:3300/0 conn(0x7fa49406a6d0 0x7fa494070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:06.332 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa4994b9640 1 --2- 172.21.15.16:0/2210842607 >> v2:172.21.15.184:3300/0 conn(0x7fa49406a6d0 0x7fa494070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:06.332 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa48bfff640 1 -- 172.21.15.16:0/2210842607 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192561517 0 0) 0x7fa49406d680 con 0x7fa4940708a0 2023-11-13T10:55:06.332 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa4994b9640 1 -- 172.21.15.16:0/2210842607 >> v1:172.21.15.16:6789/0 conn(0x7fa4940708a0 legacy=0x7fa4940b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:06.333 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa4994b9640 1 -- 172.21.15.16:0/2210842607 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49415d630 con 0x7fa49415aa90 2023-11-13T10:55:06.333 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.304+0000 7fa498cb8640 1 --2- 172.21.15.16:0/2210842607 >> v2:172.21.15.184:3300/0 conn(0x7fa49406a6d0 0x7fa494070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:06.353 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.306+0000 7fa4994b9640 1 --2- 172.21.15.16:0/2210842607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49415aa90 0x7fa49406a190 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa49406b320 tx=0x7fa49002e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce025f38c36cd3a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:06.353 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.306+0000 7fa48bfff640 1 -- 172.21.15.16:0/2210842607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49002edc0 con 0x7fa49415aa90 2023-11-13T10:55:06.353 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.306+0000 7fa48bfff640 1 -- 172.21.15.16:0/2210842607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa490036440 con 0x7fa49415aa90 2023-11-13T10:55:06.354 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.306+0000 7fa48bfff640 1 -- 172.21.15.16:0/2210842607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4900367d0 con 0x7fa49415aa90 2023-11-13T10:55:06.354 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.307+0000 7fa49a4bb640 1 -- 172.21.15.16:0/2210842607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49415aa90 msgr2=0x7fa49406a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:06.354 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.307+0000 7fa49a4bb640 1 --2- 172.21.15.16:0/2210842607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49415aa90 0x7fa49406a190 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa49406b320 tx=0x7fa49002e010 comp rx=0 tx=0).stop 2023-11-13T10:55:06.357 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.307+0000 7fa49a4bb640 1 -- 172.21.15.16:0/2210842607 shutdown_connections 2023-11-13T10:55:06.357 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.307+0000 7fa49a4bb640 1 --2- 172.21.15.16:0/2210842607 >> v2:172.21.15.184:3300/0 conn(0x7fa49406a6d0 0x7fa494070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:06.357 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.307+0000 7fa49a4bb640 1 --2- 172.21.15.16:0/2210842607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa49415aa90 0x7fa49406a190 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:06.357 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.307+0000 7fa49a4bb640 1 -- 172.21.15.16:0/2210842607 >> 172.21.15.16:0/2210842607 conn(0x7fa4940b04a0 msgr2=0x7fa4940b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:06.357 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.307+0000 7fa49a4bb640 1 -- 172.21.15.16:0/2210842607 shutdown_connections 2023-11-13T10:55:06.358 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.307+0000 7fa49a4bb640 1 -- 172.21.15.16:0/2210842607 wait complete. 2023-11-13T10:55:06.371 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.309+0000 7fa49a4bb640 1 Processor -- start 2023-11-13T10:55:06.372 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.309+0000 7fa49a4bb640 1 -- start start 2023-11-13T10:55:06.372 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa49a4bb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49406a6d0 0x7fa494149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:06.372 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa49a4bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4940708a0 0x7fa4940a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:06.373 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa49a4bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49415aa90 0x7fa494154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:06.373 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa49a4bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940b2ea0 con 0x7fa4940708a0 2023-11-13T10:55:06.373 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa49a4bb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa494149690 con 0x7fa49415aa90 2023-11-13T10:55:06.373 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa49a4bb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940a8a00 con 0x7fa49406a6d0 2023-11-13T10:55:06.373 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa498cb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4940708a0 0x7fa4940a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:06.374 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa498cb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4940708a0 0x7fa4940a8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54200/0 (socket says 172.21.15.16:54200) 2023-11-13T10:55:06.374 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa498cb8640 1 -- 172.21.15.16:0/1827745442 learned_addr learned my addr 172.21.15.16:0/1827745442 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:06.374 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa4994b9640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49406a6d0 0x7fa494149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:06.374 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa499cba640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49415aa90 0x7fa494154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:06.380 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa498cb8640 1 -- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49406a6d0 msgr2=0x7fa494149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:06.380 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa498cb8640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49406a6d0 0x7fa494149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:06.380 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa498cb8640 1 -- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49415aa90 msgr2=0x7fa494154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:06.380 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa498cb8640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49415aa90 0x7fa494154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:06.381 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa498cb8640 1 -- 172.21.15.16:0/1827745442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49000b6a0 con 0x7fa4940708a0 2023-11-13T10:55:06.381 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa4994b9640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49406a6d0 0x7fa494149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:06.381 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.310+0000 7fa499cba640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49415aa90 0x7fa494154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:06.381 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.311+0000 7fa498cb8640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4940708a0 0x7fa4940a8e60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c7f0 tx=0x7fa48400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:06.381 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.311+0000 7fa489ffb640 1 -- 172.21.15.16:0/1827745442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484007820 con 0x7fa4940708a0 2023-11-13T10:55:06.382 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.311+0000 7fa489ffb640 1 -- 172.21.15.16:0/1827745442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa484007980 con 0x7fa4940708a0 2023-11-13T10:55:06.382 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.311+0000 7fa489ffb640 1 -- 172.21.15.16:0/1827745442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484007cf0 con 0x7fa4940708a0 2023-11-13T10:55:06.382 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.311+0000 7fa49a4bb640 1 -- 172.21.15.16:0/1827745442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4940b27e0 con 0x7fa4940708a0 2023-11-13T10:55:06.382 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.311+0000 7fa49a4bb640 1 -- 172.21.15.16:0/1827745442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4940a93a0 con 0x7fa4940708a0 2023-11-13T10:55:06.382 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.312+0000 7fa49a4bb640 1 -- 172.21.15.16:0/1827745442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49404e4d0 con 0x7fa4940708a0 2023-11-13T10:55:06.383 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.312+0000 7fa489ffb640 1 -- 172.21.15.16:0/1827745442 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 2) v1 ==== 44567+0+0 (secure 0 0 0) 0x7fa4840113f0 con 0x7fa4940708a0 2023-11-13T10:55:06.383 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.313+0000 7fa489ffb640 1 -- 172.21.15.16:0/1827745442 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fa4840498e0 con 0x7fa4940708a0 2023-11-13T10:55:06.383 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.315+0000 7fa489ffb640 1 -- 172.21.15.16:0/1827745442 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa484021300 con 0x7fa4940708a0 2023-11-13T10:55:06.393 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.382+0000 7fa49a4bb640 1 -- 172.21.15.16:0/1827745442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa49406c220 con 0x7fa4940708a0 2023-11-13T10:55:06.393 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.384+0000 7fa489ffb640 1 -- 172.21.15.16:0/1827745442 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fa484018e80 con 0x7fa4940708a0 2023-11-13T10:55:06.394 INFO:teuthology.misc.health.smithi016.stdout: 2023-11-13T10:55:06.394 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:04.873821+0000","last_up_change":"0.000000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:06.440 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.385+0000 7fa49a4bb640 1 -- 172.21.15.16:0/1827745442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4940708a0 msgr2=0x7fa4940a8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:06.441 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.385+0000 7fa49a4bb640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4940708a0 0x7fa4940a8e60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c7f0 tx=0x7fa48400ccb0 comp rx=0 tx=0).stop 2023-11-13T10:55:06.441 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.385+0000 7fa49a4bb640 1 -- 172.21.15.16:0/1827745442 shutdown_connections 2023-11-13T10:55:06.441 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.385+0000 7fa49a4bb640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49415aa90 0x7fa494154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:06.441 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.385+0000 7fa49a4bb640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4940708a0 0x7fa4940a8e60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:06.441 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.385+0000 7fa49a4bb640 1 --2- 172.21.15.16:0/1827745442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49406a6d0 0x7fa494149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:06.442 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.385+0000 7fa49a4bb640 1 -- 172.21.15.16:0/1827745442 >> 172.21.15.16:0/1827745442 conn(0x7fa4940b04a0 msgr2=0x7fa494158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:06.442 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.385+0000 7fa49a4bb640 1 -- 172.21.15.16:0/1827745442 shutdown_connections 2023-11-13T10:55:06.442 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:06.385+0000 7fa49a4bb640 1 -- 172.21.15.16:0/1827745442 wait complete. 2023-11-13T10:55:06.442 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-13T10:55:06.986 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:06.982+0000 7fd6fd2fd640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-13T10:55:06.989 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-11-13T10:55:06.985+0000 7fd6fd2fd640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-13T10:55:09.008 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-11-13T10:55:09.002+0000 7f1fe900c500 -1 osd.7 0 log_to_monitors true 2023-11-13T10:55:09.166 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-11-13T10:55:09.159+0000 7f799b467500 -1 osd.4 0 log_to_monitors true 2023-11-13T10:55:09.193 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-11-13T10:55:09.186+0000 7fc4f2939500 -1 osd.5 0 log_to_monitors true 2023-11-13T10:55:09.199 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-11-13T10:55:09.196+0000 7fb857d07500 -1 osd.2 0 log_to_monitors true 2023-11-13T10:55:09.246 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-11-13T10:55:09.241+0000 7f60ec40e500 -1 osd.6 0 log_to_monitors true 2023-11-13T10:55:09.408 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-11-13T10:55:09.404+0000 7ff731c1f500 -1 osd.1 0 log_to_monitors true 2023-11-13T10:55:09.452 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-11-13T10:55:09.446+0000 7fdc729fb500 -1 osd.0 0 log_to_monitors true 2023-11-13T10:55:09.473 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-11-13T10:55:09.469+0000 7f0e11e0c500 -1 osd.3 0 log_to_monitors true 2023-11-13T10:55:11.227 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-11-13T10:55:11.219+0000 7f1fe52a1640 -1 osd.7 0 waiting for initial osdmap 2023-11-13T10:55:11.329 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-11-13T10:55:11.324+0000 7f1fdc8b4640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-13T10:55:12.868 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-11-13T10:55:12.862+0000 7fb853c97640 -1 osd.2 0 waiting for initial osdmap 2023-11-13T10:55:12.877 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-11-13T10:55:12.874+0000 7fb84babe640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-13T10:55:12.880 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-11-13T10:55:12.877+0000 7ff72d565640 -1 osd.1 0 waiting for initial osdmap 2023-11-13T10:55:12.888 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-11-13T10:55:12.882+0000 7ff72538c640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-13T10:55:12.932 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-11-13T10:55:12.928+0000 7f0e0d6cc640 -1 osd.3 0 waiting for initial osdmap 2023-11-13T10:55:12.938 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-11-13T10:55:12.935+0000 7f0e054f3640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-13T10:55:12.965 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-11-13T10:55:12.962+0000 7fdc6e98b640 -1 osd.0 0 waiting for initial osdmap 2023-11-13T10:55:12.982 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-11-13T10:55:12.978+0000 7fdc667b2640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-13T10:55:12.996 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-11-13T10:55:12.989+0000 7f60e851d640 -1 osd.6 0 waiting for initial osdmap 2023-11-13T10:55:13.002 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-11-13T10:55:12.997+0000 7f60dfb30640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-13T10:55:13.007 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-11-13T10:55:13.001+0000 7fc4ef0db640 -1 osd.5 0 waiting for initial osdmap 2023-11-13T10:55:13.021 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-11-13T10:55:13.004+0000 7f79973f5640 -1 osd.4 0 waiting for initial osdmap 2023-11-13T10:55:13.021 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-11-13T10:55:13.008+0000 7f798f21c640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-13T10:55:13.031 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-11-13T10:55:13.021+0000 7fc4e66ee640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-13T10:55:13.250 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:13.444 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.437+0000 7f818777d640 1 Processor -- start 2023-11-13T10:55:13.444 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.437+0000 7f818777d640 1 -- start start 2023-11-13T10:55:13.444 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.439+0000 7f818777d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8180155410 0x7f81800ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:13.444 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.439+0000 7f818777d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8180071b50 0x7f81800cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:13.445 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.439+0000 7f818777d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818015b500 con 0x7f81800ce990 2023-11-13T10:55:13.445 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.439+0000 7f818777d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818015b7e0 con 0x7f8180071b50 2023-11-13T10:55:13.445 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.439+0000 7f818777d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818015bac0 con 0x7f8180155410 2023-11-13T10:55:13.445 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.440+0000 7f8185d1a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8180071b50 0x7f81800cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:13.446 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.440+0000 7f8185d1a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8180071b50 0x7f81800cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59548/0 (socket says 172.21.15.16:59548) 2023-11-13T10:55:13.446 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.440+0000 7f8185d1a640 1 -- 172.21.15.16:0/3068192382 learned_addr learned my addr 172.21.15.16:0/3068192382 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:13.446 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.440+0000 7f8185519640 1 --2- 172.21.15.16:0/3068192382 >> v2:172.21.15.184:3300/0 conn(0x7f8180155410 0x7f81800ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:13.446 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.440+0000 7f8185d1a640 1 -- 172.21.15.16:0/3068192382 >> v2:172.21.15.184:3300/0 conn(0x7f8180155410 msgr2=0x7f81800ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:13.447 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.440+0000 7f8185d1a640 1 --2- 172.21.15.16:0/3068192382 >> v2:172.21.15.184:3300/0 conn(0x7f8180155410 0x7f81800ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:13.447 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.440+0000 7f8185d1a640 1 -- 172.21.15.16:0/3068192382 >> v1:172.21.15.16:6789/0 conn(0x7f81800ce990 legacy=0x7f8180071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:13.447 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.440+0000 7f8185d1a640 1 -- 172.21.15.16:0/3068192382 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818015bda0 con 0x7f8180071b50 2023-11-13T10:55:13.447 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.440+0000 7f8177fff640 1 -- 172.21.15.16:0/3068192382 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191011601 0 0) 0x7f818015b500 con 0x7f81800ce990 2023-11-13T10:55:13.453 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.441+0000 7f8185d1a640 1 --2- 172.21.15.16:0/3068192382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 0x7f81800cee40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f818006dee0 tx=0x7f817802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91aa96d066a7b28 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:13.453 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f8177fff640 1 -- 172.21.15.16:0/3068192382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817802cb20 con 0x7f8180071b50 2023-11-13T10:55:13.454 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f8177fff640 1 -- 172.21.15.16:0/3068192382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817802cc80 con 0x7f8180071b50 2023-11-13T10:55:13.454 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f8177fff640 1 -- 172.21.15.16:0/3068192382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780343f0 con 0x7f8180071b50 2023-11-13T10:55:13.454 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f818777d640 1 -- 172.21.15.16:0/3068192382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 msgr2=0x7f81800cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:13.454 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f818777d640 1 --2- 172.21.15.16:0/3068192382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 0x7f81800cee40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f818006dee0 tx=0x7f817802bf80 comp rx=0 tx=0).stop 2023-11-13T10:55:13.454 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f818777d640 1 -- 172.21.15.16:0/3068192382 shutdown_connections 2023-11-13T10:55:13.455 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f818777d640 1 --2- 172.21.15.16:0/3068192382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 0x7f81800cee40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:13.455 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f818777d640 1 --2- 172.21.15.16:0/3068192382 >> v2:172.21.15.184:3300/0 conn(0x7f8180155410 0x7f81800ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:13.455 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f818777d640 1 -- 172.21.15.16:0/3068192382 >> 172.21.15.16:0/3068192382 conn(0x7f818006a190 msgr2=0x7f818006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:13.455 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f818777d640 1 -- 172.21.15.16:0/3068192382 shutdown_connections 2023-11-13T10:55:13.455 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.442+0000 7f818777d640 1 -- 172.21.15.16:0/3068192382 wait complete. 2023-11-13T10:55:13.456 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.444+0000 7f818777d640 1 Processor -- start 2023-11-13T10:55:13.456 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.444+0000 7f818777d640 1 -- start start 2023-11-13T10:55:13.456 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.445+0000 7f818777d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 0x7f818013f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:13.456 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.445+0000 7f818777d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81800ce990 0x7f81801561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:13.456 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.445+0000 7f818777d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8180155410 0x7f8180156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:13.457 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.445+0000 7f818777d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818015aa10 con 0x7f8180071b50 2023-11-13T10:55:13.457 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.445+0000 7f818777d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180152490 con 0x7f8180155410 2023-11-13T10:55:13.457 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.445+0000 7f818777d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818015ae30 con 0x7f81800ce990 2023-11-13T10:55:13.457 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.445+0000 7f8185519640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 0x7f818013f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:13.457 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185519640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 0x7f818013f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59560/0 (socket says 172.21.15.16:59560) 2023-11-13T10:55:13.458 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185519640 1 -- 172.21.15.16:0/623183875 learned_addr learned my addr 172.21.15.16:0/623183875 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:13.458 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185d1a640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8180155410 0x7f8180156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:13.478 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8184d18640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81800ce990 0x7f81801561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:13.479 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185519640 1 -- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81800ce990 msgr2=0x7f81801561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:13.479 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185519640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81800ce990 0x7f81801561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:13.479 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185519640 1 -- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8180155410 msgr2=0x7f8180156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:13.479 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185519640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8180155410 0x7f8180156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:13.480 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185519640 1 -- 172.21.15.16:0/623183875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8178009b30 con 0x7f8180071b50 2023-11-13T10:55:13.480 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8184d18640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81800ce990 0x7f81801561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:13.480 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185d1a640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8180155410 0x7f8180156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:13.480 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.446+0000 7f8185519640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 0x7f818013f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f817c00e8a0 tx=0x7f817c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:13.481 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.447+0000 7f8175ffb640 1 -- 172.21.15.16:0/623183875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c0094f0 con 0x7f8180071b50 2023-11-13T10:55:13.481 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.447+0000 7f8175ffb640 1 -- 172.21.15.16:0/623183875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00c940 con 0x7f8180071b50 2023-11-13T10:55:13.481 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.447+0000 7f8175ffb640 1 -- 172.21.15.16:0/623183875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00ccf0 con 0x7f8180071b50 2023-11-13T10:55:13.481 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.447+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818015a070 con 0x7f8180071b50 2023-11-13T10:55:13.493 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.447+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818015a630 con 0x7f8180071b50 2023-11-13T10:55:13.494 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.448+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8180067090 con 0x7f8180071b50 2023-11-13T10:55:13.494 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.449+0000 7f8175ffb640 1 -- 172.21.15.16:0/623183875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f817c013470 con 0x7f8180071b50 2023-11-13T10:55:13.494 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.450+0000 7f8175ffb640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f816406adb0 0x7f816406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:13.508 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.450+0000 7f8175ffb640 1 -- 172.21.15.16:0/623183875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2511+0+0 (secure 0 0 0) 0x7f817c088990 con 0x7f8180071b50 2023-11-13T10:55:13.509 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.451+0000 7f8184d18640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f816406adb0 0x7f816406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:13.509 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.451+0000 7f8184d18640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f816406adb0 0x7f816406d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8180150ae0 tx=0x7f8170009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:13.515 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.455+0000 7f8175ffb640 1 -- 172.21.15.16:0/623183875 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f817c054790 con 0x7f8180071b50 2023-11-13T10:55:13.644 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.640+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81800ae1d0 con 0x7f8180071b50 2023-11-13T10:55:13.645 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.641+0000 7f8175ffb640 1 -- 172.21.15.16:0/623183875 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7566 (secure 0 0 0) 0x7f817c021020 con 0x7f8180071b50 2023-11-13T10:55:13.645 INFO:teuthology.misc.health.smithi016.stdout: 2023-11-13T10:55:13.645 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":14,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:12.262071+0000","last_up_change":"2023-11-13T10:55:12.262071+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:13.647 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f816406adb0 msgr2=0x7f816406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:13.648 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f816406adb0 0x7f816406d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8180150ae0 tx=0x7f8170009040 comp rx=0 tx=0).stop 2023-11-13T10:55:13.648 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 msgr2=0x7f818013f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:13.648 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 0x7f818013f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f817c00e8a0 tx=0x7f817c00ed60 comp rx=0 tx=0).stop 2023-11-13T10:55:13.648 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 shutdown_connections 2023-11-13T10:55:13.649 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f816406adb0 0x7f816406d270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:13.649 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8180155410 0x7f8180156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:13.649 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81800ce990 0x7f81801561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:13.649 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 --2- 172.21.15.16:0/623183875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180071b50 0x7f818013f1f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:13.649 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 >> 172.21.15.16:0/623183875 conn(0x7f818006a190 msgr2=0x7f81800b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:13.650 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 shutdown_connections 2023-11-13T10:55:13.650 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:13.644+0000 7f818777d640 1 -- 172.21.15.16:0/623183875 wait complete. 2023-11-13T10:55:13.661 DEBUG:teuthology.misc:1 of 8 OSDs are up 2023-11-13T10:55:16.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:55:16.371 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:55:20.468 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:20.664 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.659+0000 7f9b008cf640 1 Processor -- start 2023-11-13T10:55:20.664 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.660+0000 7f9b008cf640 1 -- start start 2023-11-13T10:55:20.667 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.661+0000 7f9b008cf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9afc155410 0x7f9afc0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:20.667 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.661+0000 7f9b008cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc0ce990 0x7f9afc0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:20.668 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.661+0000 7f9b008cf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc15b500 con 0x7f9afc071440 2023-11-13T10:55:20.668 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.661+0000 7f9b008cf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc15b7e0 con 0x7f9afc155410 2023-11-13T10:55:20.668 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.661+0000 7f9b008cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc15bac0 con 0x7f9afc0ce990 2023-11-13T10:55:20.668 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.662+0000 7f9afa59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9afc155410 0x7f9afc0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:20.668 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.662+0000 7f9afad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9afc071440 0x7f9afc06fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45382/0 (socket says 172.21.15.16:45382) 2023-11-13T10:55:20.669 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.662+0000 7f9afa59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9afc155410 0x7f9afc0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59572/0 (socket says 172.21.15.16:59572) 2023-11-13T10:55:20.669 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.662+0000 7f9afad9d640 1 -- 172.21.15.16:0/3506902728 learned_addr learned my addr 172.21.15.16:0/3506902728 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:20.669 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.662+0000 7f9af9d9b640 1 --2- 172.21.15.16:0/3506902728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc0ce990 0x7f9afc0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:20.670 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.662+0000 7f9afa59c640 1 -- 172.21.15.16:0/3506902728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc0ce990 msgr2=0x7f9afc0cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:20.670 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.662+0000 7f9afa59c640 1 --2- 172.21.15.16:0/3506902728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc0ce990 0x7f9afc0cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.671 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.662+0000 7f9afa59c640 1 -- 172.21.15.16:0/3506902728 >> v1:172.21.15.16:6789/0 conn(0x7f9afc071440 legacy=0x7f9afc06fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:20.671 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.662+0000 7f9afa59c640 1 -- 172.21.15.16:0/3506902728 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc15bda0 con 0x7f9afc155410 2023-11-13T10:55:20.679 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.664+0000 7f9afa59c640 1 --2- 172.21.15.16:0/3506902728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 0x7f9afc0ccc60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009d90 tx=0x7f9aec02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15525c564f0f4c07 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:20.679 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.664+0000 7f9af959a640 1 -- 172.21.15.16:0/3506902728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec02cc50 con 0x7f9afc155410 2023-11-13T10:55:20.679 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.664+0000 7f9af959a640 1 -- 172.21.15.16:0/3506902728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec02cdb0 con 0x7f9afc155410 2023-11-13T10:55:20.679 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.664+0000 7f9af959a640 1 -- 172.21.15.16:0/3506902728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec034660 con 0x7f9afc155410 2023-11-13T10:55:20.679 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.664+0000 7f9b008cf640 1 -- 172.21.15.16:0/3506902728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 msgr2=0x7f9afc0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:20.680 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.664+0000 7f9b008cf640 1 --2- 172.21.15.16:0/3506902728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 0x7f9afc0ccc60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009d90 tx=0x7f9aec02bea0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.680 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.665+0000 7f9b008cf640 1 -- 172.21.15.16:0/3506902728 shutdown_connections 2023-11-13T10:55:20.680 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.665+0000 7f9b008cf640 1 --2- 172.21.15.16:0/3506902728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc0ce990 0x7f9afc0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.680 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.665+0000 7f9b008cf640 1 --2- 172.21.15.16:0/3506902728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 0x7f9afc0ccc60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.680 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.665+0000 7f9b008cf640 1 -- 172.21.15.16:0/3506902728 >> 172.21.15.16:0/3506902728 conn(0x7f9afc06a190 msgr2=0x7f9afc06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:20.688 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.665+0000 7f9b008cf640 1 -- 172.21.15.16:0/3506902728 shutdown_connections 2023-11-13T10:55:20.688 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.665+0000 7f9b008cf640 1 -- 172.21.15.16:0/3506902728 wait complete. 2023-11-13T10:55:20.688 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.667+0000 7f9b008cf640 1 Processor -- start 2023-11-13T10:55:20.688 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.667+0000 7f9b008cf640 1 -- start start 2023-11-13T10:55:20.689 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9b008cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc071440 0x7f9afc14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:20.689 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9b008cf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9afc0ce990 0x7f9afc153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:20.689 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9b008cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 0x7f9afc14b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:20.689 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9b008cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc14ea30 con 0x7f9afc155410 2023-11-13T10:55:20.689 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9b008cf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc14f390 con 0x7f9afc0ce990 2023-11-13T10:55:20.690 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9b008cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc14ddb0 con 0x7f9afc071440 2023-11-13T10:55:20.690 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9afad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 0x7f9afc14b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:20.690 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9afad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 0x7f9afc14b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59582/0 (socket says 172.21.15.16:59582) 2023-11-13T10:55:20.690 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9afad9d640 1 -- 172.21.15.16:0/2745023849 learned_addr learned my addr 172.21.15.16:0/2745023849 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:20.690 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9afa59c640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc071440 0x7f9afc14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:20.691 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9af9d9b640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9afc0ce990 0x7f9afc153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:20.691 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9afad9d640 1 -- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc071440 msgr2=0x7f9afc14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:20.702 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9afad9d640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc071440 0x7f9afc14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.702 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9afad9d640 1 -- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9afc0ce990 msgr2=0x7f9afc153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:20.703 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9afad9d640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9afc0ce990 0x7f9afc153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.703 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.668+0000 7f9afad9d640 1 -- 172.21.15.16:0/2745023849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec009a70 con 0x7f9afc155410 2023-11-13T10:55:20.703 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.669+0000 7f9afad9d640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 0x7f9afc14b390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c420 tx=0x7f9af400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:20.703 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.669+0000 7f9af2ffd640 1 -- 172.21.15.16:0/2745023849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4005ce0 con 0x7f9afc155410 2023-11-13T10:55:20.711 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.669+0000 7f9af2ffd640 1 -- 172.21.15.16:0/2745023849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4005e40 con 0x7f9afc155410 2023-11-13T10:55:20.711 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.669+0000 7f9af2ffd640 1 -- 172.21.15.16:0/2745023849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40072d0 con 0x7f9afc155410 2023-11-13T10:55:20.712 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.669+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc14e090 con 0x7f9afc155410 2023-11-13T10:55:20.712 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.669+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc14e650 con 0x7f9afc155410 2023-11-13T10:55:20.712 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.670+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc067090 con 0x7f9afc155410 2023-11-13T10:55:20.712 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.671+0000 7f9af2ffd640 1 -- 172.21.15.16:0/2745023849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9af40074f0 con 0x7f9afc155410 2023-11-13T10:55:20.712 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.672+0000 7f9af2ffd640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9adc06ae40 0x7f9adc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:20.713 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.672+0000 7f9af2ffd640 1 -- 172.21.15.16:0/2745023849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9af4088650 con 0x7f9afc155410 2023-11-13T10:55:20.713 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.673+0000 7f9afa59c640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9adc06ae40 0x7f9adc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:20.713 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.674+0000 7f9afa59c640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9adc06ae40 0x7f9adc06d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9aec036000 tx=0x7f9aec030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:20.714 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.678+0000 7f9af2ffd640 1 -- 172.21.15.16:0/2745023849 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9af4053b90 con 0x7f9afc155410 2023-11-13T10:55:20.881 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.876+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9afc0af350 con 0x7f9afc155410 2023-11-13T10:55:20.882 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.878+0000 7f9af2ffd640 1 -- 172.21.15.16:0/2745023849 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12862 (secure 0 0 0) 0x7f9af4017090 con 0x7f9afc155410 2023-11-13T10:55:20.883 INFO:teuthology.misc.health.smithi016.stdout: 2023-11-13T10:55:20.883 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":18,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:16.902265+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:20.886 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9adc06ae40 msgr2=0x7f9adc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:20.886 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9adc06ae40 0x7f9adc06d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9aec036000 tx=0x7f9aec030000 comp rx=0 tx=0).stop 2023-11-13T10:55:20.886 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 msgr2=0x7f9afc14b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:20.887 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 0x7f9afc14b390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9af400c420 tx=0x7f9af400c8e0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.887 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 shutdown_connections 2023-11-13T10:55:20.887 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9adc06ae40 0x7f9adc06d300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.887 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155410 0x7f9afc14b390 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.887 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9afc0ce990 0x7f9afc153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.888 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 --2- 172.21.15.16:0/2745023849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9afc071440 0x7f9afc14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:20.888 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 >> 172.21.15.16:0/2745023849 conn(0x7f9afc06a190 msgr2=0x7f9afc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:20.888 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.881+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 shutdown_connections 2023-11-13T10:55:20.888 INFO:teuthology.misc.health.smithi016.stderr:2023-11-13T10:55:20.882+0000 7f9b008cf640 1 -- 172.21.15.16:0/2745023849 wait complete. 2023-11-13T10:55:20.901 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-13T10:55:20.902 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-13T10:55:20.902 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-13T10:55:20.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-13T10:55:20.912 INFO:tasks.ceph.mds.a:Started 2023-11-13T10:55:20.912 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-13T10:55:20.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-13T10:55:20.925 INFO:tasks.ceph.mds.c:Started 2023-11-13T10:55:20.926 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-13T10:55:20.926 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-13T10:55:20.941 INFO:tasks.ceph.mds.b:Started 2023-11-13T10:55:20.942 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-13T10:55:20.942 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-13T10:55:20.960 INFO:tasks.ceph.mds.d:Started 2023-11-13T10:55:20.960 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-13T10:55:20.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:21.072 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-13T10:55:21.100 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-13T10:55:21.112 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-11-13T10:55:21.118 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-11-13T10:55:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.153+0000 7f6296e61640 1 Processor -- start 2023-11-13T10:55:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.153+0000 7f6296e61640 1 -- start start 2023-11-13T10:55:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.154+0000 7f6296e61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f629015d0f0 0x7f62900d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.154+0000 7f6296e61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62900cfe20 0x7f62900d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.154+0000 7f6296e61640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62900d3c70 con 0x7f62900d0ac0 2023-11-13T10:55:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.154+0000 7f6296e61640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900c69d0 con 0x7f62900cfe20 2023-11-13T10:55:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.154+0000 7f6296e61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62900c6cb0 con 0x7f629015d0f0 2023-11-13T10:55:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.155+0000 7f628ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62900cfe20 0x7f62900d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.155+0000 7f6294bfd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f629015d0f0 0x7f62900d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.155+0000 7f62953fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f62900d0ac0 0x7f62900b3950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45398/0 (socket says 172.21.15.16:45398) 2023-11-13T10:55:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.155+0000 7f62953fe640 1 -- 172.21.15.16:0/317335834 learned_addr learned my addr 172.21.15.16:0/317335834 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.155+0000 7f628ffff640 1 -- 172.21.15.16:0/317335834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f629015d0f0 msgr2=0x7f62900d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.155+0000 7f628ffff640 1 --2- 172.21.15.16:0/317335834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f629015d0f0 0x7f62900d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.155+0000 7f628ffff640 1 -- 172.21.15.16:0/317335834 >> v1:172.21.15.16:6789/0 conn(0x7f62900d0ac0 legacy=0x7f62900b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.155+0000 7f628f7fe640 1 -- 172.21.15.16:0/317335834 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596209264 0 0) 0x7f6288001490 con 0x7f62900d0ac0 2023-11-13T10:55:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.155+0000 7f628ffff640 1 -- 172.21.15.16:0/317335834 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900c6f90 con 0x7f62900cfe20 2023-11-13T10:55:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.156+0000 7f628ffff640 1 --2- 172.21.15.16:0/317335834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62900cfe20 0x7f62900d1800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f627c0097a0 tx=0x7f627c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7403896bd859bc77 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:21.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f628f7fe640 1 -- 172.21.15.16:0/317335834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c030610 con 0x7f62900cfe20 2023-11-13T10:55:21.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f628f7fe640 1 -- 172.21.15.16:0/317335834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c030770 con 0x7f62900cfe20 2023-11-13T10:55:21.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f628f7fe640 1 -- 172.21.15.16:0/317335834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c030b00 con 0x7f62900cfe20 2023-11-13T10:55:21.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f6296e61640 1 -- 172.21.15.16:0/317335834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62900cfe20 msgr2=0x7f62900d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f6296e61640 1 --2- 172.21.15.16:0/317335834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62900cfe20 0x7f62900d1800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f627c0097a0 tx=0x7f627c02b960 comp rx=0 tx=0).stop 2023-11-13T10:55:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f6296e61640 1 -- 172.21.15.16:0/317335834 shutdown_connections 2023-11-13T10:55:21.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f6296e61640 1 --2- 172.21.15.16:0/317335834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62900cfe20 0x7f62900d1800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f6296e61640 1 --2- 172.21.15.16:0/317335834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f629015d0f0 0x7f62900d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f6296e61640 1 -- 172.21.15.16:0/317335834 >> 172.21.15.16:0/317335834 conn(0x7f62900d2ce0 msgr2=0x7f62900d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:21.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f6296e61640 1 -- 172.21.15.16:0/317335834 shutdown_connections 2023-11-13T10:55:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.157+0000 7f6296e61640 1 -- 172.21.15.16:0/317335834 wait complete. 2023-11-13T10:55:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.159+0000 7f6296e61640 1 Processor -- start 2023-11-13T10:55:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.159+0000 7f6296e61640 1 -- start start 2023-11-13T10:55:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.160+0000 7f6296e61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62900cfe20 0x7f62900b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.160+0000 7f6296e61640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62900d0ac0 0x7f629015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.160+0000 7f6296e61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629015d0f0 0x7f629006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.160+0000 7f6296e61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62901e6550 con 0x7f629015d0f0 2023-11-13T10:55:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.160+0000 7f6296e61640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62901e6830 con 0x7f62900d0ac0 2023-11-13T10:55:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.160+0000 7f6296e61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62901e6b10 con 0x7f62900cfe20 2023-11-13T10:55:21.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.160+0000 7f62953fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629015d0f0 0x7f629006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.160+0000 7f62953fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629015d0f0 0x7f629006ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59634/0 (socket says 172.21.15.16:59634) 2023-11-13T10:55:21.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.160+0000 7f62953fe640 1 -- 172.21.15.16:0/3253706311 learned_addr learned my addr 172.21.15.16:0/3253706311 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f6294bfd640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62900cfe20 0x7f62900b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f628ffff640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62900d0ac0 0x7f629015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f62953fe640 1 -- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62900cfe20 msgr2=0x7f62900b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f62953fe640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62900cfe20 0x7f62900b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f62953fe640 1 -- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62900d0ac0 msgr2=0x7f629015b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f62953fe640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62900d0ac0 0x7f629015b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f62953fe640 1 -- 172.21.15.16:0/3253706311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c009400 con 0x7f629015d0f0 2023-11-13T10:55:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f628ffff640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62900d0ac0 0x7f629015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f62953fe640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629015d0f0 0x7f629006ba50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f628800c7f0 tx=0x7f628800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.161+0000 7f628d7fa640 1 -- 172.21.15.16:0/3253706311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800a640 con 0x7f629015d0f0 2023-11-13T10:55:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.162+0000 7f628d7fa640 1 -- 172.21.15.16:0/3253706311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628800a7c0 con 0x7f629015d0f0 2023-11-13T10:55:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.162+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62901e6df0 con 0x7f629015d0f0 2023-11-13T10:55:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.162+0000 7f628d7fa640 1 -- 172.21.15.16:0/3253706311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800ab70 con 0x7f629015d0f0 2023-11-13T10:55:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.162+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62900b7d60 con 0x7f629015d0f0 2023-11-13T10:55:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.163+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62901e7310 con 0x7f629015d0f0 2023-11-13T10:55:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.164+0000 7f628d7fa640 1 -- 172.21.15.16:0/3253706311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f628800acd0 con 0x7f629015d0f0 2023-11-13T10:55:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.165+0000 7f628d7fa640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f627406ae40 0x7f627406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.165+0000 7f628d7fa640 1 -- 172.21.15.16:0/3253706311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6288088620 con 0x7f629015d0f0 2023-11-13T10:55:21.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.165+0000 7f6294bfd640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f627406ae40 0x7f627406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.166+0000 7f6294bfd640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f627406ae40 0x7f627406d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6284009a30 tx=0x7f6284008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:21.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.170+0000 7f628d7fa640 1 -- 172.21.15.16:0/3253706311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62901e7310 con 0x7f629015d0f0 2023-11-13T10:55:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.363+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62900d3c70 con 0x7f629015d0f0 2023-11-13T10:55:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.364+0000 7f628d7fa640 1 -- 172.21.15.16:0/3253706311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12862 (secure 0 0 0) 0x7f62900d3c70 con 0x7f629015d0f0 2023-11-13T10:55:21.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:21.370 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:16.902265+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.366+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f627406ae40 msgr2=0x7f627406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.366+0000 7f6296e61640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f627406ae40 0x7f627406d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6284009a30 tx=0x7f6284008040 comp rx=0 tx=0).stop 2023-11-13T10:55:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.366+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629015d0f0 msgr2=0x7f629006ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.366+0000 7f6296e61640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629015d0f0 0x7f629006ba50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f628800c7f0 tx=0x7f628800ccb0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.366+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 shutdown_connections 2023-11-13T10:55:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.366+0000 7f6296e61640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f627406ae40 0x7f627406d300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.366+0000 7f6296e61640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629015d0f0 0x7f629006ba50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.366+0000 7f6296e61640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62900d0ac0 0x7f629015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.366+0000 7f6296e61640 1 --2- 172.21.15.16:0/3253706311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62900cfe20 0x7f62900b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.367+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 >> 172.21.15.16:0/3253706311 conn(0x7f62900d2ce0 msgr2=0x7f62900ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.367+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 shutdown_connections 2023-11-13T10:55:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.367+0000 7f6296e61640 1 -- 172.21.15.16:0/3253706311 wait complete. 2023-11-13T10:55:21.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T10:55:21.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:55:21.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.627+0000 7f906a259640 1 Processor -- start 2023-11-13T10:55:21.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.627+0000 7f906a259640 1 -- start start 2023-11-13T10:55:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.629+0000 7f906a259640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d1850 0x7f90640d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.629+0000 7f906a259640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90640d2d00 0x7f90640c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.629+0000 7f906a259640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90640ce0e0 con 0x7f906415ce70 2023-11-13T10:55:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.629+0000 7f906a259640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90640ce3c0 con 0x7f90640d2d00 2023-11-13T10:55:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.629+0000 7f906a259640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90640ce6a0 con 0x7f90640d1850 2023-11-13T10:55:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.629+0000 7f9069a58640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90640d2d00 0x7f90640c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.629+0000 7f9069a58640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90640d2d00 0x7f90640c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59644/0 (socket says 172.21.15.16:59644) 2023-11-13T10:55:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.629+0000 7f9069a58640 1 -- 172.21.15.16:0/573037944 learned_addr learned my addr 172.21.15.16:0/573037944 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.629+0000 7f9068a56640 1 --2- 172.21.15.16:0/573037944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d1850 0x7f90640d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.630+0000 7f9069a58640 1 -- 172.21.15.16:0/573037944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d1850 msgr2=0x7f90640d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.630+0000 7f9069a58640 1 --2- 172.21.15.16:0/573037944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d1850 0x7f90640d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.630+0000 7f9069a58640 1 -- 172.21.15.16:0/573037944 >> v1:172.21.15.184:6789/0 conn(0x7f906415ce70 legacy=0x7f90640cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.630+0000 7f9069a58640 1 -- 172.21.15.16:0/573037944 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90640ce980 con 0x7f90640d2d00 2023-11-13T10:55:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.631+0000 7f9069a58640 1 --2- 172.21.15.16:0/573037944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90640d2d00 0x7f90640c9c30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f905c009e50 tx=0x7f905c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f5b12810ae6178 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.631+0000 7f905bfff640 1 -- 172.21.15.16:0/573037944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c02ca80 con 0x7f90640d2d00 2023-11-13T10:55:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.631+0000 7f905bfff640 1 -- 172.21.15.16:0/573037944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c02cbe0 con 0x7f90640d2d00 2023-11-13T10:55:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.631+0000 7f905bfff640 1 -- 172.21.15.16:0/573037944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0343f0 con 0x7f90640d2d00 2023-11-13T10:55:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.632+0000 7f906a259640 1 -- 172.21.15.16:0/573037944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90640d2d00 msgr2=0x7f90640c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.632+0000 7f906a259640 1 --2- 172.21.15.16:0/573037944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90640d2d00 0x7f90640c9c30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f905c009e50 tx=0x7f905c02bee0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.632+0000 7f906a259640 1 -- 172.21.15.16:0/573037944 shutdown_connections 2023-11-13T10:55:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.632+0000 7f906a259640 1 --2- 172.21.15.16:0/573037944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90640d2d00 0x7f90640c9c30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.632+0000 7f906a259640 1 --2- 172.21.15.16:0/573037944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d1850 0x7f90640d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.632+0000 7f906a259640 1 -- 172.21.15.16:0/573037944 >> 172.21.15.16:0/573037944 conn(0x7f906408c580 msgr2=0x7f906409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:21.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.632+0000 7f906a259640 1 -- 172.21.15.16:0/573037944 shutdown_connections 2023-11-13T10:55:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.632+0000 7f906a259640 1 -- 172.21.15.16:0/573037944 wait complete. 2023-11-13T10:55:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.634+0000 7f906a259640 1 Processor -- start 2023-11-13T10:55:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.635+0000 7f906a259640 1 -- start start 2023-11-13T10:55:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.635+0000 7f906a259640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90640d1850 0x7f90641f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.635+0000 7f906a259640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d2d00 0x7f90640c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f906a259640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906415ce70 0x7f90640c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f906a259640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90640c9220 con 0x7f906415ce70 2023-11-13T10:55:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f906a259640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906415e4c0 con 0x7f90640d1850 2023-11-13T10:55:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f906a259640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90640c9430 con 0x7f90640d2d00 2023-11-13T10:55:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9069a58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906415ce70 0x7f90640c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9069a58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906415ce70 0x7f90640c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59656/0 (socket says 172.21.15.16:59656) 2023-11-13T10:55:21.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9069a58640 1 -- 172.21.15.16:0/1562562570 learned_addr learned my addr 172.21.15.16:0/1562562570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:21.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9069257640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90640d1850 0x7f90641f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9068a56640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d2d00 0x7f90640c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9069a58640 1 -- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d2d00 msgr2=0x7f90640c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9069a58640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d2d00 0x7f90640c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9069a58640 1 -- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90640d1850 msgr2=0x7f90641f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9069a58640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90640d1850 0x7f90641f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9069a58640 1 -- 172.21.15.16:0/1562562570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c009b30 con 0x7f906415ce70 2023-11-13T10:55:21.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.636+0000 7f9068a56640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d2d00 0x7f90640c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.637+0000 7f9069a58640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906415ce70 0x7f90640c6290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f905c009e50 tx=0x7f905c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.637+0000 7f9059ffb640 1 -- 172.21.15.16:0/1562562570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c009510 con 0x7f906415ce70 2023-11-13T10:55:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.637+0000 7f9059ffb640 1 -- 172.21.15.16:0/1562562570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c03e450 con 0x7f906415ce70 2023-11-13T10:55:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.637+0000 7f9059ffb640 1 -- 172.21.15.16:0/1562562570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c03e800 con 0x7f906415ce70 2023-11-13T10:55:21.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.637+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906415e760 con 0x7f906415ce70 2023-11-13T10:55:21.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.637+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90640c9710 con 0x7f906415ce70 2023-11-13T10:55:21.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.638+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906415ec80 con 0x7f906415ce70 2023-11-13T10:55:21.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.639+0000 7f9059ffb640 1 -- 172.21.15.16:0/1562562570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f905c02cd50 con 0x7f906415ce70 2023-11-13T10:55:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.640+0000 7f9059ffb640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f904806ae70 0x7f904806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.640+0000 7f9059ffb640 1 -- 172.21.15.16:0/1562562570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f905c0abe20 con 0x7f906415ce70 2023-11-13T10:55:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.641+0000 7f9069257640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f904806ae70 0x7f904806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.642+0000 7f9069257640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f904806ae70 0x7f904806d330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9060004ea0 tx=0x7f9060009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:21.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.644+0000 7f9059ffb640 1 -- 172.21.15.16:0/1562562570 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f906415ec80 con 0x7f906415ce70 2023-11-13T10:55:21.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.839+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f906415ec80 con 0x7f906415ce70 2023-11-13T10:55:21.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.840+0000 7f9059ffb640 1 -- 172.21.15.16:0/1562562570 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f906415ec80 con 0x7f906415ce70 2023-11-13T10:55:21.845 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:55:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.842+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f904806ae70 msgr2=0x7f904806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.842+0000 7f906a259640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f904806ae70 0x7f904806d330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9060004ea0 tx=0x7f9060009040 comp rx=0 tx=0).stop 2023-11-13T10:55:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.842+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906415ce70 msgr2=0x7f90640c6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.842+0000 7f906a259640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906415ce70 0x7f90640c6290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f905c009e50 tx=0x7f905c004a40 comp rx=0 tx=0).stop 2023-11-13T10:55:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.842+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 shutdown_connections 2023-11-13T10:55:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.842+0000 7f906a259640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f904806ae70 0x7f904806d330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.843+0000 7f906a259640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906415ce70 0x7f90640c6290 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.843+0000 7f906a259640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90640d2d00 0x7f90640c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.843+0000 7f906a259640 1 --2- 172.21.15.16:0/1562562570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90640d1850 0x7f90641f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.843+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 >> 172.21.15.16:0/1562562570 conn(0x7f906408c580 msgr2=0x7f90640cafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.843+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 shutdown_connections 2023-11-13T10:55:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:21.843+0000 7f906a259640 1 -- 172.21.15.16:0/1562562570 wait complete. 2023-11-13T10:55:21.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:55:22.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.091+0000 7f24aafcc640 1 Processor -- start 2023-11-13T10:55:22.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.091+0000 7f24aafcc640 1 -- start start 2023-11-13T10:55:22.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.093+0000 7f24aafcc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a4070650 0x7f24a406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.093+0000 7f24aafcc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a40b3a80 con 0x7f24a406fb60 2023-11-13T10:55:22.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.093+0000 7f24aafcc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a4161880 con 0x7f24a40ae030 2023-11-13T10:55:22.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.093+0000 7f24aafcc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a4161b60 con 0x7f24a4070650 2023-11-13T10:55:22.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.093+0000 7f24a8d68640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24a406fb60 0x7f24a406ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45406/0 (socket says 172.21.15.16:45406) 2023-11-13T10:55:22.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.093+0000 7f24a8d68640 1 -- 172.21.15.16:0/816929045 learned_addr learned my addr 172.21.15.16:0/816929045 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.093+0000 7f24a9569640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a4070650 0x7f24a406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055893545 0 0) 0x7f24a40b3a80 con 0x7f24a406fb60 2023-11-13T10:55:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f248c003520 con 0x7f24a406fb60 2023-11-13T10:55:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1253449031 0 0) 0x7f248c003520 con 0x7f24a406fb60 2023-11-13T10:55:22.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a4070650 msgr2=0x7f24a406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:22.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a4070650 0x7f24a406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 >> v1:172.21.15.184:6789/0 conn(0x7f24a40ae030 legacy=0x7f24a406c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4161e40 con 0x7f24a406fb60 2023-11-13T10:55:22.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f24a0001670 con 0x7f24a406fb60 2023-11-13T10:55:22.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 >> v1:172.21.15.16:6789/0 conn(0x7f24a406fb60 legacy=0x7f24a406ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f248c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a4070650 0x7f248c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c004d00 0x7f248c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a4161b60 con 0x7f248c0039a0 2023-11-13T10:55:22.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c00ba60 con 0x7f248c004d00 2023-11-13T10:55:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c00bc20 con 0x7f24a4070650 2023-11-13T10:55:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.094+0000 7f24a8d68640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f248c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.095+0000 7f249bfff640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a4070650 0x7f248c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.095+0000 7f24a9569640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c004d00 0x7f248c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.095+0000 7f24a8d68640 1 -- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a4070650 msgr2=0x7f248c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.095+0000 7f24a8d68640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a4070650 0x7f248c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.095+0000 7f24a8d68640 1 -- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c004d00 msgr2=0x7f248c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.095+0000 7f24a8d68640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c004d00 0x7f248c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.095+0000 7f24a8d68640 1 -- 172.21.15.16:0/816929045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c003520 con 0x7f248c0039a0 2023-11-13T10:55:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.096+0000 7f24a8d68640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f248c003dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f24a0001eb0 tx=0x7f248c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acfb61fea3e5896 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.096+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0005ad0 con 0x7f248c0039a0 2023-11-13T10:55:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.096+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0005c30 con 0x7f248c0039a0 2023-11-13T10:55:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.096+0000 7f249b7fe640 1 -- 172.21.15.16:0/816929045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a002ca00 con 0x7f248c0039a0 2023-11-13T10:55:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.097+0000 7f24aafcc640 1 -- 172.21.15.16:0/816929045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 msgr2=0x7f248c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.097+0000 7f24aafcc640 1 --2- 172.21.15.16:0/816929045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f248c003dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f24a0001eb0 tx=0x7f248c0057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.097+0000 7f24a9569640 1 -- 172.21.15.16:0/816929045 reap_dead start 2023-11-13T10:55:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.097+0000 7f24aafcc640 1 -- 172.21.15.16:0/816929045 shutdown_connections 2023-11-13T10:55:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.097+0000 7f24aafcc640 1 -- 172.21.15.16:0/816929045 >> 172.21.15.16:0/816929045 conn(0x7f24a406a190 msgr2=0x7f24a406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.097+0000 7f24aafcc640 1 -- 172.21.15.16:0/816929045 shutdown_connections 2023-11-13T10:55:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.097+0000 7f24aafcc640 1 -- 172.21.15.16:0/816929045 wait complete. 2023-11-13T10:55:22.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.099+0000 7f24aafcc640 1 Processor -- start 2023-11-13T10:55:22.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.099+0000 7f24aafcc640 1 -- start start 2023-11-13T10:55:22.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24aafcc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f24a40cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24aafcc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24a4146230 0x7f24a41466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24aafcc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a414c390 0x7f24a414c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24aafcc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a4146bf0 con 0x7f248c0039a0 2023-11-13T10:55:22.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24aafcc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a414cd50 con 0x7f24a4146230 2023-11-13T10:55:22.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24aafcc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a4159140 con 0x7f24a414c390 2023-11-13T10:55:22.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24a8d68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f24a40cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24a8d68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f24a40cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59676/0 (socket says 172.21.15.16:59676) 2023-11-13T10:55:22.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24a8d68640 1 -- 172.21.15.16:0/1554936786 learned_addr learned my addr 172.21.15.16:0/1554936786 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:22.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f249bfff640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24a4146230 0x7f24a41466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.100+0000 7f24a9569640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a414c390 0x7f24a414c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24a8d68640 1 -- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a414c390 msgr2=0x7f24a414c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24a8d68640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a414c390 0x7f24a414c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24a8d68640 1 -- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24a4146230 msgr2=0x7f24a41466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24a8d68640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24a4146230 0x7f24a41466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24a8d68640 1 -- 172.21.15.16:0/1554936786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4146f20 con 0x7f248c0039a0 2023-11-13T10:55:22.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f249bfff640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24a4146230 0x7f24a41466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:22.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24a8d68640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f24a40cb1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f24a0001c40 tx=0x7f24a0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:22.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24997fa640 1 -- 172.21.15.16:0/1554936786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a002ca90 con 0x7f248c0039a0 2023-11-13T10:55:22.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24997fa640 1 -- 172.21.15.16:0/1554936786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a002cbf0 con 0x7f248c0039a0 2023-11-13T10:55:22.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24997fa640 1 -- 172.21.15.16:0/1554936786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0007070 con 0x7f248c0039a0 2023-11-13T10:55:22.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.101+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a40b0e40 con 0x7f248c0039a0 2023-11-13T10:55:22.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.102+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a41471c0 con 0x7f248c0039a0 2023-11-13T10:55:22.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.103+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a406db80 con 0x7f248c0039a0 2023-11-13T10:55:22.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.104+0000 7f24997fa640 1 -- 172.21.15.16:0/1554936786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24a0032070 con 0x7f248c0039a0 2023-11-13T10:55:22.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.104+0000 7f24997fa640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f249006ae70 0x7f249006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.105+0000 7f24997fa640 1 -- 172.21.15.16:0/1554936786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f24a00aa900 con 0x7f248c0039a0 2023-11-13T10:55:22.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.105+0000 7f249bfff640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f249006ae70 0x7f249006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.106+0000 7f249bfff640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f249006ae70 0x7f249006d330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2494000c00 tx=0x7f2494009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:22.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.109+0000 7f24997fa640 1 -- 172.21.15.16:0/1554936786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24a0075c60 con 0x7f248c0039a0 2023-11-13T10:55:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.341+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24a406db50 con 0x7f248c0039a0 2023-11-13T10:55:22.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.342+0000 7f24997fa640 1 -- 172.21.15.16:0/1554936786 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f24a0079280 con 0x7f248c0039a0 2023-11-13T10:55:22.349 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:22.350 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-13T10:55:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2023-11-13T10:55:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f249006ae70 msgr2=0x7f249006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f249006ae70 0x7f249006d330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2494000c00 tx=0x7f2494009040 comp rx=0 tx=0).stop 2023-11-13T10:55:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 msgr2=0x7f24a40cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f24a40cb1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f24a0001c40 tx=0x7f24a0001930 comp rx=0 tx=0).stop 2023-11-13T10:55:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 shutdown_connections 2023-11-13T10:55:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f249006ae70 0x7f249006d330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24a414c390 0x7f24a414c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24a4146230 0x7f24a41466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 --2- 172.21.15.16:0/1554936786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248c0039a0 0x7f24a40cb1d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 >> 172.21.15.16:0/1554936786 conn(0x7f24a406a190 msgr2=0x7f24a40712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 shutdown_connections 2023-11-13T10:55:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.345+0000 7f24aafcc640 1 -- 172.21.15.16:0/1554936786 wait complete. 2023-11-13T10:55:22.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:22.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.603+0000 7fec7186d640 1 Processor -- start 2023-11-13T10:55:22.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.604+0000 7fec7186d640 1 -- start start 2023-11-13T10:55:22.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.605+0000 7fec7186d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fec6c15cfc0 0x7fec6c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.605+0000 7fec7186d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fec6c0d2ae0 0x7fec6c0d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.605+0000 7fec7186d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c0b3cf0 con 0x7fec6c0b29d0 2023-11-13T10:55:22.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.605+0000 7fec7186d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c0b3550 con 0x7fec6c0d2ae0 2023-11-13T10:55:22.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.605+0000 7fec7186d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c0d3480 con 0x7fec6c15cfc0 2023-11-13T10:55:22.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.606+0000 7fec7106c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fec6c0b29d0 0x7fec6c0b2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45408/0 (socket says 172.21.15.16:45408) 2023-11-13T10:55:22.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.606+0000 7fec7106c640 1 -- 172.21.15.16:0/4082012888 learned_addr learned my addr 172.21.15.16:0/4082012888 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:22.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.606+0000 7fec63fff640 1 --2- 172.21.15.16:0/4082012888 >> v2:172.21.15.16:3300/0 conn(0x7fec6c0d2ae0 0x7fec6c0d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.606+0000 7fec7086b640 1 --2- 172.21.15.16:0/4082012888 >> v2:172.21.15.184:3300/0 conn(0x7fec6c15cfc0 0x7fec6c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.606+0000 7fec63fff640 1 -- 172.21.15.16:0/4082012888 >> v2:172.21.15.184:3300/0 conn(0x7fec6c15cfc0 msgr2=0x7fec6c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.606+0000 7fec63fff640 1 --2- 172.21.15.16:0/4082012888 >> v2:172.21.15.184:3300/0 conn(0x7fec6c15cfc0 0x7fec6c0cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.606+0000 7fec637fe640 1 -- 172.21.15.16:0/4082012888 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887667247 0 0) 0x7fec6c0b3cf0 con 0x7fec6c0b29d0 2023-11-13T10:55:22.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.606+0000 7fec63fff640 1 -- 172.21.15.16:0/4082012888 >> v1:172.21.15.16:6789/0 conn(0x7fec6c0b29d0 legacy=0x7fec6c0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.606+0000 7fec63fff640 1 -- 172.21.15.16:0/4082012888 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0b3730 con 0x7fec6c0d2ae0 2023-11-13T10:55:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.607+0000 7fec63fff640 1 --2- 172.21.15.16:0/4082012888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c0d2ae0 0x7fec6c0d2f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0d0680 tx=0x7fec5c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec21e2e098e83abe server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec637fe640 1 -- 172.21.15.16:0/4082012888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c02e070 con 0x7fec6c0d2ae0 2023-11-13T10:55:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec637fe640 1 -- 172.21.15.16:0/4082012888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c0315a0 con 0x7fec6c0d2ae0 2023-11-13T10:55:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec637fe640 1 -- 172.21.15.16:0/4082012888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c031930 con 0x7fec6c0d2ae0 2023-11-13T10:55:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec7186d640 1 -- 172.21.15.16:0/4082012888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c0d2ae0 msgr2=0x7fec6c0d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec7186d640 1 --2- 172.21.15.16:0/4082012888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c0d2ae0 0x7fec6c0d2f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0d0680 tx=0x7fec5c02b840 comp rx=0 tx=0).stop 2023-11-13T10:55:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec7186d640 1 -- 172.21.15.16:0/4082012888 shutdown_connections 2023-11-13T10:55:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec7186d640 1 --2- 172.21.15.16:0/4082012888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c0d2ae0 0x7fec6c0d2f40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec7186d640 1 --2- 172.21.15.16:0/4082012888 >> v2:172.21.15.184:3300/0 conn(0x7fec6c15cfc0 0x7fec6c0cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec7186d640 1 -- 172.21.15.16:0/4082012888 >> 172.21.15.16:0/4082012888 conn(0x7fec6c08c570 msgr2=0x7fec6c09be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec7186d640 1 -- 172.21.15.16:0/4082012888 shutdown_connections 2023-11-13T10:55:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.608+0000 7fec7186d640 1 -- 172.21.15.16:0/4082012888 wait complete. 2023-11-13T10:55:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.610+0000 7fec7186d640 1 Processor -- start 2023-11-13T10:55:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.610+0000 7fec7186d640 1 -- start start 2023-11-13T10:55:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.611+0000 7fec7186d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c0b29d0 0x7fec6c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.611+0000 7fec7186d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6c0d2ae0 0x7fec6c15f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.611+0000 7fec7186d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c15cfc0 0x7fec6c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.611+0000 7fec7186d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0b5b10 con 0x7fec6c15cfc0 2023-11-13T10:55:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.611+0000 7fec7186d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c1f2d90 con 0x7fec6c0b29d0 2023-11-13T10:55:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.611+0000 7fec7186d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c15fd10 con 0x7fec6c0d2ae0 2023-11-13T10:55:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7106c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c15cfc0 0x7fec6c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7106c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c15cfc0 0x7fec6c15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59704/0 (socket says 172.21.15.16:59704) 2023-11-13T10:55:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7106c640 1 -- 172.21.15.16:0/900003521 learned_addr learned my addr 172.21.15.16:0/900003521 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7086b640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c0b29d0 0x7fec6c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec63fff640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6c0d2ae0 0x7fec6c15f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7106c640 1 -- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6c0d2ae0 msgr2=0x7fec6c15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7106c640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6c0d2ae0 0x7fec6c15f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7106c640 1 -- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c0b29d0 msgr2=0x7fec6c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7106c640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c0b29d0 0x7fec6c15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7106c640 1 -- 172.21.15.16:0/900003521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c009400 con 0x7fec6c15cfc0 2023-11-13T10:55:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7086b640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c0b29d0 0x7fec6c15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.612+0000 7fec7106c640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c15cfc0 0x7fec6c15f710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec6400c450 tx=0x7fec6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.613+0000 7fec617fa640 1 -- 172.21.15.16:0/900003521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec640074e0 con 0x7fec6c15cfc0 2023-11-13T10:55:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.613+0000 7fec617fa640 1 -- 172.21.15.16:0/900003521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec64007660 con 0x7fec6c15cfc0 2023-11-13T10:55:22.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.613+0000 7fec617fa640 1 -- 172.21.15.16:0/900003521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64004a20 con 0x7fec6c15cfc0 2023-11-13T10:55:22.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.613+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6c15fff0 con 0x7fec6c15cfc0 2023-11-13T10:55:22.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.613+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c0b5730 con 0x7fec6c15cfc0 2023-11-13T10:55:22.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.614+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6c0b5c50 con 0x7fec6c15cfc0 2023-11-13T10:55:22.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.615+0000 7fec617fa640 1 -- 172.21.15.16:0/900003521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec64004b80 con 0x7fec6c15cfc0 2023-11-13T10:55:22.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.616+0000 7fec617fa640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fec5806ae40 0x7fec5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:22.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.616+0000 7fec617fa640 1 -- 172.21.15.16:0/900003521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec64088540 con 0x7fec6c15cfc0 2023-11-13T10:55:22.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.617+0000 7fec7086b640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fec5806ae40 0x7fec5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:22.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.618+0000 7fec7086b640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fec5806ae40 0x7fec5806d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec68009f50 tx=0x7fec6800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:22.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.620+0000 7fec617fa640 1 -- 172.21.15.16:0/900003521 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec6c0b5c50 con 0x7fec6c15cfc0 2023-11-13T10:55:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.816+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec6c0b5c50 con 0x7fec6c15cfc0 2023-11-13T10:55:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.818+0000 7fec617fa640 1 -- 172.21.15.16:0/900003521 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12862 (secure 0 0 0) 0x7fec6c0b5c50 con 0x7fec6c15cfc0 2023-11-13T10:55:22.826 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:22.826 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:16.902265+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:22.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fec5806ae40 msgr2=0x7fec5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fec5806ae40 0x7fec5806d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec68009f50 tx=0x7fec6800b040 comp rx=0 tx=0).stop 2023-11-13T10:55:22.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c15cfc0 msgr2=0x7fec6c15f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:22.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c15cfc0 0x7fec6c15f710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec6400c450 tx=0x7fec6400c910 comp rx=0 tx=0).stop 2023-11-13T10:55:22.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 shutdown_connections 2023-11-13T10:55:22.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fec5806ae40 0x7fec5806d300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c15cfc0 0x7fec6c15f710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec6c0d2ae0 0x7fec6c15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 --2- 172.21.15.16:0/900003521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec6c0b29d0 0x7fec6c15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 >> 172.21.15.16:0/900003521 conn(0x7fec6c08c570 msgr2=0x7fec6c06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 shutdown_connections 2023-11-13T10:55:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:22.821+0000 7fec7186d640 1 -- 172.21.15.16:0/900003521 wait complete. 2023-11-13T10:55:22.847 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T10:55:22.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:55:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.073+0000 7f4306c59640 1 Processor -- start 2023-11-13T10:55:23.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.073+0000 7f4306c59640 1 -- start start 2023-11-13T10:55:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.075+0000 7f4306c59640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43000d1850 0x7f43000d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.075+0000 7f4306c59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 0x7f43000c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.075+0000 7f4306c59640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43000ce0e0 con 0x7f430015ce70 2023-11-13T10:55:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.075+0000 7f4306c59640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43000ce3c0 con 0x7f43000d1850 2023-11-13T10:55:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.075+0000 7f4306c59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43000ce6a0 con 0x7f43000d2d00 2023-11-13T10:55:23.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.075+0000 7f4305456640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43000d1850 0x7f43000d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.076+0000 7f4305456640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43000d1850 0x7f43000d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42724/0 (socket says 172.21.15.16:42724) 2023-11-13T10:55:23.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.076+0000 7f4305456640 1 -- 172.21.15.16:0/3818375528 learned_addr learned my addr 172.21.15.16:0/3818375528 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:23.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.076+0000 7f4306458640 1 --2- 172.21.15.16:0/3818375528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 0x7f43000c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.076+0000 7f4305456640 1 -- 172.21.15.16:0/3818375528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 msgr2=0x7f43000c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:23.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.076+0000 7f4305456640 1 --2- 172.21.15.16:0/3818375528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 0x7f43000c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.076+0000 7f4305456640 1 -- 172.21.15.16:0/3818375528 >> v1:172.21.15.184:6789/0 conn(0x7f430015ce70 legacy=0x7f43000cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:23.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.076+0000 7f4305456640 1 -- 172.21.15.16:0/3818375528 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43000ce980 con 0x7f43000d1850 2023-11-13T10:55:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.077+0000 7f4305456640 1 --2- 172.21.15.16:0/3818375528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000d1850 0x7f43000d27c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009870 tx=0x7f42f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1eb554a1ab4918 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.078+0000 7f4304c55640 1 -- 172.21.15.16:0/3818375528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0031640 con 0x7f43000d1850 2023-11-13T10:55:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.078+0000 7f4304c55640 1 -- 172.21.15.16:0/3818375528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f00317a0 con 0x7f43000d1850 2023-11-13T10:55:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.078+0000 7f4304c55640 1 -- 172.21.15.16:0/3818375528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0031b30 con 0x7f43000d1850 2023-11-13T10:55:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.078+0000 7f4306c59640 1 -- 172.21.15.16:0/3818375528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000d1850 msgr2=0x7f43000d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.078+0000 7f4306c59640 1 --2- 172.21.15.16:0/3818375528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000d1850 0x7f43000d27c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009870 tx=0x7f42f002b8f0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.079+0000 7f4306c59640 1 -- 172.21.15.16:0/3818375528 shutdown_connections 2023-11-13T10:55:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.079+0000 7f4306c59640 1 --2- 172.21.15.16:0/3818375528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 0x7f43000c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.079+0000 7f4306c59640 1 --2- 172.21.15.16:0/3818375528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000d1850 0x7f43000d27c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.079+0000 7f4306c59640 1 -- 172.21.15.16:0/3818375528 >> 172.21.15.16:0/3818375528 conn(0x7f430008c580 msgr2=0x7f430009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.079+0000 7f4306c59640 1 -- 172.21.15.16:0/3818375528 shutdown_connections 2023-11-13T10:55:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.079+0000 7f4306c59640 1 -- 172.21.15.16:0/3818375528 wait complete. 2023-11-13T10:55:23.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.080+0000 7f4306c59640 1 Processor -- start 2023-11-13T10:55:23.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.081+0000 7f4306c59640 1 -- start start 2023-11-13T10:55:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.081+0000 7f4306c59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43000d1850 0x7f43000c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306c59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 0x7f43000c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306c59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f430015ce70 0x7f43000b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306c59640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43000c6c50 con 0x7f430015ce70 2023-11-13T10:55:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306c59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43000c8ae0 con 0x7f43000d1850 2023-11-13T10:55:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306c59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43000b82c0 con 0x7f43000d2d00 2023-11-13T10:55:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306458640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f430015ce70 0x7f43000b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306458640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f430015ce70 0x7f43000b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42732/0 (socket says 172.21.15.16:42732) 2023-11-13T10:55:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306458640 1 -- 172.21.15.16:0/1255976169 learned_addr learned my addr 172.21.15.16:0/1255976169 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4305456640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 0x7f43000c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4305c57640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43000d1850 0x7f43000c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306458640 1 -- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 msgr2=0x7f43000c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306458640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 0x7f43000c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306458640 1 -- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43000d1850 msgr2=0x7f43000c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306458640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43000d1850 0x7f43000c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.082+0000 7f4306458640 1 -- 172.21.15.16:0/1255976169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f00094d0 con 0x7f430015ce70 2023-11-13T10:55:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.083+0000 7f4306458640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f430015ce70 0x7f43000b7800 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42f8007e40 tx=0x7f42f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.083+0000 7f42f67fc640 1 -- 172.21.15.16:0/1255976169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f8014620 con 0x7f430015ce70 2023-11-13T10:55:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.083+0000 7f42f67fc640 1 -- 172.21.15.16:0/1255976169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f800ee90 con 0x7f430015ce70 2023-11-13T10:55:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.083+0000 7f42f67fc640 1 -- 172.21.15.16:0/1255976169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f80149e0 con 0x7f430015ce70 2023-11-13T10:55:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.083+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4300156910 con 0x7f430015ce70 2023-11-13T10:55:23.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.083+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4300156ed0 con 0x7f430015ce70 2023-11-13T10:55:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.084+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43000675b0 con 0x7f430015ce70 2023-11-13T10:55:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.085+0000 7f42f67fc640 1 -- 172.21.15.16:0/1255976169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f42f8014b40 con 0x7f430015ce70 2023-11-13T10:55:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.086+0000 7f42f67fc640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f42e406ae40 0x7f42e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.086+0000 7f42f67fc640 1 -- 172.21.15.16:0/1255976169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f42f8089ce0 con 0x7f430015ce70 2023-11-13T10:55:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.087+0000 7f4305c57640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f42e406ae40 0x7f42e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.088+0000 7f4305c57640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f42e406ae40 0x7f42e406d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42fc004ea0 tx=0x7f42fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:23.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.091+0000 7f42f67fc640 1 -- 172.21.15.16:0/1255976169 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42f80550e0 con 0x7f430015ce70 2023-11-13T10:55:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.286+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f430004e4d0 con 0x7f430015ce70 2023-11-13T10:55:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.286+0000 7f42f67fc640 1 -- 172.21.15.16:0/1255976169 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42f8058700 con 0x7f430015ce70 2023-11-13T10:55:23.291 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:55:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f42e406ae40 msgr2=0x7f42e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f42e406ae40 0x7f42e406d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42fc004ea0 tx=0x7f42fc009040 comp rx=0 tx=0).stop 2023-11-13T10:55:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f430015ce70 msgr2=0x7f43000b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f430015ce70 0x7f43000b7800 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42f8007e40 tx=0x7f42f800e4b0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 shutdown_connections 2023-11-13T10:55:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f42e406ae40 0x7f42e406d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f430015ce70 0x7f43000b7800 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43000d2d00 0x7f43000c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 --2- 172.21.15.16:0/1255976169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43000d1850 0x7f43000c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 >> 172.21.15.16:0/1255976169 conn(0x7f430008c580 msgr2=0x7f43000cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 shutdown_connections 2023-11-13T10:55:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.289+0000 7f4306c59640 1 -- 172.21.15.16:0/1255976169 wait complete. 2023-11-13T10:55:23.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-13T10:55:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.528+0000 7f2a17fff640 1 Processor -- start 2023-11-13T10:55:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.528+0000 7f2a17fff640 1 -- start start 2023-11-13T10:55:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.530+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a1815ad50 0x7f2a180cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.530+0000 7f2a17fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2a180cced0 0x7f2a180cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.530+0000 7f2a17fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a180cdea0 con 0x7f2a1806e690 2023-11-13T10:55:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.530+0000 7f2a17fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a180ce410 con 0x7f2a180cced0 2023-11-13T10:55:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.530+0000 7f2a17fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180ce7a0 con 0x7f2a1815ad50 2023-11-13T10:55:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.530+0000 7f2a177fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2a1806e690 0x7f2a1806eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51820/0 (socket says 172.21.15.16:51820) 2023-11-13T10:55:23.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.530+0000 7f2a177fe640 1 -- 172.21.15.16:0/2702909211 learned_addr learned my addr 172.21.15.16:0/2702909211 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:23.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.530+0000 7f2a167fc640 1 --2- 172.21.15.16:0/2702909211 >> v2:172.21.15.16:3300/0 conn(0x7f2a180cced0 0x7f2a180cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.530+0000 7f2a16ffd640 1 --2- 172.21.15.16:0/2702909211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a1815ad50 0x7f2a180cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.531+0000 7f2a15ffb640 1 -- 172.21.15.16:0/2702909211 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918984762 0 0) 0x7f2a180cdea0 con 0x7f2a1806e690 2023-11-13T10:55:23.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.531+0000 7f2a167fc640 1 -- 172.21.15.16:0/2702909211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a1815ad50 msgr2=0x7f2a180cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.531+0000 7f2a167fc640 1 --2- 172.21.15.16:0/2702909211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a1815ad50 0x7f2a180cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.531+0000 7f2a167fc640 1 -- 172.21.15.16:0/2702909211 >> v1:172.21.15.16:6789/0 conn(0x7f2a1806e690 legacy=0x7f2a1806eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.531+0000 7f2a167fc640 1 -- 172.21.15.16:0/2702909211 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1806fcc0 con 0x7f2a180cced0 2023-11-13T10:55:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.532+0000 7f2a167fc640 1 --2- 172.21.15.16:0/2702909211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a180cced0 0x7f2a180cd960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0097a0 tx=0x7f2a0c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e85ee971c0ac03 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.532+0000 7f2a15ffb640 1 -- 172.21.15.16:0/2702909211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c02d040 con 0x7f2a180cced0 2023-11-13T10:55:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.532+0000 7f2a15ffb640 1 -- 172.21.15.16:0/2702909211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c031810 con 0x7f2a180cced0 2023-11-13T10:55:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.532+0000 7f2a15ffb640 1 -- 172.21.15.16:0/2702909211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c031ba0 con 0x7f2a180cced0 2023-11-13T10:55:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.533+0000 7f2a17fff640 1 -- 172.21.15.16:0/2702909211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a180cced0 msgr2=0x7f2a180cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.533+0000 7f2a17fff640 1 --2- 172.21.15.16:0/2702909211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a180cced0 0x7f2a180cd960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0097a0 tx=0x7f2a0c02b8b0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.533+0000 7f2a17fff640 1 -- 172.21.15.16:0/2702909211 shutdown_connections 2023-11-13T10:55:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.533+0000 7f2a17fff640 1 --2- 172.21.15.16:0/2702909211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a180cced0 0x7f2a180cd960 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.533+0000 7f2a17fff640 1 --2- 172.21.15.16:0/2702909211 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a1815ad50 0x7f2a180cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.533+0000 7f2a17fff640 1 -- 172.21.15.16:0/2702909211 >> 172.21.15.16:0/2702909211 conn(0x7f2a180b2700 msgr2=0x7f2a180b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.533+0000 7f2a17fff640 1 -- 172.21.15.16:0/2702909211 shutdown_connections 2023-11-13T10:55:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.533+0000 7f2a17fff640 1 -- 172.21.15.16:0/2702909211 wait complete. 2023-11-13T10:55:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.535+0000 7f2a17fff640 1 Processor -- start 2023-11-13T10:55:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.535+0000 7f2a17fff640 1 -- start start 2023-11-13T10:55:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.536+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1806e690 0x7f2a180b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.536+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a180cced0 0x7f2a180b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.536+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1815ad50 0x7f2a18141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.536+0000 7f2a17fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180b43e0 con 0x7f2a1806e690 2023-11-13T10:55:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.536+0000 7f2a17fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180b37f0 con 0x7f2a1815ad50 2023-11-13T10:55:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.536+0000 7f2a17fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180b3a40 con 0x7f2a180cced0 2023-11-13T10:55:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.536+0000 7f2a16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1806e690 0x7f2a180b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.536+0000 7f2a16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1806e690 0x7f2a180b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42748/0 (socket says 172.21.15.16:42748) 2023-11-13T10:55:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.536+0000 7f2a16ffd640 1 -- 172.21.15.16:0/3667786368 learned_addr learned my addr 172.21.15.16:0/3667786368 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a177fe640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1815ad50 0x7f2a18141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a167fc640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a180cced0 0x7f2a180b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a16ffd640 1 -- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a180cced0 msgr2=0x7f2a180b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a16ffd640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a180cced0 0x7f2a180b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a16ffd640 1 -- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1815ad50 msgr2=0x7f2a18141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a16ffd640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1815ad50 0x7f2a18141ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a16ffd640 1 -- 172.21.15.16:0/3667786368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c009400 con 0x7f2a1806e690 2023-11-13T10:55:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a167fc640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a180cced0 0x7f2a180b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a177fe640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1815ad50 0x7f2a18141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a16ffd640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1806e690 0x7f2a180b4800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a08007e70 tx=0x7f2a0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:23.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f29fffff640 1 -- 172.21.15.16:0/3667786368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08014620 con 0x7f2a1806e690 2023-11-13T10:55:23.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f29fffff640 1 -- 172.21.15.16:0/3667786368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a08014780 con 0x7f2a1806e690 2023-11-13T10:55:23.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f29fffff640 1 -- 172.21.15.16:0/3667786368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08014b30 con 0x7f2a1806e690 2023-11-13T10:55:23.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.537+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a180b3d20 con 0x7f2a1806e690 2023-11-13T10:55:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.538+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a180b5690 con 0x7f2a1806e690 2023-11-13T10:55:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.539+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a180b4650 con 0x7f2a1806e690 2023-11-13T10:55:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.540+0000 7f29fffff640 1 -- 172.21.15.16:0/3667786368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a08014c90 con 0x7f2a1806e690 2023-11-13T10:55:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.540+0000 7f29fffff640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0406ae70 0x7f2a0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.541+0000 7f29fffff640 1 -- 172.21.15.16:0/3667786368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2a08089b60 con 0x7f2a1806e690 2023-11-13T10:55:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.541+0000 7f2a167fc640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0406ae70 0x7f2a0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.542+0000 7f2a167fc640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0406ae70 0x7f2a0406d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c036000 tx=0x7f2a0c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.545+0000 7f29fffff640 1 -- 172.21.15.16:0/3667786368 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a08054ff0 con 0x7f2a1806e690 2023-11-13T10:55:23.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.744+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2a180b5820 con 0x7f2a1806e690 2023-11-13T10:55:23.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.745+0000 7f29fffff640 1 -- 172.21.15.16:0/3667786368 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2a08058610 con 0x7f2a1806e690 2023-11-13T10:55:23.750 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-13T10:55:23.750 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:55:23.750 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-13T10:55:23.751 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-13T10:55:23.751 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:55:23.751 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-13T10:55:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.747+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0406ae70 msgr2=0x7f2a0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.747+0000 7f2a17fff640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0406ae70 0x7f2a0406d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c036000 tx=0x7f2a0c03c040 comp rx=0 tx=0).stop 2023-11-13T10:55:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1806e690 msgr2=0x7f2a180b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1806e690 0x7f2a180b4800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a08007e70 tx=0x7f2a0800e4b0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 shutdown_connections 2023-11-13T10:55:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0406ae70 0x7f2a0406d330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a1815ad50 0x7f2a18141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a180cced0 0x7f2a180b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 --2- 172.21.15.16:0/3667786368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a1806e690 0x7f2a180b4800 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 >> 172.21.15.16:0/3667786368 conn(0x7f2a180b2700 msgr2=0x7f2a18159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 shutdown_connections 2023-11-13T10:55:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.748+0000 7f2a17fff640 1 -- 172.21.15.16:0/3667786368 wait complete. 2023-11-13T10:55:23.775 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-13T10:55:23.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-13T10:55:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.991+0000 7f8c96d4e640 1 Processor -- start 2023-11-13T10:55:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.991+0000 7f8c96d4e640 1 -- start start 2023-11-13T10:55:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.993+0000 7f8c96d4e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c900d0dc0 0x7f8c900ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.993+0000 7f8c96d4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900ce590 0x7f8c900b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.993+0000 7f8c96d4e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c900d1540 con 0x7f8c9015d0d0 2023-11-13T10:55:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.993+0000 7f8c96d4e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c900d0570 con 0x7f8c900d0dc0 2023-11-13T10:55:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.993+0000 7f8c96d4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900d1250 con 0x7f8c900ce590 2023-11-13T10:55:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.993+0000 7f8c9554b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c900d0dc0 0x7f8c900ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.994+0000 7f8c9554b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c900d0dc0 0x7f8c900ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42754/0 (socket says 172.21.15.16:42754) 2023-11-13T10:55:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.994+0000 7f8c9554b640 1 -- 172.21.15.16:0/1528793437 learned_addr learned my addr 172.21.15.16:0/1528793437 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.994+0000 7f8c9654d640 1 --2- 172.21.15.16:0/1528793437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900ce590 0x7f8c900b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.994+0000 7f8c9554b640 1 -- 172.21.15.16:0/1528793437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900ce590 msgr2=0x7f8c900b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.994+0000 7f8c9554b640 1 --2- 172.21.15.16:0/1528793437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900ce590 0x7f8c900b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.994+0000 7f8c9554b640 1 -- 172.21.15.16:0/1528793437 >> v1:172.21.15.184:6789/0 conn(0x7f8c9015d0d0 legacy=0x7f8c900cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.994+0000 7f8c9554b640 1 -- 172.21.15.16:0/1528793437 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c900d17e0 con 0x7f8c900d0dc0 2023-11-13T10:55:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.995+0000 7f8c9554b640 1 --2- 172.21.15.16:0/1528793437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c900d0dc0 0x7f8c900ce050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009870 tx=0x7f8c8002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921169d12230eac9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c94d4a640 1 -- 172.21.15.16:0/1528793437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80030840 con 0x7f8c900d0dc0 2023-11-13T10:55:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c94d4a640 1 -- 172.21.15.16:0/1528793437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c800309a0 con 0x7f8c900d0dc0 2023-11-13T10:55:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c94d4a640 1 -- 172.21.15.16:0/1528793437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80030d30 con 0x7f8c900d0dc0 2023-11-13T10:55:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c96d4e640 1 -- 172.21.15.16:0/1528793437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c900d0dc0 msgr2=0x7f8c900ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c96d4e640 1 --2- 172.21.15.16:0/1528793437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c900d0dc0 0x7f8c900ce050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009870 tx=0x7f8c8002b980 comp rx=0 tx=0).stop 2023-11-13T10:55:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c96d4e640 1 -- 172.21.15.16:0/1528793437 shutdown_connections 2023-11-13T10:55:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c96d4e640 1 --2- 172.21.15.16:0/1528793437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900ce590 0x7f8c900b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c96d4e640 1 --2- 172.21.15.16:0/1528793437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c900d0dc0 0x7f8c900ce050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c96d4e640 1 -- 172.21.15.16:0/1528793437 >> 172.21.15.16:0/1528793437 conn(0x7f8c900d2b90 msgr2=0x7f8c900d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.996+0000 7f8c96d4e640 1 -- 172.21.15.16:0/1528793437 shutdown_connections 2023-11-13T10:55:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.997+0000 7f8c96d4e640 1 -- 172.21.15.16:0/1528793437 wait complete. 2023-11-13T10:55:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.998+0000 7f8c96d4e640 1 Processor -- start 2023-11-13T10:55:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.998+0000 7f8c96d4e640 1 -- start start 2023-11-13T10:55:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.999+0000 7f8c96d4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c900ce590 0x7f8c901f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.999+0000 7f8c96d4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9015d0d0 0x7f8c900cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.999+0000 7f8c96d4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900cc850 0x7f8c9006b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.999+0000 7f8c96d4e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9015fce0 con 0x7f8c9015d0d0 2023-11-13T10:55:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.999+0000 7f8c96d4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c901f2720 con 0x7f8c900ce590 2023-11-13T10:55:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:23.999+0000 7f8c96d4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90160170 con 0x7f8c900cc850 2023-11-13T10:55:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9554b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9015d0d0 0x7f8c900cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9554b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9015d0d0 0x7f8c900cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42768/0 (socket says 172.21.15.16:42768) 2023-11-13T10:55:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9554b640 1 -- 172.21.15.16:0/2673187608 learned_addr learned my addr 172.21.15.16:0/2673187608 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c95d4c640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c900ce590 0x7f8c901f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9654d640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900cc850 0x7f8c9006b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9554b640 1 -- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900cc850 msgr2=0x7f8c9006b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9554b640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900cc850 0x7f8c9006b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9554b640 1 -- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c900ce590 msgr2=0x7f8c901f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9554b640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c900ce590 0x7f8c901f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9554b640 1 -- 172.21.15.16:0/2673187608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800094d0 con 0x7f8c9015d0d0 2023-11-13T10:55:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.000+0000 7f8c9554b640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9015d0d0 0x7f8c900cb6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.001+0000 7f8c867fc640 1 -- 172.21.15.16:0/2673187608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80039c90 con 0x7f8c9015d0d0 2023-11-13T10:55:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.001+0000 7f8c867fc640 1 -- 172.21.15.16:0/2673187608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80039df0 con 0x7f8c9015d0d0 2023-11-13T10:55:24.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.001+0000 7f8c867fc640 1 -- 172.21.15.16:0/2673187608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80043640 con 0x7f8c9015d0d0 2023-11-13T10:55:24.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.001+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9015f0d0 con 0x7f8c9015d0d0 2023-11-13T10:55:24.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.001+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9015f690 con 0x7f8c9015d0d0 2023-11-13T10:55:24.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.002+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9015fe90 con 0x7f8c9015d0d0 2023-11-13T10:55:24.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.004+0000 7f8c867fc640 1 -- 172.21.15.16:0/2673187608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c800437a0 con 0x7f8c9015d0d0 2023-11-13T10:55:24.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.005+0000 7f8c867fc640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c7406ae40 0x7f8c7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.005+0000 7f8c867fc640 1 -- 172.21.15.16:0/2673187608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8c8003e070 con 0x7f8c9015d0d0 2023-11-13T10:55:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.005+0000 7f8c95d4c640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c7406ae40 0x7f8c7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.006+0000 7f8c95d4c640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c7406ae40 0x7f8c7406d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c000f60 tx=0x7f8c8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.009+0000 7f8c867fc640 1 -- 172.21.15.16:0/2673187608 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c9015fe90 con 0x7f8c9015d0d0 2023-11-13T10:55:24.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.207+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8c9004e4d0 con 0x7f8c9015d0d0 2023-11-13T10:55:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.802+0000 7f8c867fc640 1 -- 172.21.15.16:0/2673187608 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8c80077780 con 0x7f8c9015d0d0 2023-11-13T10:55:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-13T10:55:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.805+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c7406ae40 msgr2=0x7f8c7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.805+0000 7f8c96d4e640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c7406ae40 0x7f8c7406d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c000f60 tx=0x7f8c8c009040 comp rx=0 tx=0).stop 2023-11-13T10:55:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.805+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9015d0d0 msgr2=0x7f8c900cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.805+0000 7f8c96d4e640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9015d0d0 0x7f8c900cb6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c80004730 comp rx=0 tx=0).stop 2023-11-13T10:55:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.806+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 shutdown_connections 2023-11-13T10:55:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.806+0000 7f8c96d4e640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c7406ae40 0x7f8c7406d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.806+0000 7f8c96d4e640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c900cc850 0x7f8c9006b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.806+0000 7f8c96d4e640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c9015d0d0 0x7f8c900cb6c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.806+0000 7f8c96d4e640 1 --2- 172.21.15.16:0/2673187608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c900ce590 0x7f8c901f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.806+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 >> 172.21.15.16:0/2673187608 conn(0x7f8c900d2b90 msgr2=0x7f8c900b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.806+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 shutdown_connections 2023-11-13T10:55:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:24.806+0000 7f8c96d4e640 1 -- 172.21.15.16:0/2673187608 wait complete. 2023-11-13T10:55:24.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-13T10:55:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.051+0000 7f20e2197640 1 Processor -- start 2023-11-13T10:55:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.052+0000 7f20e2197640 1 -- start start 2023-11-13T10:55:25.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.053+0000 7f20e2197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 0x7f20dc0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.053+0000 7f20e2197640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20dc0d18f0 0x7f20dc0cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.053+0000 7f20e2197640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20dc0d0ae0 con 0x7f20dc1606c0 2023-11-13T10:55:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.053+0000 7f20e2197640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20dc0d1010 con 0x7f20dc0d18f0 2023-11-13T10:55:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.053+0000 7f20e2197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc160aa0 con 0x7f20dc0cfea0 2023-11-13T10:55:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.054+0000 7f20dbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20dc0d18f0 0x7f20dc0cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.054+0000 7f20dbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20dc0d18f0 0x7f20dc0cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42780/0 (socket says 172.21.15.16:42780) 2023-11-13T10:55:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.054+0000 7f20dbfff640 1 -- 172.21.15.16:0/1447184649 learned_addr learned my addr 172.21.15.16:0/1447184649 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.054+0000 7f20daffd640 1 --2- 172.21.15.16:0/1447184649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 0x7f20dc0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.054+0000 7f20dbfff640 1 -- 172.21.15.16:0/1447184649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 msgr2=0x7f20dc0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.054+0000 7f20dbfff640 1 --2- 172.21.15.16:0/1447184649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 0x7f20dc0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.054+0000 7f20dbfff640 1 -- 172.21.15.16:0/1447184649 >> v1:172.21.15.184:6789/0 conn(0x7f20dc1606c0 legacy=0x7f20dc0d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.054+0000 7f20dbfff640 1 -- 172.21.15.16:0/1447184649 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc0d37f0 con 0x7f20dc0d18f0 2023-11-13T10:55:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.055+0000 7f20dbfff640 1 --2- 172.21.15.16:0/1447184649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 0x7f20dc0cdf70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009870 tx=0x7f20d402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ebb6b9caa4674a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.056+0000 7f20da7fc640 1 -- 172.21.15.16:0/1447184649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4031740 con 0x7f20dc0d18f0 2023-11-13T10:55:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.056+0000 7f20da7fc640 1 -- 172.21.15.16:0/1447184649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d40318a0 con 0x7f20dc0d18f0 2023-11-13T10:55:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.056+0000 7f20da7fc640 1 -- 172.21.15.16:0/1447184649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4031c30 con 0x7f20dc0d18f0 2023-11-13T10:55:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.056+0000 7f20e2197640 1 -- 172.21.15.16:0/1447184649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 msgr2=0x7f20dc0cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.056+0000 7f20e2197640 1 --2- 172.21.15.16:0/1447184649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 0x7f20dc0cdf70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009870 tx=0x7f20d402b7e0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.056+0000 7f20e2197640 1 -- 172.21.15.16:0/1447184649 shutdown_connections 2023-11-13T10:55:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.056+0000 7f20e2197640 1 --2- 172.21.15.16:0/1447184649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 0x7f20dc0cdf70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.057+0000 7f20e2197640 1 --2- 172.21.15.16:0/1447184649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 0x7f20dc0cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.057+0000 7f20e2197640 1 -- 172.21.15.16:0/1447184649 >> 172.21.15.16:0/1447184649 conn(0x7f20dc0d2d00 msgr2=0x7f20dc0d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.057+0000 7f20e2197640 1 -- 172.21.15.16:0/1447184649 shutdown_connections 2023-11-13T10:55:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.057+0000 7f20e2197640 1 -- 172.21.15.16:0/1447184649 wait complete. 2023-11-13T10:55:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.059+0000 7f20e2197640 1 Processor -- start 2023-11-13T10:55:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.059+0000 7f20e2197640 1 -- start start 2023-11-13T10:55:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20e2197640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 0x7f20dc0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20e2197640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 0x7f20dc15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20e2197640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc1606c0 0x7f20dc0b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20e2197640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc15ae30 con 0x7f20dc0d18f0 2023-11-13T10:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20e2197640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc0b7bf0 con 0x7f20dc1606c0 2023-11-13T10:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20e2197640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc0b3210 con 0x7f20dc0cfea0 2023-11-13T10:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 0x7f20dc15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:25.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 0x7f20dc15a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42786/0 (socket says 172.21.15.16:42786) 2023-11-13T10:55:25.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20daffd640 1 -- 172.21.15.16:0/3022573162 learned_addr learned my addr 172.21.15.16:0/3022573162 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:25.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20db7fe640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 0x7f20dc0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:25.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20daffd640 1 -- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 msgr2=0x7f20dc0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:25.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20daffd640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 0x7f20dc0b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20dbfff640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc1606c0 0x7f20dc0b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:25.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.060+0000 7f20daffd640 1 -- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc1606c0 msgr2=0x7f20dc0b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:25.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.061+0000 7f20daffd640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc1606c0 0x7f20dc0b76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.061+0000 7f20daffd640 1 -- 172.21.15.16:0/3022573162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d40094d0 con 0x7f20dc0d18f0 2023-11-13T10:55:25.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.061+0000 7f20dbfff640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc1606c0 0x7f20dc0b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:25.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.061+0000 7f20daffd640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 0x7f20dc15a8f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f20c400d5a0 tx=0x7f20c400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:25.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.061+0000 7f20d3fff640 1 -- 172.21.15.16:0/3022573162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4004240 con 0x7f20dc0d18f0 2023-11-13T10:55:25.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.061+0000 7f20d3fff640 1 -- 172.21.15.16:0/3022573162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c40043a0 con 0x7f20dc0d18f0 2023-11-13T10:55:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.061+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc0b34f0 con 0x7f20dc0d18f0 2023-11-13T10:55:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.061+0000 7f20d3fff640 1 -- 172.21.15.16:0/3022573162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4003830 con 0x7f20dc0d18f0 2023-11-13T10:55:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.061+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc0b3ab0 con 0x7f20dc0d18f0 2023-11-13T10:55:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.062+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc067090 con 0x7f20dc0d18f0 2023-11-13T10:55:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.065+0000 7f20d3fff640 1 -- 172.21.15.16:0/3022573162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20c4003990 con 0x7f20dc0d18f0 2023-11-13T10:55:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.066+0000 7f20d3fff640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20bc06ae90 0x7f20bc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:25.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.066+0000 7f20d3fff640 1 -- 172.21.15.16:0/3022573162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f20c40892e0 con 0x7f20dc0d18f0 2023-11-13T10:55:25.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.066+0000 7f20db7fe640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20bc06ae90 0x7f20bc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:25.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.068+0000 7f20db7fe640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20bc06ae90 0x7f20bc06d350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f20cc007fd0 tx=0x7f20cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:25.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.070+0000 7f20d3fff640 1 -- 172.21.15.16:0/3022573162 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20c4054690 con 0x7f20dc0d18f0 2023-11-13T10:55:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.268+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f20dc0663b0 con 0x7f20dc0d18f0 2023-11-13T10:55:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.816+0000 7f20d3fff640 1 -- 172.21.15.16:0/3022573162 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7f20c4057cb0 con 0x7f20dc0d18f0 2023-11-13T10:55:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-13T10:55:25.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.821+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20bc06ae90 msgr2=0x7f20bc06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:25.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.821+0000 7f20e2197640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20bc06ae90 0x7f20bc06d350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f20cc007fd0 tx=0x7f20cc009040 comp rx=0 tx=0).stop 2023-11-13T10:55:25.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.821+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 msgr2=0x7f20dc15a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.821+0000 7f20e2197640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 0x7f20dc15a8f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f20c400d5a0 tx=0x7f20c400da60 comp rx=0 tx=0).stop 2023-11-13T10:55:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.821+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 shutdown_connections 2023-11-13T10:55:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.821+0000 7f20e2197640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20bc06ae90 0x7f20bc06d350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.821+0000 7f20e2197640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20dc1606c0 0x7f20dc0b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.821+0000 7f20e2197640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20dc0d18f0 0x7f20dc15a8f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.821+0000 7f20e2197640 1 --2- 172.21.15.16:0/3022573162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20dc0cfea0 0x7f20dc0b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.822+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 >> 172.21.15.16:0/3022573162 conn(0x7f20dc0d2d00 msgr2=0x7f20dc0d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.822+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 shutdown_connections 2023-11-13T10:55:25.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:25.823+0000 7f20e2197640 1 -- 172.21.15.16:0/3022573162 wait complete. 2023-11-13T10:55:25.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-13T10:55:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.052+0000 7f2ded147640 1 Processor -- start 2023-11-13T10:55:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.052+0000 7f2ded147640 1 -- start start 2023-11-13T10:55:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.054+0000 7f2ded147640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de815b5c0 0x7f2de8164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.054+0000 7f2ded147640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2de80b2b40 0x7f2de80ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.054+0000 7f2ded147640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de8165b40 con 0x7f2de8161640 2023-11-13T10:55:26.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.054+0000 7f2ded147640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de81652a0 con 0x7f2de80b2b40 2023-11-13T10:55:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.054+0000 7f2ded147640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8165580 con 0x7f2de815b5c0 2023-11-13T10:55:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.055+0000 7f2de77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2de8161640 0x7f2de80b2430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51828/0 (socket says 172.21.15.16:51828) 2023-11-13T10:55:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.055+0000 7f2de77fe640 1 -- 172.21.15.16:0/1728004735 learned_addr learned my addr 172.21.15.16:0/1728004735 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:26.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.055+0000 7f2dec946640 1 --2- 172.21.15.16:0/1728004735 >> v2:172.21.15.184:3300/0 conn(0x7f2de80b2b40 0x7f2de80ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:26.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.055+0000 7f2de7fff640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de815b5c0 0x7f2de8164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:26.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.055+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146689265 0 0) 0x7f2de8165b40 con 0x7f2de8161640 2023-11-13T10:55:26.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.055+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dd0003520 con 0x7f2de8161640 2023-11-13T10:55:26.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.055+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1540336969 0 0) 0x7f2dd0003520 con 0x7f2de8161640 2023-11-13T10:55:26.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de815b5c0 msgr2=0x7f2de8164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:26.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de815b5c0 0x7f2de8164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 >> v2:172.21.15.184:3300/0 conn(0x7f2de80b2b40 msgr2=0x7f2de80ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:26.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 --2- 172.21.15.16:0/1728004735 >> v2:172.21.15.184:3300/0 conn(0x7f2de80b2b40 0x7f2de80ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de8165860 con 0x7f2de8161640 2023-11-13T10:55:26.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f2ddc001670 con 0x7f2de8161640 2023-11-13T10:55:26.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 >> v1:172.21.15.16:6789/0 conn(0x7f2de8161640 legacy=0x7f2de80b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:26.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dd0003970 0x7f2dd0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:26.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de815b5c0 0x7f2dd00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2dd0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de81652a0 con 0x7f2dd0004d00 2023-11-13T10:55:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8165580 con 0x7f2dd0003970 2023-11-13T10:55:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd000b980 con 0x7f2de815b5c0 2023-11-13T10:55:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2dec946640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2dd0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de77fe640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de815b5c0 0x7f2dd00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.056+0000 7f2de7fff640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dd0003970 0x7f2dd0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.057+0000 7f2dec946640 1 -- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de815b5c0 msgr2=0x7f2dd00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.057+0000 7f2dec946640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de815b5c0 0x7f2dd00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.057+0000 7f2dec946640 1 -- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dd0003970 msgr2=0x7f2dd0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.057+0000 7f2dec946640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2dd0003970 0x7f2dd0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.057+0000 7f2dec946640 1 -- 172.21.15.16:0/1728004735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd0003520 con 0x7f2dd0004d00 2023-11-13T10:55:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.058+0000 7f2dec946640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2dd0005180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009fc0 tx=0x7f2dd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ba79da0a437695 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.058+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de002cc60 con 0x7f2dd0004d00 2023-11-13T10:55:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.058+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de000f040 con 0x7f2dd0004d00 2023-11-13T10:55:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.058+0000 7f2de6ffd640 1 -- 172.21.15.16:0/1728004735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0034440 con 0x7f2dd0004d00 2023-11-13T10:55:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.058+0000 7f2ded147640 1 -- 172.21.15.16:0/1728004735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 msgr2=0x7f2dd0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.058+0000 7f2ded147640 1 --2- 172.21.15.16:0/1728004735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2dd0005180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009fc0 tx=0x7f2dd00057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.059+0000 7f2dec946640 1 -- 172.21.15.16:0/1728004735 reap_dead start 2023-11-13T10:55:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.059+0000 7f2ded147640 1 -- 172.21.15.16:0/1728004735 shutdown_connections 2023-11-13T10:55:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.059+0000 7f2ded147640 1 -- 172.21.15.16:0/1728004735 >> 172.21.15.16:0/1728004735 conn(0x7f2de80b0970 msgr2=0x7f2de80b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.059+0000 7f2ded147640 1 -- 172.21.15.16:0/1728004735 shutdown_connections 2023-11-13T10:55:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.059+0000 7f2ded147640 1 -- 172.21.15.16:0/1728004735 wait complete. 2023-11-13T10:55:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.061+0000 7f2ded147640 1 Processor -- start 2023-11-13T10:55:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.061+0000 7f2ded147640 1 -- start start 2023-11-13T10:55:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2ded147640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2de81473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2ded147640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2de815c930 0x7f2de814c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2ded147640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de814cc10 0x7f2de80ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2ded147640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8147c80 con 0x7f2dd0004d00 2023-11-13T10:55:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2ded147640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8147f60 con 0x7f2de815c930 2023-11-13T10:55:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2ded147640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8148240 con 0x7f2de814cc10 2023-11-13T10:55:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2de7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2de81473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2de7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2de81473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42800/0 (socket says 172.21.15.16:42800) 2023-11-13T10:55:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2de7fff640 1 -- 172.21.15.16:0/2598751953 learned_addr learned my addr 172.21.15.16:0/2598751953 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2dec946640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de814cc10 0x7f2de80ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:26.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2de77fe640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2de815c930 0x7f2de814c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2de7fff640 1 -- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de814cc10 msgr2=0x7f2de80ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2de7fff640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de814cc10 0x7f2de80ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2de7fff640 1 -- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2de815c930 msgr2=0x7f2de814c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.062+0000 7f2de7fff640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2de815c930 0x7f2de814c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.063+0000 7f2de7fff640 1 -- 172.21.15.16:0/2598751953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de8148520 con 0x7f2dd0004d00 2023-11-13T10:55:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.063+0000 7f2de7fff640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2de81473e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8011540 tx=0x7f2dd8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.063+0000 7f2de4ff9640 1 -- 172.21.15.16:0/2598751953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8015930 con 0x7f2dd0004d00 2023-11-13T10:55:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.063+0000 7f2de4ff9640 1 -- 172.21.15.16:0/2598751953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd8015a90 con 0x7f2dd0004d00 2023-11-13T10:55:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.063+0000 7f2de4ff9640 1 -- 172.21.15.16:0/2598751953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8015e60 con 0x7f2dd0004d00 2023-11-13T10:55:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.063+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de8148800 con 0x7f2dd0004d00 2023-11-13T10:55:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.063+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de80cd000 con 0x7f2dd0004d00 2023-11-13T10:55:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.065+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de804e4d0 con 0x7f2dd0004d00 2023-11-13T10:55:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.066+0000 7f2de4ff9640 1 -- 172.21.15.16:0/2598751953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2dd800f460 con 0x7f2dd0004d00 2023-11-13T10:55:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.066+0000 7f2de4ff9640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.067+0000 7f2de4ff9640 1 -- 172.21.15.16:0/2598751953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2dd8089db0 con 0x7f2dd0004d00 2023-11-13T10:55:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.067+0000 7f2de77fe640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.068+0000 7f2de77fe640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2de80719f0 tx=0x7f2ddc000f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.071+0000 7f2de4ff9640 1 -- 172.21.15.16:0/2598751953 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2dd8054ec0 con 0x7f2dd0004d00 2023-11-13T10:55:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.308+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2de81610f0 con 0x7f2dd0004d00 2023-11-13T10:55:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.843+0000 7f2de4ff9640 1 -- 172.21.15.16:0/2598751953 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2dd80584e0 con 0x7f2dd0004d00 2023-11-13T10:55:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2023-11-13T10:55:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.849+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 msgr2=0x7f2dd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.849+0000 7f2ded147640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2de80719f0 tx=0x7f2ddc000f50 comp rx=0 tx=0).stop 2023-11-13T10:55:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.849+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 msgr2=0x7f2de81473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.849+0000 7f2ded147640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2de81473e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8011540 tx=0x7f2dd8011a00 comp rx=0 tx=0).stop 2023-11-13T10:55:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.850+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 shutdown_connections 2023-11-13T10:55:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.850+0000 7f2ded147640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2de80719f0 tx=0x7f2ddc000f50 comp rx=0 tx=0).stop 2023-11-13T10:55:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.850+0000 7f2ded147640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2de814cc10 0x7f2de80ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.850+0000 7f2ded147640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2de815c930 0x7f2de814c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.850+0000 7f2ded147640 1 --2- 172.21.15.16:0/2598751953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dd0004d00 0x7f2de81473e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.850+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 >> 172.21.15.16:0/2598751953 conn(0x7f2de80b0970 msgr2=0x7f2de806bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.851+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 shutdown_connections 2023-11-13T10:55:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:26.852+0000 7f2ded147640 1 -- 172.21.15.16:0/2598751953 wait complete. 2023-11-13T10:55:26.892 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-13T10:55:26.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-13T10:55:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.084+0000 7f968f226640 1 Processor -- start 2023-11-13T10:55:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.084+0000 7f968f226640 1 -- start start 2023-11-13T10:55:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f968f226640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96880ccbb0 0x7f96880ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f968f226640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f968815e0c0 0x7f96880cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f968f226640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96880ce2d0 con 0x7f96880cd920 2023-11-13T10:55:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f968f226640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96880cddb0 con 0x7f96880ccbb0 2023-11-13T10:55:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f968f226640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880ccf90 con 0x7f968815e0c0 2023-11-13T10:55:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f968cfc2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96880ccbb0 0x7f96880ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f968cfc2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96880ccbb0 0x7f96880ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42816/0 (socket says 172.21.15.16:42816) 2023-11-13T10:55:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f968cfc2640 1 -- 172.21.15.16:0/4160197182 learned_addr learned my addr 172.21.15.16:0/4160197182 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f968cfc2640 1 -- 172.21.15.16:0/4160197182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f968815e0c0 msgr2=0x7f96880cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.086+0000 7f9687fff640 1 --2- 172.21.15.16:0/4160197182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f968815e0c0 0x7f96880cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.087+0000 7f968cfc2640 1 --2- 172.21.15.16:0/4160197182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f968815e0c0 0x7f96880cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.087+0000 7f968cfc2640 1 -- 172.21.15.16:0/4160197182 >> v1:172.21.15.184:6789/0 conn(0x7f96880cd920 legacy=0x7f968815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.087+0000 7f968cfc2640 1 -- 172.21.15.16:0/4160197182 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880cf440 con 0x7f96880ccbb0 2023-11-13T10:55:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.087+0000 7f9687fff640 1 --2- 172.21.15.16:0/4160197182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f968815e0c0 0x7f96880cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.088+0000 7f968cfc2640 1 --2- 172.21.15.16:0/4160197182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880ccbb0 0x7f96880ceda0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9678009870 tx=0x7f967802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30d9a9a380f8b9c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.088+0000 7f96877fe640 1 -- 172.21.15.16:0/4160197182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678031640 con 0x7f96880ccbb0 2023-11-13T10:55:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.088+0000 7f96877fe640 1 -- 172.21.15.16:0/4160197182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96780317a0 con 0x7f96880ccbb0 2023-11-13T10:55:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.088+0000 7f96877fe640 1 -- 172.21.15.16:0/4160197182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678031b30 con 0x7f96880ccbb0 2023-11-13T10:55:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.089+0000 7f968f226640 1 -- 172.21.15.16:0/4160197182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880ccbb0 msgr2=0x7f96880ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:27.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.089+0000 7f968f226640 1 --2- 172.21.15.16:0/4160197182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880ccbb0 0x7f96880ceda0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9678009870 tx=0x7f967802b8f0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.089+0000 7f968f226640 1 -- 172.21.15.16:0/4160197182 shutdown_connections 2023-11-13T10:55:27.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.089+0000 7f968f226640 1 --2- 172.21.15.16:0/4160197182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f968815e0c0 0x7f96880cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.089+0000 7f968f226640 1 --2- 172.21.15.16:0/4160197182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880ccbb0 0x7f96880ceda0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.089+0000 7f968f226640 1 -- 172.21.15.16:0/4160197182 >> 172.21.15.16:0/4160197182 conn(0x7f96880b2690 msgr2=0x7f96880b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:27.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.089+0000 7f968f226640 1 -- 172.21.15.16:0/4160197182 shutdown_connections 2023-11-13T10:55:27.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.089+0000 7f968f226640 1 -- 172.21.15.16:0/4160197182 wait complete. 2023-11-13T10:55:27.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.091+0000 7f968f226640 1 Processor -- start 2023-11-13T10:55:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.091+0000 7f968f226640 1 -- start start 2023-11-13T10:55:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f968f226640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96880ccbb0 0x7f96881f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f968f226640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880cd920 0x7f96880b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f968f226640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f968815e0c0 0x7f96880b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f968f226640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96881f6680 con 0x7f96880cd920 2023-11-13T10:55:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f968f226640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96881f3260 con 0x7f968815e0c0 2023-11-13T10:55:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f968f226640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880c61a0 con 0x7f96880ccbb0 2023-11-13T10:55:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f9687fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880cd920 0x7f96880b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f9687fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880cd920 0x7f96880b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42826/0 (socket says 172.21.15.16:42826) 2023-11-13T10:55:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f9687fff640 1 -- 172.21.15.16:0/118405185 learned_addr learned my addr 172.21.15.16:0/118405185 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.092+0000 7f968cfc2640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96880ccbb0 0x7f96881f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f9687fff640 1 -- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96880ccbb0 msgr2=0x7f96881f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f9687fff640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96880ccbb0 0x7f96881f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f968d7c3640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f968815e0c0 0x7f96880b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f9687fff640 1 -- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f968815e0c0 msgr2=0x7f96880b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f9687fff640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f968815e0c0 0x7f96880b5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f9687fff640 1 -- 172.21.15.16:0/118405185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96780094d0 con 0x7f96880cd920 2023-11-13T10:55:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f968cfc2640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96880ccbb0 0x7f96881f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f9687fff640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880cd920 0x7f96880b52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f967000ce30 tx=0x7f967000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f96857fa640 1 -- 172.21.15.16:0/118405185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967000f890 con 0x7f96880cd920 2023-11-13T10:55:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f96857fa640 1 -- 172.21.15.16:0/118405185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967000f9f0 con 0x7f96880cd920 2023-11-13T10:55:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f96857fa640 1 -- 172.21.15.16:0/118405185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967000fd80 con 0x7f96880cd920 2023-11-13T10:55:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96880c5620 con 0x7f96880cd920 2023-11-13T10:55:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.093+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96880c5be0 con 0x7f96880cd920 2023-11-13T10:55:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.095+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96880677a0 con 0x7f96880cd920 2023-11-13T10:55:27.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.096+0000 7f96857fa640 1 -- 172.21.15.16:0/118405185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f967000b040 con 0x7f96880cd920 2023-11-13T10:55:27.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.096+0000 7f96857fa640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f966806ae40 0x7f966806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:27.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.097+0000 7f96857fa640 1 -- 172.21.15.16:0/118405185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f967008a090 con 0x7f96880cd920 2023-11-13T10:55:27.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.097+0000 7f968cfc2640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f966806ae40 0x7f966806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.098+0000 7f968cfc2640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f966806ae40 0x7f966806d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9678036000 tx=0x7f967803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.101+0000 7f96857fa640 1 -- 172.21.15.16:0/118405185 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96700550a0 con 0x7f96880cd920 2023-11-13T10:55:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.302+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f96880b31b0 con 0x7f96880cd920 2023-11-13T10:55:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.850+0000 7f96857fa640 1 -- 172.21.15.16:0/118405185 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) v1 ==== 157+0+0 (secure 0 0 0) 0x7f967001d0e0 con 0x7f96880cd920 2023-11-13T10:55:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.854+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f966806ae40 msgr2=0x7f966806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.854+0000 7f968f226640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f966806ae40 0x7f966806d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9678036000 tx=0x7f967803c040 comp rx=0 tx=0).stop 2023-11-13T10:55:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.854+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880cd920 msgr2=0x7f96880b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.854+0000 7f968f226640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880cd920 0x7f96880b52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f967000ce30 tx=0x7f967000a650 comp rx=0 tx=0).stop 2023-11-13T10:55:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.854+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 shutdown_connections 2023-11-13T10:55:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.854+0000 7f968f226640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f966806ae40 0x7f966806d300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.854+0000 7f968f226640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f968815e0c0 0x7f96880b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.854+0000 7f968f226640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96880cd920 0x7f96880b52d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.855+0000 7f968f226640 1 --2- 172.21.15.16:0/118405185 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96880ccbb0 0x7f96881f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.855+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 >> 172.21.15.16:0/118405185 conn(0x7f96880b2690 msgr2=0x7f96880af300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.855+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 shutdown_connections 2023-11-13T10:55:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:27.855+0000 7f968f226640 1 -- 172.21.15.16:0/118405185 wait complete. 2023-11-13T10:55:27.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-13T10:55:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.102+0000 7fa4661d0640 1 Processor -- start 2023-11-13T10:55:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.102+0000 7fa4661d0640 1 -- start start 2023-11-13T10:55:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.103+0000 7fa4661d0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4600d2ab0 0x7fa4600d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.103+0000 7fa4661d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4600d3450 con 0x7fa4600cd6c0 2023-11-13T10:55:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.103+0000 7fa4661d0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4600d1460 con 0x7fa460160440 2023-11-13T10:55:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.103+0000 7fa4661d0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4600d2540 con 0x7fa4600d2ab0 2023-11-13T10:55:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.104+0000 7fa4659cf640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa4600cd6c0 0x7fa4600cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51836/0 (socket says 172.21.15.16:51836) 2023-11-13T10:55:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.104+0000 7fa4659cf640 1 -- 172.21.15.16:0/880948465 learned_addr learned my addr 172.21.15.16:0/880948465 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.104+0000 7fa4649cd640 1 --2- 172.21.15.16:0/880948465 >> v2:172.21.15.184:3300/0 conn(0x7fa4600d2ab0 0x7fa4600d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.104+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828513128 0 0) 0x7fa4600d3450 con 0x7fa4600cd6c0 2023-11-13T10:55:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.104+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa44c003520 con 0x7fa4600cd6c0 2023-11-13T10:55:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1546305742 0 0) 0x7fa44c003520 con 0x7fa4600cd6c0 2023-11-13T10:55:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 >> v2:172.21.15.184:3300/0 conn(0x7fa4600d2ab0 msgr2=0x7fa4600d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 --2- 172.21.15.16:0/880948465 >> v2:172.21.15.184:3300/0 conn(0x7fa4600d2ab0 0x7fa4600d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 >> v1:172.21.15.184:6789/0 conn(0x7fa460160440 legacy=0x7fa4600b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600d0890 con 0x7fa4600cd6c0 2023-11-13T10:55:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fa4580019f0 con 0x7fa4600cd6c0 2023-11-13T10:55:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 >> v1:172.21.15.16:6789/0 conn(0x7fa4600cd6c0 legacy=0x7fa4600cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c003970 0x7fa44c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0047c0 0x7fa44c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c005180 0x7fa44c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600d2540 con 0x7fa44c0047c0 2023-11-13T10:55:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c00c020 con 0x7fa44c003970 2023-11-13T10:55:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c00c1e0 con 0x7fa44c005180 2023-11-13T10:55:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa4649cd640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0047c0 0x7fa44c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa4659cf640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c005180 0x7fa44c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa4651ce640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c003970 0x7fa44c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa4649cd640 1 -- 172.21.15.16:0/880948465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c005180 msgr2=0x7fa44c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa4649cd640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c005180 0x7fa44c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa4649cd640 1 -- 172.21.15.16:0/880948465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c003970 msgr2=0x7fa44c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.105+0000 7fa4649cd640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa44c003970 0x7fa44c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.106+0000 7fa4649cd640 1 -- 172.21.15.16:0/880948465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c003520 con 0x7fa44c0047c0 2023-11-13T10:55:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.106+0000 7fa4659cf640 1 -- 172.21.15.16:0/880948465 reap_dead start 2023-11-13T10:55:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.107+0000 7fa4649cd640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0047c0 0x7fa44c004c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa450009ac0 tx=0x7fa44c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5e9929cfba7d99 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.107+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45002c6b0 con 0x7fa44c0047c0 2023-11-13T10:55:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.107+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45002c810 con 0x7fa44c0047c0 2023-11-13T10:55:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.107+0000 7fa457fff640 1 -- 172.21.15.16:0/880948465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45002cba0 con 0x7fa44c0047c0 2023-11-13T10:55:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.108+0000 7fa4661d0640 1 -- 172.21.15.16:0/880948465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0047c0 msgr2=0x7fa44c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.108+0000 7fa4661d0640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0047c0 0x7fa44c004c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa450009ac0 tx=0x7fa44c0057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.108+0000 7fa4661d0640 1 -- 172.21.15.16:0/880948465 shutdown_connections 2023-11-13T10:55:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.108+0000 7fa4661d0640 1 --2- 172.21.15.16:0/880948465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0047c0 0x7fa44c004c40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.108+0000 7fa4661d0640 1 -- 172.21.15.16:0/880948465 >> 172.21.15.16:0/880948465 conn(0x7fa46008c560 msgr2=0x7fa46009be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:28.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.108+0000 7fa4661d0640 1 -- 172.21.15.16:0/880948465 shutdown_connections 2023-11-13T10:55:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.108+0000 7fa4661d0640 1 -- 172.21.15.16:0/880948465 wait complete. 2023-11-13T10:55:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.110+0000 7fa4661d0640 1 Processor -- start 2023-11-13T10:55:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.110+0000 7fa4661d0640 1 -- start start 2023-11-13T10:55:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4661d0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c0047c0 0x7fa4600cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4661d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460152e60 0x7fa46015d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4661d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46015da50 0x7fa46015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4661d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4601f2790 con 0x7fa460152e60 2023-11-13T10:55:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4661d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600cca00 con 0x7fa46015da50 2023-11-13T10:55:28.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4661d0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600b5870 con 0x7fa44c0047c0 2023-11-13T10:55:28.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4649cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460152e60 0x7fa46015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4649cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460152e60 0x7fa46015d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42836/0 (socket says 172.21.15.16:42836) 2023-11-13T10:55:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4649cd640 1 -- 172.21.15.16:0/2576793600 learned_addr learned my addr 172.21.15.16:0/2576793600 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4659cf640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46015da50 0x7fa46015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4649cd640 1 -- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c0047c0 msgr2=0x7fa4600cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4649cd640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c0047c0 0x7fa4600cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4649cd640 1 -- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46015da50 msgr2=0x7fa46015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4649cd640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46015da50 0x7fa46015deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.111+0000 7fa4649cd640 1 -- 172.21.15.16:0/2576793600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600b5b50 con 0x7fa460152e60 2023-11-13T10:55:28.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.112+0000 7fa4649cd640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460152e60 0x7fa46015d510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa45002c250 tx=0x7fa450004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:28.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.112+0000 7fa455ffb640 1 -- 172.21.15.16:0/2576793600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450003810 con 0x7fa460152e60 2023-11-13T10:55:28.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.112+0000 7fa455ffb640 1 -- 172.21.15.16:0/2576793600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa450003970 con 0x7fa460152e60 2023-11-13T10:55:28.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.112+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4600b5e30 con 0x7fa460152e60 2023-11-13T10:55:28.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.112+0000 7fa455ffb640 1 -- 172.21.15.16:0/2576793600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450003d00 con 0x7fa460152e60 2023-11-13T10:55:28.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.112+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4600c6910 con 0x7fa460152e60 2023-11-13T10:55:28.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.113+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4600c6ad0 con 0x7fa460152e60 2023-11-13T10:55:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.114+0000 7fa455ffb640 1 -- 172.21.15.16:0/2576793600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa450034440 con 0x7fa460152e60 2023-11-13T10:55:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.115+0000 7fa455ffb640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa44406ae40 0x7fa44406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.115+0000 7fa455ffb640 1 -- 172.21.15.16:0/2576793600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fa4500ab9b0 con 0x7fa460152e60 2023-11-13T10:55:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.116+0000 7fa4651ce640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa44406ae40 0x7fa44406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:28.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.117+0000 7fa4651ce640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa44406ae40 0x7fa44406d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa460063e10 tx=0x7fa45c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:28.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.119+0000 7fa455ffb640 1 -- 172.21.15.16:0/2576793600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa450076a30 con 0x7fa460152e60 2023-11-13T10:55:28.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:28.306+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa4601f2770 con 0x7fa460152e60 2023-11-13T10:55:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.868+0000 7fa455ffb640 1 -- 172.21.15.16:0/2576793600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) v1 ==== 228+0+0 (secure 0 0 0) 0x7fa45004c100 con 0x7fa460152e60 2023-11-13T10:55:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2023-11-13T10:55:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.872+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa44406ae40 msgr2=0x7fa44406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.872+0000 7fa4661d0640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa44406ae40 0x7fa44406d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa460063e10 tx=0x7fa45c009040 comp rx=0 tx=0).stop 2023-11-13T10:55:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.872+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460152e60 msgr2=0x7fa46015d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.872+0000 7fa4661d0640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460152e60 0x7fa46015d510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa45002c250 tx=0x7fa450004730 comp rx=0 tx=0).stop 2023-11-13T10:55:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.873+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 shutdown_connections 2023-11-13T10:55:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.873+0000 7fa4661d0640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa44406ae40 0x7fa44406d300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.873+0000 7fa4661d0640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46015da50 0x7fa46015deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.873+0000 7fa4661d0640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460152e60 0x7fa46015d510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.873+0000 7fa4661d0640 1 --2- 172.21.15.16:0/2576793600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa44c0047c0 0x7fa4600cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.873+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 >> 172.21.15.16:0/2576793600 conn(0x7fa46008c560 msgr2=0x7fa4600ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.873+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 shutdown_connections 2023-11-13T10:55:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:29.873+0000 7fa4661d0640 1 -- 172.21.15.16:0/2576793600 wait complete. 2023-11-13T10:55:29.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-13T10:55:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.095+0000 7fd7fef6c640 1 Processor -- start 2023-11-13T10:55:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.095+0000 7fd7fef6c640 1 -- start start 2023-11-13T10:55:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.096+0000 7fd7fef6c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7f80ce4c0 0x7fd7f80ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.096+0000 7fd7fef6c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80cf1f0 con 0x7fd7f80b66b0 2023-11-13T10:55:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.096+0000 7fd7fef6c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80b5bd0 con 0x7fd7f80cf580 2023-11-13T10:55:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.096+0000 7fd7fef6c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80b59c0 con 0x7fd7f80ce4c0 2023-11-13T10:55:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.097+0000 7fd7fe76b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd7f80b66b0 0x7fd7f80b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51846/0 (socket says 172.21.15.16:51846) 2023-11-13T10:55:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.097+0000 7fd7fe76b640 1 -- 172.21.15.16:0/1515211172 learned_addr learned my addr 172.21.15.16:0/1515211172 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.097+0000 7fd7fd769640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7f80ce4c0 0x7fd7f80ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.097+0000 7fd7fcf68640 1 -- 172.21.15.16:0/1515211172 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190020556 0 0) 0x7fd7f80cf1f0 con 0x7fd7f80b66b0 2023-11-13T10:55:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.097+0000 7fd7fd769640 1 -- 172.21.15.16:0/1515211172 >> v1:172.21.15.184:6789/0 conn(0x7fd7f80cf580 legacy=0x7fd7f80cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.097+0000 7fd7fd769640 1 -- 172.21.15.16:0/1515211172 >> v1:172.21.15.16:6789/0 conn(0x7fd7f80b66b0 legacy=0x7fd7f80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.097+0000 7fd7fd769640 1 -- 172.21.15.16:0/1515211172 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80b5eb0 con 0x7fd7f80ce4c0 2023-11-13T10:55:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.098+0000 7fd7fd769640 1 --2- 172.21.15.16:0/1515211172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80ce4c0 0x7fd7f80ce920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0097a0 tx=0x7fd7ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ed52d3c5fddf82 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.099+0000 7fd7fcf68640 1 -- 172.21.15.16:0/1515211172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec030610 con 0x7fd7f80ce4c0 2023-11-13T10:55:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.099+0000 7fd7fcf68640 1 -- 172.21.15.16:0/1515211172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec030770 con 0x7fd7f80ce4c0 2023-11-13T10:55:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.099+0000 7fd7fcf68640 1 -- 172.21.15.16:0/1515211172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec030b00 con 0x7fd7f80ce4c0 2023-11-13T10:55:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.099+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1515211172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80ce4c0 msgr2=0x7fd7f80ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:30.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.099+0000 7fd7fef6c640 1 --2- 172.21.15.16:0/1515211172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80ce4c0 0x7fd7f80ce920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0097a0 tx=0x7fd7ec02b960 comp rx=0 tx=0).stop 2023-11-13T10:55:30.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.099+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1515211172 shutdown_connections 2023-11-13T10:55:30.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.099+0000 7fd7fef6c640 1 --2- 172.21.15.16:0/1515211172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80ce4c0 0x7fd7f80ce920 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.099+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1515211172 >> 172.21.15.16:0/1515211172 conn(0x7fd7f80d2b70 msgr2=0x7fd7f80d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.100+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1515211172 shutdown_connections 2023-11-13T10:55:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.100+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1515211172 wait complete. 2023-11-13T10:55:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.101+0000 7fd7fef6c640 1 Processor -- start 2023-11-13T10:55:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.101+0000 7fd7fef6c640 1 -- start start 2023-11-13T10:55:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.102+0000 7fd7fef6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80b66b0 0x7fd7f81560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.102+0000 7fd7fef6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f80ce4c0 0x7fd7f81e3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.102+0000 7fd7fef6c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7f80cf580 0x7fd7f81e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.102+0000 7fd7fef6c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8155f10 con 0x7fd7f80b66b0 2023-11-13T10:55:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.102+0000 7fd7fef6c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81e4000 con 0x7fd7f80ce4c0 2023-11-13T10:55:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.102+0000 7fd7fef6c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81e42b0 con 0x7fd7f80cf580 2023-11-13T10:55:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fdf6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80b66b0 0x7fd7f81560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fdf6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80b66b0 0x7fd7f81560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42862/0 (socket says 172.21.15.16:42862) 2023-11-13T10:55:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fdf6a640 1 -- 172.21.15.16:0/1027558217 learned_addr learned my addr 172.21.15.16:0/1027558217 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fd769640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f80ce4c0 0x7fd7f81e3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fdf6a640 1 -- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7f80cf580 msgr2=0x7fd7f81e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fdf6a640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7f80cf580 0x7fd7f81e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fdf6a640 1 -- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f80ce4c0 msgr2=0x7fd7f81e3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fdf6a640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f80ce4c0 0x7fd7f81e3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fdf6a640 1 -- 172.21.15.16:0/1027558217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec009400 con 0x7fd7f80b66b0 2023-11-13T10:55:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fd769640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f80ce4c0 0x7fd7f81e3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.103+0000 7fd7fdf6a640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80b66b0 0x7fd7f81560f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c400 tx=0x7fd7f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.104+0000 7fd7ea7fc640 1 -- 172.21.15.16:0/1027558217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f40052d0 con 0x7fd7f80b66b0 2023-11-13T10:55:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.104+0000 7fd7ea7fc640 1 -- 172.21.15.16:0/1027558217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f4005430 con 0x7fd7f80b66b0 2023-11-13T10:55:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.104+0000 7fd7ea7fc640 1 -- 172.21.15.16:0/1027558217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4003830 con 0x7fd7f80b66b0 2023-11-13T10:55:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.104+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f80c7500 con 0x7fd7f80b66b0 2023-11-13T10:55:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.104+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f80c7ac0 con 0x7fd7f80b66b0 2023-11-13T10:55:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.105+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f8159b00 con 0x7fd7f80b66b0 2023-11-13T10:55:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.106+0000 7fd7ea7fc640 1 -- 172.21.15.16:0/1027558217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd7f4003990 con 0x7fd7f80b66b0 2023-11-13T10:55:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.107+0000 7fd7ea7fc640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd7dc06ae40 0x7fd7dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.107+0000 7fd7ea7fc640 1 -- 172.21.15.16:0/1027558217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fd7f4088e70 con 0x7fd7f80b66b0 2023-11-13T10:55:30.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.107+0000 7fd7fd769640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd7dc06ae40 0x7fd7dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:30.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.109+0000 7fd7fd769640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd7dc06ae40 0x7fd7dc06d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec036000 tx=0x7fd7ec03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:30.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.116+0000 7fd7ea7fc640 1 -- 172.21.15.16:0/1027558217 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7f4053c40 con 0x7fd7f80b66b0 2023-11-13T10:55:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.300+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd7f804e4d0 con 0x7fd7f80b66b0 2023-11-13T10:55:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.880+0000 7fd7ea7fc640 1 -- 172.21.15.16:0/1027558217 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd7f40170c0 con 0x7fd7f80b66b0 2023-11-13T10:55:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2023-11-13T10:55:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.883+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd7dc06ae40 msgr2=0x7fd7dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.883+0000 7fd7fef6c640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd7dc06ae40 0x7fd7dc06d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec036000 tx=0x7fd7ec03b040 comp rx=0 tx=0).stop 2023-11-13T10:55:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.883+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80b66b0 msgr2=0x7fd7f81560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.883+0000 7fd7fef6c640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80b66b0 0x7fd7f81560f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c400 tx=0x7fd7f400c8c0 comp rx=0 tx=0).stop 2023-11-13T10:55:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.884+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 shutdown_connections 2023-11-13T10:55:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.884+0000 7fd7fef6c640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd7dc06ae40 0x7fd7dc06d300 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec036000 tx=0x7fd7ec03b040 comp rx=0 tx=0).stop 2023-11-13T10:55:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.884+0000 7fd7fef6c640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7f80cf580 0x7fd7f81e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.884+0000 7fd7fef6c640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7f80ce4c0 0x7fd7f81e3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.884+0000 7fd7fef6c640 1 --2- 172.21.15.16:0/1027558217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7f80b66b0 0x7fd7f81560f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.884+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 >> 172.21.15.16:0/1027558217 conn(0x7fd7f80d2b70 msgr2=0x7fd7f815b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.886+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 shutdown_connections 2023-11-13T10:55:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:30.886+0000 7fd7fef6c640 1 -- 172.21.15.16:0/1027558217 wait complete. 2023-11-13T10:55:30.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-13T10:55:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.116+0000 7f307242e640 1 Processor -- start 2023-11-13T10:55:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.116+0000 7f307242e640 1 -- start start 2023-11-13T10:55:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.118+0000 7f307242e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f306c15ce70 0x7f306c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.118+0000 7f307242e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0ce0e0 con 0x7f306c0cdbb0 2023-11-13T10:55:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.118+0000 7f307242e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0ce3c0 con 0x7f306c0d1850 2023-11-13T10:55:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.118+0000 7f307242e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c0ce6a0 con 0x7f306c15ce70 2023-11-13T10:55:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.118+0000 7f3070c2b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f306c0cdbb0 0x7f306c0d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51856/0 (socket says 172.21.15.16:51856) 2023-11-13T10:55:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.118+0000 7f3070c2b640 1 -- 172.21.15.16:0/2447838710 learned_addr learned my addr 172.21.15.16:0/2447838710 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.118+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217119762 0 0) 0x7f306c0ce0e0 con 0x7f306c0cdbb0 2023-11-13T10:55:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.118+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3058003520 con 0x7f306c0cdbb0 2023-11-13T10:55:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.118+0000 7f307142c640 1 --2- 172.21.15.16:0/2447838710 >> v2:172.21.15.184:3300/0 conn(0x7f306c15ce70 0x7f306c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1130022563 0 0) 0x7f3058003520 con 0x7f306c0cdbb0 2023-11-13T10:55:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 >> v2:172.21.15.184:3300/0 conn(0x7f306c15ce70 msgr2=0x7f306c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 --2- 172.21.15.16:0/2447838710 >> v2:172.21.15.184:3300/0 conn(0x7f306c15ce70 0x7f306c0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 >> v1:172.21.15.184:6789/0 conn(0x7f306c0d1850 legacy=0x7f306c0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c0ce980 con 0x7f306c0cdbb0 2023-11-13T10:55:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f305c001670 con 0x7f306c0cdbb0 2023-11-13T10:55:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f307142c640 1 --2- 172.21.15.16:0/2447838710 >> v2:172.21.15.184:3300/0 conn(0x7f306c15ce70 0x7f306c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 >> v1:172.21.15.16:6789/0 conn(0x7f306c0cdbb0 legacy=0x7f306c0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 --2- 172.21.15.16:0/2447838710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3058003970 0x7f3058003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 --2- 172.21.15.16:0/2447838710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30580047c0 0x7f3058004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:31.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 --2- 172.21.15.16:0/2447838710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3058005180 0x7f305800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0ce6a0 con 0x7f3058003970 2023-11-13T10:55:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305800c0b0 con 0x7f3058005180 2023-11-13T10:55:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305800c270 con 0x7f30580047c0 2023-11-13T10:55:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.119+0000 7f307142c640 1 --2- 172.21.15.16:0/2447838710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3058003970 0x7f3058003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.120+0000 7f307142c640 1 -- 172.21.15.16:0/2447838710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30580047c0 msgr2=0x7f3058004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.120+0000 7f307142c640 1 --2- 172.21.15.16:0/2447838710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f30580047c0 0x7f3058004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.120+0000 7f307142c640 1 -- 172.21.15.16:0/2447838710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3058005180 msgr2=0x7f305800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.120+0000 7f307142c640 1 --2- 172.21.15.16:0/2447838710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3058005180 0x7f305800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.120+0000 7f307142c640 1 -- 172.21.15.16:0/2447838710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3058003520 con 0x7f3058003970 2023-11-13T10:55:31.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.120+0000 7f3071c2d640 1 -- 172.21.15.16:0/2447838710 reap_dead start 2023-11-13T10:55:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.121+0000 7f307142c640 1 --2- 172.21.15.16:0/2447838710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3058003970 0x7f3058003d70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3068006fd0 tx=0x7f30580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf71a2369cde2c87 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.121+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068004230 con 0x7f3058003970 2023-11-13T10:55:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.121+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3068004390 con 0x7f3058003970 2023-11-13T10:55:31.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.121+0000 7f3063fff640 1 -- 172.21.15.16:0/2447838710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068003830 con 0x7f3058003970 2023-11-13T10:55:31.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.121+0000 7f307242e640 1 -- 172.21.15.16:0/2447838710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3058003970 msgr2=0x7f3058003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:31.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.121+0000 7f307242e640 1 --2- 172.21.15.16:0/2447838710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3058003970 0x7f3058003d70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3068006fd0 tx=0x7f30580057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.121+0000 7f307242e640 1 -- 172.21.15.16:0/2447838710 shutdown_connections 2023-11-13T10:55:31.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.121+0000 7f307242e640 1 --2- 172.21.15.16:0/2447838710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3058003970 0x7f3058003d70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.121+0000 7f307242e640 1 -- 172.21.15.16:0/2447838710 >> 172.21.15.16:0/2447838710 conn(0x7f306c08c580 msgr2=0x7f306c09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:31.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.122+0000 7f307242e640 1 -- 172.21.15.16:0/2447838710 shutdown_connections 2023-11-13T10:55:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.122+0000 7f307242e640 1 -- 172.21.15.16:0/2447838710 wait complete. 2023-11-13T10:55:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.123+0000 7f307242e640 1 Processor -- start 2023-11-13T10:55:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.124+0000 7f307242e640 1 -- start start 2023-11-13T10:55:31.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.124+0000 7f307242e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3058003970 0x7f306c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:31.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.124+0000 7f307242e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c15ea10 0x7f306c0b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:31.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.124+0000 7f307242e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c0b7b80 0x7f306c1f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:31.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.124+0000 7f307242e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0c7760 con 0x7f306c15ea10 2023-11-13T10:55:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.124+0000 7f307242e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0b8000 con 0x7f3058003970 2023-11-13T10:55:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.124+0000 7f307242e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c15d3d0 con 0x7f306c0b7b80 2023-11-13T10:55:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f3070c2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c15ea10 0x7f306c0b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f3070c2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c15ea10 0x7f306c0b7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42874/0 (socket says 172.21.15.16:42874) 2023-11-13T10:55:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f3070c2b640 1 -- 172.21.15.16:0/56262135 learned_addr learned my addr 172.21.15.16:0/56262135 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f307142c640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3058003970 0x7f306c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f3071c2d640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c0b7b80 0x7f306c1f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f3070c2b640 1 -- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c0b7b80 msgr2=0x7f306c1f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f3070c2b640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c0b7b80 0x7f306c1f65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f3070c2b640 1 -- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3058003970 msgr2=0x7f306c0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f3070c2b640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3058003970 0x7f306c0c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.125+0000 7f3070c2b640 1 -- 172.21.15.16:0/56262135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c152170 con 0x7f306c15ea10 2023-11-13T10:55:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.126+0000 7f3070c2b640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c15ea10 0x7f306c0b7640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f305c00c6c0 tx=0x7f305c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.126+0000 7f3061ffb640 1 -- 172.21.15.16:0/56262135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c0072d0 con 0x7f306c15ea10 2023-11-13T10:55:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.126+0000 7f3061ffb640 1 -- 172.21.15.16:0/56262135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c007430 con 0x7f306c15ea10 2023-11-13T10:55:31.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.126+0000 7f3061ffb640 1 -- 172.21.15.16:0/56262135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c0047e0 con 0x7f306c15ea10 2023-11-13T10:55:31.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.126+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c0c6ae0 con 0x7f306c15ea10 2023-11-13T10:55:31.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.126+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c0c70a0 con 0x7f306c15ea10 2023-11-13T10:55:31.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.127+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c0c9a00 con 0x7f306c15ea10 2023-11-13T10:55:31.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.128+0000 7f3061ffb640 1 -- 172.21.15.16:0/56262135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f305c004940 con 0x7f306c15ea10 2023-11-13T10:55:31.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.129+0000 7f3061ffb640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f305006ae70 0x7f305006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.129+0000 7f3061ffb640 1 -- 172.21.15.16:0/56262135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f305c088b00 con 0x7f306c15ea10 2023-11-13T10:55:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.130+0000 7f307142c640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f305006ae70 0x7f305006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.131+0000 7f307142c640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f305006ae70 0x7f305006d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3068035000 tx=0x7f306803a040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.134+0000 7f3061ffb640 1 -- 172.21.15.16:0/56262135 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f305c08ae60 con 0x7f306c15ea10 2023-11-13T10:55:31.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.367+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f306c152170 con 0x7f306c15ea10 2023-11-13T10:55:31.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.887+0000 7f3061ffb640 1 -- 172.21.15.16:0/56262135 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f306c152170 con 0x7f306c15ea10 2023-11-13T10:55:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2023-11-13T10:55:31.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f305006ae70 msgr2=0x7f305006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:31.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f305006ae70 0x7f305006d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3068035000 tx=0x7f306803a040 comp rx=0 tx=0).stop 2023-11-13T10:55:31.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c15ea10 msgr2=0x7f306c0b7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:31.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c15ea10 0x7f306c0b7640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f305c00c6c0 tx=0x7f305c00cb80 comp rx=0 tx=0).stop 2023-11-13T10:55:31.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 shutdown_connections 2023-11-13T10:55:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f305006ae70 0x7f305006d330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c0b7b80 0x7f306c1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c15ea10 0x7f306c0b7640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 --2- 172.21.15.16:0/56262135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3058003970 0x7f306c0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 >> 172.21.15.16:0/56262135 conn(0x7f306c08c580 msgr2=0x7f306c15bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 shutdown_connections 2023-11-13T10:55:31.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:31.891+0000 7f307242e640 1 -- 172.21.15.16:0/56262135 wait complete. 2023-11-13T10:55:31.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:55:32.096 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-11-13T10:55:32.092+0000 7f16742f5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-13T10:55:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.132+0000 7ff992c83640 1 Processor -- start 2023-11-13T10:55:32.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.132+0000 7ff992c83640 1 -- start start 2023-11-13T10:55:32.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.134+0000 7ff992c83640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff98c06a190 0x7ff98c06a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.134+0000 7ff992c83640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c158c20 con 0x7ff98c0cb190 2023-11-13T10:55:32.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.134+0000 7ff992c83640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c0b1830 con 0x7ff98c15b190 2023-11-13T10:55:32.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.134+0000 7ff992c83640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98c06ab40 con 0x7ff98c06a190 2023-11-13T10:55:32.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.134+0000 7ff991c81640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff98c0cb190 0x7ff98c0716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51860/0 (socket says 172.21.15.16:51860) 2023-11-13T10:55:32.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.134+0000 7ff991c81640 1 -- 172.21.15.16:0/2640474930 learned_addr learned my addr 172.21.15.16:0/2640474930 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:32.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff992482640 1 --2- 172.21.15.16:0/2640474930 >> v2:172.21.15.184:3300/0 conn(0x7ff98c06a190 0x7ff98c06a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866610700 0 0) 0x7ff98c158c20 con 0x7ff98c0cb190 2023-11-13T10:55:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff978003520 con 0x7ff98c0cb190 2023-11-13T10:55:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3352806674 0 0) 0x7ff978003520 con 0x7ff98c0cb190 2023-11-13T10:55:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 >> v2:172.21.15.184:3300/0 conn(0x7ff98c06a190 msgr2=0x7ff98c06a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:32.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 --2- 172.21.15.16:0/2640474930 >> v2:172.21.15.184:3300/0 conn(0x7ff98c06a190 0x7ff98c06a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 >> v1:172.21.15.184:6789/0 conn(0x7ff98c15b190 legacy=0x7ff98c06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:32.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c15b780 con 0x7ff98c0cb190 2023-11-13T10:55:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7ff988001670 con 0x7ff98c0cb190 2023-11-13T10:55:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 >> v1:172.21.15.16:6789/0 conn(0x7ff98c0cb190 legacy=0x7ff98c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff978003970 0x7ff978003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff978004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff978005180 0x7ff97800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c06ab40 con 0x7ff9780047c0 2023-11-13T10:55:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97800c0b0 con 0x7ff978005180 2023-11-13T10:55:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.135+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97800c270 con 0x7ff978003970 2023-11-13T10:55:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.136+0000 7ff991480640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff978004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.136+0000 7ff992482640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff978005180 0x7ff97800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.136+0000 7ff991c81640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff978003970 0x7ff978003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.136+0000 7ff991480640 1 -- 172.21.15.16:0/2640474930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff978003970 msgr2=0x7ff978003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.136+0000 7ff991480640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff978003970 0x7ff978003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.136+0000 7ff991480640 1 -- 172.21.15.16:0/2640474930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff978005180 msgr2=0x7ff97800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.136+0000 7ff991480640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff978005180 0x7ff97800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.136+0000 7ff991480640 1 -- 172.21.15.16:0/2640474930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff978003520 con 0x7ff9780047c0 2023-11-13T10:55:32.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.136+0000 7ff992482640 1 -- 172.21.15.16:0/2640474930 reap_dead start 2023-11-13T10:55:32.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.137+0000 7ff991480640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff978004c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff97c003db0 tx=0x7ff9780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64adbf3a0348ac6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.137+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c032600 con 0x7ff9780047c0 2023-11-13T10:55:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.137+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c02e050 con 0x7ff9780047c0 2023-11-13T10:55:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.137+0000 7ff990c7f640 1 -- 172.21.15.16:0/2640474930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0329e0 con 0x7ff9780047c0 2023-11-13T10:55:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.138+0000 7ff992c83640 1 -- 172.21.15.16:0/2640474930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 msgr2=0x7ff978004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.138+0000 7ff992c83640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff978004c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff97c003db0 tx=0x7ff9780057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.138+0000 7ff992c83640 1 -- 172.21.15.16:0/2640474930 shutdown_connections 2023-11-13T10:55:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.138+0000 7ff992c83640 1 --2- 172.21.15.16:0/2640474930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff978004c40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.138+0000 7ff992c83640 1 -- 172.21.15.16:0/2640474930 >> 172.21.15.16:0/2640474930 conn(0x7ff98c0b04a0 msgr2=0x7ff98c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.138+0000 7ff992c83640 1 -- 172.21.15.16:0/2640474930 shutdown_connections 2023-11-13T10:55:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.138+0000 7ff992c83640 1 -- 172.21.15.16:0/2640474930 wait complete. 2023-11-13T10:55:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.140+0000 7ff992c83640 1 Processor -- start 2023-11-13T10:55:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.140+0000 7ff992c83640 1 -- start start 2023-11-13T10:55:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.141+0000 7ff992c83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff98c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.141+0000 7ff992c83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06cb30 0x7ff98c1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.141+0000 7ff992c83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0cd490 0x7ff98c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.141+0000 7ff992c83640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c0cde80 con 0x7ff9780047c0 2023-11-13T10:55:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.141+0000 7ff992c83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1f2810 con 0x7ff98c06cb30 2023-11-13T10:55:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.141+0000 7ff992c83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c0c95c0 con 0x7ff98c0cd490 2023-11-13T10:55:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.141+0000 7ff991c81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff98c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.141+0000 7ff991c81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff98c0c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42906/0 (socket says 172.21.15.16:42906) 2023-11-13T10:55:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.141+0000 7ff991c81640 1 -- 172.21.15.16:0/1544786274 learned_addr learned my addr 172.21.15.16:0/1544786274 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff992482640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0cd490 0x7ff98c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff991c81640 1 -- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0cd490 msgr2=0x7ff98c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff991480640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06cb30 0x7ff98c1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff991c81640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0cd490 0x7ff98c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff991c81640 1 -- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06cb30 msgr2=0x7ff98c1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff991c81640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06cb30 0x7ff98c1546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff991c81640 1 -- 172.21.15.16:0/1544786274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c0c8d20 con 0x7ff9780047c0 2023-11-13T10:55:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff991c81640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff98c0c8740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff98800d7e0 tx=0x7ff98800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff9827fc640 1 -- 172.21.15.16:0/1544786274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880074e0 con 0x7ff9780047c0 2023-11-13T10:55:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff9827fc640 1 -- 172.21.15.16:0/1544786274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff988007640 con 0x7ff9780047c0 2023-11-13T10:55:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff9827fc640 1 -- 172.21.15.16:0/1544786274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988004a00 con 0x7ff9780047c0 2023-11-13T10:55:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c0c9000 con 0x7ff9780047c0 2023-11-13T10:55:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.142+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c1f2a20 con 0x7ff9780047c0 2023-11-13T10:55:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.144+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98c1f2c40 con 0x7ff9780047c0 2023-11-13T10:55:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.145+0000 7ff9827fc640 1 -- 172.21.15.16:0/1544786274 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff988004b60 con 0x7ff9780047c0 2023-11-13T10:55:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.146+0000 7ff9827fc640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff97006ae90 0x7ff97006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.146+0000 7ff9827fc640 1 -- 172.21.15.16:0/1544786274 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff988089360 con 0x7ff9780047c0 2023-11-13T10:55:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.146+0000 7ff991480640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff97006ae90 0x7ff97006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.147+0000 7ff991480640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff97006ae90 0x7ff97006d350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff97c034000 tx=0x7ff97c009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:32.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.150+0000 7ff9827fc640 1 -- 172.21.15.16:0/1544786274 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff988054340 con 0x7ff9780047c0 2023-11-13T10:55:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.382+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff98c04e4d0 con 0x7ff9780047c0 2023-11-13T10:55:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.384+0000 7ff9827fc640 1 -- 172.21.15.16:0/1544786274 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7ff988057960 con 0x7ff9780047c0 2023-11-13T10:55:32.389 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:32.389 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:31.370515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-13T10:55:32.389 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-11-13T10:55:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.386+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff97006ae90 msgr2=0x7ff97006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.386+0000 7ff992c83640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff97006ae90 0x7ff97006d350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff97c034000 tx=0x7ff97c009030 comp rx=0 tx=0).stop 2023-11-13T10:55:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.386+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 msgr2=0x7ff98c0c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.386+0000 7ff992c83640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff98c0c8740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff98800d7e0 tx=0x7ff98800dca0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.387+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 shutdown_connections 2023-11-13T10:55:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.387+0000 7ff992c83640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff97006ae90 0x7ff97006d350 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.387+0000 7ff992c83640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff98c0cd490 0x7ff98c0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.387+0000 7ff992c83640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff98c06cb30 0x7ff98c1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.387+0000 7ff992c83640 1 --2- 172.21.15.16:0/1544786274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9780047c0 0x7ff98c0c8740 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.387+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 >> 172.21.15.16:0/1544786274 conn(0x7ff98c0b04a0 msgr2=0x7ff98c157ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.387+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 shutdown_connections 2023-11-13T10:55:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.387+0000 7ff992c83640 1 -- 172.21.15.16:0/1544786274 wait complete. 2023-11-13T10:55:32.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.632+0000 7f55826c9640 1 Processor -- start 2023-11-13T10:55:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.632+0000 7f55826c9640 1 -- start start 2023-11-13T10:55:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.634+0000 7f55826c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f557c155410 0x7f557c0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.634+0000 7f55826c9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f557c0ce990 0x7f557c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.634+0000 7f55826c9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c15b500 con 0x7f557c071440 2023-11-13T10:55:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.634+0000 7f55826c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557c15b7e0 con 0x7f557c155410 2023-11-13T10:55:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.634+0000 7f55826c9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557c15bac0 con 0x7f557c0ce990 2023-11-13T10:55:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.634+0000 7f557bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f557c155410 0x7f557c0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.634+0000 7f557bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f557c155410 0x7f557c0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42922/0 (socket says 172.21.15.16:42922) 2023-11-13T10:55:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.635+0000 7f557bfff640 1 -- 172.21.15.16:0/1173112420 learned_addr learned my addr 172.21.15.16:0/1173112420 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.635+0000 7f55737fe640 1 --2- 172.21.15.16:0/1173112420 >> v2:172.21.15.184:3300/0 conn(0x7f557c0ce990 0x7f557c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.635+0000 7f557bfff640 1 -- 172.21.15.16:0/1173112420 >> v2:172.21.15.184:3300/0 conn(0x7f557c0ce990 msgr2=0x7f557c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:32.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.635+0000 7f557bfff640 1 --2- 172.21.15.16:0/1173112420 >> v2:172.21.15.184:3300/0 conn(0x7f557c0ce990 0x7f557c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.635+0000 7f557bfff640 1 -- 172.21.15.16:0/1173112420 >> v1:172.21.15.184:6789/0 conn(0x7f557c071440 legacy=0x7f557c06fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:32.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.635+0000 7f557bfff640 1 -- 172.21.15.16:0/1173112420 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c15bda0 con 0x7f557c155410 2023-11-13T10:55:32.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.635+0000 7f55737fe640 1 --2- 172.21.15.16:0/1173112420 >> v2:172.21.15.184:3300/0 conn(0x7f557c0ce990 0x7f557c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.636+0000 7f557bfff640 1 --2- 172.21.15.16:0/1173112420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c155410 0x7f557c0ccc60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f557c06e000 tx=0x7f556c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312e4fbeea327ff5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.636+0000 7f557b7fe640 1 -- 172.21.15.16:0/1173112420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c031870 con 0x7f557c155410 2023-11-13T10:55:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.637+0000 7f557b7fe640 1 -- 172.21.15.16:0/1173112420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c0319d0 con 0x7f557c155410 2023-11-13T10:55:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.637+0000 7f557b7fe640 1 -- 172.21.15.16:0/1173112420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c031d60 con 0x7f557c155410 2023-11-13T10:55:32.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.637+0000 7f55826c9640 1 -- 172.21.15.16:0/1173112420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c155410 msgr2=0x7f557c0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.637+0000 7f55826c9640 1 --2- 172.21.15.16:0/1173112420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c155410 0x7f557c0ccc60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f557c06e000 tx=0x7f556c02b910 comp rx=0 tx=0).stop 2023-11-13T10:55:32.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.637+0000 7f55826c9640 1 -- 172.21.15.16:0/1173112420 shutdown_connections 2023-11-13T10:55:32.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.637+0000 7f55826c9640 1 --2- 172.21.15.16:0/1173112420 >> v2:172.21.15.184:3300/0 conn(0x7f557c0ce990 0x7f557c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.637+0000 7f55826c9640 1 --2- 172.21.15.16:0/1173112420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c155410 0x7f557c0ccc60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.637+0000 7f55826c9640 1 -- 172.21.15.16:0/1173112420 >> 172.21.15.16:0/1173112420 conn(0x7f557c06a190 msgr2=0x7f557c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:32.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.637+0000 7f55826c9640 1 -- 172.21.15.16:0/1173112420 shutdown_connections 2023-11-13T10:55:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.638+0000 7f55826c9640 1 -- 172.21.15.16:0/1173112420 wait complete. 2023-11-13T10:55:32.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.640+0000 7f55826c9640 1 Processor -- start 2023-11-13T10:55:32.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.640+0000 7f55826c9640 1 -- start start 2023-11-13T10:55:32.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.641+0000 7f55826c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557c071440 0x7f557c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.641+0000 7f55826c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c0ce990 0x7f557c13aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.641+0000 7f55826c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f557c155410 0x7f557c13b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.641+0000 7f55826c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c15c2a0 con 0x7f557c0ce990 2023-11-13T10:55:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.641+0000 7f55826c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c15c540 con 0x7f557c071440 2023-11-13T10:55:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f55826c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557c15c820 con 0x7f557c155410 2023-11-13T10:55:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f55737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c0ce990 0x7f557c13aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f55737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c0ce990 0x7f557c13aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42932/0 (socket says 172.21.15.16:42932) 2023-11-13T10:55:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f55737fe640 1 -- 172.21.15.16:0/3283592529 learned_addr learned my addr 172.21.15.16:0/3283592529 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:32.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f557bfff640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557c071440 0x7f557c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f55737fe640 1 -- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f557c155410 msgr2=0x7f557c13b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f5580c66640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f557c155410 0x7f557c13b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f55737fe640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f557c155410 0x7f557c13b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f55737fe640 1 -- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557c071440 msgr2=0x7f557c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f55737fe640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557c071440 0x7f557c06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f55737fe640 1 -- 172.21.15.16:0/3283592529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c0094d0 con 0x7f557c0ce990 2023-11-13T10:55:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.642+0000 7f557bfff640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557c071440 0x7f557c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.643+0000 7f55737fe640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c0ce990 0x7f557c13aeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f556400cf60 tx=0x7f556400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.643+0000 7f55797fa640 1 -- 172.21.15.16:0/3283592529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556400d840 con 0x7f557c0ce990 2023-11-13T10:55:32.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.643+0000 7f55797fa640 1 -- 172.21.15.16:0/3283592529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556400d9a0 con 0x7f557c0ce990 2023-11-13T10:55:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.643+0000 7f55797fa640 1 -- 172.21.15.16:0/3283592529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556400dd30 con 0x7f557c0ce990 2023-11-13T10:55:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.643+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557c15cb00 con 0x7f557c0ce990 2023-11-13T10:55:32.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.643+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557c06d7e0 con 0x7f557c0ce990 2023-11-13T10:55:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.644+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557c15cd20 con 0x7f557c0ce990 2023-11-13T10:55:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.645+0000 7f55797fa640 1 -- 172.21.15.16:0/3283592529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5564015440 con 0x7f557c0ce990 2023-11-13T10:55:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.646+0000 7f55797fa640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f555c06ae40 0x7f555c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.646+0000 7f55797fa640 1 -- 172.21.15.16:0/3283592529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f556408a2c0 con 0x7f557c0ce990 2023-11-13T10:55:32.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.647+0000 7f557bfff640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f555c06ae40 0x7f555c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.648+0000 7f557bfff640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f555c06ae40 0x7f555c06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f556c036000 tx=0x7f556c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.653+0000 7f55797fa640 1 -- 172.21.15.16:0/3283592529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5564055130 con 0x7f557c0ce990 2023-11-13T10:55:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.851+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f557c067090 con 0x7f557c0ce990 2023-11-13T10:55:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.854+0000 7f55797fa640 1 -- 172.21.15.16:0/3283592529 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f5564058750 con 0x7f557c0ce990 2023-11-13T10:55:32.860 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:32.860 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f555c06ae40 msgr2=0x7f555c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f555c06ae40 0x7f555c06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f556c036000 tx=0x7f556c03c040 comp rx=0 tx=0).stop 2023-11-13T10:55:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c0ce990 msgr2=0x7f557c13aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c0ce990 0x7f557c13aeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f556400cf60 tx=0x7f556400cf90 comp rx=0 tx=0).stop 2023-11-13T10:55:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 shutdown_connections 2023-11-13T10:55:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f555c06ae40 0x7f555c06d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f557c155410 0x7f557c13b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557c0ce990 0x7f557c13aeb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 --2- 172.21.15.16:0/3283592529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f557c071440 0x7f557c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.857+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 >> 172.21.15.16:0/3283592529 conn(0x7f557c06a190 msgr2=0x7f557c0b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.858+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 shutdown_connections 2023-11-13T10:55:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:32.858+0000 7f55826c9640 1 -- 172.21.15.16:0/3283592529 wait complete. 2023-11-13T10:55:32.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:33.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.105+0000 7efc86ffb640 1 Processor -- start 2023-11-13T10:55:33.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.106+0000 7efc86ffb640 1 -- start start 2023-11-13T10:55:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.107+0000 7efc86ffb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc800cfce0 0x7efc800b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.107+0000 7efc86ffb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc800d0c90 con 0x7efc800cda30 2023-11-13T10:55:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.107+0000 7efc86ffb640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc800d0940 con 0x7efc80160730 2023-11-13T10:55:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.107+0000 7efc86ffb640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc800b3e40 con 0x7efc800cfce0 2023-11-13T10:55:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.108+0000 7efc7ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc800cfce0 0x7efc800b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.108+0000 7efc7ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc800cfce0 0x7efc800b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36712/0 (socket says 172.21.15.16:36712) 2023-11-13T10:55:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.108+0000 7efc7ffff640 1 -- 172.21.15.16:0/1546730386 learned_addr learned my addr 172.21.15.16:0/1546730386 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.108+0000 7efc7f7fe640 1 -- 172.21.15.16:0/1546730386 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138630814 0 0) 0x7efc800d0c90 con 0x7efc800cda30 2023-11-13T10:55:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.108+0000 7efc7ffff640 1 -- 172.21.15.16:0/1546730386 >> v1:172.21.15.184:6789/0 conn(0x7efc80160730 legacy=0x7efc800cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.108+0000 7efc7ffff640 1 -- 172.21.15.16:0/1546730386 >> v1:172.21.15.16:6789/0 conn(0x7efc800cda30 legacy=0x7efc800cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.108+0000 7efc7ffff640 1 -- 172.21.15.16:0/1546730386 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800ce740 con 0x7efc800cfce0 2023-11-13T10:55:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.110+0000 7efc7ffff640 1 --2- 172.21.15.16:0/1546730386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc800cfce0 0x7efc800b5bc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc68009870 tx=0x7efc6802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c512c0bf9011107e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.110+0000 7efc7f7fe640 1 -- 172.21.15.16:0/1546730386 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68031870 con 0x7efc800cfce0 2023-11-13T10:55:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.110+0000 7efc7f7fe640 1 -- 172.21.15.16:0/1546730386 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc680319d0 con 0x7efc800cfce0 2023-11-13T10:55:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.110+0000 7efc7f7fe640 1 -- 172.21.15.16:0/1546730386 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68031d60 con 0x7efc800cfce0 2023-11-13T10:55:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.111+0000 7efc86ffb640 1 -- 172.21.15.16:0/1546730386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc800cfce0 msgr2=0x7efc800b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.111+0000 7efc86ffb640 1 --2- 172.21.15.16:0/1546730386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc800cfce0 0x7efc800b5bc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc68009870 tx=0x7efc6802b910 comp rx=0 tx=0).stop 2023-11-13T10:55:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.111+0000 7efc86ffb640 1 -- 172.21.15.16:0/1546730386 shutdown_connections 2023-11-13T10:55:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.111+0000 7efc86ffb640 1 --2- 172.21.15.16:0/1546730386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc800cfce0 0x7efc800b5bc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.111+0000 7efc86ffb640 1 -- 172.21.15.16:0/1546730386 >> 172.21.15.16:0/1546730386 conn(0x7efc8009c080 msgr2=0x7efc800d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.111+0000 7efc86ffb640 1 -- 172.21.15.16:0/1546730386 shutdown_connections 2023-11-13T10:55:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.111+0000 7efc86ffb640 1 -- 172.21.15.16:0/1546730386 wait complete. 2023-11-13T10:55:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.113+0000 7efc86ffb640 1 Processor -- start 2023-11-13T10:55:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.113+0000 7efc86ffb640 1 -- start start 2023-11-13T10:55:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.114+0000 7efc86ffb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc800cda30 0x7efc8015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.114+0000 7efc86ffb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc800cfce0 0x7efc800cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.114+0000 7efc86ffb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80160730 0x7efc800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.114+0000 7efc86ffb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc800ca1b0 con 0x7efc80160730 2023-11-13T10:55:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.114+0000 7efc86ffb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc800ca490 con 0x7efc800cfce0 2023-11-13T10:55:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.114+0000 7efc86ffb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc800ca770 con 0x7efc800cda30 2023-11-13T10:55:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.114+0000 7efc85598640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80160730 0x7efc800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc85598640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80160730 0x7efc800b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44126/0 (socket says 172.21.15.16:44126) 2023-11-13T10:55:33.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc85598640 1 -- 172.21.15.16:0/3732389625 learned_addr learned my addr 172.21.15.16:0/3732389625 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:33.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc7ffff640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc800cfce0 0x7efc800cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc84d97640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc800cda30 0x7efc8015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc85598640 1 -- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc800cda30 msgr2=0x7efc8015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc85598640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc800cda30 0x7efc8015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc85598640 1 -- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc800cfce0 msgr2=0x7efc800cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc85598640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc800cfce0 0x7efc800cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc85598640 1 -- 172.21.15.16:0/3732389625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc680094d0 con 0x7efc80160730 2023-11-13T10:55:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.115+0000 7efc85598640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80160730 0x7efc800b7820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efc7800c480 tx=0x7efc7800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.116+0000 7efc7d7fa640 1 -- 172.21.15.16:0/3732389625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78005480 con 0x7efc80160730 2023-11-13T10:55:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.116+0000 7efc7d7fa640 1 -- 172.21.15.16:0/3732389625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc780035f0 con 0x7efc80160730 2023-11-13T10:55:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.116+0000 7efc7d7fa640 1 -- 172.21.15.16:0/3732389625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780039a0 con 0x7efc80160730 2023-11-13T10:55:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.116+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc800caa50 con 0x7efc80160730 2023-11-13T10:55:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.116+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc800c8320 con 0x7efc80160730 2023-11-13T10:55:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.117+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc800caf70 con 0x7efc80160730 2023-11-13T10:55:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.118+0000 7efc7d7fa640 1 -- 172.21.15.16:0/3732389625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc78003b00 con 0x7efc80160730 2023-11-13T10:55:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.119+0000 7efc7d7fa640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6006ae40 0x7efc6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.119+0000 7efc7d7fa640 1 -- 172.21.15.16:0/3732389625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efc780894a0 con 0x7efc80160730 2023-11-13T10:55:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.120+0000 7efc84d97640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6006ae40 0x7efc6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.121+0000 7efc84d97640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6006ae40 0x7efc6006d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc70001d70 tx=0x7efc70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:33.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.123+0000 7efc7d7fa640 1 -- 172.21.15.16:0/3732389625 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc800caf70 con 0x7efc80160730 2023-11-13T10:55:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.323+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc800caf70 con 0x7efc80160730 2023-11-13T10:55:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.325+0000 7efc7d7fa640 1 -- 172.21.15.16:0/3732389625 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7efc800caf70 con 0x7efc80160730 2023-11-13T10:55:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:33.331 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.327+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6006ae40 msgr2=0x7efc6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6006ae40 0x7efc6006d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc70001d70 tx=0x7efc70009040 comp rx=0 tx=0).stop 2023-11-13T10:55:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80160730 msgr2=0x7efc800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80160730 0x7efc800b7820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efc7800c480 tx=0x7efc7800c940 comp rx=0 tx=0).stop 2023-11-13T10:55:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 shutdown_connections 2023-11-13T10:55:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6006ae40 0x7efc6006d300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80160730 0x7efc800b7820 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc800cfce0 0x7efc800cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 --2- 172.21.15.16:0/3732389625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc800cda30 0x7efc8015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 >> 172.21.15.16:0/3732389625 conn(0x7efc8009c080 msgr2=0x7efc8006ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 shutdown_connections 2023-11-13T10:55:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.328+0000 7efc86ffb640 1 -- 172.21.15.16:0/3732389625 wait complete. 2023-11-13T10:55:33.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:55:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.660+0000 7f1cadaaf640 1 Processor -- start 2023-11-13T10:55:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.661+0000 7f1cadaaf640 1 -- start start 2023-11-13T10:55:33.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.662+0000 7f1cadaaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca8071b50 0x7f1ca806fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.662+0000 7f1cadaaf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca80cd110 con 0x7f1ca80ccc60 2023-11-13T10:55:33.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.662+0000 7f1cadaaf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca815b500 con 0x7f1ca8155410 2023-11-13T10:55:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.662+0000 7f1cadaaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca815b7e0 con 0x7f1ca8071b50 2023-11-13T10:55:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1ca80ccc60 0x7f1ca8071440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54490/0 (socket says 172.21.15.16:54490) 2023-11-13T10:55:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca6ffd640 1 -- 172.21.15.16:0/490243992 learned_addr learned my addr 172.21.15.16:0/490243992 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999220885 0 0) 0x7f1ca80cd110 con 0x7f1ca80ccc60 2023-11-13T10:55:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c94003520 con 0x7f1ca80ccc60 2023-11-13T10:55:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca7fff640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca8071b50 0x7f1ca806fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2033278839 0 0) 0x7f1c94003520 con 0x7f1ca80ccc60 2023-11-13T10:55:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca8071b50 msgr2=0x7f1ca806fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca67fc640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca8071b50 0x7f1ca806fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 >> v1:172.21.15.184:6789/0 conn(0x7f1ca8155410 legacy=0x7f1ca80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca815bac0 con 0x7f1ca80ccc60 2023-11-13T10:55:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.663+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f1c90001670 con 0x7f1ca80ccc60 2023-11-13T10:55:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 >> v1:172.21.15.16:6789/0 conn(0x7f1ca80ccc60 legacy=0x7f1ca8071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca67fc640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca8071b50 0x7f1c940039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca67fc640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c94004020 0x7f1c940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca67fc640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c94004d00 0x7f1c94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca815b7e0 con 0x7f1c94004d00 2023-11-13T10:55:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9400bbc0 con 0x7f1c94004020 2023-11-13T10:55:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9400bd80 con 0x7f1ca8071b50 2023-11-13T10:55:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca7fff640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c94004d00 0x7f1c94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca77fe640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca8071b50 0x7f1c940039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca6ffd640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c94004020 0x7f1c940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca7fff640 1 -- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca8071b50 msgr2=0x7f1c940039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca7fff640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca8071b50 0x7f1c940039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca7fff640 1 -- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c94004020 msgr2=0x7f1c940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca7fff640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c94004020 0x7f1c940047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca7fff640 1 -- 172.21.15.16:0/490243992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94003520 con 0x7f1c94004d00 2023-11-13T10:55:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.664+0000 7f1ca77fe640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca8071b50 0x7f1c940039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.665+0000 7f1ca7fff640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c94004d00 0x7f1c94005180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000ba30 tx=0x7f1c940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e915eb0f17a2608f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.666+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0032650 con 0x7f1c94004d00 2023-11-13T10:55:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.666+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca00327b0 con 0x7f1c94004d00 2023-11-13T10:55:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.666+0000 7f1ca67fc640 1 -- 172.21.15.16:0/490243992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0032b80 con 0x7f1c94004d00 2023-11-13T10:55:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.666+0000 7f1cadaaf640 1 -- 172.21.15.16:0/490243992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c94004d00 msgr2=0x7f1c94005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.666+0000 7f1cadaaf640 1 --2- 172.21.15.16:0/490243992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c94004d00 0x7f1c94005180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000ba30 tx=0x7f1c940057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.666+0000 7f1ca7fff640 1 -- 172.21.15.16:0/490243992 reap_dead start 2023-11-13T10:55:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.666+0000 7f1cadaaf640 1 -- 172.21.15.16:0/490243992 shutdown_connections 2023-11-13T10:55:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.666+0000 7f1cadaaf640 1 -- 172.21.15.16:0/490243992 >> 172.21.15.16:0/490243992 conn(0x7f1ca806a190 msgr2=0x7f1ca806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.667+0000 7f1cadaaf640 1 -- 172.21.15.16:0/490243992 shutdown_connections 2023-11-13T10:55:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.667+0000 7f1cadaaf640 1 -- 172.21.15.16:0/490243992 wait complete. 2023-11-13T10:55:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.669+0000 7f1cadaaf640 1 Processor -- start 2023-11-13T10:55:33.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.669+0000 7f1cadaaf640 1 -- start start 2023-11-13T10:55:33.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1cadaaf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c94004d00 0x7f1ca813bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1cadaaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ca81f3960 0x7f1ca80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1cadaaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81f3dc0 0x7f1ca80ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1cadaaf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca81579f0 con 0x7f1ca81f3960 2023-11-13T10:55:33.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1cadaaf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca813c760 con 0x7f1c94004d00 2023-11-13T10:55:33.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1cadaaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca81f2ea0 con 0x7f1ca81f3dc0 2023-11-13T10:55:33.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1ca6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ca81f3960 0x7f1ca80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1ca6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ca81f3960 0x7f1ca80ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44156/0 (socket says 172.21.15.16:44156) 2023-11-13T10:55:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1ca6ffd640 1 -- 172.21.15.16:0/2138265280 learned_addr learned my addr 172.21.15.16:0/2138265280 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.671+0000 7f1ca77fe640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c94004d00 0x7f1ca813bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1ca7fff640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81f3dc0 0x7f1ca80ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1ca6ffd640 1 -- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81f3dc0 msgr2=0x7f1ca80ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1ca6ffd640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81f3dc0 0x7f1ca80ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1ca6ffd640 1 -- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c94004d00 msgr2=0x7f1ca813bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1ca6ffd640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c94004d00 0x7f1ca813bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1ca6ffd640 1 -- 172.21.15.16:0/2138265280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8156450 con 0x7f1ca81f3960 2023-11-13T10:55:33.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1ca7fff640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81f3dc0 0x7f1ca80ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1ca6ffd640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ca81f3960 0x7f1ca80ac8e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000d7d0 tx=0x7f1c9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1c9ffff640 1 -- 172.21.15.16:0/2138265280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c900074e0 con 0x7f1ca81f3960 2023-11-13T10:55:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1c9ffff640 1 -- 172.21.15.16:0/2138265280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c90007640 con 0x7f1ca81f3960 2023-11-13T10:55:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1c9ffff640 1 -- 172.21.15.16:0/2138265280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90004a20 con 0x7f1ca81f3960 2023-11-13T10:55:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca8156730 con 0x7f1ca81f3960 2023-11-13T10:55:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.672+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca81f3440 con 0x7f1ca81f3960 2023-11-13T10:55:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.674+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca8067090 con 0x7f1ca81f3960 2023-11-13T10:55:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.675+0000 7f1c9ffff640 1 -- 172.21.15.16:0/2138265280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c90004b80 con 0x7f1ca81f3960 2023-11-13T10:55:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.676+0000 7f1c9ffff640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c8806ae40 0x7f1c8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.676+0000 7f1c9ffff640 1 -- 172.21.15.16:0/2138265280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1c900895b0 con 0x7f1ca81f3960 2023-11-13T10:55:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.677+0000 7f1ca77fe640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c8806ae40 0x7f1c8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.678+0000 7f1ca77fe640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c8806ae40 0x7f1c8806d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c98001e40 tx=0x7f1c98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.680+0000 7f1c9ffff640 1 -- 172.21.15.16:0/2138265280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c900542f0 con 0x7f1ca81f3960 2023-11-13T10:55:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.914+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ca813c1b0 con 0x7f1ca81f3960 2023-11-13T10:55:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.915+0000 7f1c9ffff640 1 -- 172.21.15.16:0/2138265280 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f1c90019090 con 0x7f1ca81f3960 2023-11-13T10:55:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:33.923 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:31.370515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-13T10:55:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-11-13T10:55:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c8806ae40 msgr2=0x7f1c8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c8806ae40 0x7f1c8806d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c98001e40 tx=0x7f1c98009040 comp rx=0 tx=0).stop 2023-11-13T10:55:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ca81f3960 msgr2=0x7f1ca80ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ca81f3960 0x7f1ca80ac8e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000d7d0 tx=0x7f1c9000dc90 comp rx=0 tx=0).stop 2023-11-13T10:55:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 shutdown_connections 2023-11-13T10:55:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c8806ae40 0x7f1c8806d300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ca81f3dc0 0x7f1ca80ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ca81f3960 0x7f1ca80ac8e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 --2- 172.21.15.16:0/2138265280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c94004d00 0x7f1ca813bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 >> 172.21.15.16:0/2138265280 conn(0x7f1ca806a190 msgr2=0x7f1ca80ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 shutdown_connections 2023-11-13T10:55:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:33.918+0000 7f1cadaaf640 1 -- 172.21.15.16:0/2138265280 wait complete. 2023-11-13T10:55:33.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.165+0000 7f69a6cd1640 1 Processor -- start 2023-11-13T10:55:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.165+0000 7f69a6cd1640 1 -- start start 2023-11-13T10:55:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.167+0000 7f69a6cd1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69a006a8a0 0x7f69a00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.167+0000 7f69a6cd1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a006fc50 con 0x7f69a0070cf0 2023-11-13T10:55:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.167+0000 7f69a6cd1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a006ff30 con 0x7f69a015cc90 2023-11-13T10:55:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.167+0000 7f69a6cd1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a0070210 con 0x7f69a006a8a0 2023-11-13T10:55:34.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.167+0000 7f69a54ce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69a006a8a0 0x7f69a00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.167+0000 7f69a64d0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f69a0070cf0 0x7f69a00b2650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54494/0 (socket says 172.21.15.16:54494) 2023-11-13T10:55:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.167+0000 7f69a64d0640 1 -- 172.21.15.16:0/688059364 learned_addr learned my addr 172.21.15.16:0/688059364 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.168+0000 7f69a54ce640 1 -- 172.21.15.16:0/688059364 >> v1:172.21.15.184:6789/0 conn(0x7f69a015cc90 legacy=0x7f69a006a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.168+0000 7f69a54ce640 1 -- 172.21.15.16:0/688059364 >> v1:172.21.15.16:6789/0 conn(0x7f69a0070cf0 legacy=0x7f69a00b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.168+0000 7f69a54ce640 1 -- 172.21.15.16:0/688059364 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00704f0 con 0x7f69a006a8a0 2023-11-13T10:55:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.168+0000 7f69a4ccd640 1 -- 172.21.15.16:0/688059364 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937089610 0 0) 0x7f69a006fc50 con 0x7f69a0070cf0 2023-11-13T10:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.169+0000 7f69a54ce640 1 --2- 172.21.15.16:0/688059364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a006a8a0 0x7f69a00b2110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f69900097a0 tx=0x7f699002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254b9fe57aaf5265 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.169+0000 7f69a4ccd640 1 -- 172.21.15.16:0/688059364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990031830 con 0x7f69a006a8a0 2023-11-13T10:55:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.169+0000 7f69a4ccd640 1 -- 172.21.15.16:0/688059364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6990031990 con 0x7f69a006a8a0 2023-11-13T10:55:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.169+0000 7f69a4ccd640 1 -- 172.21.15.16:0/688059364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990031d20 con 0x7f69a006a8a0 2023-11-13T10:55:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.170+0000 7f69a6cd1640 1 -- 172.21.15.16:0/688059364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a006a8a0 msgr2=0x7f69a00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.170+0000 7f69a6cd1640 1 --2- 172.21.15.16:0/688059364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a006a8a0 0x7f69a00b2110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f69900097a0 tx=0x7f699002b8d0 comp rx=0 tx=0).stop 2023-11-13T10:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.170+0000 7f69a6cd1640 1 -- 172.21.15.16:0/688059364 shutdown_connections 2023-11-13T10:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.170+0000 7f69a6cd1640 1 --2- 172.21.15.16:0/688059364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a006a8a0 0x7f69a00b2110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.170+0000 7f69a6cd1640 1 -- 172.21.15.16:0/688059364 >> 172.21.15.16:0/688059364 conn(0x7f69a00b04a0 msgr2=0x7f69a00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.170+0000 7f69a6cd1640 1 -- 172.21.15.16:0/688059364 shutdown_connections 2023-11-13T10:55:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.170+0000 7f69a6cd1640 1 -- 172.21.15.16:0/688059364 wait complete. 2023-11-13T10:55:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.172+0000 7f69a6cd1640 1 Processor -- start 2023-11-13T10:55:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.172+0000 7f69a6cd1640 1 -- start start 2023-11-13T10:55:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.173+0000 7f69a6cd1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69a006a8a0 0x7f69a0140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.173+0000 7f69a6cd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a0070cf0 0x7f69a01f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.173+0000 7f69a6cd1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69a015cc90 0x7f69a01f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.173+0000 7f69a6cd1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a01f4ad0 con 0x7f69a0070cf0 2023-11-13T10:55:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.173+0000 7f69a6cd1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a01f4db0 con 0x7f69a006a8a0 2023-11-13T10:55:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.173+0000 7f69a6cd1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a01f5090 con 0x7f69a015cc90 2023-11-13T10:55:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.173+0000 7f69a54ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a0070cf0 0x7f69a01f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.173+0000 7f69a54ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a0070cf0 0x7f69a01f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44176/0 (socket says 172.21.15.16:44176) 2023-11-13T10:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.173+0000 7f69a54ce640 1 -- 172.21.15.16:0/3487689003 learned_addr learned my addr 172.21.15.16:0/3487689003 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69a5ccf640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69a006a8a0 0x7f69a0140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69a54ce640 1 -- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69a015cc90 msgr2=0x7f69a01f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69a54ce640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69a015cc90 0x7f69a01f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69a54ce640 1 -- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69a006a8a0 msgr2=0x7f69a0140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69a54ce640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69a006a8a0 0x7f69a0140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69a54ce640 1 -- 172.21.15.16:0/3487689003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6990009400 con 0x7f69a0070cf0 2023-11-13T10:55:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69a5ccf640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69a006a8a0 0x7f69a0140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69a54ce640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a0070cf0 0x7f69a01f4180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f69900097a0 tx=0x7f69900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69967fc640 1 -- 172.21.15.16:0/3487689003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699003ba70 con 0x7f69a0070cf0 2023-11-13T10:55:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69967fc640 1 -- 172.21.15.16:0/3487689003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699003bbd0 con 0x7f69a0070cf0 2023-11-13T10:55:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69967fc640 1 -- 172.21.15.16:0/3487689003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69900453f0 con 0x7f69a0070cf0 2023-11-13T10:55:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.174+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a01f5370 con 0x7f69a0070cf0 2023-11-13T10:55:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.175+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a01408c0 con 0x7f69a0070cf0 2023-11-13T10:55:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.176+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a01f5890 con 0x7f69a0070cf0 2023-11-13T10:55:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.177+0000 7f69967fc640 1 -- 172.21.15.16:0/3487689003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f699004e450 con 0x7f69a0070cf0 2023-11-13T10:55:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.178+0000 7f69967fc640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698406ae90 0x7f698406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.178+0000 7f69967fc640 1 -- 172.21.15.16:0/3487689003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f69900abaf0 con 0x7f69a0070cf0 2023-11-13T10:55:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.178+0000 7f69a5ccf640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698406ae90 0x7f698406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.179+0000 7f69a5ccf640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698406ae90 0x7f698406d350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f69a0153db0 tx=0x7f699c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.182+0000 7f69967fc640 1 -- 172.21.15.16:0/3487689003 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69a01f5890 con 0x7f69a0070cf0 2023-11-13T10:55:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.374+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69a01f5890 con 0x7f69a0070cf0 2023-11-13T10:55:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.377+0000 7f69967fc640 1 -- 172.21.15.16:0/3487689003 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f69a01f5890 con 0x7f69a0070cf0 2023-11-13T10:55:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:34.385 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698406ae90 msgr2=0x7f698406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698406ae90 0x7f698406d350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f69a0153db0 tx=0x7f699c008040 comp rx=0 tx=0).stop 2023-11-13T10:55:34.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a0070cf0 msgr2=0x7f69a01f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:34.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a0070cf0 0x7f69a01f4180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f69900097a0 tx=0x7f69900046c0 comp rx=0 tx=0).stop 2023-11-13T10:55:34.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 shutdown_connections 2023-11-13T10:55:34.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698406ae90 0x7f698406d350 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:34.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69a015cc90 0x7f69a01f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:34.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69a0070cf0 0x7f69a01f4180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:34.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 --2- 172.21.15.16:0/3487689003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69a006a8a0 0x7f69a0140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:34.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 >> 172.21.15.16:0/3487689003 conn(0x7f69a00b04a0 msgr2=0x7f69a00ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:34.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 shutdown_connections 2023-11-13T10:55:34.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:34.380+0000 7f69a6cd1640 1 -- 172.21.15.16:0/3487689003 wait complete. 2023-11-13T10:55:34.812 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-13T10:55:26.312366+0000', 'modified': '2023-11-13T10:55:31.370515+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4251}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4251': {'gid': 4251, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/3826676517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 3826676517}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 3826676517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-13T10:55:34.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-13T10:55:34.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-13T10:55:35.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.049+0000 7fe52bfff640 1 Processor -- start 2023-11-13T10:55:35.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.050+0000 7fe52bfff640 1 -- start start 2023-11-13T10:55:35.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe52c161060 0x7fe52c1621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06f970 0x7fe52c162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe52c06a190 0x7fe52c06a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52bfff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52c06ab60 con 0x7fe52c06a190 2023-11-13T10:55:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52bfff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52c162700 con 0x7fe52c161060 2023-11-13T10:55:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c0b1840 con 0x7fe52c06f970 2023-11-13T10:55:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe52c06a190 0x7fe52c06a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe52c06a190 0x7fe52c06a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44188/0 (socket says 172.21.15.16:44188) 2023-11-13T10:55:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52b7fe640 1 -- 172.21.15.16:0/1963604425 learned_addr learned my addr 172.21.15.16:0/1963604425 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe52affd640 1 --2- 172.21.15.16:0/1963604425 >> v2:172.21.15.184:3300/0 conn(0x7fe52c161060 0x7fe52c1621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.052+0000 7fe523fff640 1 --2- 172.21.15.16:0/1963604425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06f970 0x7fe52c162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.053+0000 7fe52b7fe640 1 -- 172.21.15.16:0/1963604425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06f970 msgr2=0x7fe52c162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.053+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/1963604425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06f970 0x7fe52c162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.053+0000 7fe52b7fe640 1 -- 172.21.15.16:0/1963604425 >> v2:172.21.15.184:3300/0 conn(0x7fe52c161060 msgr2=0x7fe52c1621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.053+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/1963604425 >> v2:172.21.15.184:3300/0 conn(0x7fe52c161060 0x7fe52c1621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.053+0000 7fe52b7fe640 1 -- 172.21.15.16:0/1963604425 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c071600 con 0x7fe52c06a190 2023-11-13T10:55:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.054+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/1963604425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06a190 0x7fe52c06a620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe52c06ad80 tx=0x7fe52402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b167cea1684f63 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.054+0000 7fe52a7fc640 1 -- 172.21.15.16:0/1963604425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52402ca80 con 0x7fe52c06a190 2023-11-13T10:55:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.054+0000 7fe52a7fc640 1 -- 172.21.15.16:0/1963604425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52402cbe0 con 0x7fe52c06a190 2023-11-13T10:55:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.054+0000 7fe52a7fc640 1 -- 172.21.15.16:0/1963604425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5240343f0 con 0x7fe52c06a190 2023-11-13T10:55:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.055+0000 7fe52bfff640 1 -- 172.21.15.16:0/1963604425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06a190 msgr2=0x7fe52c06a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.055+0000 7fe52bfff640 1 --2- 172.21.15.16:0/1963604425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06a190 0x7fe52c06a620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe52c06ad80 tx=0x7fe52402bee0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.055+0000 7fe52bfff640 1 -- 172.21.15.16:0/1963604425 shutdown_connections 2023-11-13T10:55:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.055+0000 7fe52bfff640 1 --2- 172.21.15.16:0/1963604425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06a190 0x7fe52c06a620 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.055+0000 7fe52bfff640 1 --2- 172.21.15.16:0/1963604425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06f970 0x7fe52c162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.055+0000 7fe52bfff640 1 --2- 172.21.15.16:0/1963604425 >> v2:172.21.15.184:3300/0 conn(0x7fe52c161060 0x7fe52c1621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.055+0000 7fe52bfff640 1 -- 172.21.15.16:0/1963604425 >> 172.21.15.16:0/1963604425 conn(0x7fe52c0b04a0 msgr2=0x7fe52c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.055+0000 7fe52bfff640 1 -- 172.21.15.16:0/1963604425 shutdown_connections 2023-11-13T10:55:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.055+0000 7fe52bfff640 1 -- 172.21.15.16:0/1963604425 wait complete. 2023-11-13T10:55:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.057+0000 7fe52bfff640 1 Processor -- start 2023-11-13T10:55:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.057+0000 7fe52bfff640 1 -- start start 2023-11-13T10:55:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.058+0000 7fe52bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06a190 0x7fe52c149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.058+0000 7fe52bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06f970 0x7fe52c1f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.058+0000 7fe52bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c161060 0x7fe52c163550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.058+0000 7fe52bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c14a2e0 con 0x7fe52c06f970 2023-11-13T10:55:35.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.058+0000 7fe52bfff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c14a7b0 con 0x7fe52c161060 2023-11-13T10:55:35.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.058+0000 7fe52bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c15d1e0 con 0x7fe52c06a190 2023-11-13T10:55:35.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.058+0000 7fe523fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06f970 0x7fe52c1f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.058+0000 7fe523fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06f970 0x7fe52c1f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44198/0 (socket says 172.21.15.16:44198) 2023-11-13T10:55:35.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.058+0000 7fe523fff640 1 -- 172.21.15.16:0/3515953544 learned_addr learned my addr 172.21.15.16:0/3515953544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe52affd640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06a190 0x7fe52c149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c161060 0x7fe52c163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe523fff640 1 -- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06a190 msgr2=0x7fe52c149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe523fff640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06a190 0x7fe52c149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe523fff640 1 -- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c161060 msgr2=0x7fe52c163550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe523fff640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c161060 0x7fe52c163550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe523fff640 1 -- 172.21.15.16:0/3515953544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524009b30 con 0x7fe52c06f970 2023-11-13T10:55:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe523fff640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06f970 0x7fe52c1f9db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe51400d800 tx=0x7fe51400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe528ff9640 1 -- 172.21.15.16:0/3515953544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5140096a0 con 0x7fe52c06f970 2023-11-13T10:55:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe528ff9640 1 -- 172.21.15.16:0/3515953544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe514009800 con 0x7fe52c06f970 2023-11-13T10:55:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe528ff9640 1 -- 172.21.15.16:0/3515953544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514009b90 con 0x7fe52c06f970 2023-11-13T10:55:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.059+0000 7fe52bfff640 1 -- 172.21.15.16:0/3515953544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52c1fa790 con 0x7fe52c06f970 2023-11-13T10:55:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.060+0000 7fe52bfff640 1 -- 172.21.15.16:0/3515953544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c15d4a0 con 0x7fe52c06f970 2023-11-13T10:55:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.062+0000 7fe528ff9640 1 -- 172.21.15.16:0/3515953544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe514009cf0 con 0x7fe52c06f970 2023-11-13T10:55:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.064+0000 7fe528ff9640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe50c06ae40 0x7fe50c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.064+0000 7fe528ff9640 1 -- 172.21.15.16:0/3515953544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe514089d80 con 0x7fe52c06f970 2023-11-13T10:55:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.064+0000 7fe52affd640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe50c06ae40 0x7fe50c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.064+0000 7fe5345a2740 1 Processor -- start 2023-11-13T10:55:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.065+0000 7fe5345a2740 1 -- start start 2023-11-13T10:55:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.065+0000 7fe52affd640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe50c06ae40 0x7fe50c06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe51c006fd0 tx=0x7fe51c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.065+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66bbe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.065+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bb8cb0 0x556d66bbd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.065+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bbd6f0 0x556d66c01670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.065+0000 7fe5345a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66b1c880 con 0x556d66bbdfb0 2023-11-13T10:55:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.065+0000 7fe5345a2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66b2bae0 con 0x556d66bb8cb0 2023-11-13T10:55:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.065+0000 7fe5345a2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d66b291d0 con 0x556d66bbd6f0 2023-11-13T10:55:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.066+0000 7fe52affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66bbe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.066+0000 7fe52affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66bbe390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44200/0 (socket says 172.21.15.16:44200) 2023-11-13T10:55:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.066+0000 7fe52affd640 1 -- 172.21.15.16:0/3953827433 learned_addr learned my addr 172.21.15.16:0/3953827433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.066+0000 7fe523fff640 1 --2- 172.21.15.16:0/3953827433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bb8cb0 0x556d66bbd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.066+0000 7fe52affd640 1 -- 172.21.15.16:0/3953827433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bbd6f0 msgr2=0x556d66c01670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.066+0000 7fe52affd640 1 --2- 172.21.15.16:0/3953827433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bbd6f0 0x556d66c01670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.066+0000 7fe52affd640 1 -- 172.21.15.16:0/3953827433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bb8cb0 msgr2=0x556d66bbd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.066+0000 7fe52affd640 1 --2- 172.21.15.16:0/3953827433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bb8cb0 0x556d66bbd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.066+0000 7fe52affd640 1 -- 172.21.15.16:0/3953827433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d66aec860 con 0x556d66bbdfb0 2023-11-13T10:55:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.067+0000 7fe52affd640 1 --2- 172.21.15.16:0/3953827433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66bbe390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00fab0 tx=0x7fe51c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2f48604db50d67 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.067+0000 7fe5217fa640 1 -- 172.21.15.16:0/3953827433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c014be0 con 0x556d66bbdfb0 2023-11-13T10:55:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.067+0000 7fe5217fa640 1 -- 172.21.15.16:0/3953827433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c019070 con 0x556d66bbdfb0 2023-11-13T10:55:35.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.067+0000 7fe5217fa640 1 -- 172.21.15.16:0/3953827433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c020440 con 0x556d66bbdfb0 2023-11-13T10:55:35.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.067+0000 7fe5345a2740 1 -- 172.21.15.16:0/3953827433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 msgr2=0x556d66bbe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.067+0000 7fe5345a2740 1 --2- 172.21.15.16:0/3953827433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66bbe390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00fab0 tx=0x7fe51c012000 comp rx=0 tx=0).stop 2023-11-13T10:55:35.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.068+0000 7fe5345a2740 1 -- 172.21.15.16:0/3953827433 shutdown_connections 2023-11-13T10:55:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.068+0000 7fe5345a2740 1 --2- 172.21.15.16:0/3953827433 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bbd6f0 0x556d66c01670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.068+0000 7fe5345a2740 1 --2- 172.21.15.16:0/3953827433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bb8cb0 0x556d66bbd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.068+0000 7fe5345a2740 1 --2- 172.21.15.16:0/3953827433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66bbe390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.068+0000 7fe5345a2740 1 -- 172.21.15.16:0/3953827433 >> 172.21.15.16:0/3953827433 conn(0x556d66b7fbb0 msgr2=0x556d66c03aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:35.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.068+0000 7fe5345a2740 1 -- 172.21.15.16:0/3953827433 shutdown_connections 2023-11-13T10:55:35.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.068+0000 7fe5345a2740 1 -- 172.21.15.16:0/3953827433 wait complete. 2023-11-13T10:55:35.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.068+0000 7fe5345a2740 1 -- start start 2023-11-13T10:55:35.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.069+0000 7fe5345a2740 1 Processor -- start 2023-11-13T10:55:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bb8cb0 0x556d66beb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bbd6f0 0x556d66be89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbdfb0 0x556d66be8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe5345a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66b24ac0 con 0x556d66bb8cb0 2023-11-13T10:55:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe5345a2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66ac12f0 con 0x556d66bbdfb0 2023-11-13T10:55:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe5345a2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d66adec70 con 0x556d66bbd6f0 2023-11-13T10:55:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe5345a2740 10 client.0 _ll_register_callbacks cb 0x556d66ae33b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T10:55:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe52affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bb8cb0 0x556d66beb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe52affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bb8cb0 0x556d66beb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44208/0 (socket says 172.21.15.16:44208) 2023-11-13T10:55:35.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe52affd640 1 -- 172.21.15.16:0/2943520567 learned_addr learned my addr 172.21.15.16:0/2943520567 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:35.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbdfb0 0x556d66be8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.070+0000 7fe523fff640 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bbd6f0 0x556d66be89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe52affd640 1 -- 172.21.15.16:0/2943520567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bbd6f0 msgr2=0x556d66be89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe52affd640 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bbd6f0 0x556d66be89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe52affd640 1 -- 172.21.15.16:0/2943520567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbdfb0 msgr2=0x556d66be8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe52affd640 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbdfb0 0x556d66be8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe52affd640 1 -- 172.21.15.16:0/2943520567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c00e040 con 0x556d66bb8cb0 2023-11-13T10:55:35.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe52affd640 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bb8cb0 0x556d66beb550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe51c022040 tx=0x7fe51c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ada6a10676b743e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe50a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T10:55:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2943520567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c020c80 con 0x556d66bb8cb0 2023-11-13T10:55:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2943520567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe51c020de0 con 0x556d66bb8cb0 2023-11-13T10:55:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.071+0000 7fe5345a2740 1 -- 172.21.15.16:0/2943520567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d66b1daa0 con 0x556d66bb8cb0 2023-11-13T10:55:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2943520567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe51c015410 con 0x556d66bb8cb0 2023-11-13T10:55:35.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe5345a2740 10 client.4401 fetch_fsmap learned FSMap version 7 2023-11-13T10:55:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe5345a2740 1 -- 172.21.15.16:0/2943520567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x556d66b23140 con 0x556d66bb8cb0 2023-11-13T10:55:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2943520567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d66b1daa0 con 0x556d66bb8cb0 2023-11-13T10:55:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2943520567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fe51c015570 con 0x556d66bb8cb0 2023-11-13T10:55:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe5345a2740 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2023-11-13T10:55:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe5345a2740 10 client.4401 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T10:55:35.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe5345a2740 20 client.4401 populate_metadata read hostname 'smithi016' 2023-11-13T10:55:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x556d66be74e0 0x556d66be78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe5345a2740 4 client.4401 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T10:55:35.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe5345a2740 1 -- 172.21.15.16:0/2943520567 --> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d66b1d7c0 con 0x556d66be74e0 2023-11-13T10:55:35.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.072+0000 7fe523fff640 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x556d66be74e0 0x556d66be78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.073+0000 7fe523fff640 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x556d66be74e0 0x556d66be78c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3da4aa73173742 server_cookie=f36563a93d52574c in_seq=0 out_seq=0 2023-11-13T10:55:35.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.073+0000 7fe50a7fc640 10 client.4401 ms_handle_connect on v2:172.21.15.16:6834/3826676517 2023-11-13T10:55:35.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2943520567 <== mds.0 v2:172.21.15.16:6834/3826676517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d66b1d7c0 con 0x556d66be74e0 2023-11-13T10:55:35.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe50a7fc640 10 client.4401 handle_command_reply: tid=0 2023-11-13T10:55:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 client.4401 shutdown 2023-11-13T10:55:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 -- 172.21.15.16:0/2943520567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bb8cb0 msgr2=0x556d66beb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bb8cb0 0x556d66beb550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe51c022040 tx=0x7fe51c004730 comp rx=0 tx=0).stop 2023-11-13T10:55:35.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 -- 172.21.15.16:0/2943520567 shutdown_connections 2023-11-13T10:55:35.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x556d66be74e0 0x556d66be78c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbdfb0 0x556d66be8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bbd6f0 0x556d66be89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2943520567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bb8cb0 0x556d66beb550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 -- 172.21.15.16:0/2943520567 >> 172.21.15.16:0/2943520567 conn(0x556d66b7fbb0 msgr2=0x556d66c03aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe50a7fc640 0 client.4401 ms_handle_reset on v2:172.21.15.16:6834/3826676517 2023-11-13T10:55:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 -- 172.21.15.16:0/2943520567 shutdown_connections 2023-11-13T10:55:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 1 -- 172.21.15.16:0/2943520567 wait complete. 2023-11-13T10:55:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.074+0000 7fe5345a2740 20 client.4401 trim_cache size 0 max 16384 2023-11-13T10:55:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.092+0000 7fe5345a2740 1 Processor -- start 2023-11-13T10:55:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.092+0000 7fe5345a2740 1 -- start start 2023-11-13T10:55:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.093+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 0x556d66bbe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.093+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbd6f0 0x556d66c674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.093+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbdfb0 0x556d66c03aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.093+0000 7fe5345a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66887f20 con 0x556d66bbd6f0 2023-11-13T10:55:35.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.093+0000 7fe5345a2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66c679e0 con 0x556d66bbdfb0 2023-11-13T10:55:35.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.093+0000 7fe5345a2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d66beb8b0 con 0x556d66bb8cb0 2023-11-13T10:55:35.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.093+0000 7fe523fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbd6f0 0x556d66c674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.093+0000 7fe523fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbd6f0 0x556d66c674a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44224/0 (socket says 172.21.15.16:44224) 2023-11-13T10:55:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.093+0000 7fe523fff640 1 -- 172.21.15.16:0/1846167370 learned_addr learned my addr 172.21.15.16:0/1846167370 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.094+0000 7fe523fff640 1 -- 172.21.15.16:0/1846167370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 msgr2=0x556d66bbe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.094+0000 7fe523fff640 1 --2- 172.21.15.16:0/1846167370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 0x556d66bbe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.094+0000 7fe523fff640 1 -- 172.21.15.16:0/1846167370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbdfb0 msgr2=0x556d66c03aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.094+0000 7fe523fff640 1 --2- 172.21.15.16:0/1846167370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbdfb0 0x556d66c03aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.094+0000 7fe523fff640 1 -- 172.21.15.16:0/1846167370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d66ac0b80 con 0x556d66bbd6f0 2023-11-13T10:55:35.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.094+0000 7fe523fff640 1 --2- 172.21.15.16:0/1846167370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbd6f0 0x556d66c674a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe514006fd0 tx=0x7fe514016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d5fbab97a536e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.094+0000 7fe521ffb640 1 -- 172.21.15.16:0/1846167370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514060b00 con 0x556d66bbd6f0 2023-11-13T10:55:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.094+0000 7fe521ffb640 1 -- 172.21.15.16:0/1846167370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51400b3c0 con 0x556d66bbd6f0 2023-11-13T10:55:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.094+0000 7fe521ffb640 1 -- 172.21.15.16:0/1846167370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514056ba0 con 0x556d66bbd6f0 2023-11-13T10:55:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.095+0000 7fe5345a2740 1 -- 172.21.15.16:0/1846167370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbd6f0 msgr2=0x556d66c674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.095+0000 7fe5345a2740 1 --2- 172.21.15.16:0/1846167370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbd6f0 0x556d66c674a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe514006fd0 tx=0x7fe514016040 comp rx=0 tx=0).stop 2023-11-13T10:55:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.095+0000 7fe5345a2740 1 -- 172.21.15.16:0/1846167370 shutdown_connections 2023-11-13T10:55:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.095+0000 7fe5345a2740 1 --2- 172.21.15.16:0/1846167370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbdfb0 0x556d66c03aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.095+0000 7fe5345a2740 1 --2- 172.21.15.16:0/1846167370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbd6f0 0x556d66c674a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.095+0000 7fe5345a2740 1 --2- 172.21.15.16:0/1846167370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 0x556d66bbe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.095+0000 7fe5345a2740 1 -- 172.21.15.16:0/1846167370 >> 172.21.15.16:0/1846167370 conn(0x556d66b7fbb0 msgr2=0x556d66bb4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.095+0000 7fe5345a2740 1 -- 172.21.15.16:0/1846167370 shutdown_connections 2023-11-13T10:55:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.095+0000 7fe5345a2740 1 -- 172.21.15.16:0/1846167370 wait complete. 2023-11-13T10:55:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.096+0000 7fe5345a2740 1 -- start start 2023-11-13T10:55:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.096+0000 7fe5345a2740 1 Processor -- start 2023-11-13T10:55:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 0x556d66c00800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbd6f0 0x556d66c03ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe5345a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66c06be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe5345a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66be2a50 con 0x556d66bbdfb0 2023-11-13T10:55:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe5345a2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66be8410 con 0x556d66bbd6f0 2023-11-13T10:55:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe5345a2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d66be7a80 con 0x556d66bb8cb0 2023-11-13T10:55:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe5345a2740 10 client.0 _ll_register_callbacks cb 0x7fe51c00f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T10:55:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66c06be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66c06be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44226/0 (socket says 172.21.15.16:44226) 2023-11-13T10:55:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe52b7fe640 1 -- 172.21.15.16:0/2363793856 learned_addr learned my addr 172.21.15.16:0/2363793856 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.097+0000 7fe523fff640 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbd6f0 0x556d66c03ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe52affd640 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 0x556d66c00800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe52b7fe640 1 -- 172.21.15.16:0/2363793856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 msgr2=0x556d66c00800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 0x556d66c00800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe52b7fe640 1 -- 172.21.15.16:0/2363793856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbd6f0 msgr2=0x556d66c03ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbd6f0 0x556d66c03ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe52b7fe640 1 -- 172.21.15.16:0/2363793856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c15d4a0 con 0x556d66bbdfb0 2023-11-13T10:55:35.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe52affd640 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 0x556d66c00800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:35.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66c06be0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe524009f40 tx=0x7fe524034600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31a962a608af349 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe50a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T10:55:35.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2363793856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524034b00 con 0x556d66bbdfb0 2023-11-13T10:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2363793856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe524034c60 con 0x556d66bbdfb0 2023-11-13T10:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.098+0000 7fe5345a2740 1 -- 172.21.15.16:0/2363793856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe51c019070 con 0x556d66bbdfb0 2023-11-13T10:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2363793856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe524034e20 con 0x556d66bbdfb0 2023-11-13T10:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe5345a2740 10 client.4407 fetch_fsmap learned FSMap version 7 2023-11-13T10:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe5345a2740 1 -- 172.21.15.16:0/2363793856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x556d66ae6170 con 0x556d66bbdfb0 2023-11-13T10:55:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2363793856 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c019070 con 0x556d66bbdfb0 2023-11-13T10:55:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2363793856 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fe524048410 con 0x556d66bbdfb0 2023-11-13T10:55:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe5345a2740 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2023-11-13T10:55:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe5345a2740 10 client.4407 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T10:55:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe5345a2740 20 client.4407 populate_metadata read hostname 'smithi016' 2023-11-13T10:55:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x556d66c123a0 0x556d66c12780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe5345a2740 4 client.4407 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-13T10:55:35.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe5345a2740 1 -- 172.21.15.16:0/2363793856 --> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556d668256c0 con 0x556d66c123a0 2023-11-13T10:55:35.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.099+0000 7fe52affd640 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x556d66c123a0 0x556d66c12780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe50a7fc640 10 client.4407 ms_handle_connect on v2:172.21.15.16:6834/3826676517 2023-11-13T10:55:35.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe52affd640 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x556d66c123a0 0x556d66c12780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa06a40dac1c1b9 server_cookie=f2b57fe208eb686c in_seq=0 out_seq=0 2023-11-13T10:55:35.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe50a7fc640 1 -- 172.21.15.16:0/2363793856 <== mds.0 v2:172.21.15.16:6834/3826676517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x556d668256c0 con 0x556d66c123a0 2023-11-13T10:55:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe50a7fc640 10 client.4407 handle_command_reply: tid=0 2023-11-13T10:55:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe5345a2740 1 client.4407 shutdown 2023-11-13T10:55:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe5345a2740 1 -- 172.21.15.16:0/2363793856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 msgr2=0x556d66c06be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66c06be0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe524009f40 tx=0x7fe524034600 comp rx=0 tx=0).stop 2023-11-13T10:55:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe5345a2740 1 -- 172.21.15.16:0/2363793856 shutdown_connections 2023-11-13T10:55:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x556d66c123a0 0x556d66c12780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d66bbdfb0 0x556d66c06be0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d66bbd6f0 0x556d66c03ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe5345a2740 1 --2- 172.21.15.16:0/2363793856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556d66bb8cb0 0x556d66c00800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe5345a2740 1 -- 172.21.15.16:0/2363793856 >> 172.21.15.16:0/2363793856 conn(0x556d66b7fbb0 msgr2=0x556d66bb4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.100+0000 7fe50a7fc640 0 client.4407 ms_handle_reset on v2:172.21.15.16:6834/3826676517 2023-11-13T10:55:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.101+0000 7fe5345a2740 1 -- 172.21.15.16:0/2363793856 shutdown_connections 2023-11-13T10:55:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.101+0000 7fe5345a2740 1 -- 172.21.15.16:0/2363793856 wait complete. 2023-11-13T10:55:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.101+0000 7fe5345a2740 20 client.4407 trim_cache size 0 max 16384 2023-11-13T10:55:35.249 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T10:55:35.249 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9", 2023-11-13T10:55:35.251 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-13T10:55:35.251 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4251, 2023-11-13T10:55:35.251 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-13T10:55:35.251 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-13T10:55:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-13T10:55:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 8.2555456019999998, 2023-11-13T10:55:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 7, 2023-11-13T10:55:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 26, 2023-11-13T10:55:35.252 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 26, 2023-11-13T10:55:35.253 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.001782554 2023-11-13T10:55:35.253 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T10:55:35.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.101+0000 7fe52bfff640 1 -- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe50c06ae40 msgr2=0x7fe50c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.101+0000 7fe52bfff640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe50c06ae40 0x7fe50c06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe51c006fd0 tx=0x7fe51c008040 comp rx=0 tx=0).stop 2023-11-13T10:55:35.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.101+0000 7fe52bfff640 1 -- 172.21.15.16:0/3515953544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06f970 msgr2=0x7fe52c1f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.101+0000 7fe52bfff640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06f970 0x7fe52c1f9db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe51400d800 tx=0x7fe51400dcc0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.102+0000 7fe52bfff640 1 -- 172.21.15.16:0/3515953544 shutdown_connections 2023-11-13T10:55:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.102+0000 7fe52bfff640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe50c06ae40 0x7fe50c06d300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.102+0000 7fe52bfff640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe52c161060 0x7fe52c163550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.102+0000 7fe52bfff640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c06f970 0x7fe52c1f9db0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.102+0000 7fe52bfff640 1 --2- 172.21.15.16:0/3515953544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe52c06a190 0x7fe52c149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.102+0000 7fe52bfff640 1 -- 172.21.15.16:0/3515953544 >> 172.21.15.16:0/3515953544 conn(0x7fe52c0b04a0 msgr2=0x7fe52c0ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.102+0000 7fe52bfff640 1 -- 172.21.15.16:0/3515953544 shutdown_connections 2023-11-13T10:55:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.102+0000 7fe52bfff640 1 -- 172.21.15.16:0/3515953544 wait complete. 2023-11-13T10:55:35.259 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-13T10:55:35.262 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-13T10:55:35.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-13T10:55:35.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.497+0000 7f3c1098f640 1 Processor -- start 2023-11-13T10:55:35.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.497+0000 7f3c1098f640 1 -- start start 2023-11-13T10:55:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.499+0000 7f3c1098f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c15b190 0x7f3c0c06f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.499+0000 7f3c1098f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0c158c20 con 0x7f3c0c0cb190 2023-11-13T10:55:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.499+0000 7f3c1098f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0c0b1830 con 0x7f3c0c06a190 2023-11-13T10:55:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.499+0000 7f3c1098f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c15b780 con 0x7f3c0c15b190 2023-11-13T10:55:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.499+0000 7f3c0ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3c0c0cb190 0x7f3c0c0716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54508/0 (socket says 172.21.15.16:54508) 2023-11-13T10:55:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.499+0000 7f3c0ad9d640 1 -- 172.21.15.16:0/3954124546 learned_addr learned my addr 172.21.15.16:0/3954124546 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c0a59c640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c15b190 0x7f3c0c06f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240892848 0 0) 0x7f3c0c158c20 con 0x7f3c0c0cb190 2023-11-13T10:55:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf4003520 con 0x7f3c0c0cb190 2023-11-13T10:55:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2164114656 0 0) 0x7f3bf4003520 con 0x7f3c0c0cb190 2023-11-13T10:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c15b190 msgr2=0x7f3c0c06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c15b190 0x7f3c0c06f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 >> v1:172.21.15.184:6789/0 conn(0x7f3c0c06a190 legacy=0x7f3c0c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c15ba60 con 0x7f3c0c0cb190 2023-11-13T10:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f3bfc001670 con 0x7f3c0c0cb190 2023-11-13T10:55:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 >> v1:172.21.15.16:6789/0 conn(0x7f3c0c0cb190 legacy=0x7f3c0c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf40039a0 0x7f3bf4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c15b190 0x7f3bf40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf4004d00 0x7f3bf4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c15b780 con 0x7f3bf40039a0 2023-11-13T10:55:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf400bab0 con 0x7f3bf4004d00 2023-11-13T10:55:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.500+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf400bc70 con 0x7f3c0c15b190 2023-11-13T10:55:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.501+0000 7f3c0ad9d640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf40039a0 0x7f3bf4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.501+0000 7f3c0a59c640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c15b190 0x7f3bf40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.501+0000 7f3c0b59e640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf4004d00 0x7f3bf4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.501+0000 7f3c0ad9d640 1 -- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c15b190 msgr2=0x7f3bf40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.501+0000 7f3c0ad9d640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c15b190 0x7f3bf40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.501+0000 7f3c0ad9d640 1 -- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf4004d00 msgr2=0x7f3bf4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.501+0000 7f3c0ad9d640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf4004d00 0x7f3bf4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.501+0000 7f3c0ad9d640 1 -- 172.21.15.16:0/3954124546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf4003520 con 0x7f3bf40039a0 2023-11-13T10:55:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.501+0000 7f3c0b59e640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf4004d00 0x7f3bf4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.502+0000 7f3c0ad9d640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf40039a0 0x7f3bf4003dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00c190 tx=0x7f3bf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36f3075c3f63d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.502+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc005ad0 con 0x7f3bf40039a0 2023-11-13T10:55:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.502+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bfc005c50 con 0x7f3bf40039a0 2023-11-13T10:55:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.502+0000 7f3c09d9b640 1 -- 172.21.15.16:0/3954124546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc02c7e0 con 0x7f3bf40039a0 2023-11-13T10:55:35.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.503+0000 7f3c1098f640 1 -- 172.21.15.16:0/3954124546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf40039a0 msgr2=0x7f3bf4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.503+0000 7f3c1098f640 1 --2- 172.21.15.16:0/3954124546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf40039a0 0x7f3bf4003dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00c190 tx=0x7f3bf40057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.503+0000 7f3c0b59e640 1 -- 172.21.15.16:0/3954124546 reap_dead start 2023-11-13T10:55:35.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.503+0000 7f3c1098f640 1 -- 172.21.15.16:0/3954124546 shutdown_connections 2023-11-13T10:55:35.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.503+0000 7f3c1098f640 1 -- 172.21.15.16:0/3954124546 >> 172.21.15.16:0/3954124546 conn(0x7f3c0c0b04a0 msgr2=0x7f3c0c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:35.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.503+0000 7f3c1098f640 1 -- 172.21.15.16:0/3954124546 shutdown_connections 2023-11-13T10:55:35.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.503+0000 7f3c1098f640 1 -- 172.21.15.16:0/3954124546 wait complete. 2023-11-13T10:55:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.505+0000 7f3c1098f640 1 Processor -- start 2023-11-13T10:55:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.505+0000 7f3c1098f640 1 -- start start 2023-11-13T10:55:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.506+0000 7f3c1098f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf40039a0 0x7f3c0c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.506+0000 7f3c1098f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c153550 0x7f3c0c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.506+0000 7f3c1098f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c0c06b9d0 0x7f3c0c06be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.506+0000 7f3c1098f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c1f2a40 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.506+0000 7f3c1098f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c15ac30 con 0x7f3bf40039a0 2023-11-13T10:55:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.506+0000 7f3c1098f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c153d60 con 0x7f3c0c153550 2023-11-13T10:55:35.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.506+0000 7f3c0b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c0c06b9d0 0x7f3c0c06be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.506+0000 7f3c0b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c0c06b9d0 0x7f3c0c06be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44240/0 (socket says 172.21.15.16:44240) 2023-11-13T10:55:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.506+0000 7f3c0b59e640 1 -- 172.21.15.16:0/3344057672 learned_addr learned my addr 172.21.15.16:0/3344057672 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3c0ad9d640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf40039a0 0x7f3c0c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3c0a59c640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c153550 0x7f3c0c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3c0b59e640 1 -- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c153550 msgr2=0x7f3c0c0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3c0b59e640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c153550 0x7f3c0c0acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3c0b59e640 1 -- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf40039a0 msgr2=0x7f3c0c15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3c0b59e640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf40039a0 0x7f3c0c15a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3c0b59e640 1 -- 172.21.15.16:0/3344057672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c06d2e0 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3c0b59e640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c0c06b9d0 0x7f3c0c06be50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0ae3e0 tx=0x7f3c0400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3bf37fe640 1 -- 172.21.15.16:0/3344057672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04006d50 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3bf37fe640 1 -- 172.21.15.16:0/3344057672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c04003970 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0c06d5c0 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.507+0000 7f3bf37fe640 1 -- 172.21.15.16:0/3344057672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04003d60 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.508+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0c158c20 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.509+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0c158db0 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.510+0000 7f3bf37fe640 1 -- 172.21.15.16:0/3344057672 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c04005070 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.511+0000 7f3bf37fe640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3bf806ae40 0x7f3bf806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.511+0000 7f3bf37fe640 1 -- 172.21.15.16:0/3344057672 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3c04089890 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.511+0000 7f3c0ad9d640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3bf806ae40 0x7f3bf806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.513+0000 7f3c0ad9d640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3bf806ae40 0x7f3bf806d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc034000 tx=0x7f3bfc009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.515+0000 7f3bf37fe640 1 -- 172.21.15.16:0/3344057672 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c040545b0 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.743+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c0c04e4d0 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.744+0000 7f3bf37fe640 1 -- 172.21.15.16:0/3344057672 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3c04057bd0 con 0x7f3c0c06b9d0 2023-11-13T10:55:35.749 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-13T10:55:35.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.746+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3bf806ae40 msgr2=0x7f3bf806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.746+0000 7f3c1098f640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3bf806ae40 0x7f3bf806d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc034000 tx=0x7f3bfc009030 comp rx=0 tx=0).stop 2023-11-13T10:55:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.746+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c0c06b9d0 msgr2=0x7f3c0c06be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.746+0000 7f3c1098f640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c0c06b9d0 0x7f3c0c06be50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0ae3e0 tx=0x7f3c0400dc00 comp rx=0 tx=0).stop 2023-11-13T10:55:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.747+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 shutdown_connections 2023-11-13T10:55:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.747+0000 7f3c1098f640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3bf806ae40 0x7f3bf806d300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.747+0000 7f3c1098f640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c0c06b9d0 0x7f3c0c06be50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.747+0000 7f3c1098f640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c0c153550 0x7f3c0c0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.747+0000 7f3c1098f640 1 --2- 172.21.15.16:0/3344057672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3bf40039a0 0x7f3c0c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:35.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.747+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 >> 172.21.15.16:0/3344057672 conn(0x7f3c0c0b04a0 msgr2=0x7f3c0c06a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.747+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 shutdown_connections 2023-11-13T10:55:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:35.747+0000 7f3c1098f640 1 -- 172.21.15.16:0/3344057672 wait complete. 2023-11-13T10:55:35.770 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-13T10:55:35.771 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-13T10:55:35.771 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-13T10:55:35.771 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-13T10:55:35.771 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-13T10:55:35.772 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-13T10:55:35.772 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-13T10:55:35.772 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-13T10:55:35.773 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-13T10:55:35.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:35.773 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-13T10:55:35.823 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:55:35.823 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:55:35.823 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:55:35.824 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:55:35.824 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-13T10:55:35.824 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:55:35.824 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:55:35.824 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:55:35.825 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-13T10:55:35.825 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:55:35.825 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:55:35.825 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-13T10:55:35.825 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-13T10:55:35.826 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:55:35.826 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-13T10:55:35.826 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-13T10:55:35.826 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42306sec preferred_lft 42306sec 2023-11-13T10:55:35.826 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-13T10:55:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:55:35.827 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:55:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:55:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-13T10:55:35.829 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-13T10:55:35.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:35.830 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:55:35.830 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:55:35.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2023-11-13T10:55:35.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2023-11-13T10:55:35.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2023-11-13T10:55:35.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-13T10:55:35.832 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:55:36.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:36.327 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-13T10:55:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-13T10:55:36.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:36.419 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-13T10:55:36.498 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-13T10:55:36.499 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-13T10:55:36.499 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-13T10:55:36.499 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-13T10:55:36.499 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-13T10:55:36.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:36.501 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:55:36.502 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:55:36.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-13T10:55:36.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-13T10:55:36.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-13T10:55:36.512 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:55:37.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:37.343 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-13T10:55:37.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:37.409 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-11-13T10:55:37.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:37.476 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:55:37.476 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:55:37.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-13T10:55:37.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-13T10:55:37.477 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:55:37.636 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-13T10:55:37.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:37.637 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:55:37.637 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:55:37.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-13T10:55:37.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-13T10:55:37.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-13T10:55:37.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-13T10:55:37.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-13T10:55:37.640 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:55:37.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:37.943 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:55:37.944 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:55:37.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-13T10:55:37.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2023-11-13T10:55:37.945 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:55:38.118 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-13T10:55:38.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:55:38.119 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-13T10:55:38.170 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-13T10:55:38.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:55:38.171 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-13T10:55:38.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T10:55:38.320 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:55:38.373 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T10:55:38.373 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T10:55:38.374 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T10:55:38.374 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T10:55:38.374 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T10:55:38.374 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T10:55:38.375 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T10:55:38.375 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T10:55:38.375 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T10:55:38.375 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T10:55:38.375 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T10:55:38.376 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:55:38.376 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T10:55:38.376 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T10:55:38.376 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T10:55:38.376 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T10:55:38.377 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:55:38.377 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T10:55:38.377 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T10:55:38.377 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:55:38.377 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:55:38.378 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:55:38.378 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T10:55:38.383 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T10:55:38.384 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T10:55:38.384 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T10:55:38.384 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T10:55:38.384 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T10:55:38.385 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T10:55:38.385 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-13T10:55:38.385 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-13T10:55:38.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:38.407 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T10:55:38.470 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-13T10:55:38.471 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-13T10:55:38.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T10:55:38.562 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:55:38.606 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T10:55:38.607 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T10:55:38.607 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T10:55:38.607 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T10:55:38.608 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T10:55:38.608 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T10:55:38.608 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T10:55:38.608 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T10:55:38.608 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T10:55:38.609 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T10:55:38.609 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T10:55:38.609 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:55:38.609 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T10:55:38.609 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T10:55:38.610 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T10:55:38.610 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T10:55:38.610 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:55:38.610 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T10:55:38.610 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T10:55:38.610 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:55:38.611 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:55:38.611 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:55:38.611 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T10:55:38.611 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T10:55:38.611 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T10:55:38.612 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T10:55:38.612 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T10:55:38.612 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T10:55:38.612 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T10:55:38.612 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-13T10:55:38.613 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-13T10:55:38.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:38.614 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T10:55:38.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2023-11-13T10:55:38.681+0000 7fd941838480 -1 init, newargv = 0x5566755ffc10 newargc=15 2023-11-13T10:55:38.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[92733]: starting ceph client 2023-11-13T10:55:38.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[92733]: starting fuse 2023-11-13T10:55:38.710 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-13T10:55:38.711 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-13T10:55:38.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-13T10:55:38.711 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-13T10:55:38.712 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-13T10:55:38.712 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T10:55:38.712 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-13T10:55:38.713 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:55:38.713 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-13T10:55:38.713 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-13T10:55:38.714 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-13T10:55:38.714 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-13T10:55:38.714 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:55:38.715 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-13T10:55:38.715 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-13T10:55:38.717 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-13T10:55:38.718 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-13T10:55:38.718 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:55:38.719 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-13T10:55:38.725 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-13T10:55:38.726 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-13T10:55:38.727 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-13T10:55:38.727 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-13T10:55:38.728 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-13T10:55:38.728 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-13T10:55:38.728 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-13T10:55:38.729 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:55:38.729 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2023-11-13T10:55:38.729 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T10:55:38.917 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.92733.asok 2023-11-13T10:55:38.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T10:55:38.937 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.92733.asok 2023-11-13T10:55:38.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:38.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.92733.asok status 2023-11-13T10:55:39.188 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T10:55:39.189 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-13T10:55:39.189 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "35f0debb3cc28955b7132ef4c509cd138a09d56b", 2023-11-13T10:55:39.190 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7288-g35f0debb (35f0debb3cc28955b7132ef4c509cd138a09d56b) reef (dev)", 2023-11-13T10:55:39.190 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2023-11-13T10:55:39.190 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-13T10:55:39.190 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-13T10:55:39.190 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "92733", 2023-11-13T10:55:39.191 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-13T10:55:39.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:55:39.191 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-13T10:55:39.191 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-13T10:55:39.191 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4419, 2023-11-13T10:55:39.192 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-13T10:55:39.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-13T10:55:39.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-13T10:55:39.192 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4419 2023-11-13T10:55:39.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:55:39.193 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T10:55:39.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T10:55:39.194 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T10:55:39.194 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3764282600 2023-11-13T10:55:39.194 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:55:39.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:55:39.195 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T10:55:39.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T10:55:39.195 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T10:55:39.195 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3764282600 2023-11-13T10:55:39.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:55:39.196 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4419 192.168.0.1:0/3764282600", 2023-11-13T10:55:39.196 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/3764282600", 2023-11-13T10:55:39.196 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-13T10:55:39.196 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 7, 2023-11-13T10:55:39.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 26, 2023-11-13T10:55:39.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-13T10:55:39.198 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-13T10:55:39.198 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-13T10:55:39.198 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T10:55:39.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:39.207 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-13T10:55:39.207 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-13T10:55:39.264 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-13T10:55:39.265 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-13T10:55:39.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:39.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-13T10:55:39.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:39.390 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-13T10:55:39.479 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2023-11-13T10:55:39.479 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-13T10:55:39.479 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:39.481 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:55:39.481 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:55:39.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.770+0000 7efe9ffff640 1 Processor -- start 2023-11-13T10:55:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.770+0000 7efe9ffff640 1 -- start start 2023-11-13T10:55:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.773+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea015b190 0x7efea006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.773+0000 7efe9ffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea0158c20 con 0x7efea006a190 2023-11-13T10:55:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.773+0000 7efe9ffff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea00b1830 con 0x7efea00cb190 2023-11-13T10:55:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.773+0000 7efe9ffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea015b780 con 0x7efea015b190 2023-11-13T10:55:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.773+0000 7efe9f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efea006a190 0x7efea00b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54524/0 (socket says 172.21.15.16:54524) 2023-11-13T10:55:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.773+0000 7efe9f7fe640 1 -- 172.21.15.16:0/3496758031 learned_addr learned my addr 172.21.15.16:0/3496758031 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.773+0000 7efe97fff640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea015b190 0x7efea006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146948340 0 0) 0x7efea0158c20 con 0x7efea006a190 2023-11-13T10:55:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe8c003520 con 0x7efea006a190 2023-11-13T10:55:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 104416916 0 0) 0x7efe8c003520 con 0x7efea006a190 2023-11-13T10:55:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea015b190 msgr2=0x7efea006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea015b190 0x7efea006f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 >> v1:172.21.15.184:6789/0 conn(0x7efea00cb190 legacy=0x7efea00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea015ba60 con 0x7efea006a190 2023-11-13T10:55:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7efe980019f0 con 0x7efea006a190 2023-11-13T10:55:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 >> v1:172.21.15.16:6789/0 conn(0x7efea006a190 legacy=0x7efea00b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea015b190 0x7efe8c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c003f20 0x7efe8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c004d00 0x7efe8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea015b780 con 0x7efe8c004d00 2023-11-13T10:55:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c00bb10 con 0x7efe8c003f20 2023-11-13T10:55:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.774+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c00bcd0 con 0x7efea015b190 2023-11-13T10:55:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.775+0000 7efe9f7fe640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c004d00 0x7efe8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.775+0000 7efe9effd640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea015b190 0x7efe8c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.775+0000 7efe97fff640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c003f20 0x7efe8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.775+0000 7efe9f7fe640 1 -- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea015b190 msgr2=0x7efe8c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.775+0000 7efe9f7fe640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea015b190 0x7efe8c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.775+0000 7efe9f7fe640 1 -- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c003f20 msgr2=0x7efe8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.775+0000 7efe9f7fe640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c003f20 0x7efe8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.775+0000 7efe9f7fe640 1 -- 172.21.15.16:0/3496758031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c003520 con 0x7efe8c004d00 2023-11-13T10:55:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.776+0000 7efe9f7fe640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c004d00 0x7efe8c005180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efe9800a820 tx=0x7efe8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77854811c0a4044 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.776+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe980072d0 con 0x7efe8c004d00 2023-11-13T10:55:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.776+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9802f070 con 0x7efe8c004d00 2023-11-13T10:55:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.776+0000 7efe9e7fc640 1 -- 172.21.15.16:0/3496758031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe980076b0 con 0x7efe8c004d00 2023-11-13T10:55:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.777+0000 7efe9ffff640 1 -- 172.21.15.16:0/3496758031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c004d00 msgr2=0x7efe8c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:39.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.777+0000 7efe9ffff640 1 --2- 172.21.15.16:0/3496758031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c004d00 0x7efe8c005180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efe9800a820 tx=0x7efe8c0057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.777+0000 7efe9f7fe640 1 -- 172.21.15.16:0/3496758031 reap_dead start 2023-11-13T10:55:39.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.777+0000 7efe9ffff640 1 -- 172.21.15.16:0/3496758031 shutdown_connections 2023-11-13T10:55:39.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.777+0000 7efe9ffff640 1 -- 172.21.15.16:0/3496758031 >> 172.21.15.16:0/3496758031 conn(0x7efea00b04a0 msgr2=0x7efea00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:39.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.777+0000 7efe9ffff640 1 -- 172.21.15.16:0/3496758031 shutdown_connections 2023-11-13T10:55:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.777+0000 7efe9ffff640 1 -- 172.21.15.16:0/3496758031 wait complete. 2023-11-13T10:55:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.779+0000 7efe9ffff640 1 Processor -- start 2023-11-13T10:55:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.779+0000 7efe9ffff640 1 -- start start 2023-11-13T10:55:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c004d00 0x7efea0150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:39.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0151730 0x7efea0154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea0145240 0x7efea015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe9ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea0155230 con 0x7efea0151730 2023-11-13T10:55:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe9ffff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea014fc80 con 0x7efe8c004d00 2023-11-13T10:55:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe9ffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea0155000 con 0x7efea0145240 2023-11-13T10:55:39.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0151730 0x7efea0154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0151730 0x7efea0154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44262/0 (socket says 172.21.15.16:44262) 2023-11-13T10:55:39.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe97fff640 1 -- 172.21.15.16:0/2829836173 learned_addr learned my addr 172.21.15.16:0/2829836173 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe9f7fe640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea0145240 0x7efea015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:39.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe9effd640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c004d00 0x7efea0150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:39.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe97fff640 1 -- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea0145240 msgr2=0x7efea015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:39.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe97fff640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea0145240 0x7efea015a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.781+0000 7efe97fff640 1 -- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c004d00 msgr2=0x7efea0150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe97fff640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c004d00 0x7efea0150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe97fff640 1 -- 172.21.15.16:0/2829836173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea01507c0 con 0x7efea0151730 2023-11-13T10:55:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe9f7fe640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea0145240 0x7efea015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe9effd640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c004d00 0x7efea0150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe97fff640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0151730 0x7efea0154ac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe88011630 tx=0x7efe88011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe977fe640 1 -- 172.21.15.16:0/2829836173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800f920 con 0x7efea0151730 2023-11-13T10:55:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe977fe640 1 -- 172.21.15.16:0/2829836173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8800fa80 con 0x7efea0151730 2023-11-13T10:55:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe977fe640 1 -- 172.21.15.16:0/2829836173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800fdf0 con 0x7efea0151730 2023-11-13T10:55:39.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea014fe90 con 0x7efea0151730 2023-11-13T10:55:39.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.782+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea0150450 con 0x7efea0151730 2023-11-13T10:55:39.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.784+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea0150950 con 0x7efea0151730 2023-11-13T10:55:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.785+0000 7efe977fe640 1 -- 172.21.15.16:0/2829836173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe88015460 con 0x7efea0151730 2023-11-13T10:55:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.785+0000 7efe977fe640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe8006ae40 0x7efe8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.786+0000 7efe977fe640 1 -- 172.21.15.16:0/2829836173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efe88089f70 con 0x7efea0151730 2023-11-13T10:55:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.786+0000 7efe9effd640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe8006ae40 0x7efe8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.787+0000 7efe9effd640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe8006ae40 0x7efe8006d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe900016b0 tx=0x7efe90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.790+0000 7efe977fe640 1 -- 172.21.15.16:0/2829836173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe88054c00 con 0x7efea0151730 2023-11-13T10:55:39.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.986+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efea004e4d0 con 0x7efea0151730 2023-11-13T10:55:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.988+0000 7efe977fe640 1 -- 172.21.15.16:0/2829836173 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7efe88022020 con 0x7efea0151730 2023-11-13T10:55:39.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:39.994 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:39.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.990+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe8006ae40 msgr2=0x7efe8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:39.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.990+0000 7efe9ffff640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe8006ae40 0x7efe8006d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe900016b0 tx=0x7efe90009040 comp rx=0 tx=0).stop 2023-11-13T10:55:39.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0151730 msgr2=0x7efea0154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:39.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0151730 0x7efea0154ac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe88011630 tx=0x7efe88011af0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 shutdown_connections 2023-11-13T10:55:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe8006ae40 0x7efe8006d300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efea0145240 0x7efea015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0151730 0x7efea0154ac0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 --2- 172.21.15.16:0/2829836173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe8c004d00 0x7efea0150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:39.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 >> 172.21.15.16:0/2829836173 conn(0x7efea00b04a0 msgr2=0x7efea006a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:39.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 shutdown_connections 2023-11-13T10:55:40.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:39.991+0000 7efe9ffff640 1 -- 172.21.15.16:0/2829836173 wait complete. 2023-11-13T10:55:40.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:55:40.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:55:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.250+0000 7f4b14ad7640 1 Processor -- start 2023-11-13T10:55:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.250+0000 7f4b14ad7640 1 -- start start 2023-11-13T10:55:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.252+0000 7f4b14ad7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b100715b0 0x7f4b1006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.252+0000 7f4b14ad7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1006bca0 con 0x7f4b100b3760 2023-11-13T10:55:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.252+0000 7f4b14ad7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1006bf80 con 0x7f4b100acff0 2023-11-13T10:55:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.252+0000 7f4b14ad7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1006c920 con 0x7f4b100715b0 2023-11-13T10:55:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.253+0000 7f4b06ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4b100b3760 0x7f4b100ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54536/0 (socket says 172.21.15.16:54536) 2023-11-13T10:55:40.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.253+0000 7f4b06ffd640 1 -- 172.21.15.16:0/784095545 learned_addr learned my addr 172.21.15.16:0/784095545 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:40.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.253+0000 7f4b0f7fe640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b100715b0 0x7f4b1006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.253+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381509595 0 0) 0x7f4b1006bca0 con 0x7f4b100b3760 2023-11-13T10:55:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.253+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4afc003520 con 0x7f4b100b3760 2023-11-13T10:55:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3382886787 0 0) 0x7f4afc003520 con 0x7f4b100b3760 2023-11-13T10:55:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b100715b0 msgr2=0x7f4b1006b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:40.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b100715b0 0x7f4b1006b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 >> v1:172.21.15.184:6789/0 conn(0x7f4b100acff0 legacy=0x7f4b1006fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1006c260 con 0x7f4b100b3760 2023-11-13T10:55:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f4af80019f0 con 0x7f4b100b3760 2023-11-13T10:55:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 >> v1:172.21.15.16:6789/0 conn(0x7f4b100b3760 legacy=0x7f4b100ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b100715b0 0x7f4afc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4afc003f70 0x7f4afc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4afc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1006c920 con 0x7f4afc004d00 2023-11-13T10:55:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc00bb70 con 0x7f4afc003f70 2023-11-13T10:55:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc00bd30 con 0x7f4b100715b0 2023-11-13T10:55:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0ffff640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4afc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0f7fe640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b100715b0 0x7f4afc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b06ffd640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4afc003f70 0x7f4afc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0ffff640 1 -- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b100715b0 msgr2=0x7f4afc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.254+0000 7f4b0ffff640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b100715b0 0x7f4afc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.255+0000 7f4b0ffff640 1 -- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4afc003f70 msgr2=0x7f4afc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:40.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.255+0000 7f4b0ffff640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4afc003f70 0x7f4afc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.255+0000 7f4b0ffff640 1 -- 172.21.15.16:0/784095545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc003520 con 0x7f4afc004d00 2023-11-13T10:55:40.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.256+0000 7f4b0ffff640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4afc005180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4b08001750 tx=0x7f4afc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa4c3f315e8be99 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:40.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.256+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b080052d0 con 0x7f4afc004d00 2023-11-13T10:55:40.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.256+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0802e050 con 0x7f4afc004d00 2023-11-13T10:55:40.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.256+0000 7f4b0effd640 1 -- 172.21.15.16:0/784095545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08003830 con 0x7f4afc004d00 2023-11-13T10:55:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.256+0000 7f4b14ad7640 1 -- 172.21.15.16:0/784095545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 msgr2=0x7f4afc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.256+0000 7f4b14ad7640 1 --2- 172.21.15.16:0/784095545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4afc005180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4b08001750 tx=0x7f4afc0057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.256+0000 7f4b0ffff640 1 -- 172.21.15.16:0/784095545 reap_dead start 2023-11-13T10:55:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.257+0000 7f4b14ad7640 1 -- 172.21.15.16:0/784095545 shutdown_connections 2023-11-13T10:55:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.257+0000 7f4b14ad7640 1 -- 172.21.15.16:0/784095545 >> 172.21.15.16:0/784095545 conn(0x7f4b100b0970 msgr2=0x7f4b100b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.257+0000 7f4b14ad7640 1 -- 172.21.15.16:0/784095545 shutdown_connections 2023-11-13T10:55:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.257+0000 7f4b14ad7640 1 -- 172.21.15.16:0/784095545 wait complete. 2023-11-13T10:55:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.258+0000 7f4b14ad7640 1 Processor -- start 2023-11-13T10:55:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.259+0000 7f4b14ad7640 1 -- start start 2023-11-13T10:55:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.259+0000 7f4b14ad7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4b1013e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b14ad7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b10141630 0x7f4b100c3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b14ad7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b10140320 0x7f4b101407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b14ad7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1013fe90 con 0x7f4afc004d00 2023-11-13T10:55:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b14ad7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b100c3e50 con 0x7f4b10140320 2023-11-13T10:55:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b14ad7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1013f5a0 con 0x7f4b10141630 2023-11-13T10:55:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4b1013e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4b1013e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44282/0 (socket says 172.21.15.16:44282) 2023-11-13T10:55:40.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b0f7fe640 1 -- 172.21.15.16:0/2910617639 learned_addr learned my addr 172.21.15.16:0/2910617639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b06ffd640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b10141630 0x7f4b100c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b0ffff640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b10140320 0x7f4b101407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b0f7fe640 1 -- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b10141630 msgr2=0x7f4b100c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b0f7fe640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b10141630 0x7f4b100c3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b0f7fe640 1 -- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b10140320 msgr2=0x7f4b101407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b0f7fe640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b10140320 0x7f4b101407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.260+0000 7f4b0f7fe640 1 -- 172.21.15.16:0/2910617639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1013f840 con 0x7f4afc004d00 2023-11-13T10:55:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.261+0000 7f4b0f7fe640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4b1013e1d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4b00011600 tx=0x7f4b00011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.261+0000 7f4b0cff9640 1 -- 172.21.15.16:0/2910617639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000f920 con 0x7f4afc004d00 2023-11-13T10:55:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.261+0000 7f4b0cff9640 1 -- 172.21.15.16:0/2910617639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0000fa80 con 0x7f4afc004d00 2023-11-13T10:55:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.261+0000 7f4b0cff9640 1 -- 172.21.15.16:0/2910617639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000fdf0 con 0x7f4afc004d00 2023-11-13T10:55:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.261+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1013fb20 con 0x7f4afc004d00 2023-11-13T10:55:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.261+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b10141da0 con 0x7f4afc004d00 2023-11-13T10:55:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.262+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1004e4d0 con 0x7f4afc004d00 2023-11-13T10:55:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.263+0000 7f4b0cff9640 1 -- 172.21.15.16:0/2910617639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b00015460 con 0x7f4afc004d00 2023-11-13T10:55:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.265+0000 7f4b0cff9640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4af006ae40 0x7f4af006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.265+0000 7f4b0cff9640 1 -- 172.21.15.16:0/2910617639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4b00089eb0 con 0x7f4afc004d00 2023-11-13T10:55:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.265+0000 7f4b06ffd640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4af006ae40 0x7f4af006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.266+0000 7f4b06ffd640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4af006ae40 0x7f4af006d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4af8002230 tx=0x7f4af80012f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.269+0000 7f4b0cff9640 1 -- 172.21.15.16:0/2910617639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b00054bd0 con 0x7f4afc004d00 2023-11-13T10:55:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.462+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b10145250 con 0x7f4afc004d00 2023-11-13T10:55:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.462+0000 7f4b0cff9640 1 -- 172.21.15.16:0/2910617639 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b000220a0 con 0x7f4afc004d00 2023-11-13T10:55:40.467 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:55:40.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4af006ae40 msgr2=0x7f4af006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4af006ae40 0x7f4af006d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4af8002230 tx=0x7f4af80012f0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 msgr2=0x7f4b1013e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4b1013e1d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4b00011600 tx=0x7f4b00011ac0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 shutdown_connections 2023-11-13T10:55:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4af006ae40 0x7f4af006d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b10140320 0x7f4b101407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b10141630 0x7f4b100c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 --2- 172.21.15.16:0/2910617639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc004d00 0x7f4b1013e1d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 >> 172.21.15.16:0/2910617639 conn(0x7f4b100b0970 msgr2=0x7f4b1006f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:40.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 shutdown_connections 2023-11-13T10:55:40.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.465+0000 7f4b14ad7640 1 -- 172.21.15.16:0/2910617639 wait complete. 2023-11-13T10:55:40.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:55:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.707+0000 7fdfb06b4640 1 Processor -- start 2023-11-13T10:55:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.707+0000 7fdfb06b4640 1 -- start start 2023-11-13T10:55:40.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.708+0000 7fdfb06b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa80ce5b0 0x7fdfa815a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfb06b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdfa80cf0e0 0x7fdfa815a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfb06b4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa815d8d0 con 0x7fdfa80ce1d0 2023-11-13T10:55:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfb06b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa815af00 con 0x7fdfa80cf0e0 2023-11-13T10:55:40.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfb06b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa815db90 con 0x7fdfa80ce5b0 2023-11-13T10:55:40.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfae450640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdfa80ce1d0 0x7fdfa80ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54538/0 (socket says 172.21.15.16:54538) 2023-11-13T10:55:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfae450640 1 -- 172.21.15.16:0/3845012065 learned_addr learned my addr 172.21.15.16:0/3845012065 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfadc4f640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa80ce5b0 0x7fdfa815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfaec51640 1 --2- 172.21.15.16:0/3845012065 >> v2:172.21.15.184:3300/0 conn(0x7fdfa80cf0e0 0x7fdfa815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952472561 0 0) 0x7fdfa815d8d0 con 0x7fdfa80ce1d0 2023-11-13T10:55:40.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.709+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf94003520 con 0x7fdfa80ce1d0 2023-11-13T10:55:40.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3143552413 0 0) 0x7fdf94003520 con 0x7fdfa80ce1d0 2023-11-13T10:55:40.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa80ce5b0 msgr2=0x7fdfa815a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:40.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa80ce5b0 0x7fdfa815a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 >> v2:172.21.15.184:3300/0 conn(0x7fdfa80cf0e0 msgr2=0x7fdfa815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:40.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 --2- 172.21.15.16:0/3845012065 >> v2:172.21.15.184:3300/0 conn(0x7fdfa80cf0e0 0x7fdfa815a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa815de70 con 0x7fdfa80ce1d0 2023-11-13T10:55:40.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fdfa4001670 con 0x7fdfa80ce1d0 2023-11-13T10:55:40.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 >> v1:172.21.15.16:6789/0 conn(0x7fdfa80ce1d0 legacy=0x7fdfa80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf94003970 0x7fdf94003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf940042d0 0x7fdf940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf94004e10 0x7fdf94005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa815af00 con 0x7fdf94004e10 2023-11-13T10:55:40.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa815db90 con 0x7fdf940042d0 2023-11-13T10:55:40.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9400bd00 con 0x7fdf94003970 2023-11-13T10:55:40.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.710+0000 7fdfaec51640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf94004e10 0x7fdf94005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.711+0000 7fdfae450640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf94003970 0x7fdf94003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.711+0000 7fdfadc4f640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf940042d0 0x7fdf940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.711+0000 7fdfaec51640 1 -- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf94003970 msgr2=0x7fdf94003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:40.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.711+0000 7fdfaec51640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf94003970 0x7fdf94003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.711+0000 7fdfaec51640 1 -- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf940042d0 msgr2=0x7fdf940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:40.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.711+0000 7fdfaec51640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf940042d0 0x7fdf940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.711+0000 7fdfaec51640 1 -- 172.21.15.16:0/3845012065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94003520 con 0x7fdf94004e10 2023-11-13T10:55:40.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.711+0000 7fdfadc4f640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf940042d0 0x7fdf940047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:40.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.712+0000 7fdfaec51640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf94004e10 0x7fdf94005290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009fc0 tx=0x7fdf940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc46955bcad8b308 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:40.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.712+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa002cc60 con 0x7fdf94004e10 2023-11-13T10:55:40.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.712+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa000f040 con 0x7fdf94004e10 2023-11-13T10:55:40.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.712+0000 7fdfad44e640 1 -- 172.21.15.16:0/3845012065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0034440 con 0x7fdf94004e10 2023-11-13T10:55:40.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.713+0000 7fdfb06b4640 1 -- 172.21.15.16:0/3845012065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf94004e10 msgr2=0x7fdf94005290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.713+0000 7fdfb06b4640 1 --2- 172.21.15.16:0/3845012065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf94004e10 0x7fdf94005290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009fc0 tx=0x7fdf940057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.713+0000 7fdfaec51640 1 -- 172.21.15.16:0/3845012065 reap_dead start 2023-11-13T10:55:40.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.713+0000 7fdfb06b4640 1 -- 172.21.15.16:0/3845012065 shutdown_connections 2023-11-13T10:55:40.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.713+0000 7fdfb06b4640 1 -- 172.21.15.16:0/3845012065 >> 172.21.15.16:0/3845012065 conn(0x7fdfa806a190 msgr2=0x7fdfa806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:40.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.713+0000 7fdfb06b4640 1 -- 172.21.15.16:0/3845012065 shutdown_connections 2023-11-13T10:55:40.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.713+0000 7fdfb06b4640 1 -- 172.21.15.16:0/3845012065 wait complete. 2023-11-13T10:55:40.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.715+0000 7fdfb06b4640 1 Processor -- start 2023-11-13T10:55:40.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.715+0000 7fdfb06b4640 1 -- start start 2023-11-13T10:55:40.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.716+0000 7fdfb06b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa81474a0 0x7fdfa8147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.716+0000 7fdfb06b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8147880 0x7fdfa8148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.716+0000 7fdfb06b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa8149bf0 0x7fdfa8150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.716+0000 7fdfb06b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa8148b50 con 0x7fdfa8149bf0 2023-11-13T10:55:40.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.716+0000 7fdfb06b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa8148e30 con 0x7fdfa81474a0 2023-11-13T10:55:40.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.716+0000 7fdfb06b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa8149110 con 0x7fdfa8147880 2023-11-13T10:55:40.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.716+0000 7fdfaec51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa8149bf0 0x7fdfa8150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.716+0000 7fdfaec51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa8149bf0 0x7fdfa8150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44312/0 (socket says 172.21.15.16:44312) 2023-11-13T10:55:40.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.716+0000 7fdfaec51640 1 -- 172.21.15.16:0/1994749859 learned_addr learned my addr 172.21.15.16:0/1994749859 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.717+0000 7fdfae450640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa81474a0 0x7fdfa8147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.717+0000 7fdfadc4f640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8147880 0x7fdfa8148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.717+0000 7fdfaec51640 1 -- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8147880 msgr2=0x7fdfa8148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.717+0000 7fdfaec51640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8147880 0x7fdfa8148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.717+0000 7fdfaec51640 1 -- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa81474a0 msgr2=0x7fdfa8147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.717+0000 7fdfaec51640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa81474a0 0x7fdfa8147de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.717+0000 7fdfaec51640 1 -- 172.21.15.16:0/1994749859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa81493f0 con 0x7fdfa8149bf0 2023-11-13T10:55:40.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.717+0000 7fdfae450640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa81474a0 0x7fdfa8147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:40.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.717+0000 7fdfaec51640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa8149bf0 0x7fdfa8150670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0037040 tx=0x7fdfa0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:40.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.718+0000 7fdf9effd640 1 -- 172.21.15.16:0/1994749859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0034d50 con 0x7fdfa8149bf0 2023-11-13T10:55:40.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.718+0000 7fdf9effd640 1 -- 172.21.15.16:0/1994749859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa0003990 con 0x7fdfa8149bf0 2023-11-13T10:55:40.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.718+0000 7fdf9effd640 1 -- 172.21.15.16:0/1994749859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0003d40 con 0x7fdfa8149bf0 2023-11-13T10:55:40.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.718+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa81496d0 con 0x7fdfa8149bf0 2023-11-13T10:55:40.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.718+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa8150bb0 con 0x7fdfa8149bf0 2023-11-13T10:55:40.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.719+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa80b36b0 con 0x7fdfa8149bf0 2023-11-13T10:55:40.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.720+0000 7fdf9effd640 1 -- 172.21.15.16:0/1994749859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdfa0004030 con 0x7fdfa8149bf0 2023-11-13T10:55:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.721+0000 7fdf9effd640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdf8c06ae40 0x7fdf8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.721+0000 7fdf9effd640 1 -- 172.21.15.16:0/1994749859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdfa00aafc0 con 0x7fdfa8149bf0 2023-11-13T10:55:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.722+0000 7fdfae450640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdf8c06ae40 0x7fdf8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.723+0000 7fdfae450640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdf8c06ae40 0x7fdf8c06d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4004840 tx=0x7fdfa4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:40.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.725+0000 7fdf9effd640 1 -- 172.21.15.16:0/1994749859 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdfa0075d90 con 0x7fdfa8149bf0 2023-11-13T10:55:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.921+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfa80cb880 con 0x7fdfa8149bf0 2023-11-13T10:55:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.922+0000 7fdf9effd640 1 -- 172.21.15.16:0/1994749859 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfa00793b0 con 0x7fdfa8149bf0 2023-11-13T10:55:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:55:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdf8c06ae40 msgr2=0x7fdf8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdf8c06ae40 0x7fdf8c06d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4004840 tx=0x7fdfa4009040 comp rx=0 tx=0).stop 2023-11-13T10:55:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa8149bf0 msgr2=0x7fdfa8150670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa8149bf0 0x7fdfa8150670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0037040 tx=0x7fdfa0009750 comp rx=0 tx=0).stop 2023-11-13T10:55:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 shutdown_connections 2023-11-13T10:55:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdf8c06ae40 0x7fdf8c06d300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfa8149bf0 0x7fdfa8150670 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8147880 0x7fdfa8148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 --2- 172.21.15.16:0/1994749859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa81474a0 0x7fdfa8147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 >> 172.21.15.16:0/1994749859 conn(0x7fdfa806a190 msgr2=0x7fdfa80b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 shutdown_connections 2023-11-13T10:55:40.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:40.925+0000 7fdfb06b4640 1 -- 172.21.15.16:0/1994749859 wait complete. 2023-11-13T10:55:40.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:55:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.181+0000 7f9beaf03640 1 Processor -- start 2023-11-13T10:55:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.181+0000 7f9beaf03640 1 -- start start 2023-11-13T10:55:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.183+0000 7f9beaf03640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9be415ce70 0x7f9be40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.183+0000 7f9beaf03640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 0x7f9be40d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.183+0000 7f9beaf03640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be40cac40 con 0x7f9be40d1850 2023-11-13T10:55:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.183+0000 7f9beaf03640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be40ce0e0 con 0x7f9be415ce70 2023-11-13T10:55:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.183+0000 7f9beaf03640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40ce3c0 con 0x7f9be40cdbb0 2023-11-13T10:55:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.183+0000 7f9be9f01640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9be415ce70 0x7f9be40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.183+0000 7f9be9f01640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9be415ce70 0x7f9be40cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44322/0 (socket says 172.21.15.16:44322) 2023-11-13T10:55:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.183+0000 7f9be9f01640 1 -- 172.21.15.16:0/1731378132 learned_addr learned my addr 172.21.15.16:0/1731378132 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.183+0000 7f9be9700640 1 --2- 172.21.15.16:0/1731378132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 0x7f9be40d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.184+0000 7f9be9f01640 1 -- 172.21.15.16:0/1731378132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 msgr2=0x7f9be40d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.184+0000 7f9be9f01640 1 --2- 172.21.15.16:0/1731378132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 0x7f9be40d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.184+0000 7f9be9f01640 1 -- 172.21.15.16:0/1731378132 >> v1:172.21.15.184:6789/0 conn(0x7f9be40d1850 legacy=0x7f9be40cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:41.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.184+0000 7f9be9f01640 1 -- 172.21.15.16:0/1731378132 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40ce6a0 con 0x7f9be415ce70 2023-11-13T10:55:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.185+0000 7f9be9f01640 1 --2- 172.21.15.16:0/1731378132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 0x7f9be40cd670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9be00098a0 tx=0x7f9be002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33361d697edd082 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.185+0000 7f9be8eff640 1 -- 172.21.15.16:0/1731378132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0031640 con 0x7f9be415ce70 2023-11-13T10:55:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.185+0000 7f9be8eff640 1 -- 172.21.15.16:0/1731378132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be00317a0 con 0x7f9be415ce70 2023-11-13T10:55:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.185+0000 7f9be8eff640 1 -- 172.21.15.16:0/1731378132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0031b30 con 0x7f9be415ce70 2023-11-13T10:55:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.186+0000 7f9beaf03640 1 -- 172.21.15.16:0/1731378132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 msgr2=0x7f9be40cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.186+0000 7f9beaf03640 1 --2- 172.21.15.16:0/1731378132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 0x7f9be40cd670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9be00098a0 tx=0x7f9be002b8f0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.186+0000 7f9beaf03640 1 -- 172.21.15.16:0/1731378132 shutdown_connections 2023-11-13T10:55:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.186+0000 7f9beaf03640 1 --2- 172.21.15.16:0/1731378132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 0x7f9be40d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.186+0000 7f9beaf03640 1 --2- 172.21.15.16:0/1731378132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 0x7f9be40cd670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.186+0000 7f9beaf03640 1 -- 172.21.15.16:0/1731378132 >> 172.21.15.16:0/1731378132 conn(0x7f9be408c580 msgr2=0x7f9be409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.186+0000 7f9beaf03640 1 -- 172.21.15.16:0/1731378132 shutdown_connections 2023-11-13T10:55:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.186+0000 7f9beaf03640 1 -- 172.21.15.16:0/1731378132 wait complete. 2023-11-13T10:55:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.188+0000 7f9beaf03640 1 Processor -- start 2023-11-13T10:55:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.188+0000 7f9beaf03640 1 -- start start 2023-11-13T10:55:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.189+0000 7f9beaf03640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 0x7f9be41520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.189+0000 7f9beaf03640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9be40d1850 0x7f9be41554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.189+0000 7f9beaf03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 0x7f9be415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.189+0000 7f9beaf03640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4155e30 con 0x7f9be415ce70 2023-11-13T10:55:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.189+0000 7f9beaf03640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41525f0 con 0x7f9be40d1850 2023-11-13T10:55:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.189+0000 7f9beaf03640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40b7750 con 0x7f9be40cdbb0 2023-11-13T10:55:41.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9bea702640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 0x7f9be415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9bea702640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 0x7f9be415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44326/0 (socket says 172.21.15.16:44326) 2023-11-13T10:55:41.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9bea702640 1 -- 172.21.15.16:0/1487820176 learned_addr learned my addr 172.21.15.16:0/1487820176 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:41.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9be9700640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9be40d1850 0x7f9be41554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9be9f01640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 0x7f9be41520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9bea702640 1 -- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 msgr2=0x7f9be41520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9bea702640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 0x7f9be41520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9bea702640 1 -- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9be40d1850 msgr2=0x7f9be41554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9bea702640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9be40d1850 0x7f9be41554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9bea702640 1 -- 172.21.15.16:0/1487820176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be00094d0 con 0x7f9be415ce70 2023-11-13T10:55:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.190+0000 7f9bea702640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 0x7f9be415f5a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00c700 tx=0x7f9bdc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.191+0000 7f9bda7fc640 1 -- 172.21.15.16:0/1487820176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00aa50 con 0x7f9be415ce70 2023-11-13T10:55:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.191+0000 7f9bda7fc640 1 -- 172.21.15.16:0/1487820176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc00abb0 con 0x7f9be415ce70 2023-11-13T10:55:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.191+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be40b6e20 con 0x7f9be415ce70 2023-11-13T10:55:41.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.191+0000 7f9bda7fc640 1 -- 172.21.15.16:0/1487820176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00ce10 con 0x7f9be415ce70 2023-11-13T10:55:41.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.191+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be40b73e0 con 0x7f9be415ce70 2023-11-13T10:55:41.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.192+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be40675b0 con 0x7f9be415ce70 2023-11-13T10:55:41.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.193+0000 7f9bda7fc640 1 -- 172.21.15.16:0/1487820176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9bdc012420 con 0x7f9be415ce70 2023-11-13T10:55:41.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.194+0000 7f9bda7fc640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bc806ae70 0x7f9bc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.194+0000 7f9bda7fc640 1 -- 172.21.15.16:0/1487820176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9bdc089d90 con 0x7f9be415ce70 2023-11-13T10:55:41.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.195+0000 7f9be9f01640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bc806ae70 0x7f9bc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.196+0000 7f9be9f01640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bc806ae70 0x7f9bc806d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9be0036000 tx=0x7f9be003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:41.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.199+0000 7f9bda7fc640 1 -- 172.21.15.16:0/1487820176 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bdc054b50 con 0x7f9be415ce70 2023-11-13T10:55:41.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.392+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9be40b78c0 con 0x7f9be415ce70 2023-11-13T10:55:41.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.393+0000 7f9bda7fc640 1 -- 172.21.15.16:0/1487820176 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9bdc016020 con 0x7f9be415ce70 2023-11-13T10:55:41.398 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:55:41.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bc806ae70 msgr2=0x7f9bc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bc806ae70 0x7f9bc806d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9be0036000 tx=0x7f9be003c040 comp rx=0 tx=0).stop 2023-11-13T10:55:41.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 msgr2=0x7f9be415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 0x7f9be415f5a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00c700 tx=0x7f9bdc00cbc0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 shutdown_connections 2023-11-13T10:55:41.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bc806ae70 0x7f9bc806d330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be415ce70 0x7f9be415f5a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9be40d1850 0x7f9be41554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 --2- 172.21.15.16:0/1487820176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9be40cdbb0 0x7f9be41520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 >> 172.21.15.16:0/1487820176 conn(0x7f9be408c580 msgr2=0x7f9be40cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 shutdown_connections 2023-11-13T10:55:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.396+0000 7f9beaf03640 1 -- 172.21.15.16:0/1487820176 wait complete. 2023-11-13T10:55:41.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:55:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.649+0000 7f7efd935640 1 Processor -- start 2023-11-13T10:55:41.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.649+0000 7f7efd935640 1 -- start start 2023-11-13T10:55:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.650+0000 7f7efd935640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ef80715b0 0x7f7ef806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7efd935640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80b3760 0x7f7ef80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7efd935640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7ef80ace20 0x7f7ef8160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7efd935640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef806c920 con 0x7f7ef80715b0 2023-11-13T10:55:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7efd935640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef80ad240 con 0x7f7ef80ace20 2023-11-13T10:55:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7efd935640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef806bca0 con 0x7f7ef80b3760 2023-11-13T10:55:41.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7efc933640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ef80715b0 0x7f7ef806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7efc933640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ef80715b0 0x7f7ef806b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44340/0 (socket says 172.21.15.16:44340) 2023-11-13T10:55:41.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7efc933640 1 -- 172.21.15.16:0/4182700658 learned_addr learned my addr 172.21.15.16:0/4182700658 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:41.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7efd134640 1 --2- 172.21.15.16:0/4182700658 >> v2:172.21.15.184:3300/0 conn(0x7f7ef80ace20 0x7f7ef8160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.651+0000 7f7eeffff640 1 --2- 172.21.15.16:0/4182700658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80b3760 0x7f7ef80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.652+0000 7f7efc933640 1 -- 172.21.15.16:0/4182700658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80b3760 msgr2=0x7f7ef80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:41.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.652+0000 7f7efc933640 1 --2- 172.21.15.16:0/4182700658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80b3760 0x7f7ef80ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.652+0000 7f7efc933640 1 -- 172.21.15.16:0/4182700658 >> v2:172.21.15.184:3300/0 conn(0x7f7ef80ace20 msgr2=0x7f7ef8160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:41.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.652+0000 7f7efc933640 1 --2- 172.21.15.16:0/4182700658 >> v2:172.21.15.184:3300/0 conn(0x7f7ef80ace20 0x7f7ef8160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.652+0000 7f7efc933640 1 -- 172.21.15.16:0/4182700658 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef806bf80 con 0x7f7ef80715b0 2023-11-13T10:55:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.653+0000 7f7efc933640 1 --2- 172.21.15.16:0/4182700658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80715b0 0x7f7ef806b6c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7ef806dbd0 tx=0x7f7ef402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea7516f7b09f9d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.653+0000 7f7eef7fe640 1 -- 172.21.15.16:0/4182700658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40317d0 con 0x7f7ef80715b0 2023-11-13T10:55:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.653+0000 7f7eef7fe640 1 -- 172.21.15.16:0/4182700658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4031930 con 0x7f7ef80715b0 2023-11-13T10:55:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.653+0000 7f7eef7fe640 1 -- 172.21.15.16:0/4182700658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4031cc0 con 0x7f7ef80715b0 2023-11-13T10:55:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.654+0000 7f7efd935640 1 -- 172.21.15.16:0/4182700658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80715b0 msgr2=0x7f7ef806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.654+0000 7f7efd935640 1 --2- 172.21.15.16:0/4182700658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80715b0 0x7f7ef806b6c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7ef806dbd0 tx=0x7f7ef402b870 comp rx=0 tx=0).stop 2023-11-13T10:55:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.654+0000 7f7efd935640 1 -- 172.21.15.16:0/4182700658 shutdown_connections 2023-11-13T10:55:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.654+0000 7f7efd935640 1 --2- 172.21.15.16:0/4182700658 >> v2:172.21.15.184:3300/0 conn(0x7f7ef80ace20 0x7f7ef8160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.654+0000 7f7efd935640 1 --2- 172.21.15.16:0/4182700658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80b3760 0x7f7ef80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.654+0000 7f7efd935640 1 --2- 172.21.15.16:0/4182700658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80715b0 0x7f7ef806b6c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.654+0000 7f7efd935640 1 -- 172.21.15.16:0/4182700658 >> 172.21.15.16:0/4182700658 conn(0x7f7ef80b0970 msgr2=0x7f7ef80b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.654+0000 7f7efd935640 1 -- 172.21.15.16:0/4182700658 shutdown_connections 2023-11-13T10:55:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.654+0000 7f7efd935640 1 -- 172.21.15.16:0/4182700658 wait complete. 2023-11-13T10:55:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.657+0000 7f7efd935640 1 Processor -- start 2023-11-13T10:55:41.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.657+0000 7f7efd935640 1 -- start start 2023-11-13T10:55:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.658+0000 7f7efd935640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80715b0 0x7f7ef80c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.658+0000 7f7efd935640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef80ace20 0x7f7ef815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.658+0000 7f7efd935640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80b3760 0x7f7ef8137620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.658+0000 7f7efd935640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef80c88a0 con 0x7f7ef80b3760 2023-11-13T10:55:41.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.658+0000 7f7efd935640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef80cd8a0 con 0x7f7ef80ace20 2023-11-13T10:55:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.658+0000 7f7efd935640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8137b60 con 0x7f7ef80715b0 2023-11-13T10:55:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.658+0000 7f7efd134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80b3760 0x7f7ef8137620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7efd134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80b3760 0x7f7ef8137620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44354/0 (socket says 172.21.15.16:44354) 2023-11-13T10:55:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7efd134640 1 -- 172.21.15.16:0/667966619 learned_addr learned my addr 172.21.15.16:0/667966619 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:41.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7efc933640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80715b0 0x7f7ef80c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7eeffff640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef80ace20 0x7f7ef815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7efd134640 1 -- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80715b0 msgr2=0x7f7ef80c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7efd134640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80715b0 0x7f7ef80c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7efd134640 1 -- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef80ace20 msgr2=0x7f7ef815c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7efd134640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef80ace20 0x7f7ef815c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7efd134640 1 -- 172.21.15.16:0/667966619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40094d0 con 0x7f7ef80b3760 2023-11-13T10:55:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.659+0000 7f7efd134640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80b3760 0x7f7ef8137620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0007d50 tx=0x7f7ef000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.660+0000 7f7eed7fa640 1 -- 172.21.15.16:0/667966619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0014420 con 0x7f7ef80b3760 2023-11-13T10:55:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.660+0000 7f7eed7fa640 1 -- 172.21.15.16:0/667966619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef00145a0 con 0x7f7ef80b3760 2023-11-13T10:55:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.660+0000 7f7eed7fa640 1 -- 172.21.15.16:0/667966619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0014950 con 0x7f7ef80b3760 2023-11-13T10:55:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.660+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef80cccd0 con 0x7f7ef80b3760 2023-11-13T10:55:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.660+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef80cd250 con 0x7f7ef80b3760 2023-11-13T10:55:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.661+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef80cda50 con 0x7f7ef80b3760 2023-11-13T10:55:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.662+0000 7f7eed7fa640 1 -- 172.21.15.16:0/667966619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ef0014ab0 con 0x7f7ef80b3760 2023-11-13T10:55:41.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.663+0000 7f7eed7fa640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ee406ae40 0x7f7ee406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.663+0000 7f7eed7fa640 1 -- 172.21.15.16:0/667966619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7ef008a1b0 con 0x7f7ef80b3760 2023-11-13T10:55:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.664+0000 7f7efc933640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ee406ae40 0x7f7ee406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.665+0000 7f7efc933640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ee406ae40 0x7f7ee406d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4036000 tx=0x7f7ef403d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:41.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.667+0000 7f7eed7fa640 1 -- 172.21.15.16:0/667966619 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7ef0054ed0 con 0x7f7ef80b3760 2023-11-13T10:55:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.871+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7ef804e4d0 con 0x7f7ef80b3760 2023-11-13T10:55:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.872+0000 7f7eed7fa640 1 -- 172.21.15.16:0/667966619 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7ef00584f0 con 0x7f7ef80b3760 2023-11-13T10:55:41.878 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:55:41.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.874+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ee406ae40 msgr2=0x7f7ee406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.874+0000 7f7efd935640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ee406ae40 0x7f7ee406d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4036000 tx=0x7f7ef403d040 comp rx=0 tx=0).stop 2023-11-13T10:55:41.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.874+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80b3760 msgr2=0x7f7ef8137620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.874+0000 7f7efd935640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80b3760 0x7f7ef8137620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0007d50 tx=0x7f7ef000e4b0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.875+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 shutdown_connections 2023-11-13T10:55:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.875+0000 7f7efd935640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ee406ae40 0x7f7ee406d300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.875+0000 7f7efd935640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ef80b3760 0x7f7ef8137620 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.875+0000 7f7efd935640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ef80ace20 0x7f7ef815c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.875+0000 7f7efd935640 1 --2- 172.21.15.16:0/667966619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ef80715b0 0x7f7ef80c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:41.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.875+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 >> 172.21.15.16:0/667966619 conn(0x7f7ef80b0970 msgr2=0x7f7ef8145c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:41.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.875+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 shutdown_connections 2023-11-13T10:55:41.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:41.875+0000 7f7efd935640 1 -- 172.21.15.16:0/667966619 wait complete. 2023-11-13T10:55:41.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:55:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.129+0000 7fb6bf294640 1 Processor -- start 2023-11-13T10:55:42.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.129+0000 7fb6bf294640 1 -- start start 2023-11-13T10:55:42.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.130+0000 7fb6bf294640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6b815d0b0 0x7fb6b80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.130+0000 7fb6bf294640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80cddb0 0x7fb6b80ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.130+0000 7fb6bf294640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb6b80d0b80 0x7fb6b80d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.130+0000 7fb6bf294640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b80cfda0 con 0x7fb6b815d0b0 2023-11-13T10:55:42.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.130+0000 7fb6bf294640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b80b29e0 con 0x7fb6b80d0b80 2023-11-13T10:55:42.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.130+0000 7fb6bf294640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80b26b0 con 0x7fb6b80cddb0 2023-11-13T10:55:42.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd030640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6b815d0b0 0x7fb6b80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd030640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6b815d0b0 0x7fb6b80cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44356/0 (socket says 172.21.15.16:44356) 2023-11-13T10:55:42.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd030640 1 -- 172.21.15.16:0/58535667 learned_addr learned my addr 172.21.15.16:0/58535667 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:42.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd831640 1 --2- 172.21.15.16:0/58535667 >> v2:172.21.15.184:3300/0 conn(0x7fb6b80d0b80 0x7fb6b80d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd030640 1 -- 172.21.15.16:0/58535667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80cddb0 msgr2=0x7fb6b80ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:42.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bc82f640 1 --2- 172.21.15.16:0/58535667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80cddb0 0x7fb6b80ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd030640 1 --2- 172.21.15.16:0/58535667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80cddb0 0x7fb6b80ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd030640 1 -- 172.21.15.16:0/58535667 >> v2:172.21.15.184:3300/0 conn(0x7fb6b80d0b80 msgr2=0x7fb6b80d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd030640 1 --2- 172.21.15.16:0/58535667 >> v2:172.21.15.184:3300/0 conn(0x7fb6b80d0b80 0x7fb6b80d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd030640 1 -- 172.21.15.16:0/58535667 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b80ce750 con 0x7fb6b815d0b0 2023-11-13T10:55:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bd831640 1 --2- 172.21.15.16:0/58535667 >> v2:172.21.15.184:3300/0 conn(0x7fb6b80d0b80 0x7fb6b80d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.131+0000 7fb6bc82f640 1 --2- 172.21.15.16:0/58535667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80cddb0 0x7fb6b80ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.132+0000 7fb6bd030640 1 --2- 172.21.15.16:0/58535667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 0x7fb6b80cd870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80a0cb0 tx=0x7fb6ac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e26db7b998c466d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.133+0000 7fb6b7fff640 1 -- 172.21.15.16:0/58535667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac02ca80 con 0x7fb6b815d0b0 2023-11-13T10:55:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.133+0000 7fb6b7fff640 1 -- 172.21.15.16:0/58535667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac02cbe0 con 0x7fb6b815d0b0 2023-11-13T10:55:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.133+0000 7fb6b7fff640 1 -- 172.21.15.16:0/58535667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0343f0 con 0x7fb6b815d0b0 2023-11-13T10:55:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.133+0000 7fb6bf294640 1 -- 172.21.15.16:0/58535667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 msgr2=0x7fb6b80cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.133+0000 7fb6bf294640 1 --2- 172.21.15.16:0/58535667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 0x7fb6b80cd870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80a0cb0 tx=0x7fb6ac02bee0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.134+0000 7fb6bf294640 1 -- 172.21.15.16:0/58535667 shutdown_connections 2023-11-13T10:55:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.134+0000 7fb6bf294640 1 --2- 172.21.15.16:0/58535667 >> v2:172.21.15.184:3300/0 conn(0x7fb6b80d0b80 0x7fb6b80d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.134+0000 7fb6bf294640 1 --2- 172.21.15.16:0/58535667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80cddb0 0x7fb6b80ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.134+0000 7fb6bf294640 1 --2- 172.21.15.16:0/58535667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 0x7fb6b80cd870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.134+0000 7fb6bf294640 1 -- 172.21.15.16:0/58535667 >> 172.21.15.16:0/58535667 conn(0x7fb6b80d2da0 msgr2=0x7fb6b80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.134+0000 7fb6bf294640 1 -- 172.21.15.16:0/58535667 shutdown_connections 2023-11-13T10:55:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.134+0000 7fb6bf294640 1 -- 172.21.15.16:0/58535667 wait complete. 2023-11-13T10:55:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.135+0000 7fb6bf294640 1 Processor -- start 2023-11-13T10:55:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.136+0000 7fb6bf294640 1 -- start start 2023-11-13T10:55:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bf294640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6b80cddb0 0x7fb6b81f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bf294640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80d0b80 0x7fb6b806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bf294640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 0x7fb6b806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bf294640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80b8330 con 0x7fb6b815d0b0 2023-11-13T10:55:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bf294640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80ca660 con 0x7fb6b80cddb0 2023-11-13T10:55:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bf294640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80ca900 con 0x7fb6b80d0b80 2023-11-13T10:55:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bd831640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 0x7fb6b806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bd831640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 0x7fb6b806c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44366/0 (socket says 172.21.15.16:44366) 2023-11-13T10:55:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bd831640 1 -- 172.21.15.16:0/3748238500 learned_addr learned my addr 172.21.15.16:0/3748238500 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bd030640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6b80cddb0 0x7fb6b81f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bc82f640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80d0b80 0x7fb6b806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.137+0000 7fb6bd831640 1 -- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80d0b80 msgr2=0x7fb6b806bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6bd831640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80d0b80 0x7fb6b806bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6bd831640 1 -- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6b80cddb0 msgr2=0x7fb6b81f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6bd831640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6b80cddb0 0x7fb6b81f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6bd831640 1 -- 172.21.15.16:0/3748238500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac009b30 con 0x7fb6b815d0b0 2023-11-13T10:55:42.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6bd030640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6b80cddb0 0x7fb6b81f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6bc82f640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80d0b80 0x7fb6b806bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6bd831640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 0x7fb6b806c280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000c7c0 tx=0x7fb6b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6b5ffb640 1 -- 172.21.15.16:0/3748238500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0004230 con 0x7fb6b815d0b0 2023-11-13T10:55:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6b5ffb640 1 -- 172.21.15.16:0/3748238500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b0004390 con 0x7fb6b815d0b0 2023-11-13T10:55:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6b5ffb640 1 -- 172.21.15.16:0/3748238500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0007860 con 0x7fb6b815d0b0 2023-11-13T10:55:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b80b7a90 con 0x7fb6b815d0b0 2023-11-13T10:55:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.138+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b80b8050 con 0x7fb6b815d0b0 2023-11-13T10:55:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.140+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b80caac0 con 0x7fb6b815d0b0 2023-11-13T10:55:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.140+0000 7fb6b5ffb640 1 -- 172.21.15.16:0/3748238500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6b00079c0 con 0x7fb6b815d0b0 2023-11-13T10:55:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.141+0000 7fb6b5ffb640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb69c06ae40 0x7fb69c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.141+0000 7fb6b5ffb640 1 -- 172.21.15.16:0/3748238500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb6b0089ac0 con 0x7fb6b815d0b0 2023-11-13T10:55:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.142+0000 7fb6bd030640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb69c06ae40 0x7fb69c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.143+0000 7fb6bd030640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb69c06ae40 0x7fb69c06d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac035000 tx=0x7fb6ac030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.145+0000 7fb6b5ffb640 1 -- 172.21.15.16:0/3748238500 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6b00549d0 con 0x7fb6b815d0b0 2023-11-13T10:55:42.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.351+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb6b8067090 con 0x7fb6b815d0b0 2023-11-13T10:55:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.352+0000 7fb6b5ffb640 1 -- 172.21.15.16:0/3748238500 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb6b0057ff0 con 0x7fb6b815d0b0 2023-11-13T10:55:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:55:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb69c06ae40 msgr2=0x7fb69c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb69c06ae40 0x7fb69c06d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac035000 tx=0x7fb6ac030000 comp rx=0 tx=0).stop 2023-11-13T10:55:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 msgr2=0x7fb6b806c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 0x7fb6b806c280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000c7c0 tx=0x7fb6b000cc80 comp rx=0 tx=0).stop 2023-11-13T10:55:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 shutdown_connections 2023-11-13T10:55:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb69c06ae40 0x7fb69c06d300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6b815d0b0 0x7fb6b806c280 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6b80d0b80 0x7fb6b806bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 --2- 172.21.15.16:0/3748238500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6b80cddb0 0x7fb6b81f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 >> 172.21.15.16:0/3748238500 conn(0x7fb6b80d2da0 msgr2=0x7fb6b80b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 shutdown_connections 2023-11-13T10:55:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.355+0000 7fb6bf294640 1 -- 172.21.15.16:0/3748238500 wait complete. 2023-11-13T10:55:42.378 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:55:42.379 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:55:42.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.655+0000 7f2a2b025640 1 Processor -- start 2023-11-13T10:55:42.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.655+0000 7f2a2b025640 1 -- start start 2023-11-13T10:55:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.657+0000 7f2a2b025640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2a2415cfc0 0x7f2a240cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.657+0000 7f2a2b025640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a240b30e0 0x7f2a240d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.657+0000 7f2a2b025640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a240b3cf0 con 0x7f2a240d2ae0 2023-11-13T10:55:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.657+0000 7f2a2b025640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a240d3480 con 0x7f2a2415cfc0 2023-11-13T10:55:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.657+0000 7f2a2b025640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240b3730 con 0x7f2a240b30e0 2023-11-13T10:55:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.657+0000 7f2a29822640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2a240d2ae0 0x7f2a240b29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54542/0 (socket says 172.21.15.16:54542) 2023-11-13T10:55:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.657+0000 7f2a29822640 1 -- 172.21.15.16:0/3956558539 learned_addr learned my addr 172.21.15.16:0/3956558539 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.657+0000 7f2a2a023640 1 --2- 172.21.15.16:0/3956558539 >> v2:172.21.15.184:3300/0 conn(0x7f2a2415cfc0 0x7f2a240cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a2a824640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a240b30e0 0x7f2a240d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149059014 0 0) 0x7f2a240b3cf0 con 0x7f2a240d2ae0 2023-11-13T10:55:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a10003520 con 0x7f2a240d2ae0 2023-11-13T10:55:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1322880360 0 0) 0x7f2a10003520 con 0x7f2a240d2ae0 2023-11-13T10:55:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a240b30e0 msgr2=0x7f2a240d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a240b30e0 0x7f2a240d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 >> v2:172.21.15.184:3300/0 conn(0x7f2a2415cfc0 msgr2=0x7f2a240cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 --2- 172.21.15.16:0/3956558539 >> v2:172.21.15.184:3300/0 conn(0x7f2a2415cfc0 0x7f2a240cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240b3a10 con 0x7f2a240d2ae0 2023-11-13T10:55:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f2a14001670 con 0x7f2a240d2ae0 2023-11-13T10:55:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 >> v1:172.21.15.16:6789/0 conn(0x7f2a240d2ae0 legacy=0x7f2a240b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a240b30e0 0x7f2a10003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a10003ef0 0x7f2a100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a10004d00 0x7f2a10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240d3480 con 0x7f2a10004d00 2023-11-13T10:55:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240b3730 con 0x7f2a10003ef0 2023-11-13T10:55:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.658+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1000b9d0 con 0x7f2a240b30e0 2023-11-13T10:55:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.659+0000 7f2a2a824640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a10004d00 0x7f2a10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.659+0000 7f2a2a023640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a240b30e0 0x7f2a10003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.659+0000 7f2a29822640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a10003ef0 0x7f2a100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.659+0000 7f2a2a824640 1 -- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a240b30e0 msgr2=0x7f2a10003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.659+0000 7f2a2a824640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a240b30e0 0x7f2a10003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.659+0000 7f2a2a824640 1 -- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a10003ef0 msgr2=0x7f2a100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.659+0000 7f2a2a824640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a10003ef0 0x7f2a100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.659+0000 7f2a2a824640 1 -- 172.21.15.16:0/3956558539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a10003520 con 0x7f2a10004d00 2023-11-13T10:55:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.659+0000 7f2a2a023640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a240b30e0 0x7f2a10003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.660+0000 7f2a2a824640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a10004d00 0x7f2a10005180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00e170 tx=0x7f2a100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcc9fb1dedff646 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.661+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c02cbf0 con 0x7f2a10004d00 2023-11-13T10:55:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.661+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c00f040 con 0x7f2a10004d00 2023-11-13T10:55:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.661+0000 7f2a29021640 1 -- 172.21.15.16:0/3956558539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c034440 con 0x7f2a10004d00 2023-11-13T10:55:42.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.661+0000 7f2a2b025640 1 -- 172.21.15.16:0/3956558539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a10004d00 msgr2=0x7f2a10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.661+0000 7f2a2b025640 1 --2- 172.21.15.16:0/3956558539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a10004d00 0x7f2a10005180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00e170 tx=0x7f2a100057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.661+0000 7f2a2a824640 1 -- 172.21.15.16:0/3956558539 reap_dead start 2023-11-13T10:55:42.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.661+0000 7f2a2b025640 1 -- 172.21.15.16:0/3956558539 shutdown_connections 2023-11-13T10:55:42.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.661+0000 7f2a2b025640 1 -- 172.21.15.16:0/3956558539 >> 172.21.15.16:0/3956558539 conn(0x7f2a2408c570 msgr2=0x7f2a2409be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:42.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.661+0000 7f2a2b025640 1 -- 172.21.15.16:0/3956558539 shutdown_connections 2023-11-13T10:55:42.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.662+0000 7f2a2b025640 1 -- 172.21.15.16:0/3956558539 wait complete. 2023-11-13T10:55:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.663+0000 7f2a2b025640 1 Processor -- start 2023-11-13T10:55:42.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.664+0000 7f2a2b025640 1 -- start start 2023-11-13T10:55:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.664+0000 7f2a2b025640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a10004d00 0x7f2a241f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.664+0000 7f2a2b025640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a240af980 0x7f2a24158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.664+0000 7f2a2b025640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240afe00 0x7f2a24159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.664+0000 7f2a2b025640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a241f6ab0 con 0x7f2a240afe00 2023-11-13T10:55:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.664+0000 7f2a2b025640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240ccb00 con 0x7f2a240af980 2023-11-13T10:55:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.664+0000 7f2a2b025640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240ca670 con 0x7f2a10004d00 2023-11-13T10:55:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a824640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240afe00 0x7f2a24159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a824640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240afe00 0x7f2a24159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44386/0 (socket says 172.21.15.16:44386) 2023-11-13T10:55:42.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a824640 1 -- 172.21.15.16:0/3387314169 learned_addr learned my addr 172.21.15.16:0/3387314169 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:42.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a023640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a10004d00 0x7f2a241f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a824640 1 -- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a10004d00 msgr2=0x7f2a241f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a29822640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a240af980 0x7f2a24158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a824640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a10004d00 0x7f2a241f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a824640 1 -- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a240af980 msgr2=0x7f2a24158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a824640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a240af980 0x7f2a24158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a824640 1 -- 172.21.15.16:0/3387314169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240ca950 con 0x7f2a240afe00 2023-11-13T10:55:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.665+0000 7f2a2a023640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a10004d00 0x7f2a241f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.666+0000 7f2a2a824640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240afe00 0x7f2a24159220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00e170 tx=0x7f2a1c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.666+0000 7f2a1affd640 1 -- 172.21.15.16:0/3387314169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c0349c0 con 0x7f2a240afe00 2023-11-13T10:55:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.666+0000 7f2a1affd640 1 -- 172.21.15.16:0/3387314169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c034b20 con 0x7f2a240afe00 2023-11-13T10:55:42.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.666+0000 7f2a1affd640 1 -- 172.21.15.16:0/3387314169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c003990 con 0x7f2a240afe00 2023-11-13T10:55:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.666+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a240cac30 con 0x7f2a240afe00 2023-11-13T10:55:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.666+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a240cb1f0 con 0x7f2a240afe00 2023-11-13T10:55:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.667+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2404e4d0 con 0x7f2a240afe00 2023-11-13T10:55:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.668+0000 7f2a1affd640 1 -- 172.21.15.16:0/3387314169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a1c047460 con 0x7f2a240afe00 2023-11-13T10:55:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.670+0000 7f2a1affd640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0806ae40 0x7f2a0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.670+0000 7f2a1affd640 1 -- 172.21.15.16:0/3387314169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2a1c0ab820 con 0x7f2a240afe00 2023-11-13T10:55:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.670+0000 7f2a2a023640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0806ae40 0x7f2a0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.671+0000 7f2a2a023640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0806ae40 0x7f2a0806d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009b00 tx=0x7f2a20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.674+0000 7f2a1affd640 1 -- 172.21.15.16:0/3387314169 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a1c0765d0 con 0x7f2a240afe00 2023-11-13T10:55:42.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.878+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a2415e750 con 0x7f2a240afe00 2023-11-13T10:55:42.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.879+0000 7f2a1affd640 1 -- 172.21.15.16:0/3387314169 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f2a1c079bf0 con 0x7f2a240afe00 2023-11-13T10:55:42.884 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:42.885 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:42.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0806ae40 msgr2=0x7f2a0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0806ae40 0x7f2a0806d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009b00 tx=0x7f2a20008040 comp rx=0 tx=0).stop 2023-11-13T10:55:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240afe00 msgr2=0x7f2a24159220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240afe00 0x7f2a24159220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00e170 tx=0x7f2a1c009750 comp rx=0 tx=0).stop 2023-11-13T10:55:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 shutdown_connections 2023-11-13T10:55:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2a0806ae40 0x7f2a0806d300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a240afe00 0x7f2a24159220 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a240af980 0x7f2a24158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 --2- 172.21.15.16:0/3387314169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a10004d00 0x7f2a241f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 >> 172.21.15.16:0/3387314169 conn(0x7f2a2408c570 msgr2=0x7f2a2406a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 shutdown_connections 2023-11-13T10:55:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:42.882+0000 7f2a2b025640 1 -- 172.21.15.16:0/3387314169 wait complete. 2023-11-13T10:55:42.912 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:55:42.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:55:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.146+0000 7f2982d28640 1 Processor -- start 2023-11-13T10:55:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.146+0000 7f2982d28640 1 -- start start 2023-11-13T10:55:43.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.147+0000 7f2982d28640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f297c15ce70 0x7f297c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.147+0000 7f2982d28640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f297c0d1850 0x7f297c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.147+0000 7f2982d28640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c0ce0e0 con 0x7f297c0cdbb0 2023-11-13T10:55:43.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.147+0000 7f2982d28640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0ce3c0 con 0x7f297c15ce70 2023-11-13T10:55:43.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.147+0000 7f2982d28640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0ce6a0 con 0x7f297c0d1850 2023-11-13T10:55:43.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.147+0000 7f2981d26640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f297c15ce70 0x7f297c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.147+0000 7f2981d26640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f297c15ce70 0x7f297c0cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45970/0 (socket says 172.21.15.16:45970) 2023-11-13T10:55:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.147+0000 7f2981d26640 1 -- 172.21.15.16:0/59931379 learned_addr learned my addr 172.21.15.16:0/59931379 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.148+0000 7f2982527640 1 --2- 172.21.15.16:0/59931379 >> v2:172.21.15.184:3300/0 conn(0x7f297c0d1850 0x7f297c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.148+0000 7f2981d26640 1 -- 172.21.15.16:0/59931379 >> v2:172.21.15.184:3300/0 conn(0x7f297c0d1850 msgr2=0x7f297c0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.148+0000 7f2981d26640 1 --2- 172.21.15.16:0/59931379 >> v2:172.21.15.184:3300/0 conn(0x7f297c0d1850 0x7f297c0c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.148+0000 7f2981d26640 1 -- 172.21.15.16:0/59931379 >> v1:172.21.15.184:6789/0 conn(0x7f297c0cdbb0 legacy=0x7f297c0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.148+0000 7f2981d26640 1 -- 172.21.15.16:0/59931379 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0ce980 con 0x7f297c15ce70 2023-11-13T10:55:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.148+0000 7f2982527640 1 --2- 172.21.15.16:0/59931379 >> v2:172.21.15.184:3300/0 conn(0x7f297c0d1850 0x7f297c0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:43.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.149+0000 7f2981d26640 1 --2- 172.21.15.16:0/59931379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c15ce70 0x7f297c0cd670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f297c08d170 tx=0x7f296c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67408368dbe627dc server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.149+0000 7f2980d24640 1 -- 172.21.15.16:0/59931379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c02ca80 con 0x7f297c15ce70 2023-11-13T10:55:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.149+0000 7f2980d24640 1 -- 172.21.15.16:0/59931379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c02cbe0 con 0x7f297c15ce70 2023-11-13T10:55:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.149+0000 7f2980d24640 1 -- 172.21.15.16:0/59931379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0343f0 con 0x7f297c15ce70 2023-11-13T10:55:43.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.150+0000 7f2982d28640 1 -- 172.21.15.16:0/59931379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c15ce70 msgr2=0x7f297c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.150+0000 7f2982d28640 1 --2- 172.21.15.16:0/59931379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c15ce70 0x7f297c0cd670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f297c08d170 tx=0x7f296c02bee0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.150+0000 7f2982d28640 1 -- 172.21.15.16:0/59931379 shutdown_connections 2023-11-13T10:55:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.150+0000 7f2982d28640 1 --2- 172.21.15.16:0/59931379 >> v2:172.21.15.184:3300/0 conn(0x7f297c0d1850 0x7f297c0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.150+0000 7f2982d28640 1 --2- 172.21.15.16:0/59931379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c15ce70 0x7f297c0cd670 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.150+0000 7f2982d28640 1 -- 172.21.15.16:0/59931379 >> 172.21.15.16:0/59931379 conn(0x7f297c08c580 msgr2=0x7f297c09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:43.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.150+0000 7f2982d28640 1 -- 172.21.15.16:0/59931379 shutdown_connections 2023-11-13T10:55:43.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.150+0000 7f2982d28640 1 -- 172.21.15.16:0/59931379 wait complete. 2023-11-13T10:55:43.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.152+0000 7f2982d28640 1 Processor -- start 2023-11-13T10:55:43.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.152+0000 7f2982d28640 1 -- start start 2023-11-13T10:55:43.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.153+0000 7f2982d28640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f297c0cdbb0 0x7f297c1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.153+0000 7f2982d28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c0d1850 0x7f297c151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.153+0000 7f2982d28640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c15ce70 0x7f297c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.153+0000 7f2982d28640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0b73d0 con 0x7f297c0d1850 2023-11-13T10:55:43.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.153+0000 7f2982d28640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0b8180 con 0x7f297c15ce70 2023-11-13T10:55:43.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.153+0000 7f2982d28640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0b7600 con 0x7f297c0cdbb0 2023-11-13T10:55:43.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.153+0000 7f2981525640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c0d1850 0x7f297c151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.153+0000 7f2981525640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c0d1850 0x7f297c151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45986/0 (socket says 172.21.15.16:45986) 2023-11-13T10:55:43.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.153+0000 7f2981525640 1 -- 172.21.15.16:0/918429385 learned_addr learned my addr 172.21.15.16:0/918429385 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:43.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.154+0000 7f2981d26640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f297c0cdbb0 0x7f297c1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.154+0000 7f2982527640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c15ce70 0x7f297c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.154+0000 7f2981525640 1 -- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f297c0cdbb0 msgr2=0x7f297c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.154+0000 7f2981525640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f297c0cdbb0 0x7f297c1563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.154+0000 7f2981525640 1 -- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c15ce70 msgr2=0x7f297c158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.154+0000 7f2981525640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c15ce70 0x7f297c158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.154+0000 7f2981525640 1 -- 172.21.15.16:0/918429385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c009b30 con 0x7f297c0d1850 2023-11-13T10:55:43.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.154+0000 7f2981525640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c0d1850 0x7f297c151cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f297000c400 tx=0x7f297000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:43.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.155+0000 7f297a7fc640 1 -- 172.21.15.16:0/918429385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29700035f0 con 0x7f297c0d1850 2023-11-13T10:55:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.155+0000 7f297a7fc640 1 -- 172.21.15.16:0/918429385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970003750 con 0x7f297c0d1850 2023-11-13T10:55:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.155+0000 7f297a7fc640 1 -- 172.21.15.16:0/918429385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970003b40 con 0x7f297c0d1850 2023-11-13T10:55:43.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.155+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c0b78e0 con 0x7f297c0d1850 2023-11-13T10:55:43.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.155+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c0b7ea0 con 0x7f297c0d1850 2023-11-13T10:55:43.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.156+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c0c65c0 con 0x7f297c0d1850 2023-11-13T10:55:43.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.157+0000 7f297a7fc640 1 -- 172.21.15.16:0/918429385 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2970003ca0 con 0x7f297c0d1850 2023-11-13T10:55:43.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.158+0000 7f297a7fc640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f296806ae40 0x7f296806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.158+0000 7f297a7fc640 1 -- 172.21.15.16:0/918429385 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2970088a50 con 0x7f297c0d1850 2023-11-13T10:55:43.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.159+0000 7f2981d26640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f296806ae40 0x7f296806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.160+0000 7f2981d26640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f296806ae40 0x7f296806d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f296c035000 tx=0x7f296c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:43.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.164+0000 7f297a7fc640 1 -- 172.21.15.16:0/918429385 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f297c0c65c0 con 0x7f297c0d1850 2023-11-13T10:55:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.357+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f297c0c65c0 con 0x7f297c0d1850 2023-11-13T10:55:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.358+0000 7f297a7fc640 1 -- 172.21.15.16:0/918429385 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f297c0c65c0 con 0x7f297c0d1850 2023-11-13T10:55:43.363 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:55:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f296806ae40 msgr2=0x7f296806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f296806ae40 0x7f296806d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f296c035000 tx=0x7f296c030000 comp rx=0 tx=0).stop 2023-11-13T10:55:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c0d1850 msgr2=0x7f297c151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c0d1850 0x7f297c151cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f297000c400 tx=0x7f297000c8c0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 shutdown_connections 2023-11-13T10:55:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f296806ae40 0x7f296806d300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f297c15ce70 0x7f297c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f297c0d1850 0x7f297c151cc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 --2- 172.21.15.16:0/918429385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f297c0cdbb0 0x7f297c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 >> 172.21.15.16:0/918429385 conn(0x7f297c08c580 msgr2=0x7f297c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.361+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 shutdown_connections 2023-11-13T10:55:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.362+0000 7f2982d28640 1 -- 172.21.15.16:0/918429385 wait complete. 2023-11-13T10:55:43.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:55:43.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.617+0000 7f5a7b67c640 1 Processor -- start 2023-11-13T10:55:43.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.617+0000 7f5a7b67c640 1 -- start start 2023-11-13T10:55:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.619+0000 7f5a7b67c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a7415e0e0 0x7f5a7406c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.619+0000 7f5a7b67c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7406d910 con 0x7f5a740719b0 2023-11-13T10:55:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.619+0000 7f5a7b67c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7406db80 con 0x7f5a7406c930 2023-11-13T10:55:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.619+0000 7f5a7b67c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a740cf0c0 con 0x7f5a7415e0e0 2023-11-13T10:55:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.619+0000 7f5a78c17640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a7415e0e0 0x7f5a7406c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.619+0000 7f5a78c17640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a7415e0e0 0x7f5a7406c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45998/0 (socket says 172.21.15.16:45998) 2023-11-13T10:55:43.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.619+0000 7f5a79418640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a740719b0 0x7f5a740b2120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56736/0 (socket says 172.21.15.16:56736) 2023-11-13T10:55:43.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.619+0000 7f5a78c17640 1 -- 172.21.15.16:0/3730626967 learned_addr learned my addr 172.21.15.16:0/3730626967 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.620+0000 7f5a78c17640 1 -- 172.21.15.16:0/3730626967 >> v1:172.21.15.184:6789/0 conn(0x7f5a7406c930 legacy=0x7f5a74161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.620+0000 7f5a78c17640 1 -- 172.21.15.16:0/3730626967 >> v1:172.21.15.16:6789/0 conn(0x7f5a740719b0 legacy=0x7f5a740b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.620+0000 7f5a78c17640 1 -- 172.21.15.16:0/3730626967 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7406de60 con 0x7f5a7415e0e0 2023-11-13T10:55:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.620+0000 7f5a6bfff640 1 -- 172.21.15.16:0/3730626967 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140421392 0 0) 0x7f5a7406d910 con 0x7f5a740719b0 2023-11-13T10:55:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.621+0000 7f5a78c17640 1 --2- 172.21.15.16:0/3730626967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7415e0e0 0x7f5a7406c3f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009870 tx=0x7f5a6402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbffab62ec9ea5c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.621+0000 7f5a6bfff640 1 -- 172.21.15.16:0/3730626967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6402f850 con 0x7f5a7415e0e0 2023-11-13T10:55:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.621+0000 7f5a6bfff640 1 -- 172.21.15.16:0/3730626967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6402f9b0 con 0x7f5a7415e0e0 2023-11-13T10:55:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.621+0000 7f5a6bfff640 1 -- 172.21.15.16:0/3730626967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6402fd40 con 0x7f5a7415e0e0 2023-11-13T10:55:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.622+0000 7f5a7b67c640 1 -- 172.21.15.16:0/3730626967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7415e0e0 msgr2=0x7f5a7406c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.622+0000 7f5a7b67c640 1 --2- 172.21.15.16:0/3730626967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7415e0e0 0x7f5a7406c3f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009870 tx=0x7f5a6402ba30 comp rx=0 tx=0).stop 2023-11-13T10:55:43.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.622+0000 7f5a7b67c640 1 -- 172.21.15.16:0/3730626967 shutdown_connections 2023-11-13T10:55:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.622+0000 7f5a7b67c640 1 --2- 172.21.15.16:0/3730626967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7415e0e0 0x7f5a7406c3f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.622+0000 7f5a7b67c640 1 -- 172.21.15.16:0/3730626967 >> 172.21.15.16:0/3730626967 conn(0x7f5a74070890 msgr2=0x7f5a740b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.622+0000 7f5a7b67c640 1 -- 172.21.15.16:0/3730626967 shutdown_connections 2023-11-13T10:55:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.622+0000 7f5a7b67c640 1 -- 172.21.15.16:0/3730626967 wait complete. 2023-11-13T10:55:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.624+0000 7f5a7b67c640 1 Processor -- start 2023-11-13T10:55:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.624+0000 7f5a7b67c640 1 -- start start 2023-11-13T10:55:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.625+0000 7f5a7b67c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a7406c930 0x7f5a74098aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.625+0000 7f5a7b67c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a740719b0 0x7f5a7415f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.625+0000 7f5a7b67c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a7415e0e0 0x7f5a7415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.625+0000 7f5a7b67c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741f2f80 con 0x7f5a740719b0 2023-11-13T10:55:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.625+0000 7f5a7b67c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741f3260 con 0x7f5a7406c930 2023-11-13T10:55:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.625+0000 7f5a7b67c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741f3540 con 0x7f5a7415e0e0 2023-11-13T10:55:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.625+0000 7f5a78c17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a740719b0 0x7f5a7415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.625+0000 7f5a78c17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a740719b0 0x7f5a7415f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46008/0 (socket says 172.21.15.16:46008) 2023-11-13T10:55:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.625+0000 7f5a78c17640 1 -- 172.21.15.16:0/1837994013 learned_addr learned my addr 172.21.15.16:0/1837994013 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a79418640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a7406c930 0x7f5a74098aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a79c19640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a7415e0e0 0x7f5a7415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a78c17640 1 -- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a7415e0e0 msgr2=0x7f5a7415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a78c17640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a7415e0e0 0x7f5a7415faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a78c17640 1 -- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a7406c930 msgr2=0x7f5a74098aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a78c17640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a7406c930 0x7f5a74098aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a78c17640 1 -- 172.21.15.16:0/1837994013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640094d0 con 0x7f5a740719b0 2023-11-13T10:55:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a79c19640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a7415e0e0 0x7f5a7415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:43.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a79418640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a7406c930 0x7f5a74098aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:43.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a78c17640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a740719b0 0x7f5a7415f560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a64000c00 tx=0x7f5a640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:43.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a69ffb640 1 -- 172.21.15.16:0/1837994013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6403b040 con 0x7f5a740719b0 2023-11-13T10:55:43.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a69ffb640 1 -- 172.21.15.16:0/1837994013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a64037d50 con 0x7f5a740719b0 2023-11-13T10:55:43.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a69ffb640 1 -- 172.21.15.16:0/1837994013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64004270 con 0x7f5a740719b0 2023-11-13T10:55:43.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.626+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a741f3820 con 0x7f5a740719b0 2023-11-13T10:55:43.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.627+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a741564a0 con 0x7f5a740719b0 2023-11-13T10:55:43.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.628+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7409d480 con 0x7f5a740719b0 2023-11-13T10:55:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.629+0000 7f5a69ffb640 1 -- 172.21.15.16:0/1837994013 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a640043d0 con 0x7f5a740719b0 2023-11-13T10:55:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.630+0000 7f5a69ffb640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a5806ae40 0x7f5a5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.630+0000 7f5a69ffb640 1 -- 172.21.15.16:0/1837994013 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5a640ac930 con 0x7f5a740719b0 2023-11-13T10:55:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.631+0000 7f5a79418640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a5806ae40 0x7f5a5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.631+0000 7f5a79418640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a5806ae40 0x7f5a5806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5a70000f60 tx=0x7f5a70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.634+0000 7f5a69ffb640 1 -- 172.21.15.16:0/1837994013 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a640776e0 con 0x7f5a740719b0 2023-11-13T10:55:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.828+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a74072cc0 con 0x7f5a740719b0 2023-11-13T10:55:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.829+0000 7f5a69ffb640 1 -- 172.21.15.16:0/1837994013 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a6407ad00 con 0x7f5a740719b0 2023-11-13T10:55:43.834 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:55:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a5806ae40 msgr2=0x7f5a5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a5806ae40 0x7f5a5806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5a70000f60 tx=0x7f5a70009040 comp rx=0 tx=0).stop 2023-11-13T10:55:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a740719b0 msgr2=0x7f5a7415f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a740719b0 0x7f5a7415f560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a64000c00 tx=0x7f5a640046c0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 shutdown_connections 2023-11-13T10:55:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a5806ae40 0x7f5a5806d300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a7415e0e0 0x7f5a7415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a740719b0 0x7f5a7415f560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 --2- 172.21.15.16:0/1837994013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a7406c930 0x7f5a74098aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 >> 172.21.15.16:0/1837994013 conn(0x7f5a74070890 msgr2=0x7f5a7415d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 shutdown_connections 2023-11-13T10:55:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:43.832+0000 7f5a7b67c640 1 -- 172.21.15.16:0/1837994013 wait complete. 2023-11-13T10:55:43.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:55:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.097+0000 7fe34490e640 1 Processor -- start 2023-11-13T10:55:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.097+0000 7fe34490e640 1 -- start start 2023-11-13T10:55:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.099+0000 7fe34490e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe34015b8e0 0x7fe34006c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.099+0000 7fe34490e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe34015bcc0 0x7fe3400cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.099+0000 7fe34490e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3400ce090 0x7fe3400cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.099+0000 7fe34490e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3400ce540 con 0x7fe3400ce090 2023-11-13T10:55:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.099+0000 7fe34490e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34006b860 con 0x7fe34015b8e0 2023-11-13T10:55:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.099+0000 7fe34490e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34006bb40 con 0x7fe34015bcc0 2023-11-13T10:55:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe33f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3400ce090 0x7fe3400cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe33f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3400ce090 0x7fe3400cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46014/0 (socket says 172.21.15.16:46014) 2023-11-13T10:55:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe33f59e640 1 -- 172.21.15.16:0/46285416 learned_addr learned my addr 172.21.15.16:0/46285416 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe33ed9d640 1 --2- 172.21.15.16:0/46285416 >> v2:172.21.15.184:3300/0 conn(0x7fe34015b8e0 0x7fe34006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe337fff640 1 --2- 172.21.15.16:0/46285416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe34015bcc0 0x7fe3400cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe33f59e640 1 -- 172.21.15.16:0/46285416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe34015bcc0 msgr2=0x7fe3400cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe33f59e640 1 --2- 172.21.15.16:0/46285416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe34015bcc0 0x7fe3400cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe33f59e640 1 -- 172.21.15.16:0/46285416 >> v2:172.21.15.184:3300/0 conn(0x7fe34015b8e0 msgr2=0x7fe34006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe33f59e640 1 --2- 172.21.15.16:0/46285416 >> v2:172.21.15.184:3300/0 conn(0x7fe34015b8e0 0x7fe34006c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.100+0000 7fe33f59e640 1 -- 172.21.15.16:0/46285416 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34006be20 con 0x7fe3400ce090 2023-11-13T10:55:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.101+0000 7fe33f59e640 1 --2- 172.21.15.16:0/46285416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400ce090 0x7fe3400cbae0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe338009e50 tx=0x7fe33802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17b02ce0001a0e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:44.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.102+0000 7fe33e59c640 1 -- 172.21.15.16:0/46285416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33802ca80 con 0x7fe3400ce090 2023-11-13T10:55:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.102+0000 7fe33e59c640 1 -- 172.21.15.16:0/46285416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33802cbe0 con 0x7fe3400ce090 2023-11-13T10:55:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.102+0000 7fe33e59c640 1 -- 172.21.15.16:0/46285416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3380343f0 con 0x7fe3400ce090 2023-11-13T10:55:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.102+0000 7fe34490e640 1 -- 172.21.15.16:0/46285416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400ce090 msgr2=0x7fe3400cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.102+0000 7fe34490e640 1 --2- 172.21.15.16:0/46285416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400ce090 0x7fe3400cbae0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe338009e50 tx=0x7fe33802bee0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.103+0000 7fe34490e640 1 -- 172.21.15.16:0/46285416 shutdown_connections 2023-11-13T10:55:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.103+0000 7fe34490e640 1 --2- 172.21.15.16:0/46285416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400ce090 0x7fe3400cbae0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.103+0000 7fe34490e640 1 --2- 172.21.15.16:0/46285416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe34015bcc0 0x7fe3400cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.103+0000 7fe34490e640 1 --2- 172.21.15.16:0/46285416 >> v2:172.21.15.184:3300/0 conn(0x7fe34015b8e0 0x7fe34006c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.103+0000 7fe34490e640 1 -- 172.21.15.16:0/46285416 >> 172.21.15.16:0/46285416 conn(0x7fe3400b0970 msgr2=0x7fe3400b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.103+0000 7fe34490e640 1 -- 172.21.15.16:0/46285416 shutdown_connections 2023-11-13T10:55:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.103+0000 7fe34490e640 1 -- 172.21.15.16:0/46285416 wait complete. 2023-11-13T10:55:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.105+0000 7fe34490e640 1 Processor -- start 2023-11-13T10:55:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.105+0000 7fe34490e640 1 -- start start 2023-11-13T10:55:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.106+0000 7fe34490e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3400ce090 0x7fe3401459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.106+0000 7fe34490e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe34015b8e0 0x7fe3401483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.106+0000 7fe34490e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340148930 0x7fe3400ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.106+0000 7fe34490e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401493f0 con 0x7fe340148930 2023-11-13T10:55:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.106+0000 7fe34490e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe340148e30 con 0x7fe3400ce090 2023-11-13T10:55:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.106+0000 7fe34490e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe340149110 con 0x7fe34015b8e0 2023-11-13T10:55:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.107+0000 7fe33f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340148930 0x7fe3400ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.107+0000 7fe33f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340148930 0x7fe3400ad460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46024/0 (socket says 172.21.15.16:46024) 2023-11-13T10:55:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.107+0000 7fe33f59e640 1 -- 172.21.15.16:0/4110296215 learned_addr learned my addr 172.21.15.16:0/4110296215 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.107+0000 7fe33f59e640 1 -- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe34015b8e0 msgr2=0x7fe3401483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.107+0000 7fe33f59e640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe34015b8e0 0x7fe3401483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.107+0000 7fe33f59e640 1 -- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3400ce090 msgr2=0x7fe3401459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.107+0000 7fe33f59e640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3400ce090 0x7fe3401459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.107+0000 7fe33f59e640 1 -- 172.21.15.16:0/4110296215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe338009b30 con 0x7fe340148930 2023-11-13T10:55:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.107+0000 7fe33f59e640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340148930 0x7fe3400ad460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe338009e50 tx=0x7fe338004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.108+0000 7fe3377fe640 1 -- 172.21.15.16:0/4110296215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338009510 con 0x7fe340148930 2023-11-13T10:55:44.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.108+0000 7fe3377fe640 1 -- 172.21.15.16:0/4110296215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33803e450 con 0x7fe340148930 2023-11-13T10:55:44.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.108+0000 7fe3377fe640 1 -- 172.21.15.16:0/4110296215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33803e800 con 0x7fe340148930 2023-11-13T10:55:44.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.108+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3400a8650 con 0x7fe340148930 2023-11-13T10:55:44.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.108+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe340146390 con 0x7fe340148930 2023-11-13T10:55:44.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.109+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34004e4d0 con 0x7fe340148930 2023-11-13T10:55:44.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.110+0000 7fe3377fe640 1 -- 172.21.15.16:0/4110296215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe33802cd50 con 0x7fe340148930 2023-11-13T10:55:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.111+0000 7fe3377fe640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.111+0000 7fe3377fe640 1 -- 172.21.15.16:0/4110296215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe3380ac3c0 con 0x7fe340148930 2023-11-13T10:55:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.112+0000 7fe33ed9d640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.113+0000 7fe33ed9d640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe330009870 tx=0x7fe330008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.115+0000 7fe3377fe640 1 -- 172.21.15.16:0/4110296215 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe338077180 con 0x7fe340148930 2023-11-13T10:55:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.309+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3400ab1e0 con 0x7fe340148930 2023-11-13T10:55:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.310+0000 7fe3377fe640 1 -- 172.21.15.16:0/4110296215 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe33807a7a0 con 0x7fe340148930 2023-11-13T10:55:44.315 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:55:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 msgr2=0x7fe32006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe330009870 tx=0x7fe330008040 comp rx=0 tx=0).stop 2023-11-13T10:55:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340148930 msgr2=0x7fe3400ad460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340148930 0x7fe3400ad460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe338009e50 tx=0x7fe338004a40 comp rx=0 tx=0).stop 2023-11-13T10:55:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 shutdown_connections 2023-11-13T10:55:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe340148930 0x7fe3400ad460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe34015b8e0 0x7fe3401483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 --2- 172.21.15.16:0/4110296215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3400ce090 0x7fe3401459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.313+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 >> 172.21.15.16:0/4110296215 conn(0x7fe3400b0970 msgr2=0x7fe34015d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.314+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 shutdown_connections 2023-11-13T10:55:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.314+0000 7fe34490e640 1 -- 172.21.15.16:0/4110296215 wait complete. 2023-11-13T10:55:44.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:55:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.559+0000 7fa0b7fff640 1 Processor -- start 2023-11-13T10:55:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.559+0000 7fa0b7fff640 1 -- start start 2023-11-13T10:55:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.561+0000 7fa0b7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0b815b8e0 0x7fa0b806c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.561+0000 7fa0b7fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa0b815bcc0 0x7fa0b80cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.561+0000 7fa0b7fff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b806b860 con 0x7fa0b80ce090 2023-11-13T10:55:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.561+0000 7fa0b7fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b80ce540 con 0x7fa0b815b8e0 2023-11-13T10:55:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.561+0000 7fa0b7fff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b806bb40 con 0x7fa0b815bcc0 2023-11-13T10:55:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.561+0000 7fa0b6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0b815b8e0 0x7fa0b806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.561+0000 7fa0b6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0b815b8e0 0x7fa0b806c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46030/0 (socket says 172.21.15.16:46030) 2023-11-13T10:55:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.561+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/3266073552 learned_addr learned my addr 172.21.15.16:0/3266073552 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.562+0000 7fa0b67fc640 1 --2- 172.21.15.16:0/3266073552 >> v2:172.21.15.184:3300/0 conn(0x7fa0b815bcc0 0x7fa0b80cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.562+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/3266073552 >> v2:172.21.15.184:3300/0 conn(0x7fa0b815bcc0 msgr2=0x7fa0b80cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.562+0000 7fa0b6ffd640 1 --2- 172.21.15.16:0/3266073552 >> v2:172.21.15.184:3300/0 conn(0x7fa0b815bcc0 0x7fa0b80cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.562+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/3266073552 >> v1:172.21.15.184:6789/0 conn(0x7fa0b80ce090 legacy=0x7fa0b80cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.562+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/3266073552 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b806be20 con 0x7fa0b815b8e0 2023-11-13T10:55:44.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.563+0000 7fa0b6ffd640 1 --2- 172.21.15.16:0/3266073552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 0x7fa0b806c900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8071d20 tx=0x7fa0a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6e685d23546d7b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:44.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.563+0000 7fa0b5ffb640 1 -- 172.21.15.16:0/3266073552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8031870 con 0x7fa0b815b8e0 2023-11-13T10:55:44.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.563+0000 7fa0b5ffb640 1 -- 172.21.15.16:0/3266073552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a80319d0 con 0x7fa0b815b8e0 2023-11-13T10:55:44.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.563+0000 7fa0b5ffb640 1 -- 172.21.15.16:0/3266073552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8031d60 con 0x7fa0b815b8e0 2023-11-13T10:55:44.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.564+0000 7fa0b7fff640 1 -- 172.21.15.16:0/3266073552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 msgr2=0x7fa0b806c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.564+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/3266073552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 0x7fa0b806c900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8071d20 tx=0x7fa0a802b910 comp rx=0 tx=0).stop 2023-11-13T10:55:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.564+0000 7fa0b7fff640 1 -- 172.21.15.16:0/3266073552 shutdown_connections 2023-11-13T10:55:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.564+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/3266073552 >> v2:172.21.15.184:3300/0 conn(0x7fa0b815bcc0 0x7fa0b80cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.564+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/3266073552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 0x7fa0b806c900 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.564+0000 7fa0b7fff640 1 -- 172.21.15.16:0/3266073552 >> 172.21.15.16:0/3266073552 conn(0x7fa0b80b0970 msgr2=0x7fa0b80b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.564+0000 7fa0b7fff640 1 -- 172.21.15.16:0/3266073552 shutdown_connections 2023-11-13T10:55:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.564+0000 7fa0b7fff640 1 -- 172.21.15.16:0/3266073552 wait complete. 2023-11-13T10:55:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.566+0000 7fa0b7fff640 1 Processor -- start 2023-11-13T10:55:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.566+0000 7fa0b7fff640 1 -- start start 2023-11-13T10:55:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.567+0000 7fa0b7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b80ce090 0x7fa0b8070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.567+0000 7fa0b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 0x7fa0b8070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.567+0000 7fa0b7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b806fd10 0x7fa0b81e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.567+0000 7fa0b7fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80aedf0 con 0x7fa0b815b8e0 2023-11-13T10:55:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.567+0000 7fa0b7fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80af090 con 0x7fa0b80ce090 2023-11-13T10:55:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.567+0000 7fa0b7fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80af370 con 0x7fa0b806fd10 2023-11-13T10:55:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.567+0000 7fa0b67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 0x7fa0b8070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.567+0000 7fa0b67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 0x7fa0b8070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46036/0 (socket says 172.21.15.16:46036) 2023-11-13T10:55:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.567+0000 7fa0b67fc640 1 -- 172.21.15.16:0/510149302 learned_addr learned my addr 172.21.15.16:0/510149302 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa0b6ffd640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b80ce090 0x7fa0b8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa0b77fe640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b806fd10 0x7fa0b81e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa0b67fc640 1 -- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b806fd10 msgr2=0x7fa0b81e4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa0b67fc640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b806fd10 0x7fa0b81e4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa0b67fc640 1 -- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b80ce090 msgr2=0x7fa0b8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa0b67fc640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b80ce090 0x7fa0b8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa0b67fc640 1 -- 172.21.15.16:0/510149302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a80094d0 con 0x7fa0b815b8e0 2023-11-13T10:55:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa0b77fe640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b806fd10 0x7fa0b81e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa0b67fc640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 0x7fa0b8070890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00d800 tx=0x7fa0ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.568+0000 7fa09f7fe640 1 -- 172.21.15.16:0/510149302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00b6c0 con 0x7fa0b815b8e0 2023-11-13T10:55:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.569+0000 7fa09f7fe640 1 -- 172.21.15.16:0/510149302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac00b820 con 0x7fa0b815b8e0 2023-11-13T10:55:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.569+0000 7fa09f7fe640 1 -- 172.21.15.16:0/510149302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00bbb0 con 0x7fa0b815b8e0 2023-11-13T10:55:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.569+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b80af650 con 0x7fa0b815b8e0 2023-11-13T10:55:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.569+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b814b310 con 0x7fa0b815b8e0 2023-11-13T10:55:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.570+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b804e4d0 con 0x7fa0b815b8e0 2023-11-13T10:55:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.571+0000 7fa09f7fe640 1 -- 172.21.15.16:0/510149302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0ac00bd10 con 0x7fa0b815b8e0 2023-11-13T10:55:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.572+0000 7fa09f7fe640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0a406ae70 0x7fa0a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.572+0000 7fa09f7fe640 1 -- 172.21.15.16:0/510149302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa0ac089e30 con 0x7fa0b815b8e0 2023-11-13T10:55:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.572+0000 7fa0b6ffd640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0a406ae70 0x7fa0a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.573+0000 7fa0b6ffd640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0a406ae70 0x7fa0a406d330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8036000 tx=0x7fa0a803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.576+0000 7fa09f7fe640 1 -- 172.21.15.16:0/510149302 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0ac054bf0 con 0x7fa0b815b8e0 2023-11-13T10:55:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.767+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa0b8070130 con 0x7fa0b815b8e0 2023-11-13T10:55:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.768+0000 7fa09f7fe640 1 -- 172.21.15.16:0/510149302 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa0ac058210 con 0x7fa0b815b8e0 2023-11-13T10:55:44.772 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:55:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0a406ae70 msgr2=0x7fa0a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0a406ae70 0x7fa0a406d330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8036000 tx=0x7fa0a803c040 comp rx=0 tx=0).stop 2023-11-13T10:55:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 msgr2=0x7fa0b8070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 0x7fa0b8070890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00d800 tx=0x7fa0ac00dcc0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 shutdown_connections 2023-11-13T10:55:44.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0a406ae70 0x7fa0a406d330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0b806fd10 0x7fa0b81e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815b8e0 0x7fa0b8070890 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/510149302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0b80ce090 0x7fa0b8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:44.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 >> 172.21.15.16:0/510149302 conn(0x7fa0b80b0970 msgr2=0x7fa0b80cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 shutdown_connections 2023-11-13T10:55:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:44.770+0000 7fa0b7fff640 1 -- 172.21.15.16:0/510149302 wait complete. 2023-11-13T10:55:44.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:55:45.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.013+0000 7f5f0c9c8640 1 Processor -- start 2023-11-13T10:55:45.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.014+0000 7f5f0c9c8640 1 -- start start 2023-11-13T10:55:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5f0c9c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f0815cc90 0x7f5f0806a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5f0c9c8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f08070cf0 0x7f5f0806a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:45.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5f0c9c8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0806fc50 con 0x7f5f08071320 2023-11-13T10:55:45.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5f0c9c8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0806ff30 con 0x7f5f08070cf0 2023-11-13T10:55:45.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5f0c9c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f08070210 con 0x7f5f0815cc90 2023-11-13T10:55:45.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5efffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f08070cf0 0x7f5f0806a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:45.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5f0759e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5f08071320 0x7f5f0806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56752/0 (socket says 172.21.15.16:56752) 2023-11-13T10:55:45.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5f0759e640 1 -- 172.21.15.16:0/3840410486 learned_addr learned my addr 172.21.15.16:0/3840410486 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5f06d9d640 1 --2- 172.21.15.16:0/3840410486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f0815cc90 0x7f5f0806a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5efffff640 1 -- 172.21.15.16:0/3840410486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f0815cc90 msgr2=0x7f5f0806a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5efffff640 1 --2- 172.21.15.16:0/3840410486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f0815cc90 0x7f5f0806a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.016+0000 7f5efffff640 1 -- 172.21.15.16:0/3840410486 >> v1:172.21.15.16:6789/0 conn(0x7f5f08071320 legacy=0x7f5f0806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.017+0000 7f5efffff640 1 -- 172.21.15.16:0/3840410486 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f080704f0 con 0x7f5f08070cf0 2023-11-13T10:55:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.017+0000 7f5f06d9d640 1 --2- 172.21.15.16:0/3840410486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f0815cc90 0x7f5f0806a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.018+0000 7f5efffff640 1 --2- 172.21.15.16:0/3840410486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f08070cf0 0x7f5f0806a6d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00097a0 tx=0x7f5ef002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e145a6a6426cd731 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:45.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.018+0000 7f5f0659c640 1 -- 172.21.15.16:0/3840410486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef002d040 con 0x7f5f08070cf0 2023-11-13T10:55:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.018+0000 7f5f0659c640 1 -- 172.21.15.16:0/3840410486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef0031810 con 0x7f5f08070cf0 2023-11-13T10:55:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.018+0000 7f5f0659c640 1 -- 172.21.15.16:0/3840410486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0031ba0 con 0x7f5f08070cf0 2023-11-13T10:55:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.019+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/3840410486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f08070cf0 msgr2=0x7f5f0806a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.019+0000 7f5f0c9c8640 1 --2- 172.21.15.16:0/3840410486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f08070cf0 0x7f5f0806a6d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00097a0 tx=0x7f5ef002b8b0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.019+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/3840410486 shutdown_connections 2023-11-13T10:55:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.019+0000 7f5f0c9c8640 1 --2- 172.21.15.16:0/3840410486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f08070cf0 0x7f5f0806a6d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.019+0000 7f5f0c9c8640 1 --2- 172.21.15.16:0/3840410486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f0815cc90 0x7f5f0806a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.019+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/3840410486 >> 172.21.15.16:0/3840410486 conn(0x7f5f080b04a0 msgr2=0x7f5f080b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:45.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.019+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/3840410486 shutdown_connections 2023-11-13T10:55:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.019+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/3840410486 wait complete. 2023-11-13T10:55:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.020+0000 7f5f0c9c8640 1 Processor -- start 2023-11-13T10:55:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.021+0000 7f5f0c9c8640 1 -- start start 2023-11-13T10:55:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0c9c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f08070cf0 0x7f5f081f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0c9c8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f08071320 0x7f5f0814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0c9c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f0815cc90 0x7f5f0815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0c9c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0815d1f0 con 0x7f5f0815cc90 2023-11-13T10:55:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0c9c8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081f67d0 con 0x7f5f08070cf0 2023-11-13T10:55:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0c9c8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081f6a70 con 0x7f5f08071320 2023-11-13T10:55:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f0815cc90 0x7f5f0815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f0815cc90 0x7f5f0815d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46058/0 (socket says 172.21.15.16:46058) 2023-11-13T10:55:45.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0759e640 1 -- 172.21.15.16:0/2842314015 learned_addr learned my addr 172.21.15.16:0/2842314015 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:45.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f06d9d640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f08070cf0 0x7f5f081f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5efffff640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f08071320 0x7f5f0814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0759e640 1 -- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f08071320 msgr2=0x7f5f0814b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0759e640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f08071320 0x7f5f0814b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0759e640 1 -- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f08070cf0 msgr2=0x7f5f081f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0759e640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f08070cf0 0x7f5f081f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f0759e640 1 -- 172.21.15.16:0/2842314015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0009400 con 0x7f5f0815cc90 2023-11-13T10:55:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.022+0000 7f5f06d9d640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f08070cf0 0x7f5f081f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.023+0000 7f5f0759e640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f0815cc90 0x7f5f0815d9b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000cbc0 tx=0x7f5f0000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.023+0000 7f5eff7fe640 1 -- 172.21.15.16:0/2842314015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000ad40 con 0x7f5f0815cc90 2023-11-13T10:55:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.023+0000 7f5eff7fe640 1 -- 172.21.15.16:0/2842314015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f00012400 con 0x7f5f0815cc90 2023-11-13T10:55:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.023+0000 7f5eff7fe640 1 -- 172.21.15.16:0/2842314015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000aea0 con 0x7f5f0815cc90 2023-11-13T10:55:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.023+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f081f6d50 con 0x7f5f0815cc90 2023-11-13T10:55:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.023+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0815d490 con 0x7f5f0815cc90 2023-11-13T10:55:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.024+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0815d680 con 0x7f5f0815cc90 2023-11-13T10:55:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.025+0000 7f5eff7fe640 1 -- 172.21.15.16:0/2842314015 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f00007070 con 0x7f5f0815cc90 2023-11-13T10:55:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.026+0000 7f5eff7fe640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5ee806ae40 0x7f5ee806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.026+0000 7f5eff7fe640 1 -- 172.21.15.16:0/2842314015 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5f00089800 con 0x7f5f0815cc90 2023-11-13T10:55:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.027+0000 7f5f06d9d640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5ee806ae40 0x7f5ee806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.028+0000 7f5f06d9d640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5ee806ae40 0x7f5ee806d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8006fd0 tx=0x7f5ef8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.030+0000 7f5eff7fe640 1 -- 172.21.15.16:0/2842314015 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f000545d0 con 0x7f5f0815cc90 2023-11-13T10:55:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.235+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5f0804e4d0 con 0x7f5f0815cc90 2023-11-13T10:55:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.236+0000 7f5eff7fe640 1 -- 172.21.15.16:0/2842314015 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5f00057bf0 con 0x7f5f0815cc90 2023-11-13T10:55:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:55:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5ee806ae40 msgr2=0x7f5ee806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:45.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5ee806ae40 0x7f5ee806d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8006fd0 tx=0x7f5ef8008040 comp rx=0 tx=0).stop 2023-11-13T10:55:45.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f0815cc90 msgr2=0x7f5f0815d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:45.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f0815cc90 0x7f5f0815d9b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000cbc0 tx=0x7f5f0000a430 comp rx=0 tx=0).stop 2023-11-13T10:55:45.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 shutdown_connections 2023-11-13T10:55:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5ee806ae40 0x7f5ee806d300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f0815cc90 0x7f5f0815d9b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f08071320 0x7f5f0814b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 --2- 172.21.15.16:0/2842314015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f08070cf0 0x7f5f081f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:45.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 >> 172.21.15.16:0/2842314015 conn(0x7f5f080b04a0 msgr2=0x7f5f080ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:45.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 shutdown_connections 2023-11-13T10:55:45.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:45.239+0000 7f5f0c9c8640 1 -- 172.21.15.16:0/2842314015 wait complete. 2023-11-13T10:55:45.262 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-13T10:55:45.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:45.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-13T10:55:45.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[92733]: fuse finished with error 0 and tester_r 0 2023-11-13T10:55:45.354 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T10:55:46.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:55:46.577 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:55:51.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T10:55:51.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:55:51.457 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-13T10:55:51.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.771+0000 7f96c2578640 1 Processor -- start 2023-11-13T10:55:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.771+0000 7f96c2578640 1 -- start start 2023-11-13T10:55:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.772+0000 7f96c2578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc15cf20 0x7f96bc06cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.773+0000 7f96c2578640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f96bc0cecc0 0x7f96bc070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.773+0000 7f96c2578640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc0b2900 con 0x7f96bc06bb90 2023-11-13T10:55:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.773+0000 7f96c2578640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc15d300 con 0x7f96bc0cecc0 2023-11-13T10:55:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.773+0000 7f96c2578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0b1fd0 con 0x7f96bc15cf20 2023-11-13T10:55:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.773+0000 7f96bb7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f96bc06bb90 0x7f96bc0ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56768/0 (socket says 172.21.15.16:56768) 2023-11-13T10:55:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.773+0000 7f96bb7fe640 1 -- 172.21.15.16:0/1255494620 learned_addr learned my addr 172.21.15.16:0/1255494620 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.773+0000 7f96bbfff640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc15cf20 0x7f96bc06cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.773+0000 7f96c0b15640 1 --2- 172.21.15.16:0/1255494620 >> v2:172.21.15.184:3300/0 conn(0x7f96bc0cecc0 0x7f96bc070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.773+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088264696 0 0) 0x7f96bc0b2900 con 0x7f96bc06bb90 2023-11-13T10:55:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96a4003520 con 0x7f96bc06bb90 2023-11-13T10:55:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 781699177 0 0) 0x7f96a4003520 con 0x7f96bc06bb90 2023-11-13T10:55:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc15cf20 msgr2=0x7f96bc06cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc15cf20 0x7f96bc06cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 >> v2:172.21.15.184:3300/0 conn(0x7f96bc0cecc0 msgr2=0x7f96bc070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 --2- 172.21.15.16:0/1255494620 >> v2:172.21.15.184:3300/0 conn(0x7f96bc0cecc0 0x7f96bc070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc0b22b0 con 0x7f96bc06bb90 2023-11-13T10:55:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f96b00019f0 con 0x7f96bc06bb90 2023-11-13T10:55:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 >> v1:172.21.15.16:6789/0 conn(0x7f96bc06bb90 legacy=0x7f96bc0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a4003970 0x7f96a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a40047c0 0x7f96a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc15cf20 0x7f96a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0b1fd0 con 0x7f96a40047c0 2023-11-13T10:55:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a400b980 con 0x7f96a4003970 2023-11-13T10:55:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a400bac0 con 0x7f96bc15cf20 2023-11-13T10:55:51.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.774+0000 7f96bb7fe640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a40047c0 0x7f96a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:51.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.775+0000 7f96bbfff640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a4003970 0x7f96a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.775+0000 7f96c0b15640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc15cf20 0x7f96a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.775+0000 7f96bb7fe640 1 -- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc15cf20 msgr2=0x7f96a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.775+0000 7f96bb7fe640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc15cf20 0x7f96a4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.775+0000 7f96bb7fe640 1 -- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a4003970 msgr2=0x7f96a4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.775+0000 7f96bb7fe640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a4003970 0x7f96a4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.775+0000 7f96bb7fe640 1 -- 172.21.15.16:0/1255494620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a4003520 con 0x7f96a40047c0 2023-11-13T10:55:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.775+0000 7f96c0b15640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc15cf20 0x7f96a4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.776+0000 7f96bb7fe640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a40047c0 0x7f96a4004c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96b000a790 tx=0x7f96a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcf85c37b7282d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.776+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0034430 con 0x7f96a40047c0 2023-11-13T10:55:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.776+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b002f070 con 0x7f96a40047c0 2023-11-13T10:55:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.776+0000 7f96baffd640 1 -- 172.21.15.16:0/1255494620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b00347f0 con 0x7f96a40047c0 2023-11-13T10:55:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.777+0000 7f96c2578640 1 -- 172.21.15.16:0/1255494620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a40047c0 msgr2=0x7f96a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.777+0000 7f96c2578640 1 --2- 172.21.15.16:0/1255494620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a40047c0 0x7f96a4004c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96b000a790 tx=0x7f96a40057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:51.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.777+0000 7f96c0b15640 1 -- 172.21.15.16:0/1255494620 reap_dead start 2023-11-13T10:55:51.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.777+0000 7f96c2578640 1 -- 172.21.15.16:0/1255494620 shutdown_connections 2023-11-13T10:55:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.777+0000 7f96c2578640 1 -- 172.21.15.16:0/1255494620 >> 172.21.15.16:0/1255494620 conn(0x7f96bc06a190 msgr2=0x7f96bc06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.777+0000 7f96c2578640 1 -- 172.21.15.16:0/1255494620 shutdown_connections 2023-11-13T10:55:51.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.777+0000 7f96c2578640 1 -- 172.21.15.16:0/1255494620 wait complete. 2023-11-13T10:55:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.780+0000 7f96c2578640 1 Processor -- start 2023-11-13T10:55:51.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.780+0000 7f96c2578640 1 -- start start 2023-11-13T10:55:51.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.780+0000 7f96c2578640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a40047c0 0x7f96bc151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:51.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.780+0000 7f96c2578640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc1521c0 0x7f96bc158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:51.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.780+0000 7f96c2578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc1539f0 0x7f96bc153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:51.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.780+0000 7f96c2578640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc159990 con 0x7f96bc1521c0 2023-11-13T10:55:51.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.780+0000 7f96c2578640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc151fd0 con 0x7f96a40047c0 2023-11-13T10:55:51.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.780+0000 7f96c2578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc158d80 con 0x7f96bc1539f0 2023-11-13T10:55:51.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc1521c0 0x7f96bc158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:51.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc1521c0 0x7f96bc158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46080/0 (socket says 172.21.15.16:46080) 2023-11-13T10:55:51.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bb7fe640 1 -- 172.21.15.16:0/602919569 learned_addr learned my addr 172.21.15.16:0/602919569 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:51.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bbfff640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a40047c0 0x7f96bc151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:51.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96c0b15640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc1539f0 0x7f96bc153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:51.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bb7fe640 1 -- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc1539f0 msgr2=0x7f96bc153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:51.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bb7fe640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc1539f0 0x7f96bc153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:51.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bb7fe640 1 -- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a40047c0 msgr2=0x7f96bc151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:51.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bb7fe640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a40047c0 0x7f96bc151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:51.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bb7fe640 1 -- 172.21.15.16:0/602919569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc159060 con 0x7f96bc1521c0 2023-11-13T10:55:51.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.781+0000 7f96bbfff640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a40047c0 0x7f96bc151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:51.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.782+0000 7f96bb7fe640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc1521c0 0x7f96bc158430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f96b002cbf0 tx=0x7f96b0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:51.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.782+0000 7f96b8ff9640 1 -- 172.21.15.16:0/602919569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0044030 con 0x7f96bc1521c0 2023-11-13T10:55:51.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.782+0000 7f96b8ff9640 1 -- 172.21.15.16:0/602919569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b0004b60 con 0x7f96bc1521c0 2023-11-13T10:55:51.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.782+0000 7f96b8ff9640 1 -- 172.21.15.16:0/602919569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0034430 con 0x7f96bc1521c0 2023-11-13T10:55:51.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.782+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc159340 con 0x7f96bc1521c0 2023-11-13T10:55:51.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.782+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc158970 con 0x7f96bc1521c0 2023-11-13T10:55:51.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.783+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc067090 con 0x7f96bc1521c0 2023-11-13T10:55:51.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.784+0000 7f96b8ff9640 1 -- 172.21.15.16:0/602919569 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f96b0047070 con 0x7f96bc1521c0 2023-11-13T10:55:51.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.785+0000 7f96b8ff9640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f96a806ae40 0x7f96a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:51.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.785+0000 7f96b8ff9640 1 -- 172.21.15.16:0/602919569 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f96b0045070 con 0x7f96bc1521c0 2023-11-13T10:55:51.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.786+0000 7f96bbfff640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f96a806ae40 0x7f96a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:51.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.787+0000 7f96bbfff640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f96a806ae40 0x7f96a806d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0099f0 tx=0x7f96ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:51.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.790+0000 7f96b8ff9640 1 -- 172.21.15.16:0/602919569 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96b00769f0 con 0x7f96bc1521c0 2023-11-13T10:55:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.990+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96bc067090 con 0x7f96bc1521c0 2023-11-13T10:55:51.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.992+0000 7f96b8ff9640 1 -- 172.21.15.16:0/602919569 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f96b007a010 con 0x7f96bc1521c0 2023-11-13T10:55:51.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:51.999 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.995+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f96a806ae40 msgr2=0x7f96a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.995+0000 7f96c2578640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f96a806ae40 0x7f96a806d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0099f0 tx=0x7f96ac008040 comp rx=0 tx=0).stop 2023-11-13T10:55:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.995+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc1521c0 msgr2=0x7f96bc158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.995+0000 7f96c2578640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc1521c0 0x7f96bc158430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f96b002cbf0 tx=0x7f96b0001cb0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.995+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 shutdown_connections 2023-11-13T10:55:52.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.995+0000 7f96c2578640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f96a806ae40 0x7f96a806d300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.995+0000 7f96c2578640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96bc1539f0 0x7f96bc153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.995+0000 7f96c2578640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96bc1521c0 0x7f96bc158430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.995+0000 7f96c2578640 1 --2- 172.21.15.16:0/602919569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96a40047c0 0x7f96bc151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.996+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 >> 172.21.15.16:0/602919569 conn(0x7f96bc06a190 msgr2=0x7f96bc0aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:52.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.996+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 shutdown_connections 2023-11-13T10:55:52.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:51.996+0000 7f96c2578640 1 -- 172.21.15.16:0/602919569 wait complete. 2023-11-13T10:55:52.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:52.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.237+0000 7fe3acd35640 1 Processor -- start 2023-11-13T10:55:52.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.237+0000 7fe3acd35640 1 -- start start 2023-11-13T10:55:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.239+0000 7fe3acd35640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe3a815cfc0 0x7fe3a80cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.239+0000 7fe3acd35640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80b30e0 0x7fe3a80d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.239+0000 7fe3acd35640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80b3cf0 con 0x7fe3a80d2ae0 2023-11-13T10:55:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.239+0000 7fe3acd35640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a80d3480 con 0x7fe3a815cfc0 2023-11-13T10:55:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.239+0000 7fe3acd35640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80b3730 con 0x7fe3a80b30e0 2023-11-13T10:55:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.239+0000 7fe3a7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80b30e0 0x7fe3a80d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.239+0000 7fe3a7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80b30e0 0x7fe3a80d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:60196/0 (socket says 172.21.15.16:60196) 2023-11-13T10:55:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.239+0000 7fe3a7fff640 1 -- 172.21.15.16:0/1445549096 learned_addr learned my addr 172.21.15.16:0/1445549096 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.239+0000 7fe3a77fe640 1 --2- 172.21.15.16:0/1445549096 >> v2:172.21.15.184:3300/0 conn(0x7fe3a815cfc0 0x7fe3a80cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.240+0000 7fe3a7fff640 1 -- 172.21.15.16:0/1445549096 >> v2:172.21.15.184:3300/0 conn(0x7fe3a815cfc0 msgr2=0x7fe3a80cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:52.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.240+0000 7fe3a7fff640 1 --2- 172.21.15.16:0/1445549096 >> v2:172.21.15.184:3300/0 conn(0x7fe3a815cfc0 0x7fe3a80cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.240+0000 7fe3a7fff640 1 -- 172.21.15.16:0/1445549096 >> v1:172.21.15.184:6789/0 conn(0x7fe3a80d2ae0 legacy=0x7fe3a80b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.240+0000 7fe3a7fff640 1 -- 172.21.15.16:0/1445549096 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a80b3a10 con 0x7fe3a80b30e0 2023-11-13T10:55:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.241+0000 7fe3a7fff640 1 --2- 172.21.15.16:0/1445549096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80b30e0 0x7fe3a80d2f40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009870 tx=0x7fe3a002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c5d564536224867 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.242+0000 7fe3a67fc640 1 -- 172.21.15.16:0/1445549096 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0031640 con 0x7fe3a80b30e0 2023-11-13T10:55:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.242+0000 7fe3a67fc640 1 -- 172.21.15.16:0/1445549096 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a00317a0 con 0x7fe3a80b30e0 2023-11-13T10:55:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.242+0000 7fe3a67fc640 1 -- 172.21.15.16:0/1445549096 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0031b30 con 0x7fe3a80b30e0 2023-11-13T10:55:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.243+0000 7fe3acd35640 1 -- 172.21.15.16:0/1445549096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80b30e0 msgr2=0x7fe3a80d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.243+0000 7fe3acd35640 1 --2- 172.21.15.16:0/1445549096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80b30e0 0x7fe3a80d2f40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009870 tx=0x7fe3a002b8f0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.243+0000 7fe3acd35640 1 -- 172.21.15.16:0/1445549096 shutdown_connections 2023-11-13T10:55:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.243+0000 7fe3acd35640 1 --2- 172.21.15.16:0/1445549096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80b30e0 0x7fe3a80d2f40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.243+0000 7fe3acd35640 1 --2- 172.21.15.16:0/1445549096 >> v2:172.21.15.184:3300/0 conn(0x7fe3a815cfc0 0x7fe3a80cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.243+0000 7fe3acd35640 1 -- 172.21.15.16:0/1445549096 >> 172.21.15.16:0/1445549096 conn(0x7fe3a808c570 msgr2=0x7fe3a809be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.243+0000 7fe3acd35640 1 -- 172.21.15.16:0/1445549096 shutdown_connections 2023-11-13T10:55:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.244+0000 7fe3acd35640 1 -- 172.21.15.16:0/1445549096 wait complete. 2023-11-13T10:55:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.245+0000 7fe3acd35640 1 Processor -- start 2023-11-13T10:55:52.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.246+0000 7fe3acd35640 1 -- start start 2023-11-13T10:55:52.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.246+0000 7fe3acd35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80b30e0 0x7fe3a815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3acd35640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80d2ae0 0x7fe3a81570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3acd35640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a815cfc0 0x7fe3a80b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3acd35640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80b6bd0 con 0x7fe3a80b30e0 2023-11-13T10:55:52.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3acd35640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a815e010 con 0x7fe3a815cfc0 2023-11-13T10:55:52.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3acd35640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a815e2f0 con 0x7fe3a80d2ae0 2023-11-13T10:55:52.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80b30e0 0x7fe3a815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80b30e0 0x7fe3a815f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46084/0 (socket says 172.21.15.16:46084) 2023-11-13T10:55:52.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a77fe640 1 -- 172.21.15.16:0/2501825010 learned_addr learned my addr 172.21.15.16:0/2501825010 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:52.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a7fff640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a815cfc0 0x7fe3a80b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a6ffd640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80d2ae0 0x7fe3a81570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a77fe640 1 -- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80d2ae0 msgr2=0x7fe3a81570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a77fe640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80d2ae0 0x7fe3a81570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a77fe640 1 -- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a815cfc0 msgr2=0x7fe3a80b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a77fe640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a815cfc0 0x7fe3a80b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a77fe640 1 -- 172.21.15.16:0/2501825010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00094d0 con 0x7fe3a80b30e0 2023-11-13T10:55:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3a77fe640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80b30e0 0x7fe3a815f0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe39800efc0 tx=0x7fe39800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe39ffff640 1 -- 172.21.15.16:0/2501825010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398009b80 con 0x7fe3a80b30e0 2023-11-13T10:55:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe39ffff640 1 -- 172.21.15.16:0/2501825010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39800f040 con 0x7fe3a80b30e0 2023-11-13T10:55:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe39ffff640 1 -- 172.21.15.16:0/2501825010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980146a0 con 0x7fe3a80b30e0 2023-11-13T10:55:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a815e5d0 con 0x7fe3a80b30e0 2023-11-13T10:55:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.248+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a815eb90 con 0x7fe3a80b30e0 2023-11-13T10:55:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.249+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a804e4d0 con 0x7fe3a80b30e0 2023-11-13T10:55:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.250+0000 7fe39ffff640 1 -- 172.21.15.16:0/2501825010 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe398014800 con 0x7fe3a80b30e0 2023-11-13T10:55:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.251+0000 7fe39ffff640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe38806ae70 0x7fe38806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.251+0000 7fe39ffff640 1 -- 172.21.15.16:0/2501825010 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe39808a780 con 0x7fe3a80b30e0 2023-11-13T10:55:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.252+0000 7fe3a6ffd640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe38806ae70 0x7fe38806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.254+0000 7fe3a6ffd640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe38806ae70 0x7fe38806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe390007fd0 tx=0x7fe390009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:52.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.256+0000 7fe39ffff640 1 -- 172.21.15.16:0/2501825010 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3980554a0 con 0x7fe3a80b30e0 2023-11-13T10:55:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.452+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3a80b8b00 con 0x7fe3a80b30e0 2023-11-13T10:55:52.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.453+0000 7fe39ffff640 1 -- 172.21.15.16:0/2501825010 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7fe398058ac0 con 0x7fe3a80b30e0 2023-11-13T10:55:52.458 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:52.459 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe38806ae70 msgr2=0x7fe38806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe38806ae70 0x7fe38806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe390007fd0 tx=0x7fe390009040 comp rx=0 tx=0).stop 2023-11-13T10:55:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80b30e0 msgr2=0x7fe3a815f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80b30e0 0x7fe3a815f0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe39800efc0 tx=0x7fe39800c490 comp rx=0 tx=0).stop 2023-11-13T10:55:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 shutdown_connections 2023-11-13T10:55:52.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe38806ae70 0x7fe38806d330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3a815cfc0 0x7fe3a80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3a80d2ae0 0x7fe3a81570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 --2- 172.21.15.16:0/2501825010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80b30e0 0x7fe3a815f0b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.456+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 >> 172.21.15.16:0/2501825010 conn(0x7fe3a808c570 msgr2=0x7fe3a80b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:52.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.457+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 shutdown_connections 2023-11-13T10:55:52.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.457+0000 7fe3acd35640 1 -- 172.21.15.16:0/2501825010 wait complete. 2023-11-13T10:55:52.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-13T10:55:52.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.710+0000 7fd11bf16640 1 Processor -- start 2023-11-13T10:55:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.710+0000 7fd11bf16640 1 -- start start 2023-11-13T10:55:52.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.711+0000 7fd11bf16640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1140ce550 0x7fd1140ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.711+0000 7fd11bf16640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1140ceef0 con 0x7fd114144570 2023-11-13T10:55:52.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.711+0000 7fd11bf16640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd114144950 con 0x7fd1140b19d0 2023-11-13T10:55:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.711+0000 7fd11bf16640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1140ce300 con 0x7fd1140ce550 2023-11-13T10:55:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.712+0000 7fd119cb2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd114144570 0x7fd11415f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56774/0 (socket says 172.21.15.16:56774) 2023-11-13T10:55:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.712+0000 7fd119cb2640 1 -- 172.21.15.16:0/1204333931 learned_addr learned my addr 172.21.15.16:0/1204333931 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.712+0000 7fd1194b1640 1 --2- 172.21.15.16:0/1204333931 >> v2:172.21.15.16:3300/0 conn(0x7fd1140ce550 0x7fd1140ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.712+0000 7fd1194b1640 1 -- 172.21.15.16:0/1204333931 >> v1:172.21.15.184:6789/0 conn(0x7fd1140b19d0 legacy=0x7fd1140b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.712+0000 7fd1194b1640 1 -- 172.21.15.16:0/1204333931 >> v1:172.21.15.16:6789/0 conn(0x7fd114144570 legacy=0x7fd11415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.712+0000 7fd1194b1640 1 -- 172.21.15.16:0/1204333931 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1140cd260 con 0x7fd1140ce550 2023-11-13T10:55:52.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.712+0000 7fd118cb0640 1 -- 172.21.15.16:0/1204333931 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850868207 0 0) 0x7fd1140ceef0 con 0x7fd114144570 2023-11-13T10:55:52.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.714+0000 7fd1194b1640 1 --2- 172.21.15.16:0/1204333931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1140ce550 0x7fd1140ce9b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1040097a0 tx=0x7fd10402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb98088fa681160 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:52.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.714+0000 7fd118cb0640 1 -- 172.21.15.16:0/1204333931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104031830 con 0x7fd1140ce550 2023-11-13T10:55:52.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.714+0000 7fd118cb0640 1 -- 172.21.15.16:0/1204333931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd104031990 con 0x7fd1140ce550 2023-11-13T10:55:52.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.714+0000 7fd118cb0640 1 -- 172.21.15.16:0/1204333931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104031d20 con 0x7fd1140ce550 2023-11-13T10:55:52.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.715+0000 7fd11bf16640 1 -- 172.21.15.16:0/1204333931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1140ce550 msgr2=0x7fd1140ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.715+0000 7fd11bf16640 1 --2- 172.21.15.16:0/1204333931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1140ce550 0x7fd1140ce9b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1040097a0 tx=0x7fd10402b8d0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.715+0000 7fd11bf16640 1 -- 172.21.15.16:0/1204333931 shutdown_connections 2023-11-13T10:55:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.715+0000 7fd11bf16640 1 --2- 172.21.15.16:0/1204333931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1140ce550 0x7fd1140ce9b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.715+0000 7fd11bf16640 1 -- 172.21.15.16:0/1204333931 >> 172.21.15.16:0/1204333931 conn(0x7fd11406a190 msgr2=0x7fd11406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.715+0000 7fd11bf16640 1 -- 172.21.15.16:0/1204333931 shutdown_connections 2023-11-13T10:55:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.715+0000 7fd11bf16640 1 -- 172.21.15.16:0/1204333931 wait complete. 2023-11-13T10:55:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.717+0000 7fd11bf16640 1 Processor -- start 2023-11-13T10:55:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.717+0000 7fd11bf16640 1 -- start start 2023-11-13T10:55:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.718+0000 7fd11bf16640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1140b19d0 0x7fd1141391c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.718+0000 7fd11bf16640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1140ce550 0x7fd11415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.718+0000 7fd11bf16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd114144570 0x7fd1141e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.718+0000 7fd11bf16640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11415d5b0 con 0x7fd114144570 2023-11-13T10:55:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.718+0000 7fd11bf16640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11415d9d0 con 0x7fd1140ce550 2023-11-13T10:55:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.718+0000 7fd11bf16640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11415c930 con 0x7fd1140b19d0 2023-11-13T10:55:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.718+0000 7fd11a4b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd114144570 0x7fd1141e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.718+0000 7fd11a4b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd114144570 0x7fd1141e8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46108/0 (socket says 172.21.15.16:46108) 2023-11-13T10:55:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.718+0000 7fd11a4b3640 1 -- 172.21.15.16:0/3595808172 learned_addr learned my addr 172.21.15.16:0/3595808172 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd1194b1640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1140ce550 0x7fd11415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd119cb2640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1140b19d0 0x7fd1141391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd11a4b3640 1 -- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1140b19d0 msgr2=0x7fd1141391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd11a4b3640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1140b19d0 0x7fd1141391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd11a4b3640 1 -- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1140ce550 msgr2=0x7fd11415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd11a4b3640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1140ce550 0x7fd11415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd11a4b3640 1 -- 172.21.15.16:0/3595808172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd104009400 con 0x7fd114144570 2023-11-13T10:55:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd11a4b3640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd114144570 0x7fd1141e8c90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00c410 tx=0x7fd10c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:52.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd10a7fc640 1 -- 172.21.15.16:0/3595808172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c011790 con 0x7fd114144570 2023-11-13T10:55:52.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd10a7fc640 1 -- 172.21.15.16:0/3595808172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10c0118f0 con 0x7fd114144570 2023-11-13T10:55:52.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd10a7fc640 1 -- 172.21.15.16:0/3595808172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c011ce0 con 0x7fd114144570 2023-11-13T10:55:52.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.719+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11415cc10 con 0x7fd114144570 2023-11-13T10:55:52.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.720+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11415d1d0 con 0x7fd114144570 2023-11-13T10:55:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.720+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11415d6f0 con 0x7fd114144570 2023-11-13T10:55:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.721+0000 7fd10a7fc640 1 -- 172.21.15.16:0/3595808172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd10c01c460 con 0x7fd114144570 2023-11-13T10:55:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.723+0000 7fd10a7fc640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd0f806ae40 0x7fd0f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.723+0000 7fd10a7fc640 1 -- 172.21.15.16:0/3595808172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd10c0895b0 con 0x7fd114144570 2023-11-13T10:55:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.724+0000 7fd119cb2640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd0f806ae40 0x7fd0f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.725+0000 7fd119cb2640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd0f806ae40 0x7fd0f806d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd110007ed0 tx=0x7fd110007e60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.728+0000 7fd10a7fc640 1 -- 172.21.15.16:0/3595808172 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd11415d6f0 con 0x7fd114144570 2023-11-13T10:55:52.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:52.966+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd11415d6f0 con 0x7fd114144570 2023-11-13T10:55:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.063+0000 7fd10a7fc640 1 -- 172.21.15.16:0/3595808172 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd11415d6f0 con 0x7fd114144570 2023-11-13T10:55:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd0f806ae40 msgr2=0x7fd0f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd0f806ae40 0x7fd0f806d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd110007ed0 tx=0x7fd110007e60 comp rx=0 tx=0).stop 2023-11-13T10:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd114144570 msgr2=0x7fd1141e8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd114144570 0x7fd1141e8c90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00c410 tx=0x7fd10c00c8d0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 shutdown_connections 2023-11-13T10:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd0f806ae40 0x7fd0f806d300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd114144570 0x7fd1141e8c90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1140ce550 0x7fd11415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 --2- 172.21.15.16:0/3595808172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1140b19d0 0x7fd1141391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.066+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 >> 172.21.15.16:0/3595808172 conn(0x7fd11406a190 msgr2=0x7fd1140adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.067+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 shutdown_connections 2023-11-13T10:55:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.067+0000 7fd11bf16640 1 -- 172.21.15.16:0/3595808172 wait complete. 2023-11-13T10:55:53.094 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-13T10:55:53.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-13T10:55:53.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.308+0000 7fe0064a2640 1 Processor -- start 2023-11-13T10:55:53.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.308+0000 7fe0064a2640 1 -- start start 2023-11-13T10:55:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.310+0000 7fe0064a2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0000d1a10 0x7fe0000ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.310+0000 7fe0064a2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe000160890 0x7fe0000b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.310+0000 7fe0064a2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0000cd910 con 0x7fe0001604b0 2023-11-13T10:55:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.310+0000 7fe0064a2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0000cdb40 con 0x7fe000160890 2023-11-13T10:55:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.310+0000 7fe0064a2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000cde20 con 0x7fe0000d1a10 2023-11-13T10:55:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.310+0000 7fe004a3f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe000160890 0x7fe0000b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.310+0000 7fdfff7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0000d1a10 0x7fe0000ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.310+0000 7fe004a3f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe000160890 0x7fe0000b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38658/0 (socket says 172.21.15.16:38658) 2023-11-13T10:55:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.311+0000 7fe004a3f640 1 -- 172.21.15.16:0/355149238 learned_addr learned my addr 172.21.15.16:0/355149238 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.311+0000 7fe004a3f640 1 -- 172.21.15.16:0/355149238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0000d1a10 msgr2=0x7fe0000ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.311+0000 7fe004a3f640 1 --2- 172.21.15.16:0/355149238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0000d1a10 0x7fe0000ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.311+0000 7fe004a3f640 1 -- 172.21.15.16:0/355149238 >> v1:172.21.15.184:6789/0 conn(0x7fe0001604b0 legacy=0x7fe0000d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:53.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.311+0000 7fe004a3f640 1 -- 172.21.15.16:0/355149238 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000ce100 con 0x7fe000160890 2023-11-13T10:55:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.312+0000 7fe004a3f640 1 --2- 172.21.15.16:0/355149238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe000160890 0x7fe0000b2270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009e50 tx=0x7fdff802bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e56fe74195b31aea server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.313+0000 7fdffeffd640 1 -- 172.21.15.16:0/355149238 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff802cb90 con 0x7fe000160890 2023-11-13T10:55:53.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.313+0000 7fdffeffd640 1 -- 172.21.15.16:0/355149238 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff802ccf0 con 0x7fe000160890 2023-11-13T10:55:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.313+0000 7fdffeffd640 1 -- 172.21.15.16:0/355149238 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8034610 con 0x7fe000160890 2023-11-13T10:55:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.313+0000 7fe0064a2640 1 -- 172.21.15.16:0/355149238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe000160890 msgr2=0x7fe0000b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.313+0000 7fe0064a2640 1 --2- 172.21.15.16:0/355149238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe000160890 0x7fe0000b2270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009e50 tx=0x7fdff802bff0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.314+0000 7fe0064a2640 1 -- 172.21.15.16:0/355149238 shutdown_connections 2023-11-13T10:55:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.314+0000 7fe0064a2640 1 --2- 172.21.15.16:0/355149238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe000160890 0x7fe0000b2270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.314+0000 7fe0064a2640 1 --2- 172.21.15.16:0/355149238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0000d1a10 0x7fe0000ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.314+0000 7fe0064a2640 1 -- 172.21.15.16:0/355149238 >> 172.21.15.16:0/355149238 conn(0x7fe0000d2d20 msgr2=0x7fe0000d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.314+0000 7fe0064a2640 1 -- 172.21.15.16:0/355149238 shutdown_connections 2023-11-13T10:55:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.314+0000 7fe0064a2640 1 -- 172.21.15.16:0/355149238 wait complete. 2023-11-13T10:55:53.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.315+0000 7fe0064a2640 1 Processor -- start 2023-11-13T10:55:53.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.316+0000 7fe0064a2640 1 -- start start 2023-11-13T10:55:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fe0064a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0000d1a10 0x7fe00014e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fe0064a2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0001604b0 0x7fe0000b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fe0064a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe000160890 0x7fe0000c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fe0064a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c7840 con 0x7fe0000d1a10 2023-11-13T10:55:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fe0064a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c7bd0 con 0x7fe000160890 2023-11-13T10:55:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fe0064a2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c7280 con 0x7fe0001604b0 2023-11-13T10:55:53.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdffffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0000d1a10 0x7fe00014e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:53.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdffffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0000d1a10 0x7fe00014e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55840/0 (socket says 172.21.15.16:55840) 2023-11-13T10:55:53.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdffffff640 1 -- 172.21.15.16:0/3925247984 learned_addr learned my addr 172.21.15.16:0/3925247984 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:53.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdfff7fe640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0001604b0 0x7fe0000b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:53.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fe004a3f640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe000160890 0x7fe0000c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:53.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdffffff640 1 -- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0001604b0 msgr2=0x7fe0000b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:53.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdffffff640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0001604b0 0x7fe0000b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdffffff640 1 -- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe000160890 msgr2=0x7fe0000c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:53.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdffffff640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe000160890 0x7fe0000c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdffffff640 1 -- 172.21.15.16:0/3925247984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff8009b30 con 0x7fe0000d1a10 2023-11-13T10:55:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.317+0000 7fdfff7fe640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0001604b0 0x7fe0000b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.318+0000 7fdffffff640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0000d1a10 0x7fe00014e020 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c450 tx=0x7fdff000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.318+0000 7fdffcff9640 1 -- 172.21.15.16:0/3925247984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00052d0 con 0x7fe0000d1a10 2023-11-13T10:55:53.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.318+0000 7fdffcff9640 1 -- 172.21.15.16:0/3925247984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff0005430 con 0x7fe0000d1a10 2023-11-13T10:55:53.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.318+0000 7fdffcff9640 1 -- 172.21.15.16:0/3925247984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0012870 con 0x7fe0000d1a10 2023-11-13T10:55:53.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.318+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0000c7560 con 0x7fe0000d1a10 2023-11-13T10:55:53.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.318+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0001f6280 con 0x7fe0000d1a10 2023-11-13T10:55:53.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.319+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0001f6410 con 0x7fe0000d1a10 2023-11-13T10:55:53.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.320+0000 7fdffcff9640 1 -- 172.21.15.16:0/3925247984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdff00129d0 con 0x7fe0000d1a10 2023-11-13T10:55:53.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.321+0000 7fdffcff9640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfec06ae40 0x7fdfec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:53.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.321+0000 7fdffcff9640 1 -- 172.21.15.16:0/3925247984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdff0089410 con 0x7fe0000d1a10 2023-11-13T10:55:53.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.322+0000 7fdfff7fe640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfec06ae40 0x7fdfec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:53.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.323+0000 7fdfff7fe640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfec06ae40 0x7fdfec06d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0000726f0 tx=0x7fdff4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:53.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.325+0000 7fdffcff9640 1 -- 172.21.15.16:0/3925247984 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdff00541c0 con 0x7fe0000d1a10 2023-11-13T10:55:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:53.559+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe0000c7a30 con 0x7fe0000d1a10 2023-11-13T10:55:54.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.067+0000 7fdffcff9640 1 -- 172.21.15.16:0/3925247984 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe0000c7a30 con 0x7fe0000d1a10 2023-11-13T10:55:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.070+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfec06ae40 msgr2=0x7fdfec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.070+0000 7fe0064a2640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfec06ae40 0x7fdfec06d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0000726f0 tx=0x7fdff4008040 comp rx=0 tx=0).stop 2023-11-13T10:55:54.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.070+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0000d1a10 msgr2=0x7fe00014e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.070+0000 7fe0064a2640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0000d1a10 0x7fe00014e020 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c450 tx=0x7fdff000c910 comp rx=0 tx=0).stop 2023-11-13T10:55:54.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.071+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 shutdown_connections 2023-11-13T10:55:54.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.071+0000 7fe0064a2640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfec06ae40 0x7fdfec06d300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.071+0000 7fe0064a2640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe000160890 0x7fe0000c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.071+0000 7fe0064a2640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0001604b0 0x7fe0000b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.071+0000 7fe0064a2640 1 --2- 172.21.15.16:0/3925247984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0000d1a10 0x7fe00014e020 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.071+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 >> 172.21.15.16:0/3925247984 conn(0x7fe0000d2d20 msgr2=0x7fe0000b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:54.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.071+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 shutdown_connections 2023-11-13T10:55:54.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.071+0000 7fe0064a2640 1 -- 172.21.15.16:0/3925247984 wait complete. 2023-11-13T10:55:54.096 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-13T10:55:54.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:55:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.332+0000 7f1766bed640 1 Processor -- start 2023-11-13T10:55:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.332+0000 7f1766bed640 1 -- start start 2023-11-13T10:55:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.334+0000 7f1766bed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1760070290 0x7f17600b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.334+0000 7f1766bed640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17600adc80 0x7f17600b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.334+0000 7f1766bed640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17600b18d0 con 0x7f1760159230 2023-11-13T10:55:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.334+0000 7f1766bed640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17600caf10 con 0x7f1760070290 2023-11-13T10:55:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.334+0000 7f1766bed640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176006cea0 con 0x7f17600adc80 2023-11-13T10:55:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.334+0000 7f17653ea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1760070290 0x7f17600b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.334+0000 7f17653ea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1760070290 0x7f17600b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55842/0 (socket says 172.21.15.16:55842) 2023-11-13T10:55:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.334+0000 7f17653ea640 1 -- 172.21.15.16:0/3626092904 learned_addr learned my addr 172.21.15.16:0/3626092904 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.335+0000 7f17663ec640 1 --2- 172.21.15.16:0/3626092904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17600adc80 0x7f17600b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.335+0000 7f17653ea640 1 -- 172.21.15.16:0/3626092904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17600adc80 msgr2=0x7f17600b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.335+0000 7f17653ea640 1 --2- 172.21.15.16:0/3626092904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17600adc80 0x7f17600b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.335+0000 7f17653ea640 1 -- 172.21.15.16:0/3626092904 >> v1:172.21.15.184:6789/0 conn(0x7f1760159230 legacy=0x7f176006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.335+0000 7f17653ea640 1 -- 172.21.15.16:0/3626092904 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176006d9c0 con 0x7f1760070290 2023-11-13T10:55:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.336+0000 7f17653ea640 1 --2- 172.21.15.16:0/3626092904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 0x7f17600b1d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1750009870 tx=0x7f175002ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4455b0611a07ebbb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.336+0000 7f1764be9640 1 -- 172.21.15.16:0/3626092904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750030820 con 0x7f1760070290 2023-11-13T10:55:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.336+0000 7f1764be9640 1 -- 172.21.15.16:0/3626092904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1750030980 con 0x7f1760070290 2023-11-13T10:55:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.336+0000 7f1764be9640 1 -- 172.21.15.16:0/3626092904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750030d10 con 0x7f1760070290 2023-11-13T10:55:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.337+0000 7f1766bed640 1 -- 172.21.15.16:0/3626092904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 msgr2=0x7f17600b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.337+0000 7f1766bed640 1 --2- 172.21.15.16:0/3626092904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 0x7f17600b1d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1750009870 tx=0x7f175002ca00 comp rx=0 tx=0).stop 2023-11-13T10:55:54.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.337+0000 7f1766bed640 1 -- 172.21.15.16:0/3626092904 shutdown_connections 2023-11-13T10:55:54.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.337+0000 7f1766bed640 1 --2- 172.21.15.16:0/3626092904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17600adc80 0x7f17600b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.337+0000 7f1766bed640 1 --2- 172.21.15.16:0/3626092904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 0x7f17600b1d60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.337+0000 7f1766bed640 1 -- 172.21.15.16:0/3626092904 >> 172.21.15.16:0/3626092904 conn(0x7f17600b04a0 msgr2=0x7f17600b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:54.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.337+0000 7f1766bed640 1 -- 172.21.15.16:0/3626092904 shutdown_connections 2023-11-13T10:55:54.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.337+0000 7f1766bed640 1 -- 172.21.15.16:0/3626092904 wait complete. 2023-11-13T10:55:54.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.339+0000 7f1766bed640 1 Processor -- start 2023-11-13T10:55:54.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.339+0000 7f1766bed640 1 -- start start 2023-11-13T10:55:54.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.340+0000 7f1766bed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 0x7f176014d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.340+0000 7f1766bed640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17600adc80 0x7f176014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.340+0000 7f1766bed640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1760159230 0x7f17600ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.340+0000 7f1766bed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17600cc2f0 con 0x7f1760070290 2023-11-13T10:55:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.340+0000 7f1766bed640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17600cc540 con 0x7f17600adc80 2023-11-13T10:55:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.340+0000 7f1766bed640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17600cc820 con 0x7f1760159230 2023-11-13T10:55:54.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.340+0000 7f1765beb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 0x7f176014d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.340+0000 7f1765beb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 0x7f176014d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55856/0 (socket says 172.21.15.16:55856) 2023-11-13T10:55:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.340+0000 7f1765beb640 1 -- 172.21.15.16:0/2080438838 learned_addr learned my addr 172.21.15.16:0/2080438838 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f17653ea640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17600adc80 0x7f176014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f17663ec640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1760159230 0x7f17600ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f1765beb640 1 -- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1760159230 msgr2=0x7f17600ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f1765beb640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1760159230 0x7f17600ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f1765beb640 1 -- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17600adc80 msgr2=0x7f176014c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f1765beb640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17600adc80 0x7f176014c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f1765beb640 1 -- 172.21.15.16:0/2080438838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17500094d0 con 0x7f1760070290 2023-11-13T10:55:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f17663ec640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1760159230 0x7f17600ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f1765beb640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 0x7f176014d2d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b3f0 tx=0x7f175c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f17567fc640 1 -- 172.21.15.16:0/2080438838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c011830 con 0x7f1760070290 2023-11-13T10:55:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f17567fc640 1 -- 172.21.15.16:0/2080438838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c011990 con 0x7f1760070290 2023-11-13T10:55:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.341+0000 7f17567fc640 1 -- 172.21.15.16:0/2080438838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c011d40 con 0x7f1760070290 2023-11-13T10:55:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.342+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17600ccb00 con 0x7f1760070290 2023-11-13T10:55:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.342+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176014cf10 con 0x7f1760070290 2023-11-13T10:55:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.343+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176014d100 con 0x7f1760070290 2023-11-13T10:55:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.344+0000 7f17567fc640 1 -- 172.21.15.16:0/2080438838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f175c01a400 con 0x7f1760070290 2023-11-13T10:55:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.345+0000 7f17567fc640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f174406ae40 0x7f174406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.345+0000 7f17567fc640 1 -- 172.21.15.16:0/2080438838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f175c0897c0 con 0x7f1760070290 2023-11-13T10:55:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.346+0000 7f17653ea640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f174406ae40 0x7f174406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.347+0000 7f17653ea640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f174406ae40 0x7f174406d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1750037000 tx=0x7f175003a040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:54.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.350+0000 7f17567fc640 1 -- 172.21.15.16:0/2080438838 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f175c0544e0 con 0x7f1760070290 2023-11-13T10:55:54.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.587+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f176004e4d0 con 0x7f1760070290 2023-11-13T10:55:54.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.589+0000 7f17567fc640 1 -- 172.21.15.16:0/2080438838 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f175c057b00 con 0x7f1760070290 2023-11-13T10:55:54.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:54.593 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:53.560913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-13T10:55:54.594 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-11-13T10:55:54.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.591+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f174406ae40 msgr2=0x7f174406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.591+0000 7f1766bed640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f174406ae40 0x7f174406d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1750037000 tx=0x7f175003a040 comp rx=0 tx=0).stop 2023-11-13T10:55:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.591+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 msgr2=0x7f176014d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.591+0000 7f1766bed640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 0x7f176014d2d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b3f0 tx=0x7f175c00b8b0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.592+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 shutdown_connections 2023-11-13T10:55:54.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.592+0000 7f1766bed640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f174406ae40 0x7f174406d300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.592+0000 7f1766bed640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1760159230 0x7f17600ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.592+0000 7f1766bed640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17600adc80 0x7f176014c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.592+0000 7f1766bed640 1 --2- 172.21.15.16:0/2080438838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1760070290 0x7f176014d2d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.592+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 >> 172.21.15.16:0/2080438838 conn(0x7f17600b04a0 msgr2=0x7f1760151540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:54.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.592+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 shutdown_connections 2023-11-13T10:55:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.592+0000 7f1766bed640 1 -- 172.21.15.16:0/2080438838 wait complete. 2023-11-13T10:55:54.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:54.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.843+0000 7fd1cb59e640 1 Processor -- start 2023-11-13T10:55:54.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.843+0000 7fd1cb59e640 1 -- start start 2023-11-13T10:55:54.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.845+0000 7fd1cb59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd1cc161060 0x7fd1cc1621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.845+0000 7fd1cb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc06f970 0x7fd1cc162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.845+0000 7fd1cb59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc162700 con 0x7fd1cc06a190 2023-11-13T10:55:54.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.845+0000 7fd1cb59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b1840 con 0x7fd1cc161060 2023-11-13T10:55:54.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.845+0000 7fd1cb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc071600 con 0x7fd1cc06f970 2023-11-13T10:55:54.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.846+0000 7fd1c9d9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc06f970 0x7fd1cc162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.846+0000 7fd1ca59c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd1cc161060 0x7fd1cc1621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.846+0000 7fd1c9d9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc06f970 0x7fd1cc162900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50224/0 (socket says 172.21.15.16:50224) 2023-11-13T10:55:54.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.846+0000 7fd1c9d9b640 1 -- 172.21.15.16:0/2870277406 learned_addr learned my addr 172.21.15.16:0/2870277406 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:54.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.846+0000 7fd1c9d9b640 1 -- 172.21.15.16:0/2870277406 >> v2:172.21.15.184:3300/0 conn(0x7fd1cc161060 msgr2=0x7fd1cc1621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:54.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.846+0000 7fd1c9d9b640 1 --2- 172.21.15.16:0/2870277406 >> v2:172.21.15.184:3300/0 conn(0x7fd1cc161060 0x7fd1cc1621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.846+0000 7fd1c9d9b640 1 -- 172.21.15.16:0/2870277406 >> v1:172.21.15.184:6789/0 conn(0x7fd1cc06a190 legacy=0x7fd1cc0b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.846+0000 7fd1c9d9b640 1 -- 172.21.15.16:0/2870277406 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0718e0 con 0x7fd1cc06f970 2023-11-13T10:55:54.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.848+0000 7fd1c9d9b640 1 --2- 172.21.15.16:0/2870277406 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc06f970 0x7fd1cc162900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009e50 tx=0x7fd1b402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96855b2b40360304 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:54.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.849+0000 7fd1c959a640 1 -- 172.21.15.16:0/2870277406 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b402cb00 con 0x7fd1cc06f970 2023-11-13T10:55:54.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.849+0000 7fd1c959a640 1 -- 172.21.15.16:0/2870277406 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b402cc60 con 0x7fd1cc06f970 2023-11-13T10:55:54.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.849+0000 7fd1c959a640 1 -- 172.21.15.16:0/2870277406 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40343f0 con 0x7fd1cc06f970 2023-11-13T10:55:54.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.849+0000 7fd1cb59e640 1 -- 172.21.15.16:0/2870277406 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc06f970 msgr2=0x7fd1cc162900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.849+0000 7fd1cb59e640 1 --2- 172.21.15.16:0/2870277406 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc06f970 0x7fd1cc162900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009e50 tx=0x7fd1b402bf60 comp rx=0 tx=0).stop 2023-11-13T10:55:54.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.850+0000 7fd1cb59e640 1 -- 172.21.15.16:0/2870277406 shutdown_connections 2023-11-13T10:55:54.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.850+0000 7fd1cb59e640 1 --2- 172.21.15.16:0/2870277406 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc06f970 0x7fd1cc162900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.850+0000 7fd1cb59e640 1 --2- 172.21.15.16:0/2870277406 >> v2:172.21.15.184:3300/0 conn(0x7fd1cc161060 0x7fd1cc1621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.850+0000 7fd1cb59e640 1 -- 172.21.15.16:0/2870277406 >> 172.21.15.16:0/2870277406 conn(0x7fd1cc0b04a0 msgr2=0x7fd1cc0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:54.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.850+0000 7fd1cb59e640 1 -- 172.21.15.16:0/2870277406 shutdown_connections 2023-11-13T10:55:54.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.850+0000 7fd1cb59e640 1 -- 172.21.15.16:0/2870277406 wait complete. 2023-11-13T10:55:54.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.852+0000 7fd1cb59e640 1 Processor -- start 2023-11-13T10:55:54.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.852+0000 7fd1cb59e640 1 -- start start 2023-11-13T10:55:54.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1cb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc06a190 0x7fd1cc0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1cb59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc06f970 0x7fd1cc06bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1cb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc161060 0x7fd1cc15f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1cb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15e650 con 0x7fd1cc06a190 2023-11-13T10:55:54.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1cb59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc160670 con 0x7fd1cc06f970 2023-11-13T10:55:54.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1cb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15e8f0 con 0x7fd1cc161060 2023-11-13T10:55:54.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc06a190 0x7fd1cc0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc06a190 0x7fd1cc0a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55868/0 (socket says 172.21.15.16:55868) 2023-11-13T10:55:54.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1ca59c640 1 -- 172.21.15.16:0/1896095418 learned_addr learned my addr 172.21.15.16:0/1896095418 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:54.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.853+0000 7fd1cad9d640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc161060 0x7fd1cc15f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1c9d9b640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc06f970 0x7fd1cc06bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1ca59c640 1 -- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc161060 msgr2=0x7fd1cc15f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc161060 0x7fd1cc15f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1ca59c640 1 -- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc06f970 msgr2=0x7fd1cc06bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:54.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc06f970 0x7fd1cc06bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:54.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1ca59c640 1 -- 172.21.15.16:0/1896095418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b4009b30 con 0x7fd1cc06a190 2023-11-13T10:55:54.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1c9d9b640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc06f970 0x7fd1cc06bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1ca59c640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc06a190 0x7fd1cc0a84d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00e9b0 tx=0x7fd1bc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1c2ffd640 1 -- 172.21.15.16:0/1896095418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc009670 con 0x7fd1cc06a190 2023-11-13T10:55:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1c2ffd640 1 -- 172.21.15.16:0/1896095418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc00c9a0 con 0x7fd1cc06a190 2023-11-13T10:55:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1c2ffd640 1 -- 172.21.15.16:0/1896095418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00cd50 con 0x7fd1cc06a190 2023-11-13T10:55:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.854+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc15ebd0 con 0x7fd1cc06a190 2023-11-13T10:55:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.855+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc158660 con 0x7fd1cc06a190 2023-11-13T10:55:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.856+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc04e4d0 con 0x7fd1cc06a190 2023-11-13T10:55:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.857+0000 7fd1c2ffd640 1 -- 172.21.15.16:0/1896095418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1bc013470 con 0x7fd1cc06a190 2023-11-13T10:55:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.858+0000 7fd1c2ffd640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1ac06ae40 0x7fd1ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.858+0000 7fd1c2ffd640 1 -- 172.21.15.16:0/1896095418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd1bc089ac0 con 0x7fd1cc06a190 2023-11-13T10:55:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.858+0000 7fd1c9d9b640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1ac06ae40 0x7fd1ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.859+0000 7fd1c9d9b640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1ac06ae40 0x7fd1ac06d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4035000 tx=0x7fd1b4030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:54.862+0000 7fd1c2ffd640 1 -- 172.21.15.16:0/1896095418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1bc054910 con 0x7fd1cc06a190 2023-11-13T10:55:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.065+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1cc0b33e0 con 0x7fd1cc06a190 2023-11-13T10:55:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.068+0000 7fd1c2ffd640 1 -- 172.21.15.16:0/1896095418 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7fd1bc017090 con 0x7fd1cc06a190 2023-11-13T10:55:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:55.073 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1ac06ae40 msgr2=0x7fd1ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1ac06ae40 0x7fd1ac06d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4035000 tx=0x7fd1b4030000 comp rx=0 tx=0).stop 2023-11-13T10:55:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc06a190 msgr2=0x7fd1cc0a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc06a190 0x7fd1cc0a84d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00e9b0 tx=0x7fd1bc00ee70 comp rx=0 tx=0).stop 2023-11-13T10:55:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 shutdown_connections 2023-11-13T10:55:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1ac06ae40 0x7fd1ac06d300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1cc161060 0x7fd1cc15f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1cc06f970 0x7fd1cc06bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 --2- 172.21.15.16:0/1896095418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1cc06a190 0x7fd1cc0a84d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 >> 172.21.15.16:0/1896095418 conn(0x7fd1cc0b04a0 msgr2=0x7fd1cc162fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 shutdown_connections 2023-11-13T10:55:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.071+0000 7fd1cb59e640 1 -- 172.21.15.16:0/1896095418 wait complete. 2023-11-13T10:55:55.094 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-13T10:55:55.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T10:55:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.319+0000 7f5be644f640 1 Processor -- start 2023-11-13T10:55:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.319+0000 7f5be644f640 1 -- start start 2023-11-13T10:55:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5be644f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5be00ccc60 0x7f5be0071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5be644f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5be0071980 0x7f5be006fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5be644f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00cd110 con 0x7f5be0155410 2023-11-13T10:55:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5be644f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be0071e10 con 0x7f5be00ccc60 2023-11-13T10:55:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5be644f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be015b500 con 0x7f5be0071980 2023-11-13T10:55:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5bd7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5be00ccc60 0x7f5be0071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5bd7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5be00ccc60 0x7f5be0071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55882/0 (socket says 172.21.15.16:55882) 2023-11-13T10:55:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5bd7fff640 1 -- 172.21.15.16:0/1112001393 learned_addr learned my addr 172.21.15.16:0/1112001393 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:55.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5be49ec640 1 --2- 172.21.15.16:0/1112001393 >> v2:172.21.15.184:3300/0 conn(0x7f5be0071980 0x7f5be006fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5bd7fff640 1 -- 172.21.15.16:0/1112001393 >> v2:172.21.15.184:3300/0 conn(0x7f5be0071980 msgr2=0x7f5be006fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:55.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5bd7fff640 1 --2- 172.21.15.16:0/1112001393 >> v2:172.21.15.184:3300/0 conn(0x7f5be0071980 0x7f5be006fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5bd7fff640 1 -- 172.21.15.16:0/1112001393 >> v1:172.21.15.184:6789/0 conn(0x7f5be0155410 legacy=0x7f5be00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.321+0000 7f5bd7fff640 1 -- 172.21.15.16:0/1112001393 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be015b7e0 con 0x7f5be00ccc60 2023-11-13T10:55:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.323+0000 7f5bd7fff640 1 --2- 172.21.15.16:0/1112001393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be00ccc60 0x7f5be0071440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5be015e700 tx=0x7f5bc802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a7557f8323781 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.323+0000 7f5bdf7fe640 1 -- 172.21.15.16:0/1112001393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8031870 con 0x7f5be00ccc60 2023-11-13T10:55:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.323+0000 7f5bdf7fe640 1 -- 172.21.15.16:0/1112001393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc80319d0 con 0x7f5be00ccc60 2023-11-13T10:55:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.323+0000 7f5bdf7fe640 1 -- 172.21.15.16:0/1112001393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8031d60 con 0x7f5be00ccc60 2023-11-13T10:55:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.324+0000 7f5be644f640 1 -- 172.21.15.16:0/1112001393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be00ccc60 msgr2=0x7f5be0071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.324+0000 7f5be644f640 1 --2- 172.21.15.16:0/1112001393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be00ccc60 0x7f5be0071440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5be015e700 tx=0x7f5bc802b910 comp rx=0 tx=0).stop 2023-11-13T10:55:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.324+0000 7f5be644f640 1 -- 172.21.15.16:0/1112001393 shutdown_connections 2023-11-13T10:55:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.324+0000 7f5be644f640 1 --2- 172.21.15.16:0/1112001393 >> v2:172.21.15.184:3300/0 conn(0x7f5be0071980 0x7f5be006fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.324+0000 7f5be644f640 1 --2- 172.21.15.16:0/1112001393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be00ccc60 0x7f5be0071440 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.324+0000 7f5be644f640 1 -- 172.21.15.16:0/1112001393 >> 172.21.15.16:0/1112001393 conn(0x7f5be006a190 msgr2=0x7f5be006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.324+0000 7f5be644f640 1 -- 172.21.15.16:0/1112001393 shutdown_connections 2023-11-13T10:55:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.324+0000 7f5be644f640 1 -- 172.21.15.16:0/1112001393 wait complete. 2023-11-13T10:55:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.326+0000 7f5be644f640 1 Processor -- start 2023-11-13T10:55:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.326+0000 7f5be644f640 1 -- start start 2023-11-13T10:55:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.327+0000 7f5be644f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0071980 0x7f5be01f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.327+0000 7f5be644f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be00ccc60 0x7f5be0148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.327+0000 7f5be644f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5be0155410 0x7f5be00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.327+0000 7f5be644f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be006c400 con 0x7f5be0071980 2023-11-13T10:55:55.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.327+0000 7f5be644f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be01f74c0 con 0x7f5be00ccc60 2023-11-13T10:55:55.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.327+0000 7f5be644f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be01f6c20 con 0x7f5be0155410 2023-11-13T10:55:55.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.327+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0071980 0x7f5be01f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.327+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0071980 0x7f5be01f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55894/0 (socket says 172.21.15.16:55894) 2023-11-13T10:55:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.327+0000 7f5bdffff640 1 -- 172.21.15.16:0/1759694316 learned_addr learned my addr 172.21.15.16:0/1759694316 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5be49ec640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5be0155410 0x7f5be00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5bdffff640 1 -- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5be0155410 msgr2=0x7f5be00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5bdffff640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5be0155410 0x7f5be00ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5bdffff640 1 -- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be00ccc60 msgr2=0x7f5be0148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5bdffff640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be00ccc60 0x7f5be0148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5bdffff640 1 -- 172.21.15.16:0/1759694316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80094d0 con 0x7f5be0071980 2023-11-13T10:55:55.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5bdffff640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0071980 0x7f5be01f65d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c440 tx=0x7f5bd000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:55.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5bdd7fa640 1 -- 172.21.15.16:0/1759694316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0005480 con 0x7f5be0071980 2023-11-13T10:55:55.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5bdd7fa640 1 -- 172.21.15.16:0/1759694316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd00035f0 con 0x7f5be0071980 2023-11-13T10:55:55.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5bdd7fa640 1 -- 172.21.15.16:0/1759694316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00039c0 con 0x7f5be0071980 2023-11-13T10:55:55.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.328+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be01f6f00 con 0x7f5be0071980 2023-11-13T10:55:55.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.329+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be00b3820 con 0x7f5be0071980 2023-11-13T10:55:55.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.330+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be0067090 con 0x7f5be0071980 2023-11-13T10:55:55.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.331+0000 7f5bdd7fa640 1 -- 172.21.15.16:0/1759694316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bd0003b20 con 0x7f5be0071980 2023-11-13T10:55:55.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.332+0000 7f5bdd7fa640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5bc006ae70 0x7f5bc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.332+0000 7f5bdd7fa640 1 -- 172.21.15.16:0/1759694316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5bd0088a00 con 0x7f5be0071980 2023-11-13T10:55:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.332+0000 7f5bd7fff640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5bc006ae70 0x7f5bc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.333+0000 7f5bd7fff640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5bc006ae70 0x7f5bc006d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8036000 tx=0x7f5bc8030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.336+0000 7f5bdd7fa640 1 -- 172.21.15.16:0/1759694316 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5bd00537c0 con 0x7f5be0071980 2023-11-13T10:55:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.524+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5be015b1b0 con 0x7f5bc006ae70 2023-11-13T10:55:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.558+0000 7f5bdd7fa640 1 -- 172.21.15.16:0/1759694316 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345077 (secure 0 0 0) 0x7f5bc8031c40 con 0x7f5bc006ae70 2023-11-13T10:55:55.566 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T10:55:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5bc006ae70 msgr2=0x7f5bc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5bc006ae70 0x7f5bc006d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8036000 tx=0x7f5bc8030000 comp rx=0 tx=0).stop 2023-11-13T10:55:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0071980 msgr2=0x7f5be01f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0071980 0x7f5be01f65d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c440 tx=0x7f5bd000c900 comp rx=0 tx=0).stop 2023-11-13T10:55:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 shutdown_connections 2023-11-13T10:55:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5bc006ae70 0x7f5bc006d330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5be0155410 0x7f5be00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5be00ccc60 0x7f5be0148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 --2- 172.21.15.16:0/1759694316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be0071980 0x7f5be01f65d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 >> 172.21.15.16:0/1759694316 conn(0x7f5be006a190 msgr2=0x7f5be00ae450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 shutdown_connections 2023-11-13T10:55:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.564+0000 7f5be644f640 1 -- 172.21.15.16:0/1759694316 wait complete. 2023-11-13T10:55:55.586 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-11-13T10:55:53.977320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232004,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742716,"statfs":{"total":773094113280,"available":772856541184,"internally_reserved":0,"allocated":3416064,"data_stored":2031150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004894"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894617+0000","last_change":"2023-11-13T10:55:25.848955+0000","last_active":"2023-11-13T10:55:30.894617+0000","last_peered":"2023-11-13T10:55:30.894617+0000","last_clean":"2023-11-13T10:55:30.894617+0000","last_became_active":"2023-11-13T10:55:25.848644+0000","last_became_peered":"2023-11-13T10:55:25.848644+0000","last_unstale":"2023-11-13T10:55:30.894617+0000","last_undegraded":"2023-11-13T10:55:30.894617+0000","last_fullsized":"2023-11-13T10:55:30.894617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:40.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911163+0000","last_change":"2023-11-13T10:55:26.862834+0000","last_active":"2023-11-13T10:55:30.911163+0000","last_peered":"2023-11-13T10:55:30.911163+0000","last_clean":"2023-11-13T10:55:30.911163+0000","last_became_active":"2023-11-13T10:55:26.861881+0000","last_became_peered":"2023-11-13T10:55:26.861881+0000","last_unstale":"2023-11-13T10:55:30.911163+0000","last_undegraded":"2023-11-13T10:55:30.911163+0000","last_fullsized":"2023-11-13T10:55:30.911163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:47.501708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906481+0000","last_change":"2023-11-13T10:55:30.929671+0000","last_active":"2023-11-13T10:55:31.906481+0000","last_peered":"2023-11-13T10:55:31.906481+0000","last_clean":"2023-11-13T10:55:31.906481+0000","last_became_active":"2023-11-13T10:55:30.929229+0000","last_became_peered":"2023-11-13T10:55:30.929229+0000","last_unstale":"2023-11-13T10:55:31.906481+0000","last_undegraded":"2023-11-13T10:55:31.906481+0000","last_fullsized":"2023-11-13T10:55:31.906481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:46.695309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911269+0000","last_change":"2023-11-13T10:55:25.855701+0000","last_active":"2023-11-13T10:55:30.911269+0000","last_peered":"2023-11-13T10:55:30.911269+0000","last_clean":"2023-11-13T10:55:30.911269+0000","last_became_active":"2023-11-13T10:55:25.855360+0000","last_became_peered":"2023-11-13T10:55:25.855360+0000","last_unstale":"2023-11-13T10:55:30.911269+0000","last_undegraded":"2023-11-13T10:55:30.911269+0000","last_fullsized":"2023-11-13T10:55:30.911269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:52.352286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912877+0000","last_change":"2023-11-13T10:55:26.848124+0000","last_active":"2023-11-13T10:55:30.912877+0000","last_peered":"2023-11-13T10:55:30.912877+0000","last_clean":"2023-11-13T10:55:30.912877+0000","last_became_active":"2023-11-13T10:55:26.847478+0000","last_became_peered":"2023-11-13T10:55:26.847478+0000","last_unstale":"2023-11-13T10:55:30.912877+0000","last_undegraded":"2023-11-13T10:55:30.912877+0000","last_fullsized":"2023-11-13T10:55:30.912877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:06.034946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.903358+0000","last_change":"2023-11-13T10:55:30.918537+0000","last_active":"2023-11-13T10:55:31.903358+0000","last_peered":"2023-11-13T10:55:31.903358+0000","last_clean":"2023-11-13T10:55:31.903358+0000","last_became_active":"2023-11-13T10:55:30.918092+0000","last_became_peered":"2023-11-13T10:55:30.918092+0000","last_unstale":"2023-11-13T10:55:31.903358+0000","last_undegraded":"2023-11-13T10:55:31.903358+0000","last_fullsized":"2023-11-13T10:55:31.903358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:50:02.751900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898103+0000","last_change":"2023-11-13T10:55:25.846685+0000","last_active":"2023-11-13T10:55:30.898103+0000","last_peered":"2023-11-13T10:55:30.898103+0000","last_clean":"2023-11-13T10:55:30.898103+0000","last_became_active":"2023-11-13T10:55:25.846370+0000","last_became_peered":"2023-11-13T10:55:25.846370+0000","last_unstale":"2023-11-13T10:55:30.898103+0000","last_undegraded":"2023-11-13T10:55:30.898103+0000","last_fullsized":"2023-11-13T10:55:30.898103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896163+0000","last_change":"2023-11-13T10:55:26.867122+0000","last_active":"2023-11-13T10:55:30.896163+0000","last_peered":"2023-11-13T10:55:30.896163+0000","last_clean":"2023-11-13T10:55:30.896163+0000","last_became_active":"2023-11-13T10:55:26.866669+0000","last_became_peered":"2023-11-13T10:55:26.866669+0000","last_unstale":"2023-11-13T10:55:30.896163+0000","last_undegraded":"2023-11-13T10:55:30.896163+0000","last_fullsized":"2023-11-13T10:55:30.896163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:25.074197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904843+0000","last_change":"2023-11-13T10:55:30.918484+0000","last_active":"2023-11-13T10:55:31.904843+0000","last_peered":"2023-11-13T10:55:31.904843+0000","last_clean":"2023-11-13T10:55:31.904843+0000","last_became_active":"2023-11-13T10:55:30.917997+0000","last_became_peered":"2023-11-13T10:55:30.917997+0000","last_unstale":"2023-11-13T10:55:31.904843+0000","last_undegraded":"2023-11-13T10:55:31.904843+0000","last_fullsized":"2023-11-13T10:55:31.904843+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:17.817768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900603+0000","last_change":"2023-11-13T10:55:25.851246+0000","last_active":"2023-11-13T10:55:30.900603+0000","last_peered":"2023-11-13T10:55:30.900603+0000","last_clean":"2023-11-13T10:55:30.900603+0000","last_became_active":"2023-11-13T10:55:25.850865+0000","last_became_peered":"2023-11-13T10:55:25.850865+0000","last_unstale":"2023-11-13T10:55:30.900603+0000","last_undegraded":"2023-11-13T10:55:30.900603+0000","last_fullsized":"2023-11-13T10:55:30.900603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:30.550401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898058+0000","last_change":"2023-11-13T10:55:26.866532+0000","last_active":"2023-11-13T10:55:30.898058+0000","last_peered":"2023-11-13T10:55:30.898058+0000","last_clean":"2023-11-13T10:55:30.898058+0000","last_became_active":"2023-11-13T10:55:26.866199+0000","last_became_peered":"2023-11-13T10:55:26.866199+0000","last_unstale":"2023-11-13T10:55:30.898058+0000","last_undegraded":"2023-11-13T10:55:30.898058+0000","last_fullsized":"2023-11-13T10:55:30.898058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:18.260379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905023+0000","last_change":"2023-11-13T10:55:30.919396+0000","last_active":"2023-11-13T10:55:31.905023+0000","last_peered":"2023-11-13T10:55:31.905023+0000","last_clean":"2023-11-13T10:55:31.905023+0000","last_became_active":"2023-11-13T10:55:30.918942+0000","last_became_peered":"2023-11-13T10:55:30.918942+0000","last_unstale":"2023-11-13T10:55:31.905023+0000","last_undegraded":"2023-11-13T10:55:31.905023+0000","last_fullsized":"2023-11-13T10:55:31.905023+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:51.562560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898635+0000","last_change":"2023-11-13T10:55:26.867788+0000","last_active":"2023-11-13T10:55:30.898635+0000","last_peered":"2023-11-13T10:55:30.898635+0000","last_clean":"2023-11-13T10:55:30.898635+0000","last_became_active":"2023-11-13T10:55:26.865720+0000","last_became_peered":"2023-11-13T10:55:26.865720+0000","last_unstale":"2023-11-13T10:55:30.898635+0000","last_undegraded":"2023-11-13T10:55:30.898635+0000","last_fullsized":"2023-11-13T10:55:30.898635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:04:53.206295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899232+0000","last_change":"2023-11-13T10:55:25.848578+0000","last_active":"2023-11-13T10:55:30.899232+0000","last_peered":"2023-11-13T10:55:30.899232+0000","last_clean":"2023-11-13T10:55:30.899232+0000","last_became_active":"2023-11-13T10:55:25.848218+0000","last_became_peered":"2023-11-13T10:55:25.848218+0000","last_unstale":"2023-11-13T10:55:30.899232+0000","last_undegraded":"2023-11-13T10:55:30.899232+0000","last_fullsized":"2023-11-13T10:55:30.899232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.882851+0000","last_change":"2023-11-13T10:55:30.927208+0000","last_active":"2023-11-13T10:55:31.882851+0000","last_peered":"2023-11-13T10:55:31.882851+0000","last_clean":"2023-11-13T10:55:31.882851+0000","last_became_active":"2023-11-13T10:55:30.926587+0000","last_became_peered":"2023-11-13T10:55:30.926587+0000","last_unstale":"2023-11-13T10:55:31.882851+0000","last_undegraded":"2023-11-13T10:55:31.882851+0000","last_fullsized":"2023-11-13T10:55:31.882851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:58:30.075199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911307+0000","last_change":"2023-11-13T10:55:26.863647+0000","last_active":"2023-11-13T10:55:30.911307+0000","last_peered":"2023-11-13T10:55:30.911307+0000","last_clean":"2023-11-13T10:55:30.911307+0000","last_became_active":"2023-11-13T10:55:26.863171+0000","last_became_peered":"2023-11-13T10:55:26.863171+0000","last_unstale":"2023-11-13T10:55:30.911307+0000","last_undegraded":"2023-11-13T10:55:30.911307+0000","last_fullsized":"2023-11-13T10:55:30.911307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:44:59.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913133+0000","last_change":"2023-11-13T10:55:25.861244+0000","last_active":"2023-11-13T10:55:30.913133+0000","last_peered":"2023-11-13T10:55:30.913133+0000","last_clean":"2023-11-13T10:55:30.913133+0000","last_became_active":"2023-11-13T10:55:25.860949+0000","last_became_peered":"2023-11-13T10:55:25.860949+0000","last_unstale":"2023-11-13T10:55:30.913133+0000","last_undegraded":"2023-11-13T10:55:30.913133+0000","last_fullsized":"2023-11-13T10:55:30.913133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:05.431553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.902438+0000","last_change":"2023-11-13T10:55:30.922731+0000","last_active":"2023-11-13T10:55:31.902438+0000","last_peered":"2023-11-13T10:55:31.902438+0000","last_clean":"2023-11-13T10:55:31.902438+0000","last_became_active":"2023-11-13T10:55:30.922328+0000","last_became_peered":"2023-11-13T10:55:30.922328+0000","last_unstale":"2023-11-13T10:55:31.902438+0000","last_undegraded":"2023-11-13T10:55:31.902438+0000","last_fullsized":"2023-11-13T10:55:31.902438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:57:17.670747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897411+0000","last_change":"2023-11-13T10:55:26.857718+0000","last_active":"2023-11-13T10:55:30.897411+0000","last_peered":"2023-11-13T10:55:30.897411+0000","last_clean":"2023-11-13T10:55:30.897411+0000","last_became_active":"2023-11-13T10:55:26.852435+0000","last_became_peered":"2023-11-13T10:55:26.852435+0000","last_unstale":"2023-11-13T10:55:30.897411+0000","last_undegraded":"2023-11-13T10:55:30.897411+0000","last_fullsized":"2023-11-13T10:55:30.897411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:45.182794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892276+0000","last_change":"2023-11-13T10:55:25.858975+0000","last_active":"2023-11-13T10:55:30.892276+0000","last_peered":"2023-11-13T10:55:30.892276+0000","last_clean":"2023-11-13T10:55:30.892276+0000","last_became_active":"2023-11-13T10:55:25.858642+0000","last_became_peered":"2023-11-13T10:55:25.858642+0000","last_unstale":"2023-11-13T10:55:30.892276+0000","last_undegraded":"2023-11-13T10:55:30.892276+0000","last_fullsized":"2023-11-13T10:55:30.892276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.908730+0000","last_change":"2023-11-13T10:55:30.931892+0000","last_active":"2023-11-13T10:55:31.908730+0000","last_peered":"2023-11-13T10:55:31.908730+0000","last_clean":"2023-11-13T10:55:31.908730+0000","last_became_active":"2023-11-13T10:55:30.931357+0000","last_became_peered":"2023-11-13T10:55:30.931357+0000","last_unstale":"2023-11-13T10:55:31.908730+0000","last_undegraded":"2023-11-13T10:55:31.908730+0000","last_fullsized":"2023-11-13T10:55:31.908730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:07:10.060960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897798+0000","last_change":"2023-11-13T10:55:26.855031+0000","last_active":"2023-11-13T10:55:30.897798+0000","last_peered":"2023-11-13T10:55:30.897798+0000","last_clean":"2023-11-13T10:55:30.897798+0000","last_became_active":"2023-11-13T10:55:26.854761+0000","last_became_peered":"2023-11-13T10:55:26.854761+0000","last_unstale":"2023-11-13T10:55:30.897798+0000","last_undegraded":"2023-11-13T10:55:30.897798+0000","last_fullsized":"2023-11-13T10:55:30.897798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:27.863825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898689+0000","last_change":"2023-11-13T10:55:25.848611+0000","last_active":"2023-11-13T10:55:30.898689+0000","last_peered":"2023-11-13T10:55:30.898689+0000","last_clean":"2023-11-13T10:55:30.898689+0000","last_became_active":"2023-11-13T10:55:25.848277+0000","last_became_peered":"2023-11-13T10:55:25.848277+0000","last_unstale":"2023-11-13T10:55:30.898689+0000","last_undegraded":"2023-11-13T10:55:30.898689+0000","last_fullsized":"2023-11-13T10:55:30.898689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905715+0000","last_change":"2023-11-13T10:55:30.920124+0000","last_active":"2023-11-13T10:55:31.905715+0000","last_peered":"2023-11-13T10:55:31.905715+0000","last_clean":"2023-11-13T10:55:31.905715+0000","last_became_active":"2023-11-13T10:55:30.919731+0000","last_became_peered":"2023-11-13T10:55:30.919731+0000","last_unstale":"2023-11-13T10:55:31.905715+0000","last_undegraded":"2023-11-13T10:55:31.905715+0000","last_fullsized":"2023-11-13T10:55:31.905715+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899898+0000","last_change":"2023-11-13T10:55:26.860831+0000","last_active":"2023-11-13T10:55:30.899898+0000","last_peered":"2023-11-13T10:55:30.899898+0000","last_clean":"2023-11-13T10:55:30.899898+0000","last_became_active":"2023-11-13T10:55:26.860527+0000","last_became_peered":"2023-11-13T10:55:26.860527+0000","last_unstale":"2023-11-13T10:55:30.899898+0000","last_undegraded":"2023-11-13T10:55:30.899898+0000","last_fullsized":"2023-11-13T10:55:30.899898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.774455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899998+0000","last_change":"2023-11-13T10:55:25.846361+0000","last_active":"2023-11-13T10:55:30.899998+0000","last_peered":"2023-11-13T10:55:30.899998+0000","last_clean":"2023-11-13T10:55:30.899998+0000","last_became_active":"2023-11-13T10:55:25.846031+0000","last_became_peered":"2023-11-13T10:55:25.846031+0000","last_unstale":"2023-11-13T10:55:30.899998+0000","last_undegraded":"2023-11-13T10:55:30.899998+0000","last_fullsized":"2023-11-13T10:55:30.899998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:44.868236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904780+0000","last_change":"2023-11-13T10:55:30.910764+0000","last_active":"2023-11-13T10:55:31.904780+0000","last_peered":"2023-11-13T10:55:31.904780+0000","last_clean":"2023-11-13T10:55:31.904780+0000","last_became_active":"2023-11-13T10:55:30.910363+0000","last_became_peered":"2023-11-13T10:55:30.910363+0000","last_unstale":"2023-11-13T10:55:31.904780+0000","last_undegraded":"2023-11-13T10:55:31.904780+0000","last_fullsized":"2023-11-13T10:55:31.904780+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:33.673828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.893835+0000","last_change":"2023-11-13T10:55:26.859968+0000","last_active":"2023-11-13T10:55:30.893835+0000","last_peered":"2023-11-13T10:55:30.893835+0000","last_clean":"2023-11-13T10:55:30.893835+0000","last_became_active":"2023-11-13T10:55:26.859344+0000","last_became_peered":"2023-11-13T10:55:26.859344+0000","last_unstale":"2023-11-13T10:55:30.893835+0000","last_undegraded":"2023-11-13T10:55:30.893835+0000","last_fullsized":"2023-11-13T10:55:30.893835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.107404+0000","last_change":"2023-11-13T10:55:25.857948+0000","last_active":"2023-11-13T10:55:31.107404+0000","last_peered":"2023-11-13T10:55:31.107404+0000","last_clean":"2023-11-13T10:55:31.107404+0000","last_became_active":"2023-11-13T10:55:25.857670+0000","last_became_peered":"2023-11-13T10:55:25.857670+0000","last_unstale":"2023-11-13T10:55:31.107404+0000","last_undegraded":"2023-11-13T10:55:31.107404+0000","last_fullsized":"2023-11-13T10:55:31.107404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:11.016234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883636+0000","last_change":"2023-11-13T10:55:30.918361+0000","last_active":"2023-11-13T10:55:31.883636+0000","last_peered":"2023-11-13T10:55:31.883636+0000","last_clean":"2023-11-13T10:55:31.883636+0000","last_became_active":"2023-11-13T10:55:30.912409+0000","last_became_peered":"2023-11-13T10:55:30.912409+0000","last_unstale":"2023-11-13T10:55:31.883636+0000","last_undegraded":"2023-11-13T10:55:31.883636+0000","last_fullsized":"2023-11-13T10:55:31.883636+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:05.903305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913447+0000","last_change":"2023-11-13T10:55:26.864360+0000","last_active":"2023-11-13T10:55:30.913447+0000","last_peered":"2023-11-13T10:55:30.913447+0000","last_clean":"2023-11-13T10:55:30.913447+0000","last_became_active":"2023-11-13T10:55:26.863969+0000","last_became_peered":"2023-11-13T10:55:26.863969+0000","last_unstale":"2023-11-13T10:55:30.913447+0000","last_undegraded":"2023-11-13T10:55:30.913447+0000","last_fullsized":"2023-11-13T10:55:30.913447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:45.182794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898332+0000","last_change":"2023-11-13T10:55:25.850034+0000","last_active":"2023-11-13T10:55:30.898332+0000","last_peered":"2023-11-13T10:55:30.898332+0000","last_clean":"2023-11-13T10:55:30.898332+0000","last_became_active":"2023-11-13T10:55:25.849670+0000","last_became_peered":"2023-11-13T10:55:25.849670+0000","last_unstale":"2023-11-13T10:55:30.898332+0000","last_undegraded":"2023-11-13T10:55:30.898332+0000","last_fullsized":"2023-11-13T10:55:30.898332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:42.787670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905007+0000","last_change":"2023-11-13T10:55:30.921622+0000","last_active":"2023-11-13T10:55:31.905007+0000","last_peered":"2023-11-13T10:55:31.905007+0000","last_clean":"2023-11-13T10:55:31.905007+0000","last_became_active":"2023-11-13T10:55:30.921092+0000","last_became_peered":"2023-11-13T10:55:30.921092+0000","last_unstale":"2023-11-13T10:55:31.905007+0000","last_undegraded":"2023-11-13T10:55:31.905007+0000","last_fullsized":"2023-11-13T10:55:31.905007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:49.941362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886149+0000","last_change":"2023-11-13T10:55:26.858625+0000","last_active":"2023-11-13T10:55:30.886149+0000","last_peered":"2023-11-13T10:55:30.886149+0000","last_clean":"2023-11-13T10:55:30.886149+0000","last_became_active":"2023-11-13T10:55:26.858318+0000","last_became_peered":"2023-11-13T10:55:26.858318+0000","last_unstale":"2023-11-13T10:55:30.886149+0000","last_undegraded":"2023-11-13T10:55:30.886149+0000","last_fullsized":"2023-11-13T10:55:30.886149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:43.105401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909064+0000","last_change":"2023-11-13T10:55:25.862368+0000","last_active":"2023-11-13T10:55:30.909064+0000","last_peered":"2023-11-13T10:55:30.909064+0000","last_clean":"2023-11-13T10:55:30.909064+0000","last_became_active":"2023-11-13T10:55:25.862060+0000","last_became_peered":"2023-11-13T10:55:25.862060+0000","last_unstale":"2023-11-13T10:55:30.909064+0000","last_undegraded":"2023-11-13T10:55:30.909064+0000","last_fullsized":"2023-11-13T10:55:30.909064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:09.162069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904580+0000","last_change":"2023-11-13T10:55:30.923556+0000","last_active":"2023-11-13T10:55:31.904580+0000","last_peered":"2023-11-13T10:55:31.904580+0000","last_clean":"2023-11-13T10:55:31.904580+0000","last_became_active":"2023-11-13T10:55:30.923070+0000","last_became_peered":"2023-11-13T10:55:30.923070+0000","last_unstale":"2023-11-13T10:55:31.904580+0000","last_undegraded":"2023-11-13T10:55:31.904580+0000","last_fullsized":"2023-11-13T10:55:31.904580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:51.562560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902612+0000","last_change":"2023-11-13T10:55:26.864760+0000","last_active":"2023-11-13T10:55:30.902612+0000","last_peered":"2023-11-13T10:55:30.902612+0000","last_clean":"2023-11-13T10:55:30.902612+0000","last_became_active":"2023-11-13T10:55:26.864419+0000","last_became_peered":"2023-11-13T10:55:26.864419+0000","last_unstale":"2023-11-13T10:55:30.902612+0000","last_undegraded":"2023-11-13T10:55:30.902612+0000","last_fullsized":"2023-11-13T10:55:30.902612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:01.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898908+0000","last_change":"2023-11-13T10:55:25.856907+0000","last_active":"2023-11-13T10:55:30.898908+0000","last_peered":"2023-11-13T10:55:30.898908+0000","last_clean":"2023-11-13T10:55:30.898908+0000","last_became_active":"2023-11-13T10:55:25.856545+0000","last_became_peered":"2023-11-13T10:55:25.856545+0000","last_unstale":"2023-11-13T10:55:30.898908+0000","last_undegraded":"2023-11-13T10:55:30.898908+0000","last_fullsized":"2023-11-13T10:55:30.898908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:28.600702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906072+0000","last_change":"2023-11-13T10:55:30.921637+0000","last_active":"2023-11-13T10:55:31.906072+0000","last_peered":"2023-11-13T10:55:31.906072+0000","last_clean":"2023-11-13T10:55:31.906072+0000","last_became_active":"2023-11-13T10:55:30.921177+0000","last_became_peered":"2023-11-13T10:55:30.921177+0000","last_unstale":"2023-11-13T10:55:31.906072+0000","last_undegraded":"2023-11-13T10:55:31.906072+0000","last_fullsized":"2023-11-13T10:55:31.906072+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903258+0000","last_change":"2023-11-13T10:55:26.851557+0000","last_active":"2023-11-13T10:55:30.903258+0000","last_peered":"2023-11-13T10:55:30.903258+0000","last_clean":"2023-11-13T10:55:30.903258+0000","last_became_active":"2023-11-13T10:55:26.850792+0000","last_became_peered":"2023-11-13T10:55:26.850792+0000","last_unstale":"2023-11-13T10:55:30.903258+0000","last_undegraded":"2023-11-13T10:55:30.903258+0000","last_fullsized":"2023-11-13T10:55:30.903258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911327+0000","last_change":"2023-11-13T10:55:25.855880+0000","last_active":"2023-11-13T10:55:30.911327+0000","last_peered":"2023-11-13T10:55:30.911327+0000","last_clean":"2023-11-13T10:55:30.911327+0000","last_became_active":"2023-11-13T10:55:25.855543+0000","last_became_peered":"2023-11-13T10:55:25.855543+0000","last_unstale":"2023-11-13T10:55:30.911327+0000","last_undegraded":"2023-11-13T10:55:30.911327+0000","last_fullsized":"2023-11-13T10:55:30.911327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:02:28.405745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.907032+0000","last_change":"2023-11-13T10:55:30.925168+0000","last_active":"2023-11-13T10:55:31.907032+0000","last_peered":"2023-11-13T10:55:31.907032+0000","last_clean":"2023-11-13T10:55:31.907032+0000","last_became_active":"2023-11-13T10:55:30.924675+0000","last_became_peered":"2023-11-13T10:55:30.924675+0000","last_unstale":"2023-11-13T10:55:31.907032+0000","last_undegraded":"2023-11-13T10:55:31.907032+0000","last_fullsized":"2023-11-13T10:55:31.907032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:25.376399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911697+0000","last_change":"2023-11-13T10:55:26.863984+0000","last_active":"2023-11-13T10:55:30.911697+0000","last_peered":"2023-11-13T10:55:30.911697+0000","last_clean":"2023-11-13T10:55:30.911697+0000","last_became_active":"2023-11-13T10:55:26.862398+0000","last_became_peered":"2023-11-13T10:55:26.862398+0000","last_unstale":"2023-11-13T10:55:30.911697+0000","last_undegraded":"2023-11-13T10:55:30.911697+0000","last_fullsized":"2023-11-13T10:55:30.911697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898919+0000","last_change":"2023-11-13T10:55:25.857004+0000","last_active":"2023-11-13T10:55:30.898919+0000","last_peered":"2023-11-13T10:55:30.898919+0000","last_clean":"2023-11-13T10:55:30.898919+0000","last_became_active":"2023-11-13T10:55:25.856718+0000","last_became_peered":"2023-11-13T10:55:25.856718+0000","last_unstale":"2023-11-13T10:55:30.898919+0000","last_undegraded":"2023-11-13T10:55:30.898919+0000","last_fullsized":"2023-11-13T10:55:30.898919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:28.167168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888398+0000","last_change":"2023-11-13T10:55:30.923781+0000","last_active":"2023-11-13T10:55:31.888398+0000","last_peered":"2023-11-13T10:55:31.888398+0000","last_clean":"2023-11-13T10:55:31.888398+0000","last_became_active":"2023-11-13T10:55:30.922714+0000","last_became_peered":"2023-11-13T10:55:30.922714+0000","last_unstale":"2023-11-13T10:55:31.888398+0000","last_undegraded":"2023-11-13T10:55:31.888398+0000","last_fullsized":"2023-11-13T10:55:31.888398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:05:31.401669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899007+0000","last_change":"2023-11-13T10:55:26.862991+0000","last_active":"2023-11-13T10:55:30.899007+0000","last_peered":"2023-11-13T10:55:30.899007+0000","last_clean":"2023-11-13T10:55:30.899007+0000","last_became_active":"2023-11-13T10:55:26.861100+0000","last_became_peered":"2023-11-13T10:55:26.861100+0000","last_unstale":"2023-11-13T10:55:30.899007+0000","last_undegraded":"2023-11-13T10:55:30.899007+0000","last_fullsized":"2023-11-13T10:55:30.899007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899324+0000","last_change":"2023-11-13T10:55:25.848422+0000","last_active":"2023-11-13T10:55:30.899324+0000","last_peered":"2023-11-13T10:55:30.899324+0000","last_clean":"2023-11-13T10:55:30.899324+0000","last_became_active":"2023-11-13T10:55:25.847809+0000","last_became_peered":"2023-11-13T10:55:25.847809+0000","last_unstale":"2023-11-13T10:55:30.899324+0000","last_undegraded":"2023-11-13T10:55:30.899324+0000","last_fullsized":"2023-11-13T10:55:30.899324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892816+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T10:55:30.892816+0000","last_peered":"2023-11-13T10:55:30.892816+0000","last_clean":"2023-11-13T10:55:30.892816+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T10:55:30.892816+0000","last_undegraded":"2023-11-13T10:55:30.892816+0000","last_fullsized":"2023-11-13T10:55:30.892816+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883757+0000","last_change":"2023-11-13T10:55:30.907496+0000","last_active":"2023-11-13T10:55:31.883757+0000","last_peered":"2023-11-13T10:55:31.883757+0000","last_clean":"2023-11-13T10:55:31.883757+0000","last_became_active":"2023-11-13T10:55:30.905911+0000","last_became_peered":"2023-11-13T10:55:30.905911+0000","last_unstale":"2023-11-13T10:55:31.883757+0000","last_undegraded":"2023-11-13T10:55:31.883757+0000","last_fullsized":"2023-11-13T10:55:31.883757+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:26:30.660465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898140+0000","last_change":"2023-11-13T10:55:26.865638+0000","last_active":"2023-11-13T10:55:30.898140+0000","last_peered":"2023-11-13T10:55:30.898140+0000","last_clean":"2023-11-13T10:55:30.898140+0000","last_became_active":"2023-11-13T10:55:26.864005+0000","last_became_peered":"2023-11-13T10:55:26.864005+0000","last_unstale":"2023-11-13T10:55:30.898140+0000","last_undegraded":"2023-11-13T10:55:30.898140+0000","last_fullsized":"2023-11-13T10:55:30.898140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:18:03.109671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.914054+0000","last_change":"2023-11-13T10:55:25.860502+0000","last_active":"2023-11-13T10:55:30.914054+0000","last_peered":"2023-11-13T10:55:30.914054+0000","last_clean":"2023-11-13T10:55:30.914054+0000","last_became_active":"2023-11-13T10:55:25.860157+0000","last_became_peered":"2023-11-13T10:55:25.860157+0000","last_unstale":"2023-11-13T10:55:30.914054+0000","last_undegraded":"2023-11-13T10:55:30.914054+0000","last_fullsized":"2023-11-13T10:55:30.914054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:42.093127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905171+0000","last_change":"2023-11-13T10:55:30.909998+0000","last_active":"2023-11-13T10:55:31.905171+0000","last_peered":"2023-11-13T10:55:31.905171+0000","last_clean":"2023-11-13T10:55:31.905171+0000","last_became_active":"2023-11-13T10:55:30.909623+0000","last_became_peered":"2023-11-13T10:55:30.909623+0000","last_unstale":"2023-11-13T10:55:31.905171+0000","last_undegraded":"2023-11-13T10:55:31.905171+0000","last_fullsized":"2023-11-13T10:55:31.905171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:47.860796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900512+0000","last_change":"2023-11-13T10:55:26.867419+0000","last_active":"2023-11-13T10:55:30.900512+0000","last_peered":"2023-11-13T10:55:30.900512+0000","last_clean":"2023-11-13T10:55:30.900512+0000","last_became_active":"2023-11-13T10:55:26.864802+0000","last_became_peered":"2023-11-13T10:55:26.864802+0000","last_unstale":"2023-11-13T10:55:30.900512+0000","last_undegraded":"2023-11-13T10:55:30.900512+0000","last_fullsized":"2023-11-13T10:55:30.900512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909289+0000","last_change":"2023-11-13T10:55:25.852321+0000","last_active":"2023-11-13T10:55:30.909289+0000","last_peered":"2023-11-13T10:55:30.909289+0000","last_clean":"2023-11-13T10:55:30.909289+0000","last_became_active":"2023-11-13T10:55:25.850605+0000","last_became_peered":"2023-11-13T10:55:25.850605+0000","last_unstale":"2023-11-13T10:55:30.909289+0000","last_undegraded":"2023-11-13T10:55:30.909289+0000","last_fullsized":"2023-11-13T10:55:30.909289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.887690+0000","last_change":"2023-11-13T10:55:30.913188+0000","last_active":"2023-11-13T10:55:31.887690+0000","last_peered":"2023-11-13T10:55:31.887690+0000","last_clean":"2023-11-13T10:55:31.887690+0000","last_became_active":"2023-11-13T10:55:30.912748+0000","last_became_peered":"2023-11-13T10:55:30.912748+0000","last_unstale":"2023-11-13T10:55:31.887690+0000","last_undegraded":"2023-11-13T10:55:31.887690+0000","last_fullsized":"2023-11-13T10:55:31.887690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:20.801658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902130+0000","last_change":"2023-11-13T10:55:26.867097+0000","last_active":"2023-11-13T10:55:30.902130+0000","last_peered":"2023-11-13T10:55:30.902130+0000","last_clean":"2023-11-13T10:55:30.902130+0000","last_became_active":"2023-11-13T10:55:26.866770+0000","last_became_peered":"2023-11-13T10:55:26.866770+0000","last_unstale":"2023-11-13T10:55:30.902130+0000","last_undegraded":"2023-11-13T10:55:30.902130+0000","last_fullsized":"2023-11-13T10:55:30.902130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:06.034946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900035+0000","last_change":"2023-11-13T10:55:25.855464+0000","last_active":"2023-11-13T10:55:30.900035+0000","last_peered":"2023-11-13T10:55:30.900035+0000","last_clean":"2023-11-13T10:55:30.900035+0000","last_became_active":"2023-11-13T10:55:25.853955+0000","last_became_peered":"2023-11-13T10:55:25.853955+0000","last_unstale":"2023-11-13T10:55:30.900035+0000","last_undegraded":"2023-11-13T10:55:30.900035+0000","last_fullsized":"2023-11-13T10:55:30.900035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.881810+0000","last_change":"2023-11-13T10:55:30.915984+0000","last_active":"2023-11-13T10:55:31.881810+0000","last_peered":"2023-11-13T10:55:31.881810+0000","last_clean":"2023-11-13T10:55:31.881810+0000","last_became_active":"2023-11-13T10:55:30.915438+0000","last_became_peered":"2023-11-13T10:55:30.915438+0000","last_unstale":"2023-11-13T10:55:31.881810+0000","last_undegraded":"2023-11-13T10:55:31.881810+0000","last_fullsized":"2023-11-13T10:55:31.881810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:06:39.428736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887086+0000","last_change":"2023-11-13T10:55:26.874069+0000","last_active":"2023-11-13T10:55:30.887086+0000","last_peered":"2023-11-13T10:55:30.887086+0000","last_clean":"2023-11-13T10:55:30.887086+0000","last_became_active":"2023-11-13T10:55:26.873537+0000","last_became_peered":"2023-11-13T10:55:26.873537+0000","last_unstale":"2023-11-13T10:55:30.887086+0000","last_undegraded":"2023-11-13T10:55:30.887086+0000","last_fullsized":"2023-11-13T10:55:30.887086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:22.528830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902191+0000","last_change":"2023-11-13T10:55:25.853256+0000","last_active":"2023-11-13T10:55:30.902191+0000","last_peered":"2023-11-13T10:55:30.902191+0000","last_clean":"2023-11-13T10:55:30.902191+0000","last_became_active":"2023-11-13T10:55:25.852947+0000","last_became_peered":"2023-11-13T10:55:25.852947+0000","last_unstale":"2023-11-13T10:55:30.902191+0000","last_undegraded":"2023-11-13T10:55:30.902191+0000","last_fullsized":"2023-11-13T10:55:30.902191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:15.762181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904654+0000","last_change":"2023-11-13T10:55:30.912617+0000","last_active":"2023-11-13T10:55:31.904654+0000","last_peered":"2023-11-13T10:55:31.904654+0000","last_clean":"2023-11-13T10:55:31.904654+0000","last_became_active":"2023-11-13T10:55:30.912019+0000","last_became_peered":"2023-11-13T10:55:30.912019+0000","last_unstale":"2023-11-13T10:55:31.904654+0000","last_undegraded":"2023-11-13T10:55:31.904654+0000","last_fullsized":"2023-11-13T10:55:31.904654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:16.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897711+0000","last_change":"2023-11-13T10:55:26.868126+0000","last_active":"2023-11-13T10:55:30.897711+0000","last_peered":"2023-11-13T10:55:30.897711+0000","last_clean":"2023-11-13T10:55:30.897711+0000","last_became_active":"2023-11-13T10:55:26.867810+0000","last_became_peered":"2023-11-13T10:55:26.867810+0000","last_unstale":"2023-11-13T10:55:30.897711+0000","last_undegraded":"2023-11-13T10:55:30.897711+0000","last_fullsized":"2023-11-13T10:55:30.897711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:16.740806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897637+0000","last_change":"2023-11-13T10:55:25.851131+0000","last_active":"2023-11-13T10:55:30.897637+0000","last_peered":"2023-11-13T10:55:30.897637+0000","last_clean":"2023-11-13T10:55:30.897637+0000","last_became_active":"2023-11-13T10:55:25.850827+0000","last_became_peered":"2023-11-13T10:55:25.850827+0000","last_unstale":"2023-11-13T10:55:30.897637+0000","last_undegraded":"2023-11-13T10:55:30.897637+0000","last_fullsized":"2023-11-13T10:55:30.897637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:05.431553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888952+0000","last_change":"2023-11-13T10:55:30.924096+0000","last_active":"2023-11-13T10:55:31.888952+0000","last_peered":"2023-11-13T10:55:31.888952+0000","last_clean":"2023-11-13T10:55:31.888952+0000","last_became_active":"2023-11-13T10:55:30.923375+0000","last_became_peered":"2023-11-13T10:55:30.923375+0000","last_unstale":"2023-11-13T10:55:31.888952+0000","last_undegraded":"2023-11-13T10:55:31.888952+0000","last_fullsized":"2023-11-13T10:55:31.888952+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:18:07.170524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892700+0000","last_change":"2023-11-13T10:55:26.852339+0000","last_active":"2023-11-13T10:55:30.892700+0000","last_peered":"2023-11-13T10:55:30.892700+0000","last_clean":"2023-11-13T10:55:30.892700+0000","last_became_active":"2023-11-13T10:55:26.852036+0000","last_became_peered":"2023-11-13T10:55:26.852036+0000","last_unstale":"2023-11-13T10:55:30.892700+0000","last_undegraded":"2023-11-13T10:55:30.892700+0000","last_fullsized":"2023-11-13T10:55:30.892700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:41:12.172895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898608+0000","last_change":"2023-11-13T10:55:25.854134+0000","last_active":"2023-11-13T10:55:30.898608+0000","last_peered":"2023-11-13T10:55:30.898608+0000","last_clean":"2023-11-13T10:55:30.898608+0000","last_became_active":"2023-11-13T10:55:25.853195+0000","last_became_peered":"2023-11-13T10:55:25.853195+0000","last_unstale":"2023-11-13T10:55:30.898608+0000","last_undegraded":"2023-11-13T10:55:30.898608+0000","last_fullsized":"2023-11-13T10:55:30.898608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:09.162069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888654+0000","last_change":"2023-11-13T10:55:30.914433+0000","last_active":"2023-11-13T10:55:31.888654+0000","last_peered":"2023-11-13T10:55:31.888654+0000","last_clean":"2023-11-13T10:55:31.888654+0000","last_became_active":"2023-11-13T10:55:30.913947+0000","last_became_peered":"2023-11-13T10:55:30.913947+0000","last_unstale":"2023-11-13T10:55:31.888654+0000","last_undegraded":"2023-11-13T10:55:31.888654+0000","last_fullsized":"2023-11-13T10:55:31.888654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:22.321232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899143+0000","last_change":"2023-11-13T10:55:26.866642+0000","last_active":"2023-11-13T10:55:30.899143+0000","last_peered":"2023-11-13T10:55:30.899143+0000","last_clean":"2023-11-13T10:55:30.899143+0000","last_became_active":"2023-11-13T10:55:26.866294+0000","last_became_peered":"2023-11-13T10:55:26.866294+0000","last_unstale":"2023-11-13T10:55:30.899143+0000","last_undegraded":"2023-11-13T10:55:30.899143+0000","last_fullsized":"2023-11-13T10:55:30.899143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:01.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898826+0000","last_change":"2023-11-13T10:55:25.848632+0000","last_active":"2023-11-13T10:55:30.898826+0000","last_peered":"2023-11-13T10:55:30.898826+0000","last_clean":"2023-11-13T10:55:30.898826+0000","last_became_active":"2023-11-13T10:55:25.848324+0000","last_became_peered":"2023-11-13T10:55:25.848324+0000","last_unstale":"2023-11-13T10:55:30.898826+0000","last_undegraded":"2023-11-13T10:55:30.898826+0000","last_fullsized":"2023-11-13T10:55:30.898826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:30.550401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905683+0000","last_change":"2023-11-13T10:55:30.931468+0000","last_active":"2023-11-13T10:55:31.905683+0000","last_peered":"2023-11-13T10:55:31.905683+0000","last_clean":"2023-11-13T10:55:31.905683+0000","last_became_active":"2023-11-13T10:55:30.930565+0000","last_became_peered":"2023-11-13T10:55:30.930565+0000","last_unstale":"2023-11-13T10:55:31.905683+0000","last_undegraded":"2023-11-13T10:55:31.905683+0000","last_fullsized":"2023-11-13T10:55:31.905683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:47.860796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912956+0000","last_change":"2023-11-13T10:55:26.866047+0000","last_active":"2023-11-13T10:55:30.912956+0000","last_peered":"2023-11-13T10:55:30.912956+0000","last_clean":"2023-11-13T10:55:30.912956+0000","last_became_active":"2023-11-13T10:55:26.865641+0000","last_became_peered":"2023-11-13T10:55:26.865641+0000","last_unstale":"2023-11-13T10:55:30.912956+0000","last_undegraded":"2023-11-13T10:55:30.912956+0000","last_fullsized":"2023-11-13T10:55:30.912956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:12.028508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911415+0000","last_change":"2023-11-13T10:55:25.860552+0000","last_active":"2023-11-13T10:55:30.911415+0000","last_peered":"2023-11-13T10:55:30.911415+0000","last_clean":"2023-11-13T10:55:30.911415+0000","last_became_active":"2023-11-13T10:55:25.859833+0000","last_became_peered":"2023-11-13T10:55:25.859833+0000","last_unstale":"2023-11-13T10:55:30.911415+0000","last_undegraded":"2023-11-13T10:55:30.911415+0000","last_fullsized":"2023-11-13T10:55:30.911415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:03:57.732657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883273+0000","last_change":"2023-11-13T10:55:30.926972+0000","last_active":"2023-11-13T10:55:31.883273+0000","last_peered":"2023-11-13T10:55:31.883273+0000","last_clean":"2023-11-13T10:55:31.883273+0000","last_became_active":"2023-11-13T10:55:30.926026+0000","last_became_peered":"2023-11-13T10:55:30.926026+0000","last_unstale":"2023-11-13T10:55:31.883273+0000","last_undegraded":"2023-11-13T10:55:31.883273+0000","last_fullsized":"2023-11-13T10:55:31.883273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:41:10.341680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897975+0000","last_change":"2023-11-13T10:55:26.860060+0000","last_active":"2023-11-13T10:55:30.897975+0000","last_peered":"2023-11-13T10:55:30.897975+0000","last_clean":"2023-11-13T10:55:30.897975+0000","last_became_active":"2023-11-13T10:55:26.859793+0000","last_became_peered":"2023-11-13T10:55:26.859793+0000","last_unstale":"2023-11-13T10:55:30.897975+0000","last_undegraded":"2023-11-13T10:55:30.897975+0000","last_fullsized":"2023-11-13T10:55:30.897975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:13.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:45.351786+0000","last_change":"2023-11-13T10:55:25.857501+0000","last_active":"2023-11-13T10:55:45.351786+0000","last_peered":"2023-11-13T10:55:45.351786+0000","last_clean":"2023-11-13T10:55:45.351786+0000","last_became_active":"2023-11-13T10:55:25.853910+0000","last_became_peered":"2023-11-13T10:55:25.853910+0000","last_unstale":"2023-11-13T10:55:45.351786+0000","last_undegraded":"2023-11-13T10:55:45.351786+0000","last_fullsized":"2023-11-13T10:55:45.351786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:41:11.160621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883442+0000","last_change":"2023-11-13T10:55:30.921872+0000","last_active":"2023-11-13T10:55:31.883442+0000","last_peered":"2023-11-13T10:55:31.883442+0000","last_clean":"2023-11-13T10:55:31.883442+0000","last_became_active":"2023-11-13T10:55:30.921430+0000","last_became_peered":"2023-11-13T10:55:30.921430+0000","last_unstale":"2023-11-13T10:55:31.883442+0000","last_undegraded":"2023-11-13T10:55:31.883442+0000","last_fullsized":"2023-11-13T10:55:31.883442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:25.376399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899011+0000","last_change":"2023-11-13T10:55:26.870526+0000","last_active":"2023-11-13T10:55:30.899011+0000","last_peered":"2023-11-13T10:55:30.899011+0000","last_clean":"2023-11-13T10:55:30.899011+0000","last_became_active":"2023-11-13T10:55:26.870044+0000","last_became_peered":"2023-11-13T10:55:26.870044+0000","last_unstale":"2023-11-13T10:55:30.899011+0000","last_undegraded":"2023-11-13T10:55:30.899011+0000","last_fullsized":"2023-11-13T10:55:30.899011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:05:27.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.915525+0000","last_change":"2023-11-13T10:55:25.863136+0000","last_active":"2023-11-13T10:55:30.915525+0000","last_peered":"2023-11-13T10:55:30.915525+0000","last_clean":"2023-11-13T10:55:30.915525+0000","last_became_active":"2023-11-13T10:55:25.862731+0000","last_became_peered":"2023-11-13T10:55:25.862731+0000","last_unstale":"2023-11-13T10:55:30.915525+0000","last_undegraded":"2023-11-13T10:55:30.915525+0000","last_fullsized":"2023-11-13T10:55:30.915525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:50.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905039+0000","last_change":"2023-11-13T10:55:30.920930+0000","last_active":"2023-11-13T10:55:31.905039+0000","last_peered":"2023-11-13T10:55:31.905039+0000","last_clean":"2023-11-13T10:55:31.905039+0000","last_became_active":"2023-11-13T10:55:30.920498+0000","last_became_peered":"2023-11-13T10:55:30.920498+0000","last_unstale":"2023-11-13T10:55:31.905039+0000","last_undegraded":"2023-11-13T10:55:31.905039+0000","last_fullsized":"2023-11-13T10:55:31.905039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:49.941362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912962+0000","last_change":"2023-11-13T10:55:26.860086+0000","last_active":"2023-11-13T10:55:30.912962+0000","last_peered":"2023-11-13T10:55:30.912962+0000","last_clean":"2023-11-13T10:55:30.912962+0000","last_became_active":"2023-11-13T10:55:26.859816+0000","last_became_peered":"2023-11-13T10:55:26.859816+0000","last_unstale":"2023-11-13T10:55:30.912962+0000","last_undegraded":"2023-11-13T10:55:30.912962+0000","last_fullsized":"2023-11-13T10:55:30.912962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902678+0000","last_change":"2023-11-13T10:55:25.846097+0000","last_active":"2023-11-13T10:55:30.902678+0000","last_peered":"2023-11-13T10:55:30.902678+0000","last_clean":"2023-11-13T10:55:30.902678+0000","last_became_active":"2023-11-13T10:55:25.845770+0000","last_became_peered":"2023-11-13T10:55:25.845770+0000","last_unstale":"2023-11-13T10:55:30.902678+0000","last_undegraded":"2023-11-13T10:55:30.902678+0000","last_fullsized":"2023-11-13T10:55:30.902678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:16.023464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906043+0000","last_change":"2023-11-13T10:55:30.925552+0000","last_active":"2023-11-13T10:55:31.906043+0000","last_peered":"2023-11-13T10:55:31.906043+0000","last_clean":"2023-11-13T10:55:31.906043+0000","last_became_active":"2023-11-13T10:55:30.925120+0000","last_became_peered":"2023-11-13T10:55:30.925120+0000","last_unstale":"2023-11-13T10:55:31.906043+0000","last_undegraded":"2023-11-13T10:55:31.906043+0000","last_fullsized":"2023-11-13T10:55:31.906043+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:54:40.287372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886993+0000","last_change":"2023-11-13T10:55:26.876001+0000","last_active":"2023-11-13T10:55:30.886993+0000","last_peered":"2023-11-13T10:55:30.886993+0000","last_clean":"2023-11-13T10:55:30.886993+0000","last_became_active":"2023-11-13T10:55:26.875523+0000","last_became_peered":"2023-11-13T10:55:26.875523+0000","last_unstale":"2023-11-13T10:55:30.886993+0000","last_undegraded":"2023-11-13T10:55:30.886993+0000","last_fullsized":"2023-11-13T10:55:30.886993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:17.901362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896671+0000","last_change":"2023-11-13T10:55:25.848415+0000","last_active":"2023-11-13T10:55:30.896671+0000","last_peered":"2023-11-13T10:55:30.896671+0000","last_clean":"2023-11-13T10:55:30.896671+0000","last_became_active":"2023-11-13T10:55:25.848102+0000","last_became_peered":"2023-11-13T10:55:25.848102+0000","last_unstale":"2023-11-13T10:55:30.896671+0000","last_undegraded":"2023-11-13T10:55:30.896671+0000","last_fullsized":"2023-11-13T10:55:30.896671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T10:56:29.442219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883284+0000","last_change":"2023-11-13T10:55:30.918365+0000","last_active":"2023-11-13T10:55:31.883284+0000","last_peered":"2023-11-13T10:55:31.883284+0000","last_clean":"2023-11-13T10:55:31.883284+0000","last_became_active":"2023-11-13T10:55:30.917240+0000","last_became_peered":"2023-11-13T10:55:30.917240+0000","last_unstale":"2023-11-13T10:55:31.883284+0000","last_undegraded":"2023-11-13T10:55:31.883284+0000","last_fullsized":"2023-11-13T10:55:31.883284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884505+0000","last_change":"2023-11-13T10:55:30.924912+0000","last_active":"2023-11-13T10:55:31.884505+0000","last_peered":"2023-11-13T10:55:31.884505+0000","last_clean":"2023-11-13T10:55:31.884505+0000","last_became_active":"2023-11-13T10:55:30.919446+0000","last_became_peered":"2023-11-13T10:55:30.919446+0000","last_unstale":"2023-11-13T10:55:31.884505+0000","last_undegraded":"2023-11-13T10:55:31.884505+0000","last_fullsized":"2023-11-13T10:55:31.884505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:31.924677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892009+0000","last_change":"2023-11-13T10:55:26.852233+0000","last_active":"2023-11-13T10:55:30.892009+0000","last_peered":"2023-11-13T10:55:30.892009+0000","last_clean":"2023-11-13T10:55:30.892009+0000","last_became_active":"2023-11-13T10:55:26.849611+0000","last_became_peered":"2023-11-13T10:55:26.849611+0000","last_unstale":"2023-11-13T10:55:30.892009+0000","last_undegraded":"2023-11-13T10:55:30.892009+0000","last_fullsized":"2023-11-13T10:55:30.892009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:53.364560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898832+0000","last_change":"2023-11-13T10:55:25.856192+0000","last_active":"2023-11-13T10:55:30.898832+0000","last_peered":"2023-11-13T10:55:30.898832+0000","last_clean":"2023-11-13T10:55:30.898832+0000","last_became_active":"2023-11-13T10:55:25.852741+0000","last_became_peered":"2023-11-13T10:55:25.852741+0000","last_unstale":"2023-11-13T10:55:30.898832+0000","last_undegraded":"2023-11-13T10:55:30.898832+0000","last_fullsized":"2023-11-13T10:55:30.898832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:15.762181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884097+0000","last_change":"2023-11-13T10:55:30.910075+0000","last_active":"2023-11-13T10:55:31.884097+0000","last_peered":"2023-11-13T10:55:31.884097+0000","last_clean":"2023-11-13T10:55:31.884097+0000","last_became_active":"2023-11-13T10:55:30.909468+0000","last_became_peered":"2023-11-13T10:55:30.909468+0000","last_unstale":"2023-11-13T10:55:31.884097+0000","last_undegraded":"2023-11-13T10:55:31.884097+0000","last_fullsized":"2023-11-13T10:55:31.884097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:04:21.045625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903339+0000","last_change":"2023-11-13T10:55:26.856127+0000","last_active":"2023-11-13T10:55:30.903339+0000","last_peered":"2023-11-13T10:55:30.903339+0000","last_clean":"2023-11-13T10:55:30.903339+0000","last_became_active":"2023-11-13T10:55:26.855805+0000","last_became_peered":"2023-11-13T10:55:26.855805+0000","last_unstale":"2023-11-13T10:55:30.903339+0000","last_undegraded":"2023-11-13T10:55:30.903339+0000","last_fullsized":"2023-11-13T10:55:30.903339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:13.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911772+0000","last_change":"2023-11-13T10:55:25.859736+0000","last_active":"2023-11-13T10:55:30.911772+0000","last_peered":"2023-11-13T10:55:30.911772+0000","last_clean":"2023-11-13T10:55:30.911772+0000","last_became_active":"2023-11-13T10:55:25.859335+0000","last_became_peered":"2023-11-13T10:55:25.859335+0000","last_unstale":"2023-11-13T10:55:30.911772+0000","last_undegraded":"2023-11-13T10:55:30.911772+0000","last_fullsized":"2023-11-13T10:55:30.911772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:24.061923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898735+0000","last_change":"2023-11-13T10:55:25.847310+0000","last_active":"2023-11-13T10:55:30.898735+0000","last_peered":"2023-11-13T10:55:30.898735+0000","last_clean":"2023-11-13T10:55:30.898735+0000","last_became_active":"2023-11-13T10:55:25.846471+0000","last_became_peered":"2023-11-13T10:55:25.846471+0000","last_unstale":"2023-11-13T10:55:30.898735+0000","last_undegraded":"2023-11-13T10:55:30.898735+0000","last_fullsized":"2023-11-13T10:55:30.898735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:57.678774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894953+0000","last_change":"2023-11-13T10:55:26.865930+0000","last_active":"2023-11-13T10:55:30.894953+0000","last_peered":"2023-11-13T10:55:30.894953+0000","last_clean":"2023-11-13T10:55:30.894953+0000","last_became_active":"2023-11-13T10:55:26.865603+0000","last_became_peered":"2023-11-13T10:55:26.865603+0000","last_unstale":"2023-11-13T10:55:30.894953+0000","last_undegraded":"2023-11-13T10:55:30.894953+0000","last_fullsized":"2023-11-13T10:55:30.894953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:31.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884103+0000","last_change":"2023-11-13T10:55:30.922058+0000","last_active":"2023-11-13T10:55:31.884103+0000","last_peered":"2023-11-13T10:55:31.884103+0000","last_clean":"2023-11-13T10:55:31.884103+0000","last_became_active":"2023-11-13T10:55:30.921389+0000","last_became_peered":"2023-11-13T10:55:30.921389+0000","last_unstale":"2023-11-13T10:55:31.884103+0000","last_undegraded":"2023-11-13T10:55:31.884103+0000","last_fullsized":"2023-11-13T10:55:31.884103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:54.585715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909071+0000","last_change":"2023-11-13T10:55:25.859601+0000","last_active":"2023-11-13T10:55:30.909071+0000","last_peered":"2023-11-13T10:55:30.909071+0000","last_clean":"2023-11-13T10:55:30.909071+0000","last_became_active":"2023-11-13T10:55:25.859320+0000","last_became_peered":"2023-11-13T10:55:25.859320+0000","last_unstale":"2023-11-13T10:55:30.909071+0000","last_undegraded":"2023-11-13T10:55:30.909071+0000","last_fullsized":"2023-11-13T10:55:30.909071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.901962+0000","last_change":"2023-11-13T10:55:26.844138+0000","last_active":"2023-11-13T10:55:30.901962+0000","last_peered":"2023-11-13T10:55:30.901962+0000","last_clean":"2023-11-13T10:55:30.901962+0000","last_became_active":"2023-11-13T10:55:26.843850+0000","last_became_peered":"2023-11-13T10:55:26.843850+0000","last_unstale":"2023-11-13T10:55:30.901962+0000","last_undegraded":"2023-11-13T10:55:30.901962+0000","last_fullsized":"2023-11-13T10:55:30.901962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:43.105401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.907248+0000","last_change":"2023-11-13T10:55:30.929718+0000","last_active":"2023-11-13T10:55:31.907248+0000","last_peered":"2023-11-13T10:55:31.907248+0000","last_clean":"2023-11-13T10:55:31.907248+0000","last_became_active":"2023-11-13T10:55:30.929223+0000","last_became_peered":"2023-11-13T10:55:30.929223+0000","last_unstale":"2023-11-13T10:55:31.907248+0000","last_undegraded":"2023-11-13T10:55:31.907248+0000","last_fullsized":"2023-11-13T10:55:31.907248+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:28:22.239712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886249+0000","last_change":"2023-11-13T10:55:26.858768+0000","last_active":"2023-11-13T10:55:30.886249+0000","last_peered":"2023-11-13T10:55:30.886249+0000","last_clean":"2023-11-13T10:55:30.886249+0000","last_became_active":"2023-11-13T10:55:26.858493+0000","last_became_peered":"2023-11-13T10:55:26.858493+0000","last_unstale":"2023-11-13T10:55:30.886249+0000","last_undegraded":"2023-11-13T10:55:30.886249+0000","last_fullsized":"2023-11-13T10:55:30.886249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:25.074197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911186+0000","last_change":"2023-11-13T10:55:26.866587+0000","last_active":"2023-11-13T10:55:30.911186+0000","last_peered":"2023-11-13T10:55:30.911186+0000","last_clean":"2023-11-13T10:55:30.911186+0000","last_became_active":"2023-11-13T10:55:26.866277+0000","last_became_peered":"2023-11-13T10:55:26.866277+0000","last_unstale":"2023-11-13T10:55:30.911186+0000","last_undegraded":"2023-11-13T10:55:30.911186+0000","last_fullsized":"2023-11-13T10:55:30.911186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:49:42.713290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898545+0000","last_change":"2023-11-13T10:55:26.869354+0000","last_active":"2023-11-13T10:55:30.898545+0000","last_peered":"2023-11-13T10:55:30.898545+0000","last_clean":"2023-11-13T10:55:30.898545+0000","last_became_active":"2023-11-13T10:55:26.869040+0000","last_became_peered":"2023-11-13T10:55:26.869040+0000","last_unstale":"2023-11-13T10:55:30.898545+0000","last_undegraded":"2023-11-13T10:55:30.898545+0000","last_fullsized":"2023-11-13T10:55:30.898545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:50.524863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.908909+0000","last_change":"2023-11-13T10:55:26.861905+0000","last_active":"2023-11-13T10:55:30.908909+0000","last_peered":"2023-11-13T10:55:30.908909+0000","last_clean":"2023-11-13T10:55:30.908909+0000","last_became_active":"2023-11-13T10:55:26.861613+0000","last_became_peered":"2023-11-13T10:55:26.861613+0000","last_unstale":"2023-11-13T10:55:30.908909+0000","last_undegraded":"2023-11-13T10:55:30.908909+0000","last_fullsized":"2023-11-13T10:55:30.908909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:31.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913352+0000","last_change":"2023-11-13T10:55:26.873181+0000","last_active":"2023-11-13T10:55:30.913352+0000","last_peered":"2023-11-13T10:55:30.913352+0000","last_clean":"2023-11-13T10:55:30.913352+0000","last_became_active":"2023-11-13T10:55:26.872868+0000","last_became_peered":"2023-11-13T10:55:26.872868+0000","last_unstale":"2023-11-13T10:55:30.913352+0000","last_undegraded":"2023-11-13T10:55:30.913352+0000","last_fullsized":"2023-11-13T10:55:30.913352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:10.174343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.915906+0000","last_change":"2023-11-13T10:55:26.855441+0000","last_active":"2023-11-13T10:55:30.915906+0000","last_peered":"2023-11-13T10:55:30.915906+0000","last_clean":"2023-11-13T10:55:30.915906+0000","last_became_active":"2023-11-13T10:55:26.855101+0000","last_became_peered":"2023-11-13T10:55:26.855101+0000","last_unstale":"2023-11-13T10:55:30.915906+0000","last_undegraded":"2023-11-13T10:55:30.915906+0000","last_fullsized":"2023-11-13T10:55:30.915906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:53.364560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886903+0000","last_change":"2023-11-13T10:55:26.864741+0000","last_active":"2023-11-13T10:55:30.886903+0000","last_peered":"2023-11-13T10:55:30.886903+0000","last_clean":"2023-11-13T10:55:30.886903+0000","last_became_active":"2023-11-13T10:55:26.863408+0000","last_became_peered":"2023-11-13T10:55:26.863408+0000","last_unstale":"2023-11-13T10:55:30.886903+0000","last_undegraded":"2023-11-13T10:55:30.886903+0000","last_fullsized":"2023-11-13T10:55:30.886903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:12.028508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912752+0000","last_change":"2023-11-13T10:55:26.856400+0000","last_active":"2023-11-13T10:55:30.912752+0000","last_peered":"2023-11-13T10:55:30.912752+0000","last_clean":"2023-11-13T10:55:30.912752+0000","last_became_active":"2023-11-13T10:55:26.853838+0000","last_became_peered":"2023-11-13T10:55:26.853838+0000","last_unstale":"2023-11-13T10:55:30.912752+0000","last_undegraded":"2023-11-13T10:55:30.912752+0000","last_fullsized":"2023-11-13T10:55:30.912752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:45.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898736+0000","last_change":"2023-11-13T10:55:26.854049+0000","last_active":"2023-11-13T10:55:30.898736+0000","last_peered":"2023-11-13T10:55:30.898736+0000","last_clean":"2023-11-13T10:55:30.898736+0000","last_became_active":"2023-11-13T10:55:26.853767+0000","last_became_peered":"2023-11-13T10:55:26.853767+0000","last_unstale":"2023-11-13T10:55:30.898736+0000","last_undegraded":"2023-11-13T10:55:30.898736+0000","last_fullsized":"2023-11-13T10:55:30.898736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903782+0000","last_change":"2023-11-13T10:55:26.855717+0000","last_active":"2023-11-13T10:55:30.903782+0000","last_peered":"2023-11-13T10:55:30.903782+0000","last_clean":"2023-11-13T10:55:30.903782+0000","last_became_active":"2023-11-13T10:55:26.853617+0000","last_became_peered":"2023-11-13T10:55:26.853617+0000","last_unstale":"2023-11-13T10:55:30.903782+0000","last_undegraded":"2023-11-13T10:55:30.903782+0000","last_fullsized":"2023-11-13T10:55:30.903782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.895441+0000","last_change":"2023-11-13T10:55:26.857625+0000","last_active":"2023-11-13T10:55:30.895441+0000","last_peered":"2023-11-13T10:55:30.895441+0000","last_clean":"2023-11-13T10:55:30.895441+0000","last_became_active":"2023-11-13T10:55:26.857214+0000","last_became_peered":"2023-11-13T10:55:26.857214+0000","last_unstale":"2023-11-13T10:55:30.895441+0000","last_undegraded":"2023-11-13T10:55:30.895441+0000","last_fullsized":"2023-11-13T10:55:30.895441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:06.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894989+0000","last_change":"2023-11-13T10:55:26.870935+0000","last_active":"2023-11-13T10:55:30.894989+0000","last_peered":"2023-11-13T10:55:30.894989+0000","last_clean":"2023-11-13T10:55:30.894989+0000","last_became_active":"2023-11-13T10:55:26.870322+0000","last_became_peered":"2023-11-13T10:55:26.870322+0000","last_unstale":"2023-11-13T10:55:30.894989+0000","last_undegraded":"2023-11-13T10:55:30.894989+0000","last_fullsized":"2023-11-13T10:55:30.894989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:17.035738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900412+0000","last_change":"2023-11-13T10:55:26.867707+0000","last_active":"2023-11-13T10:55:30.900412+0000","last_peered":"2023-11-13T10:55:30.900412+0000","last_clean":"2023-11-13T10:55:30.900412+0000","last_became_active":"2023-11-13T10:55:26.864751+0000","last_became_peered":"2023-11-13T10:55:26.864751+0000","last_unstale":"2023-11-13T10:55:30.900412+0000","last_undegraded":"2023-11-13T10:55:30.900412+0000","last_fullsized":"2023-11-13T10:55:30.900412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:43.799944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898242+0000","last_change":"2023-11-13T10:55:26.855923+0000","last_active":"2023-11-13T10:55:30.898242+0000","last_peered":"2023-11-13T10:55:30.898242+0000","last_clean":"2023-11-13T10:55:30.898242+0000","last_became_active":"2023-11-13T10:55:26.855638+0000","last_became_peered":"2023-11-13T10:55:26.855638+0000","last_unstale":"2023-11-13T10:55:30.898242+0000","last_undegraded":"2023-11-13T10:55:30.898242+0000","last_fullsized":"2023-11-13T10:55:30.898242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:40:26.611005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.916348+0000","last_change":"2023-11-13T10:55:26.870707+0000","last_active":"2023-11-13T10:55:30.916348+0000","last_peered":"2023-11-13T10:55:30.916348+0000","last_clean":"2023-11-13T10:55:30.916348+0000","last_became_active":"2023-11-13T10:55:26.870417+0000","last_became_peered":"2023-11-13T10:55:26.870417+0000","last_unstale":"2023-11-13T10:55:30.916348+0000","last_undegraded":"2023-11-13T10:55:30.916348+0000","last_fullsized":"2023-11-13T10:55:30.916348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:24:16.048814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899902+0000","last_change":"2023-11-13T10:55:26.873122+0000","last_active":"2023-11-13T10:55:30.899902+0000","last_peered":"2023-11-13T10:55:30.899902+0000","last_clean":"2023-11-13T10:55:30.899902+0000","last_became_active":"2023-11-13T10:55:26.872823+0000","last_became_peered":"2023-11-13T10:55:26.872823+0000","last_unstale":"2023-11-13T10:55:30.899902+0000","last_undegraded":"2023-11-13T10:55:30.899902+0000","last_fullsized":"2023-11-13T10:55:30.899902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:09:58.900197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898638+0000","last_change":"2023-11-13T10:55:26.869223+0000","last_active":"2023-11-13T10:55:30.898638+0000","last_peered":"2023-11-13T10:55:30.898638+0000","last_clean":"2023-11-13T10:55:30.898638+0000","last_became_active":"2023-11-13T10:55:26.868919+0000","last_became_peered":"2023-11-13T10:55:26.868919+0000","last_unstale":"2023-11-13T10:55:30.898638+0000","last_undegraded":"2023-11-13T10:55:30.898638+0000","last_fullsized":"2023-11-13T10:55:30.898638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:57:13.609895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903525+0000","last_change":"2023-11-13T10:55:26.865259+0000","last_active":"2023-11-13T10:55:30.903525+0000","last_peered":"2023-11-13T10:55:30.903525+0000","last_clean":"2023-11-13T10:55:30.903525+0000","last_became_active":"2023-11-13T10:55:26.864675+0000","last_became_peered":"2023-11-13T10:55:26.864675+0000","last_unstale":"2023-11-13T10:55:30.903525+0000","last_undegraded":"2023-11-13T10:55:30.903525+0000","last_fullsized":"2023-11-13T10:55:30.903525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:17.035738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912854+0000","last_change":"2023-11-13T10:55:26.863002+0000","last_active":"2023-11-13T10:55:30.912854+0000","last_peered":"2023-11-13T10:55:30.912854+0000","last_clean":"2023-11-13T10:55:30.912854+0000","last_became_active":"2023-11-13T10:55:26.862257+0000","last_became_peered":"2023-11-13T10:55:26.862257+0000","last_unstale":"2023-11-13T10:55:30.912854+0000","last_undegraded":"2023-11-13T10:55:30.912854+0000","last_fullsized":"2023-11-13T10:55:30.912854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:17.901362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.914921+0000","last_change":"2023-11-13T10:55:26.872307+0000","last_active":"2023-11-13T10:55:30.914921+0000","last_peered":"2023-11-13T10:55:30.914921+0000","last_clean":"2023-11-13T10:55:30.914921+0000","last_became_active":"2023-11-13T10:55:26.872087+0000","last_became_peered":"2023-11-13T10:55:26.872087+0000","last_unstale":"2023-11-13T10:55:30.914921+0000","last_undegraded":"2023-11-13T10:55:30.914921+0000","last_fullsized":"2023-11-13T10:55:30.914921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:22.528830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900326+0000","last_change":"2023-11-13T10:55:26.872483+0000","last_active":"2023-11-13T10:55:30.900326+0000","last_peered":"2023-11-13T10:55:30.900326+0000","last_clean":"2023-11-13T10:55:30.900326+0000","last_became_active":"2023-11-13T10:55:26.872186+0000","last_became_peered":"2023-11-13T10:55:26.872186+0000","last_unstale":"2023-11-13T10:55:30.900326+0000","last_undegraded":"2023-11-13T10:55:30.900326+0000","last_fullsized":"2023-11-13T10:55:30.900326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:47.501708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897861+0000","last_change":"2023-11-13T10:55:26.862513+0000","last_active":"2023-11-13T10:55:30.897861+0000","last_peered":"2023-11-13T10:55:30.897861+0000","last_clean":"2023-11-13T10:55:30.897861+0000","last_became_active":"2023-11-13T10:55:26.862167+0000","last_became_peered":"2023-11-13T10:55:26.862167+0000","last_unstale":"2023-11-13T10:55:30.897861+0000","last_undegraded":"2023-11-13T10:55:30.897861+0000","last_fullsized":"2023-11-13T10:55:30.897861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902597+0000","last_change":"2023-11-13T10:55:26.861425+0000","last_active":"2023-11-13T10:55:30.902597+0000","last_peered":"2023-11-13T10:55:30.902597+0000","last_clean":"2023-11-13T10:55:30.902597+0000","last_became_active":"2023-11-13T10:55:26.861132+0000","last_became_peered":"2023-11-13T10:55:26.861132+0000","last_unstale":"2023-11-13T10:55:30.902597+0000","last_undegraded":"2023-11-13T10:55:30.902597+0000","last_fullsized":"2023-11-13T10:55:30.902597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:29.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896561+0000","last_change":"2023-11-13T10:55:26.858332+0000","last_active":"2023-11-13T10:55:30.896561+0000","last_peered":"2023-11-13T10:55:30.896561+0000","last_clean":"2023-11-13T10:55:30.896561+0000","last_became_active":"2023-11-13T10:55:26.858052+0000","last_became_peered":"2023-11-13T10:55:26.858052+0000","last_unstale":"2023-11-13T10:55:30.896561+0000","last_undegraded":"2023-11-13T10:55:30.896561+0000","last_fullsized":"2023-11-13T10:55:30.896561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903167+0000","last_change":"2023-11-13T10:55:26.863579+0000","last_active":"2023-11-13T10:55:30.903167+0000","last_peered":"2023-11-13T10:55:30.903167+0000","last_clean":"2023-11-13T10:55:30.903167+0000","last_became_active":"2023-11-13T10:55:26.863247+0000","last_became_peered":"2023-11-13T10:55:26.863247+0000","last_unstale":"2023-11-13T10:55:30.903167+0000","last_undegraded":"2023-11-13T10:55:30.903167+0000","last_fullsized":"2023-11-13T10:55:30.903167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.774455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898503+0000","last_change":"2023-11-13T10:55:26.856219+0000","last_active":"2023-11-13T10:55:30.898503+0000","last_peered":"2023-11-13T10:55:30.898503+0000","last_clean":"2023-11-13T10:55:30.898503+0000","last_became_active":"2023-11-13T10:55:26.855933+0000","last_became_peered":"2023-11-13T10:55:26.855933+0000","last_unstale":"2023-11-13T10:55:30.898503+0000","last_undegraded":"2023-11-13T10:55:30.898503+0000","last_fullsized":"2023-11-13T10:55:30.898503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887319+0000","last_change":"2023-11-13T10:55:26.853802+0000","last_active":"2023-11-13T10:55:30.887319+0000","last_peered":"2023-11-13T10:55:30.887319+0000","last_clean":"2023-11-13T10:55:30.887319+0000","last_became_active":"2023-11-13T10:55:26.853463+0000","last_became_peered":"2023-11-13T10:55:26.853463+0000","last_unstale":"2023-11-13T10:55:30.887319+0000","last_undegraded":"2023-11-13T10:55:30.887319+0000","last_fullsized":"2023-11-13T10:55:30.887319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:06.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902537+0000","last_change":"2023-11-13T10:55:26.865339+0000","last_active":"2023-11-13T10:55:30.902537+0000","last_peered":"2023-11-13T10:55:30.902537+0000","last_clean":"2023-11-13T10:55:30.902537+0000","last_became_active":"2023-11-13T10:55:26.865069+0000","last_became_peered":"2023-11-13T10:55:26.865069+0000","last_unstale":"2023-11-13T10:55:30.902537+0000","last_undegraded":"2023-11-13T10:55:30.902537+0000","last_fullsized":"2023-11-13T10:55:30.902537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:50.524863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887342+0000","last_change":"2023-11-13T10:55:26.857434+0000","last_active":"2023-11-13T10:55:30.887342+0000","last_peered":"2023-11-13T10:55:30.887342+0000","last_clean":"2023-11-13T10:55:30.887342+0000","last_became_active":"2023-11-13T10:55:26.857164+0000","last_became_peered":"2023-11-13T10:55:26.857164+0000","last_unstale":"2023-11-13T10:55:30.887342+0000","last_undegraded":"2023-11-13T10:55:30.887342+0000","last_fullsized":"2023-11-13T10:55:30.887342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909271+0000","last_change":"2023-11-13T10:55:26.845954+0000","last_active":"2023-11-13T10:55:30.909271+0000","last_peered":"2023-11-13T10:55:30.909271+0000","last_clean":"2023-11-13T10:55:30.909271+0000","last_became_active":"2023-11-13T10:55:26.845505+0000","last_became_peered":"2023-11-13T10:55:26.845505+0000","last_unstale":"2023-11-13T10:55:30.909271+0000","last_undegraded":"2023-11-13T10:55:30.909271+0000","last_fullsized":"2023-11-13T10:55:30.909271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:43.799944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911061+0000","last_change":"2023-11-13T10:55:26.857565+0000","last_active":"2023-11-13T10:55:30.911061+0000","last_peered":"2023-11-13T10:55:30.911061+0000","last_clean":"2023-11-13T10:55:30.911061+0000","last_became_active":"2023-11-13T10:55:26.857291+0000","last_became_peered":"2023-11-13T10:55:26.857291+0000","last_unstale":"2023-11-13T10:55:30.911061+0000","last_undegraded":"2023-11-13T10:55:30.911061+0000","last_fullsized":"2023-11-13T10:55:30.911061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:29.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902086+0000","last_change":"2023-11-13T10:55:26.858620+0000","last_active":"2023-11-13T10:55:30.902086+0000","last_peered":"2023-11-13T10:55:30.902086+0000","last_clean":"2023-11-13T10:55:30.902086+0000","last_became_active":"2023-11-13T10:55:26.858348+0000","last_became_peered":"2023-11-13T10:55:26.858348+0000","last_unstale":"2023-11-13T10:55:30.902086+0000","last_undegraded":"2023-11-13T10:55:30.902086+0000","last_fullsized":"2023-11-13T10:55:30.902086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:45.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":695602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":109655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.643}]}]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":105724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":108983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7949999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.655}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":695580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]}]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.859}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.95}]}]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.46}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T10:55:55.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T10:55:55.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.849+0000 7f6427373640 1 Processor -- start 2023-11-13T10:55:55.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.849+0000 7f6427373640 1 -- start start 2023-11-13T10:55:55.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.851+0000 7f6427373640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f642015cf20 0x7f642006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.851+0000 7f6427373640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64200cecc0 0x7f6420070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.851+0000 7f6427373640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64200b2900 con 0x7f642006bb90 2023-11-13T10:55:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.851+0000 7f6427373640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642015d300 con 0x7f642015cf20 2023-11-13T10:55:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.851+0000 7f6427373640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64200b1fd0 con 0x7f64200cecc0 2023-11-13T10:55:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.851+0000 7f642490e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f642006bb90 0x7f64200ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34466/0 (socket says 172.21.15.16:34466) 2023-11-13T10:55:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.851+0000 7f642490e640 1 -- 172.21.15.16:0/2471404693 learned_addr learned my addr 172.21.15.16:0/2471404693 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.852+0000 7f6425910640 1 --2- 172.21.15.16:0/2471404693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64200cecc0 0x7f6420070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.852+0000 7f642510f640 1 --2- 172.21.15.16:0/2471404693 >> v2:172.21.15.184:3300/0 conn(0x7f642015cf20 0x7f642006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.852+0000 7f6417fff640 1 -- 172.21.15.16:0/2471404693 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571304567 0 0) 0x7f64200b2900 con 0x7f642006bb90 2023-11-13T10:55:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.852+0000 7f6417fff640 1 -- 172.21.15.16:0/2471404693 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f640c003520 con 0x7f642006bb90 2023-11-13T10:55:55.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.852+0000 7f6425910640 1 -- 172.21.15.16:0/2471404693 >> v2:172.21.15.184:3300/0 conn(0x7f642015cf20 msgr2=0x7f642006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:55.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.852+0000 7f6425910640 1 --2- 172.21.15.16:0/2471404693 >> v2:172.21.15.184:3300/0 conn(0x7f642015cf20 0x7f642006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.852+0000 7f6425910640 1 -- 172.21.15.16:0/2471404693 >> v1:172.21.15.16:6789/0 conn(0x7f642006bb90 legacy=0x7f64200ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.852+0000 7f6425910640 1 -- 172.21.15.16:0/2471404693 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64200b22b0 con 0x7f64200cecc0 2023-11-13T10:55:55.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.852+0000 7f642510f640 1 --2- 172.21.15.16:0/2471404693 >> v2:172.21.15.184:3300/0 conn(0x7f642015cf20 0x7f642006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.854+0000 7f6425910640 1 --2- 172.21.15.16:0/2471404693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64200cecc0 0x7f6420070330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f64180097a0 tx=0x7f641802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72a261f8bb147acf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.854+0000 7f6417fff640 1 -- 172.21.15.16:0/2471404693 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641802d040 con 0x7f64200cecc0 2023-11-13T10:55:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.854+0000 7f6417fff640 1 -- 172.21.15.16:0/2471404693 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6418031810 con 0x7f64200cecc0 2023-11-13T10:55:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.854+0000 7f6417fff640 1 -- 172.21.15.16:0/2471404693 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418031b80 con 0x7f64200cecc0 2023-11-13T10:55:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.855+0000 7f6427373640 1 -- 172.21.15.16:0/2471404693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64200cecc0 msgr2=0x7f6420070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.855+0000 7f6427373640 1 --2- 172.21.15.16:0/2471404693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64200cecc0 0x7f6420070330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f64180097a0 tx=0x7f641802b8b0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.855+0000 7f6427373640 1 -- 172.21.15.16:0/2471404693 shutdown_connections 2023-11-13T10:55:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.855+0000 7f6427373640 1 --2- 172.21.15.16:0/2471404693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64200cecc0 0x7f6420070330 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.855+0000 7f6427373640 1 --2- 172.21.15.16:0/2471404693 >> v2:172.21.15.184:3300/0 conn(0x7f642015cf20 0x7f642006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.855+0000 7f6427373640 1 -- 172.21.15.16:0/2471404693 >> 172.21.15.16:0/2471404693 conn(0x7f642006a190 msgr2=0x7f642006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:55.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.855+0000 7f6427373640 1 -- 172.21.15.16:0/2471404693 shutdown_connections 2023-11-13T10:55:55.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.855+0000 7f6427373640 1 -- 172.21.15.16:0/2471404693 wait complete. 2023-11-13T10:55:55.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.857+0000 7f6427373640 1 Processor -- start 2023-11-13T10:55:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.857+0000 7f6427373640 1 -- start start 2023-11-13T10:55:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.858+0000 7f6427373640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f642006bb90 0x7f642015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.858+0000 7f6427373640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200cecc0 0x7f642015f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.858+0000 7f6427373640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f642015cf20 0x7f64200b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.858+0000 7f6427373640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642015fd00 con 0x7f64200cecc0 2023-11-13T10:55:55.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.858+0000 7f6427373640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642015f410 con 0x7f642006bb90 2023-11-13T10:55:55.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.858+0000 7f6427373640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6420151e90 con 0x7f642015cf20 2023-11-13T10:55:55.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642490e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200cecc0 0x7f642015f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642490e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200cecc0 0x7f642015f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55896/0 (socket says 172.21.15.16:55896) 2023-11-13T10:55:55.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642490e640 1 -- 172.21.15.16:0/2280125592 learned_addr learned my addr 172.21.15.16:0/2280125592 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:55.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642510f640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f642006bb90 0x7f642015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642490e640 1 -- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f642015cf20 msgr2=0x7f64200b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642490e640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f642015cf20 0x7f64200b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642490e640 1 -- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f642006bb90 msgr2=0x7f642015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642490e640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f642006bb90 0x7f642015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642490e640 1 -- 172.21.15.16:0/2280125592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6418009400 con 0x7f64200cecc0 2023-11-13T10:55:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.859+0000 7f642510f640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f642006bb90 0x7f642015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.860+0000 7f642490e640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200cecc0 0x7f642015f7c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f641000c400 tx=0x7f641000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.860+0000 7f6415ffb640 1 -- 172.21.15.16:0/2280125592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410005ce0 con 0x7f64200cecc0 2023-11-13T10:55:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.860+0000 7f6415ffb640 1 -- 172.21.15.16:0/2280125592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6410005e60 con 0x7f64200cecc0 2023-11-13T10:55:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.860+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6420152130 con 0x7f64200cecc0 2023-11-13T10:55:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.860+0000 7f6415ffb640 1 -- 172.21.15.16:0/2280125592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64100072f0 con 0x7f64200cecc0 2023-11-13T10:55:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.860+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64201526f0 con 0x7f64200cecc0 2023-11-13T10:55:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.861+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6420067090 con 0x7f64200cecc0 2023-11-13T10:55:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.862+0000 7f6415ffb640 1 -- 172.21.15.16:0/2280125592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6410007450 con 0x7f64200cecc0 2023-11-13T10:55:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.863+0000 7f6415ffb640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640406ae40 0x7f640406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.863+0000 7f6415ffb640 1 -- 172.21.15.16:0/2280125592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6410088f60 con 0x7f64200cecc0 2023-11-13T10:55:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.864+0000 7f642510f640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640406ae40 0x7f640406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.865+0000 7f642510f640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640406ae40 0x7f640406d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f641c009540 tx=0x7f641c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:55.868+0000 7f6415ffb640 1 -- 172.21.15.16:0/2280125592 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6410053d10 con 0x7f64200cecc0 2023-11-13T10:55:56.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.061+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f64200b3540 con 0x7f640406ae40 2023-11-13T10:55:56.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.087+0000 7f6415ffb640 1 -- 172.21.15.16:0/2280125592 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345075 (secure 0 0 0) 0x7f641c00f040 con 0x7f640406ae40 2023-11-13T10:55:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T10:55:56.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640406ae40 msgr2=0x7f640406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640406ae40 0x7f640406d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f641c009540 tx=0x7f641c00b040 comp rx=0 tx=0).stop 2023-11-13T10:55:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200cecc0 msgr2=0x7f642015f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200cecc0 0x7f642015f7c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f641000c400 tx=0x7f641000c8c0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 shutdown_connections 2023-11-13T10:55:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640406ae40 0x7f640406d300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f642015cf20 0x7f64200b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64200cecc0 0x7f642015f7c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 --2- 172.21.15.16:0/2280125592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f642006bb90 0x7f642015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 >> 172.21.15.16:0/2280125592 conn(0x7f642006a190 msgr2=0x7f64200aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.096+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 shutdown_connections 2023-11-13T10:55:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.097+0000 7f6427373640 1 -- 172.21.15.16:0/2280125592 wait complete. 2023-11-13T10:55:56.122 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-11-13T10:55:55.978106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232004,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742716,"statfs":{"total":773094113280,"available":772856541184,"internally_reserved":0,"allocated":3416064,"data_stored":2031150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005037"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894617+0000","last_change":"2023-11-13T10:55:25.848955+0000","last_active":"2023-11-13T10:55:30.894617+0000","last_peered":"2023-11-13T10:55:30.894617+0000","last_clean":"2023-11-13T10:55:30.894617+0000","last_became_active":"2023-11-13T10:55:25.848644+0000","last_became_peered":"2023-11-13T10:55:25.848644+0000","last_unstale":"2023-11-13T10:55:30.894617+0000","last_undegraded":"2023-11-13T10:55:30.894617+0000","last_fullsized":"2023-11-13T10:55:30.894617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:40.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911163+0000","last_change":"2023-11-13T10:55:26.862834+0000","last_active":"2023-11-13T10:55:30.911163+0000","last_peered":"2023-11-13T10:55:30.911163+0000","last_clean":"2023-11-13T10:55:30.911163+0000","last_became_active":"2023-11-13T10:55:26.861881+0000","last_became_peered":"2023-11-13T10:55:26.861881+0000","last_unstale":"2023-11-13T10:55:30.911163+0000","last_undegraded":"2023-11-13T10:55:30.911163+0000","last_fullsized":"2023-11-13T10:55:30.911163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:47.501708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906481+0000","last_change":"2023-11-13T10:55:30.929671+0000","last_active":"2023-11-13T10:55:31.906481+0000","last_peered":"2023-11-13T10:55:31.906481+0000","last_clean":"2023-11-13T10:55:31.906481+0000","last_became_active":"2023-11-13T10:55:30.929229+0000","last_became_peered":"2023-11-13T10:55:30.929229+0000","last_unstale":"2023-11-13T10:55:31.906481+0000","last_undegraded":"2023-11-13T10:55:31.906481+0000","last_fullsized":"2023-11-13T10:55:31.906481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:46.695309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911269+0000","last_change":"2023-11-13T10:55:25.855701+0000","last_active":"2023-11-13T10:55:30.911269+0000","last_peered":"2023-11-13T10:55:30.911269+0000","last_clean":"2023-11-13T10:55:30.911269+0000","last_became_active":"2023-11-13T10:55:25.855360+0000","last_became_peered":"2023-11-13T10:55:25.855360+0000","last_unstale":"2023-11-13T10:55:30.911269+0000","last_undegraded":"2023-11-13T10:55:30.911269+0000","last_fullsized":"2023-11-13T10:55:30.911269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:52.352286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912877+0000","last_change":"2023-11-13T10:55:26.848124+0000","last_active":"2023-11-13T10:55:30.912877+0000","last_peered":"2023-11-13T10:55:30.912877+0000","last_clean":"2023-11-13T10:55:30.912877+0000","last_became_active":"2023-11-13T10:55:26.847478+0000","last_became_peered":"2023-11-13T10:55:26.847478+0000","last_unstale":"2023-11-13T10:55:30.912877+0000","last_undegraded":"2023-11-13T10:55:30.912877+0000","last_fullsized":"2023-11-13T10:55:30.912877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:06.034946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.903358+0000","last_change":"2023-11-13T10:55:30.918537+0000","last_active":"2023-11-13T10:55:31.903358+0000","last_peered":"2023-11-13T10:55:31.903358+0000","last_clean":"2023-11-13T10:55:31.903358+0000","last_became_active":"2023-11-13T10:55:30.918092+0000","last_became_peered":"2023-11-13T10:55:30.918092+0000","last_unstale":"2023-11-13T10:55:31.903358+0000","last_undegraded":"2023-11-13T10:55:31.903358+0000","last_fullsized":"2023-11-13T10:55:31.903358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:50:02.751900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898103+0000","last_change":"2023-11-13T10:55:25.846685+0000","last_active":"2023-11-13T10:55:30.898103+0000","last_peered":"2023-11-13T10:55:30.898103+0000","last_clean":"2023-11-13T10:55:30.898103+0000","last_became_active":"2023-11-13T10:55:25.846370+0000","last_became_peered":"2023-11-13T10:55:25.846370+0000","last_unstale":"2023-11-13T10:55:30.898103+0000","last_undegraded":"2023-11-13T10:55:30.898103+0000","last_fullsized":"2023-11-13T10:55:30.898103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896163+0000","last_change":"2023-11-13T10:55:26.867122+0000","last_active":"2023-11-13T10:55:30.896163+0000","last_peered":"2023-11-13T10:55:30.896163+0000","last_clean":"2023-11-13T10:55:30.896163+0000","last_became_active":"2023-11-13T10:55:26.866669+0000","last_became_peered":"2023-11-13T10:55:26.866669+0000","last_unstale":"2023-11-13T10:55:30.896163+0000","last_undegraded":"2023-11-13T10:55:30.896163+0000","last_fullsized":"2023-11-13T10:55:30.896163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:25.074197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904843+0000","last_change":"2023-11-13T10:55:30.918484+0000","last_active":"2023-11-13T10:55:31.904843+0000","last_peered":"2023-11-13T10:55:31.904843+0000","last_clean":"2023-11-13T10:55:31.904843+0000","last_became_active":"2023-11-13T10:55:30.917997+0000","last_became_peered":"2023-11-13T10:55:30.917997+0000","last_unstale":"2023-11-13T10:55:31.904843+0000","last_undegraded":"2023-11-13T10:55:31.904843+0000","last_fullsized":"2023-11-13T10:55:31.904843+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:17.817768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900603+0000","last_change":"2023-11-13T10:55:25.851246+0000","last_active":"2023-11-13T10:55:30.900603+0000","last_peered":"2023-11-13T10:55:30.900603+0000","last_clean":"2023-11-13T10:55:30.900603+0000","last_became_active":"2023-11-13T10:55:25.850865+0000","last_became_peered":"2023-11-13T10:55:25.850865+0000","last_unstale":"2023-11-13T10:55:30.900603+0000","last_undegraded":"2023-11-13T10:55:30.900603+0000","last_fullsized":"2023-11-13T10:55:30.900603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:30.550401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898058+0000","last_change":"2023-11-13T10:55:26.866532+0000","last_active":"2023-11-13T10:55:30.898058+0000","last_peered":"2023-11-13T10:55:30.898058+0000","last_clean":"2023-11-13T10:55:30.898058+0000","last_became_active":"2023-11-13T10:55:26.866199+0000","last_became_peered":"2023-11-13T10:55:26.866199+0000","last_unstale":"2023-11-13T10:55:30.898058+0000","last_undegraded":"2023-11-13T10:55:30.898058+0000","last_fullsized":"2023-11-13T10:55:30.898058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:18.260379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905023+0000","last_change":"2023-11-13T10:55:30.919396+0000","last_active":"2023-11-13T10:55:31.905023+0000","last_peered":"2023-11-13T10:55:31.905023+0000","last_clean":"2023-11-13T10:55:31.905023+0000","last_became_active":"2023-11-13T10:55:30.918942+0000","last_became_peered":"2023-11-13T10:55:30.918942+0000","last_unstale":"2023-11-13T10:55:31.905023+0000","last_undegraded":"2023-11-13T10:55:31.905023+0000","last_fullsized":"2023-11-13T10:55:31.905023+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:51.562560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898635+0000","last_change":"2023-11-13T10:55:26.867788+0000","last_active":"2023-11-13T10:55:30.898635+0000","last_peered":"2023-11-13T10:55:30.898635+0000","last_clean":"2023-11-13T10:55:30.898635+0000","last_became_active":"2023-11-13T10:55:26.865720+0000","last_became_peered":"2023-11-13T10:55:26.865720+0000","last_unstale":"2023-11-13T10:55:30.898635+0000","last_undegraded":"2023-11-13T10:55:30.898635+0000","last_fullsized":"2023-11-13T10:55:30.898635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:04:53.206295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899232+0000","last_change":"2023-11-13T10:55:25.848578+0000","last_active":"2023-11-13T10:55:30.899232+0000","last_peered":"2023-11-13T10:55:30.899232+0000","last_clean":"2023-11-13T10:55:30.899232+0000","last_became_active":"2023-11-13T10:55:25.848218+0000","last_became_peered":"2023-11-13T10:55:25.848218+0000","last_unstale":"2023-11-13T10:55:30.899232+0000","last_undegraded":"2023-11-13T10:55:30.899232+0000","last_fullsized":"2023-11-13T10:55:30.899232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.882851+0000","last_change":"2023-11-13T10:55:30.927208+0000","last_active":"2023-11-13T10:55:31.882851+0000","last_peered":"2023-11-13T10:55:31.882851+0000","last_clean":"2023-11-13T10:55:31.882851+0000","last_became_active":"2023-11-13T10:55:30.926587+0000","last_became_peered":"2023-11-13T10:55:30.926587+0000","last_unstale":"2023-11-13T10:55:31.882851+0000","last_undegraded":"2023-11-13T10:55:31.882851+0000","last_fullsized":"2023-11-13T10:55:31.882851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:58:30.075199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911307+0000","last_change":"2023-11-13T10:55:26.863647+0000","last_active":"2023-11-13T10:55:30.911307+0000","last_peered":"2023-11-13T10:55:30.911307+0000","last_clean":"2023-11-13T10:55:30.911307+0000","last_became_active":"2023-11-13T10:55:26.863171+0000","last_became_peered":"2023-11-13T10:55:26.863171+0000","last_unstale":"2023-11-13T10:55:30.911307+0000","last_undegraded":"2023-11-13T10:55:30.911307+0000","last_fullsized":"2023-11-13T10:55:30.911307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:44:59.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913133+0000","last_change":"2023-11-13T10:55:25.861244+0000","last_active":"2023-11-13T10:55:30.913133+0000","last_peered":"2023-11-13T10:55:30.913133+0000","last_clean":"2023-11-13T10:55:30.913133+0000","last_became_active":"2023-11-13T10:55:25.860949+0000","last_became_peered":"2023-11-13T10:55:25.860949+0000","last_unstale":"2023-11-13T10:55:30.913133+0000","last_undegraded":"2023-11-13T10:55:30.913133+0000","last_fullsized":"2023-11-13T10:55:30.913133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:05.431553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.902438+0000","last_change":"2023-11-13T10:55:30.922731+0000","last_active":"2023-11-13T10:55:31.902438+0000","last_peered":"2023-11-13T10:55:31.902438+0000","last_clean":"2023-11-13T10:55:31.902438+0000","last_became_active":"2023-11-13T10:55:30.922328+0000","last_became_peered":"2023-11-13T10:55:30.922328+0000","last_unstale":"2023-11-13T10:55:31.902438+0000","last_undegraded":"2023-11-13T10:55:31.902438+0000","last_fullsized":"2023-11-13T10:55:31.902438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:57:17.670747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897411+0000","last_change":"2023-11-13T10:55:26.857718+0000","last_active":"2023-11-13T10:55:30.897411+0000","last_peered":"2023-11-13T10:55:30.897411+0000","last_clean":"2023-11-13T10:55:30.897411+0000","last_became_active":"2023-11-13T10:55:26.852435+0000","last_became_peered":"2023-11-13T10:55:26.852435+0000","last_unstale":"2023-11-13T10:55:30.897411+0000","last_undegraded":"2023-11-13T10:55:30.897411+0000","last_fullsized":"2023-11-13T10:55:30.897411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:45.182794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892276+0000","last_change":"2023-11-13T10:55:25.858975+0000","last_active":"2023-11-13T10:55:30.892276+0000","last_peered":"2023-11-13T10:55:30.892276+0000","last_clean":"2023-11-13T10:55:30.892276+0000","last_became_active":"2023-11-13T10:55:25.858642+0000","last_became_peered":"2023-11-13T10:55:25.858642+0000","last_unstale":"2023-11-13T10:55:30.892276+0000","last_undegraded":"2023-11-13T10:55:30.892276+0000","last_fullsized":"2023-11-13T10:55:30.892276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.908730+0000","last_change":"2023-11-13T10:55:30.931892+0000","last_active":"2023-11-13T10:55:31.908730+0000","last_peered":"2023-11-13T10:55:31.908730+0000","last_clean":"2023-11-13T10:55:31.908730+0000","last_became_active":"2023-11-13T10:55:30.931357+0000","last_became_peered":"2023-11-13T10:55:30.931357+0000","last_unstale":"2023-11-13T10:55:31.908730+0000","last_undegraded":"2023-11-13T10:55:31.908730+0000","last_fullsized":"2023-11-13T10:55:31.908730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:07:10.060960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897798+0000","last_change":"2023-11-13T10:55:26.855031+0000","last_active":"2023-11-13T10:55:30.897798+0000","last_peered":"2023-11-13T10:55:30.897798+0000","last_clean":"2023-11-13T10:55:30.897798+0000","last_became_active":"2023-11-13T10:55:26.854761+0000","last_became_peered":"2023-11-13T10:55:26.854761+0000","last_unstale":"2023-11-13T10:55:30.897798+0000","last_undegraded":"2023-11-13T10:55:30.897798+0000","last_fullsized":"2023-11-13T10:55:30.897798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:27.863825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898689+0000","last_change":"2023-11-13T10:55:25.848611+0000","last_active":"2023-11-13T10:55:30.898689+0000","last_peered":"2023-11-13T10:55:30.898689+0000","last_clean":"2023-11-13T10:55:30.898689+0000","last_became_active":"2023-11-13T10:55:25.848277+0000","last_became_peered":"2023-11-13T10:55:25.848277+0000","last_unstale":"2023-11-13T10:55:30.898689+0000","last_undegraded":"2023-11-13T10:55:30.898689+0000","last_fullsized":"2023-11-13T10:55:30.898689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905715+0000","last_change":"2023-11-13T10:55:30.920124+0000","last_active":"2023-11-13T10:55:31.905715+0000","last_peered":"2023-11-13T10:55:31.905715+0000","last_clean":"2023-11-13T10:55:31.905715+0000","last_became_active":"2023-11-13T10:55:30.919731+0000","last_became_peered":"2023-11-13T10:55:30.919731+0000","last_unstale":"2023-11-13T10:55:31.905715+0000","last_undegraded":"2023-11-13T10:55:31.905715+0000","last_fullsized":"2023-11-13T10:55:31.905715+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899898+0000","last_change":"2023-11-13T10:55:26.860831+0000","last_active":"2023-11-13T10:55:30.899898+0000","last_peered":"2023-11-13T10:55:30.899898+0000","last_clean":"2023-11-13T10:55:30.899898+0000","last_became_active":"2023-11-13T10:55:26.860527+0000","last_became_peered":"2023-11-13T10:55:26.860527+0000","last_unstale":"2023-11-13T10:55:30.899898+0000","last_undegraded":"2023-11-13T10:55:30.899898+0000","last_fullsized":"2023-11-13T10:55:30.899898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.774455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899998+0000","last_change":"2023-11-13T10:55:25.846361+0000","last_active":"2023-11-13T10:55:30.899998+0000","last_peered":"2023-11-13T10:55:30.899998+0000","last_clean":"2023-11-13T10:55:30.899998+0000","last_became_active":"2023-11-13T10:55:25.846031+0000","last_became_peered":"2023-11-13T10:55:25.846031+0000","last_unstale":"2023-11-13T10:55:30.899998+0000","last_undegraded":"2023-11-13T10:55:30.899998+0000","last_fullsized":"2023-11-13T10:55:30.899998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:44.868236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904780+0000","last_change":"2023-11-13T10:55:30.910764+0000","last_active":"2023-11-13T10:55:31.904780+0000","last_peered":"2023-11-13T10:55:31.904780+0000","last_clean":"2023-11-13T10:55:31.904780+0000","last_became_active":"2023-11-13T10:55:30.910363+0000","last_became_peered":"2023-11-13T10:55:30.910363+0000","last_unstale":"2023-11-13T10:55:31.904780+0000","last_undegraded":"2023-11-13T10:55:31.904780+0000","last_fullsized":"2023-11-13T10:55:31.904780+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:33.673828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.893835+0000","last_change":"2023-11-13T10:55:26.859968+0000","last_active":"2023-11-13T10:55:30.893835+0000","last_peered":"2023-11-13T10:55:30.893835+0000","last_clean":"2023-11-13T10:55:30.893835+0000","last_became_active":"2023-11-13T10:55:26.859344+0000","last_became_peered":"2023-11-13T10:55:26.859344+0000","last_unstale":"2023-11-13T10:55:30.893835+0000","last_undegraded":"2023-11-13T10:55:30.893835+0000","last_fullsized":"2023-11-13T10:55:30.893835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.107404+0000","last_change":"2023-11-13T10:55:25.857948+0000","last_active":"2023-11-13T10:55:31.107404+0000","last_peered":"2023-11-13T10:55:31.107404+0000","last_clean":"2023-11-13T10:55:31.107404+0000","last_became_active":"2023-11-13T10:55:25.857670+0000","last_became_peered":"2023-11-13T10:55:25.857670+0000","last_unstale":"2023-11-13T10:55:31.107404+0000","last_undegraded":"2023-11-13T10:55:31.107404+0000","last_fullsized":"2023-11-13T10:55:31.107404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:11.016234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883636+0000","last_change":"2023-11-13T10:55:30.918361+0000","last_active":"2023-11-13T10:55:31.883636+0000","last_peered":"2023-11-13T10:55:31.883636+0000","last_clean":"2023-11-13T10:55:31.883636+0000","last_became_active":"2023-11-13T10:55:30.912409+0000","last_became_peered":"2023-11-13T10:55:30.912409+0000","last_unstale":"2023-11-13T10:55:31.883636+0000","last_undegraded":"2023-11-13T10:55:31.883636+0000","last_fullsized":"2023-11-13T10:55:31.883636+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:05.903305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913447+0000","last_change":"2023-11-13T10:55:26.864360+0000","last_active":"2023-11-13T10:55:30.913447+0000","last_peered":"2023-11-13T10:55:30.913447+0000","last_clean":"2023-11-13T10:55:30.913447+0000","last_became_active":"2023-11-13T10:55:26.863969+0000","last_became_peered":"2023-11-13T10:55:26.863969+0000","last_unstale":"2023-11-13T10:55:30.913447+0000","last_undegraded":"2023-11-13T10:55:30.913447+0000","last_fullsized":"2023-11-13T10:55:30.913447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:45.182794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898332+0000","last_change":"2023-11-13T10:55:25.850034+0000","last_active":"2023-11-13T10:55:30.898332+0000","last_peered":"2023-11-13T10:55:30.898332+0000","last_clean":"2023-11-13T10:55:30.898332+0000","last_became_active":"2023-11-13T10:55:25.849670+0000","last_became_peered":"2023-11-13T10:55:25.849670+0000","last_unstale":"2023-11-13T10:55:30.898332+0000","last_undegraded":"2023-11-13T10:55:30.898332+0000","last_fullsized":"2023-11-13T10:55:30.898332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:42.787670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905007+0000","last_change":"2023-11-13T10:55:30.921622+0000","last_active":"2023-11-13T10:55:31.905007+0000","last_peered":"2023-11-13T10:55:31.905007+0000","last_clean":"2023-11-13T10:55:31.905007+0000","last_became_active":"2023-11-13T10:55:30.921092+0000","last_became_peered":"2023-11-13T10:55:30.921092+0000","last_unstale":"2023-11-13T10:55:31.905007+0000","last_undegraded":"2023-11-13T10:55:31.905007+0000","last_fullsized":"2023-11-13T10:55:31.905007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:49.941362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886149+0000","last_change":"2023-11-13T10:55:26.858625+0000","last_active":"2023-11-13T10:55:30.886149+0000","last_peered":"2023-11-13T10:55:30.886149+0000","last_clean":"2023-11-13T10:55:30.886149+0000","last_became_active":"2023-11-13T10:55:26.858318+0000","last_became_peered":"2023-11-13T10:55:26.858318+0000","last_unstale":"2023-11-13T10:55:30.886149+0000","last_undegraded":"2023-11-13T10:55:30.886149+0000","last_fullsized":"2023-11-13T10:55:30.886149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:43.105401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909064+0000","last_change":"2023-11-13T10:55:25.862368+0000","last_active":"2023-11-13T10:55:30.909064+0000","last_peered":"2023-11-13T10:55:30.909064+0000","last_clean":"2023-11-13T10:55:30.909064+0000","last_became_active":"2023-11-13T10:55:25.862060+0000","last_became_peered":"2023-11-13T10:55:25.862060+0000","last_unstale":"2023-11-13T10:55:30.909064+0000","last_undegraded":"2023-11-13T10:55:30.909064+0000","last_fullsized":"2023-11-13T10:55:30.909064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:09.162069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904580+0000","last_change":"2023-11-13T10:55:30.923556+0000","last_active":"2023-11-13T10:55:31.904580+0000","last_peered":"2023-11-13T10:55:31.904580+0000","last_clean":"2023-11-13T10:55:31.904580+0000","last_became_active":"2023-11-13T10:55:30.923070+0000","last_became_peered":"2023-11-13T10:55:30.923070+0000","last_unstale":"2023-11-13T10:55:31.904580+0000","last_undegraded":"2023-11-13T10:55:31.904580+0000","last_fullsized":"2023-11-13T10:55:31.904580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:51.562560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902612+0000","last_change":"2023-11-13T10:55:26.864760+0000","last_active":"2023-11-13T10:55:30.902612+0000","last_peered":"2023-11-13T10:55:30.902612+0000","last_clean":"2023-11-13T10:55:30.902612+0000","last_became_active":"2023-11-13T10:55:26.864419+0000","last_became_peered":"2023-11-13T10:55:26.864419+0000","last_unstale":"2023-11-13T10:55:30.902612+0000","last_undegraded":"2023-11-13T10:55:30.902612+0000","last_fullsized":"2023-11-13T10:55:30.902612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:01.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898908+0000","last_change":"2023-11-13T10:55:25.856907+0000","last_active":"2023-11-13T10:55:30.898908+0000","last_peered":"2023-11-13T10:55:30.898908+0000","last_clean":"2023-11-13T10:55:30.898908+0000","last_became_active":"2023-11-13T10:55:25.856545+0000","last_became_peered":"2023-11-13T10:55:25.856545+0000","last_unstale":"2023-11-13T10:55:30.898908+0000","last_undegraded":"2023-11-13T10:55:30.898908+0000","last_fullsized":"2023-11-13T10:55:30.898908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:28.600702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906072+0000","last_change":"2023-11-13T10:55:30.921637+0000","last_active":"2023-11-13T10:55:31.906072+0000","last_peered":"2023-11-13T10:55:31.906072+0000","last_clean":"2023-11-13T10:55:31.906072+0000","last_became_active":"2023-11-13T10:55:30.921177+0000","last_became_peered":"2023-11-13T10:55:30.921177+0000","last_unstale":"2023-11-13T10:55:31.906072+0000","last_undegraded":"2023-11-13T10:55:31.906072+0000","last_fullsized":"2023-11-13T10:55:31.906072+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903258+0000","last_change":"2023-11-13T10:55:26.851557+0000","last_active":"2023-11-13T10:55:30.903258+0000","last_peered":"2023-11-13T10:55:30.903258+0000","last_clean":"2023-11-13T10:55:30.903258+0000","last_became_active":"2023-11-13T10:55:26.850792+0000","last_became_peered":"2023-11-13T10:55:26.850792+0000","last_unstale":"2023-11-13T10:55:30.903258+0000","last_undegraded":"2023-11-13T10:55:30.903258+0000","last_fullsized":"2023-11-13T10:55:30.903258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911327+0000","last_change":"2023-11-13T10:55:25.855880+0000","last_active":"2023-11-13T10:55:30.911327+0000","last_peered":"2023-11-13T10:55:30.911327+0000","last_clean":"2023-11-13T10:55:30.911327+0000","last_became_active":"2023-11-13T10:55:25.855543+0000","last_became_peered":"2023-11-13T10:55:25.855543+0000","last_unstale":"2023-11-13T10:55:30.911327+0000","last_undegraded":"2023-11-13T10:55:30.911327+0000","last_fullsized":"2023-11-13T10:55:30.911327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:02:28.405745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.907032+0000","last_change":"2023-11-13T10:55:30.925168+0000","last_active":"2023-11-13T10:55:31.907032+0000","last_peered":"2023-11-13T10:55:31.907032+0000","last_clean":"2023-11-13T10:55:31.907032+0000","last_became_active":"2023-11-13T10:55:30.924675+0000","last_became_peered":"2023-11-13T10:55:30.924675+0000","last_unstale":"2023-11-13T10:55:31.907032+0000","last_undegraded":"2023-11-13T10:55:31.907032+0000","last_fullsized":"2023-11-13T10:55:31.907032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:25.376399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911697+0000","last_change":"2023-11-13T10:55:26.863984+0000","last_active":"2023-11-13T10:55:30.911697+0000","last_peered":"2023-11-13T10:55:30.911697+0000","last_clean":"2023-11-13T10:55:30.911697+0000","last_became_active":"2023-11-13T10:55:26.862398+0000","last_became_peered":"2023-11-13T10:55:26.862398+0000","last_unstale":"2023-11-13T10:55:30.911697+0000","last_undegraded":"2023-11-13T10:55:30.911697+0000","last_fullsized":"2023-11-13T10:55:30.911697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898919+0000","last_change":"2023-11-13T10:55:25.857004+0000","last_active":"2023-11-13T10:55:30.898919+0000","last_peered":"2023-11-13T10:55:30.898919+0000","last_clean":"2023-11-13T10:55:30.898919+0000","last_became_active":"2023-11-13T10:55:25.856718+0000","last_became_peered":"2023-11-13T10:55:25.856718+0000","last_unstale":"2023-11-13T10:55:30.898919+0000","last_undegraded":"2023-11-13T10:55:30.898919+0000","last_fullsized":"2023-11-13T10:55:30.898919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:28.167168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888398+0000","last_change":"2023-11-13T10:55:30.923781+0000","last_active":"2023-11-13T10:55:31.888398+0000","last_peered":"2023-11-13T10:55:31.888398+0000","last_clean":"2023-11-13T10:55:31.888398+0000","last_became_active":"2023-11-13T10:55:30.922714+0000","last_became_peered":"2023-11-13T10:55:30.922714+0000","last_unstale":"2023-11-13T10:55:31.888398+0000","last_undegraded":"2023-11-13T10:55:31.888398+0000","last_fullsized":"2023-11-13T10:55:31.888398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:05:31.401669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899007+0000","last_change":"2023-11-13T10:55:26.862991+0000","last_active":"2023-11-13T10:55:30.899007+0000","last_peered":"2023-11-13T10:55:30.899007+0000","last_clean":"2023-11-13T10:55:30.899007+0000","last_became_active":"2023-11-13T10:55:26.861100+0000","last_became_peered":"2023-11-13T10:55:26.861100+0000","last_unstale":"2023-11-13T10:55:30.899007+0000","last_undegraded":"2023-11-13T10:55:30.899007+0000","last_fullsized":"2023-11-13T10:55:30.899007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899324+0000","last_change":"2023-11-13T10:55:25.848422+0000","last_active":"2023-11-13T10:55:30.899324+0000","last_peered":"2023-11-13T10:55:30.899324+0000","last_clean":"2023-11-13T10:55:30.899324+0000","last_became_active":"2023-11-13T10:55:25.847809+0000","last_became_peered":"2023-11-13T10:55:25.847809+0000","last_unstale":"2023-11-13T10:55:30.899324+0000","last_undegraded":"2023-11-13T10:55:30.899324+0000","last_fullsized":"2023-11-13T10:55:30.899324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892816+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T10:55:30.892816+0000","last_peered":"2023-11-13T10:55:30.892816+0000","last_clean":"2023-11-13T10:55:30.892816+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T10:55:30.892816+0000","last_undegraded":"2023-11-13T10:55:30.892816+0000","last_fullsized":"2023-11-13T10:55:30.892816+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883757+0000","last_change":"2023-11-13T10:55:30.907496+0000","last_active":"2023-11-13T10:55:31.883757+0000","last_peered":"2023-11-13T10:55:31.883757+0000","last_clean":"2023-11-13T10:55:31.883757+0000","last_became_active":"2023-11-13T10:55:30.905911+0000","last_became_peered":"2023-11-13T10:55:30.905911+0000","last_unstale":"2023-11-13T10:55:31.883757+0000","last_undegraded":"2023-11-13T10:55:31.883757+0000","last_fullsized":"2023-11-13T10:55:31.883757+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:26:30.660465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898140+0000","last_change":"2023-11-13T10:55:26.865638+0000","last_active":"2023-11-13T10:55:30.898140+0000","last_peered":"2023-11-13T10:55:30.898140+0000","last_clean":"2023-11-13T10:55:30.898140+0000","last_became_active":"2023-11-13T10:55:26.864005+0000","last_became_peered":"2023-11-13T10:55:26.864005+0000","last_unstale":"2023-11-13T10:55:30.898140+0000","last_undegraded":"2023-11-13T10:55:30.898140+0000","last_fullsized":"2023-11-13T10:55:30.898140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:18:03.109671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.914054+0000","last_change":"2023-11-13T10:55:25.860502+0000","last_active":"2023-11-13T10:55:30.914054+0000","last_peered":"2023-11-13T10:55:30.914054+0000","last_clean":"2023-11-13T10:55:30.914054+0000","last_became_active":"2023-11-13T10:55:25.860157+0000","last_became_peered":"2023-11-13T10:55:25.860157+0000","last_unstale":"2023-11-13T10:55:30.914054+0000","last_undegraded":"2023-11-13T10:55:30.914054+0000","last_fullsized":"2023-11-13T10:55:30.914054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:42.093127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905171+0000","last_change":"2023-11-13T10:55:30.909998+0000","last_active":"2023-11-13T10:55:31.905171+0000","last_peered":"2023-11-13T10:55:31.905171+0000","last_clean":"2023-11-13T10:55:31.905171+0000","last_became_active":"2023-11-13T10:55:30.909623+0000","last_became_peered":"2023-11-13T10:55:30.909623+0000","last_unstale":"2023-11-13T10:55:31.905171+0000","last_undegraded":"2023-11-13T10:55:31.905171+0000","last_fullsized":"2023-11-13T10:55:31.905171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:47.860796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900512+0000","last_change":"2023-11-13T10:55:26.867419+0000","last_active":"2023-11-13T10:55:30.900512+0000","last_peered":"2023-11-13T10:55:30.900512+0000","last_clean":"2023-11-13T10:55:30.900512+0000","last_became_active":"2023-11-13T10:55:26.864802+0000","last_became_peered":"2023-11-13T10:55:26.864802+0000","last_unstale":"2023-11-13T10:55:30.900512+0000","last_undegraded":"2023-11-13T10:55:30.900512+0000","last_fullsized":"2023-11-13T10:55:30.900512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909289+0000","last_change":"2023-11-13T10:55:25.852321+0000","last_active":"2023-11-13T10:55:30.909289+0000","last_peered":"2023-11-13T10:55:30.909289+0000","last_clean":"2023-11-13T10:55:30.909289+0000","last_became_active":"2023-11-13T10:55:25.850605+0000","last_became_peered":"2023-11-13T10:55:25.850605+0000","last_unstale":"2023-11-13T10:55:30.909289+0000","last_undegraded":"2023-11-13T10:55:30.909289+0000","last_fullsized":"2023-11-13T10:55:30.909289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.887690+0000","last_change":"2023-11-13T10:55:30.913188+0000","last_active":"2023-11-13T10:55:31.887690+0000","last_peered":"2023-11-13T10:55:31.887690+0000","last_clean":"2023-11-13T10:55:31.887690+0000","last_became_active":"2023-11-13T10:55:30.912748+0000","last_became_peered":"2023-11-13T10:55:30.912748+0000","last_unstale":"2023-11-13T10:55:31.887690+0000","last_undegraded":"2023-11-13T10:55:31.887690+0000","last_fullsized":"2023-11-13T10:55:31.887690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:20.801658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902130+0000","last_change":"2023-11-13T10:55:26.867097+0000","last_active":"2023-11-13T10:55:30.902130+0000","last_peered":"2023-11-13T10:55:30.902130+0000","last_clean":"2023-11-13T10:55:30.902130+0000","last_became_active":"2023-11-13T10:55:26.866770+0000","last_became_peered":"2023-11-13T10:55:26.866770+0000","last_unstale":"2023-11-13T10:55:30.902130+0000","last_undegraded":"2023-11-13T10:55:30.902130+0000","last_fullsized":"2023-11-13T10:55:30.902130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:06.034946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900035+0000","last_change":"2023-11-13T10:55:25.855464+0000","last_active":"2023-11-13T10:55:30.900035+0000","last_peered":"2023-11-13T10:55:30.900035+0000","last_clean":"2023-11-13T10:55:30.900035+0000","last_became_active":"2023-11-13T10:55:25.853955+0000","last_became_peered":"2023-11-13T10:55:25.853955+0000","last_unstale":"2023-11-13T10:55:30.900035+0000","last_undegraded":"2023-11-13T10:55:30.900035+0000","last_fullsized":"2023-11-13T10:55:30.900035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.881810+0000","last_change":"2023-11-13T10:55:30.915984+0000","last_active":"2023-11-13T10:55:31.881810+0000","last_peered":"2023-11-13T10:55:31.881810+0000","last_clean":"2023-11-13T10:55:31.881810+0000","last_became_active":"2023-11-13T10:55:30.915438+0000","last_became_peered":"2023-11-13T10:55:30.915438+0000","last_unstale":"2023-11-13T10:55:31.881810+0000","last_undegraded":"2023-11-13T10:55:31.881810+0000","last_fullsized":"2023-11-13T10:55:31.881810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:06:39.428736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887086+0000","last_change":"2023-11-13T10:55:26.874069+0000","last_active":"2023-11-13T10:55:30.887086+0000","last_peered":"2023-11-13T10:55:30.887086+0000","last_clean":"2023-11-13T10:55:30.887086+0000","last_became_active":"2023-11-13T10:55:26.873537+0000","last_became_peered":"2023-11-13T10:55:26.873537+0000","last_unstale":"2023-11-13T10:55:30.887086+0000","last_undegraded":"2023-11-13T10:55:30.887086+0000","last_fullsized":"2023-11-13T10:55:30.887086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:22.528830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902191+0000","last_change":"2023-11-13T10:55:25.853256+0000","last_active":"2023-11-13T10:55:30.902191+0000","last_peered":"2023-11-13T10:55:30.902191+0000","last_clean":"2023-11-13T10:55:30.902191+0000","last_became_active":"2023-11-13T10:55:25.852947+0000","last_became_peered":"2023-11-13T10:55:25.852947+0000","last_unstale":"2023-11-13T10:55:30.902191+0000","last_undegraded":"2023-11-13T10:55:30.902191+0000","last_fullsized":"2023-11-13T10:55:30.902191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:15.762181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904654+0000","last_change":"2023-11-13T10:55:30.912617+0000","last_active":"2023-11-13T10:55:31.904654+0000","last_peered":"2023-11-13T10:55:31.904654+0000","last_clean":"2023-11-13T10:55:31.904654+0000","last_became_active":"2023-11-13T10:55:30.912019+0000","last_became_peered":"2023-11-13T10:55:30.912019+0000","last_unstale":"2023-11-13T10:55:31.904654+0000","last_undegraded":"2023-11-13T10:55:31.904654+0000","last_fullsized":"2023-11-13T10:55:31.904654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:16.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897711+0000","last_change":"2023-11-13T10:55:26.868126+0000","last_active":"2023-11-13T10:55:30.897711+0000","last_peered":"2023-11-13T10:55:30.897711+0000","last_clean":"2023-11-13T10:55:30.897711+0000","last_became_active":"2023-11-13T10:55:26.867810+0000","last_became_peered":"2023-11-13T10:55:26.867810+0000","last_unstale":"2023-11-13T10:55:30.897711+0000","last_undegraded":"2023-11-13T10:55:30.897711+0000","last_fullsized":"2023-11-13T10:55:30.897711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:16.740806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897637+0000","last_change":"2023-11-13T10:55:25.851131+0000","last_active":"2023-11-13T10:55:30.897637+0000","last_peered":"2023-11-13T10:55:30.897637+0000","last_clean":"2023-11-13T10:55:30.897637+0000","last_became_active":"2023-11-13T10:55:25.850827+0000","last_became_peered":"2023-11-13T10:55:25.850827+0000","last_unstale":"2023-11-13T10:55:30.897637+0000","last_undegraded":"2023-11-13T10:55:30.897637+0000","last_fullsized":"2023-11-13T10:55:30.897637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:05.431553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888952+0000","last_change":"2023-11-13T10:55:30.924096+0000","last_active":"2023-11-13T10:55:31.888952+0000","last_peered":"2023-11-13T10:55:31.888952+0000","last_clean":"2023-11-13T10:55:31.888952+0000","last_became_active":"2023-11-13T10:55:30.923375+0000","last_became_peered":"2023-11-13T10:55:30.923375+0000","last_unstale":"2023-11-13T10:55:31.888952+0000","last_undegraded":"2023-11-13T10:55:31.888952+0000","last_fullsized":"2023-11-13T10:55:31.888952+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:18:07.170524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892700+0000","last_change":"2023-11-13T10:55:26.852339+0000","last_active":"2023-11-13T10:55:30.892700+0000","last_peered":"2023-11-13T10:55:30.892700+0000","last_clean":"2023-11-13T10:55:30.892700+0000","last_became_active":"2023-11-13T10:55:26.852036+0000","last_became_peered":"2023-11-13T10:55:26.852036+0000","last_unstale":"2023-11-13T10:55:30.892700+0000","last_undegraded":"2023-11-13T10:55:30.892700+0000","last_fullsized":"2023-11-13T10:55:30.892700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:41:12.172895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898608+0000","last_change":"2023-11-13T10:55:25.854134+0000","last_active":"2023-11-13T10:55:30.898608+0000","last_peered":"2023-11-13T10:55:30.898608+0000","last_clean":"2023-11-13T10:55:30.898608+0000","last_became_active":"2023-11-13T10:55:25.853195+0000","last_became_peered":"2023-11-13T10:55:25.853195+0000","last_unstale":"2023-11-13T10:55:30.898608+0000","last_undegraded":"2023-11-13T10:55:30.898608+0000","last_fullsized":"2023-11-13T10:55:30.898608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:09.162069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888654+0000","last_change":"2023-11-13T10:55:30.914433+0000","last_active":"2023-11-13T10:55:31.888654+0000","last_peered":"2023-11-13T10:55:31.888654+0000","last_clean":"2023-11-13T10:55:31.888654+0000","last_became_active":"2023-11-13T10:55:30.913947+0000","last_became_peered":"2023-11-13T10:55:30.913947+0000","last_unstale":"2023-11-13T10:55:31.888654+0000","last_undegraded":"2023-11-13T10:55:31.888654+0000","last_fullsized":"2023-11-13T10:55:31.888654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:22.321232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899143+0000","last_change":"2023-11-13T10:55:26.866642+0000","last_active":"2023-11-13T10:55:30.899143+0000","last_peered":"2023-11-13T10:55:30.899143+0000","last_clean":"2023-11-13T10:55:30.899143+0000","last_became_active":"2023-11-13T10:55:26.866294+0000","last_became_peered":"2023-11-13T10:55:26.866294+0000","last_unstale":"2023-11-13T10:55:30.899143+0000","last_undegraded":"2023-11-13T10:55:30.899143+0000","last_fullsized":"2023-11-13T10:55:30.899143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:01.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898826+0000","last_change":"2023-11-13T10:55:25.848632+0000","last_active":"2023-11-13T10:55:30.898826+0000","last_peered":"2023-11-13T10:55:30.898826+0000","last_clean":"2023-11-13T10:55:30.898826+0000","last_became_active":"2023-11-13T10:55:25.848324+0000","last_became_peered":"2023-11-13T10:55:25.848324+0000","last_unstale":"2023-11-13T10:55:30.898826+0000","last_undegraded":"2023-11-13T10:55:30.898826+0000","last_fullsized":"2023-11-13T10:55:30.898826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:30.550401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905683+0000","last_change":"2023-11-13T10:55:30.931468+0000","last_active":"2023-11-13T10:55:31.905683+0000","last_peered":"2023-11-13T10:55:31.905683+0000","last_clean":"2023-11-13T10:55:31.905683+0000","last_became_active":"2023-11-13T10:55:30.930565+0000","last_became_peered":"2023-11-13T10:55:30.930565+0000","last_unstale":"2023-11-13T10:55:31.905683+0000","last_undegraded":"2023-11-13T10:55:31.905683+0000","last_fullsized":"2023-11-13T10:55:31.905683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:47.860796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912956+0000","last_change":"2023-11-13T10:55:26.866047+0000","last_active":"2023-11-13T10:55:30.912956+0000","last_peered":"2023-11-13T10:55:30.912956+0000","last_clean":"2023-11-13T10:55:30.912956+0000","last_became_active":"2023-11-13T10:55:26.865641+0000","last_became_peered":"2023-11-13T10:55:26.865641+0000","last_unstale":"2023-11-13T10:55:30.912956+0000","last_undegraded":"2023-11-13T10:55:30.912956+0000","last_fullsized":"2023-11-13T10:55:30.912956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:12.028508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911415+0000","last_change":"2023-11-13T10:55:25.860552+0000","last_active":"2023-11-13T10:55:30.911415+0000","last_peered":"2023-11-13T10:55:30.911415+0000","last_clean":"2023-11-13T10:55:30.911415+0000","last_became_active":"2023-11-13T10:55:25.859833+0000","last_became_peered":"2023-11-13T10:55:25.859833+0000","last_unstale":"2023-11-13T10:55:30.911415+0000","last_undegraded":"2023-11-13T10:55:30.911415+0000","last_fullsized":"2023-11-13T10:55:30.911415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:03:57.732657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883273+0000","last_change":"2023-11-13T10:55:30.926972+0000","last_active":"2023-11-13T10:55:31.883273+0000","last_peered":"2023-11-13T10:55:31.883273+0000","last_clean":"2023-11-13T10:55:31.883273+0000","last_became_active":"2023-11-13T10:55:30.926026+0000","last_became_peered":"2023-11-13T10:55:30.926026+0000","last_unstale":"2023-11-13T10:55:31.883273+0000","last_undegraded":"2023-11-13T10:55:31.883273+0000","last_fullsized":"2023-11-13T10:55:31.883273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:41:10.341680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897975+0000","last_change":"2023-11-13T10:55:26.860060+0000","last_active":"2023-11-13T10:55:30.897975+0000","last_peered":"2023-11-13T10:55:30.897975+0000","last_clean":"2023-11-13T10:55:30.897975+0000","last_became_active":"2023-11-13T10:55:26.859793+0000","last_became_peered":"2023-11-13T10:55:26.859793+0000","last_unstale":"2023-11-13T10:55:30.897975+0000","last_undegraded":"2023-11-13T10:55:30.897975+0000","last_fullsized":"2023-11-13T10:55:30.897975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:13.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:45.351786+0000","last_change":"2023-11-13T10:55:25.857501+0000","last_active":"2023-11-13T10:55:45.351786+0000","last_peered":"2023-11-13T10:55:45.351786+0000","last_clean":"2023-11-13T10:55:45.351786+0000","last_became_active":"2023-11-13T10:55:25.853910+0000","last_became_peered":"2023-11-13T10:55:25.853910+0000","last_unstale":"2023-11-13T10:55:45.351786+0000","last_undegraded":"2023-11-13T10:55:45.351786+0000","last_fullsized":"2023-11-13T10:55:45.351786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:41:11.160621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883442+0000","last_change":"2023-11-13T10:55:30.921872+0000","last_active":"2023-11-13T10:55:31.883442+0000","last_peered":"2023-11-13T10:55:31.883442+0000","last_clean":"2023-11-13T10:55:31.883442+0000","last_became_active":"2023-11-13T10:55:30.921430+0000","last_became_peered":"2023-11-13T10:55:30.921430+0000","last_unstale":"2023-11-13T10:55:31.883442+0000","last_undegraded":"2023-11-13T10:55:31.883442+0000","last_fullsized":"2023-11-13T10:55:31.883442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:25.376399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899011+0000","last_change":"2023-11-13T10:55:26.870526+0000","last_active":"2023-11-13T10:55:30.899011+0000","last_peered":"2023-11-13T10:55:30.899011+0000","last_clean":"2023-11-13T10:55:30.899011+0000","last_became_active":"2023-11-13T10:55:26.870044+0000","last_became_peered":"2023-11-13T10:55:26.870044+0000","last_unstale":"2023-11-13T10:55:30.899011+0000","last_undegraded":"2023-11-13T10:55:30.899011+0000","last_fullsized":"2023-11-13T10:55:30.899011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:05:27.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.915525+0000","last_change":"2023-11-13T10:55:25.863136+0000","last_active":"2023-11-13T10:55:30.915525+0000","last_peered":"2023-11-13T10:55:30.915525+0000","last_clean":"2023-11-13T10:55:30.915525+0000","last_became_active":"2023-11-13T10:55:25.862731+0000","last_became_peered":"2023-11-13T10:55:25.862731+0000","last_unstale":"2023-11-13T10:55:30.915525+0000","last_undegraded":"2023-11-13T10:55:30.915525+0000","last_fullsized":"2023-11-13T10:55:30.915525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:50.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905039+0000","last_change":"2023-11-13T10:55:30.920930+0000","last_active":"2023-11-13T10:55:31.905039+0000","last_peered":"2023-11-13T10:55:31.905039+0000","last_clean":"2023-11-13T10:55:31.905039+0000","last_became_active":"2023-11-13T10:55:30.920498+0000","last_became_peered":"2023-11-13T10:55:30.920498+0000","last_unstale":"2023-11-13T10:55:31.905039+0000","last_undegraded":"2023-11-13T10:55:31.905039+0000","last_fullsized":"2023-11-13T10:55:31.905039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:49.941362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912962+0000","last_change":"2023-11-13T10:55:26.860086+0000","last_active":"2023-11-13T10:55:30.912962+0000","last_peered":"2023-11-13T10:55:30.912962+0000","last_clean":"2023-11-13T10:55:30.912962+0000","last_became_active":"2023-11-13T10:55:26.859816+0000","last_became_peered":"2023-11-13T10:55:26.859816+0000","last_unstale":"2023-11-13T10:55:30.912962+0000","last_undegraded":"2023-11-13T10:55:30.912962+0000","last_fullsized":"2023-11-13T10:55:30.912962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902678+0000","last_change":"2023-11-13T10:55:25.846097+0000","last_active":"2023-11-13T10:55:30.902678+0000","last_peered":"2023-11-13T10:55:30.902678+0000","last_clean":"2023-11-13T10:55:30.902678+0000","last_became_active":"2023-11-13T10:55:25.845770+0000","last_became_peered":"2023-11-13T10:55:25.845770+0000","last_unstale":"2023-11-13T10:55:30.902678+0000","last_undegraded":"2023-11-13T10:55:30.902678+0000","last_fullsized":"2023-11-13T10:55:30.902678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:16.023464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906043+0000","last_change":"2023-11-13T10:55:30.925552+0000","last_active":"2023-11-13T10:55:31.906043+0000","last_peered":"2023-11-13T10:55:31.906043+0000","last_clean":"2023-11-13T10:55:31.906043+0000","last_became_active":"2023-11-13T10:55:30.925120+0000","last_became_peered":"2023-11-13T10:55:30.925120+0000","last_unstale":"2023-11-13T10:55:31.906043+0000","last_undegraded":"2023-11-13T10:55:31.906043+0000","last_fullsized":"2023-11-13T10:55:31.906043+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:54:40.287372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886993+0000","last_change":"2023-11-13T10:55:26.876001+0000","last_active":"2023-11-13T10:55:30.886993+0000","last_peered":"2023-11-13T10:55:30.886993+0000","last_clean":"2023-11-13T10:55:30.886993+0000","last_became_active":"2023-11-13T10:55:26.875523+0000","last_became_peered":"2023-11-13T10:55:26.875523+0000","last_unstale":"2023-11-13T10:55:30.886993+0000","last_undegraded":"2023-11-13T10:55:30.886993+0000","last_fullsized":"2023-11-13T10:55:30.886993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:17.901362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896671+0000","last_change":"2023-11-13T10:55:25.848415+0000","last_active":"2023-11-13T10:55:30.896671+0000","last_peered":"2023-11-13T10:55:30.896671+0000","last_clean":"2023-11-13T10:55:30.896671+0000","last_became_active":"2023-11-13T10:55:25.848102+0000","last_became_peered":"2023-11-13T10:55:25.848102+0000","last_unstale":"2023-11-13T10:55:30.896671+0000","last_undegraded":"2023-11-13T10:55:30.896671+0000","last_fullsized":"2023-11-13T10:55:30.896671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T10:56:29.442219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883284+0000","last_change":"2023-11-13T10:55:30.918365+0000","last_active":"2023-11-13T10:55:31.883284+0000","last_peered":"2023-11-13T10:55:31.883284+0000","last_clean":"2023-11-13T10:55:31.883284+0000","last_became_active":"2023-11-13T10:55:30.917240+0000","last_became_peered":"2023-11-13T10:55:30.917240+0000","last_unstale":"2023-11-13T10:55:31.883284+0000","last_undegraded":"2023-11-13T10:55:31.883284+0000","last_fullsized":"2023-11-13T10:55:31.883284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884505+0000","last_change":"2023-11-13T10:55:30.924912+0000","last_active":"2023-11-13T10:55:31.884505+0000","last_peered":"2023-11-13T10:55:31.884505+0000","last_clean":"2023-11-13T10:55:31.884505+0000","last_became_active":"2023-11-13T10:55:30.919446+0000","last_became_peered":"2023-11-13T10:55:30.919446+0000","last_unstale":"2023-11-13T10:55:31.884505+0000","last_undegraded":"2023-11-13T10:55:31.884505+0000","last_fullsized":"2023-11-13T10:55:31.884505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:31.924677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892009+0000","last_change":"2023-11-13T10:55:26.852233+0000","last_active":"2023-11-13T10:55:30.892009+0000","last_peered":"2023-11-13T10:55:30.892009+0000","last_clean":"2023-11-13T10:55:30.892009+0000","last_became_active":"2023-11-13T10:55:26.849611+0000","last_became_peered":"2023-11-13T10:55:26.849611+0000","last_unstale":"2023-11-13T10:55:30.892009+0000","last_undegraded":"2023-11-13T10:55:30.892009+0000","last_fullsized":"2023-11-13T10:55:30.892009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:53.364560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898832+0000","last_change":"2023-11-13T10:55:25.856192+0000","last_active":"2023-11-13T10:55:30.898832+0000","last_peered":"2023-11-13T10:55:30.898832+0000","last_clean":"2023-11-13T10:55:30.898832+0000","last_became_active":"2023-11-13T10:55:25.852741+0000","last_became_peered":"2023-11-13T10:55:25.852741+0000","last_unstale":"2023-11-13T10:55:30.898832+0000","last_undegraded":"2023-11-13T10:55:30.898832+0000","last_fullsized":"2023-11-13T10:55:30.898832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:15.762181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884097+0000","last_change":"2023-11-13T10:55:30.910075+0000","last_active":"2023-11-13T10:55:31.884097+0000","last_peered":"2023-11-13T10:55:31.884097+0000","last_clean":"2023-11-13T10:55:31.884097+0000","last_became_active":"2023-11-13T10:55:30.909468+0000","last_became_peered":"2023-11-13T10:55:30.909468+0000","last_unstale":"2023-11-13T10:55:31.884097+0000","last_undegraded":"2023-11-13T10:55:31.884097+0000","last_fullsized":"2023-11-13T10:55:31.884097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:04:21.045625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903339+0000","last_change":"2023-11-13T10:55:26.856127+0000","last_active":"2023-11-13T10:55:30.903339+0000","last_peered":"2023-11-13T10:55:30.903339+0000","last_clean":"2023-11-13T10:55:30.903339+0000","last_became_active":"2023-11-13T10:55:26.855805+0000","last_became_peered":"2023-11-13T10:55:26.855805+0000","last_unstale":"2023-11-13T10:55:30.903339+0000","last_undegraded":"2023-11-13T10:55:30.903339+0000","last_fullsized":"2023-11-13T10:55:30.903339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:13.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911772+0000","last_change":"2023-11-13T10:55:25.859736+0000","last_active":"2023-11-13T10:55:30.911772+0000","last_peered":"2023-11-13T10:55:30.911772+0000","last_clean":"2023-11-13T10:55:30.911772+0000","last_became_active":"2023-11-13T10:55:25.859335+0000","last_became_peered":"2023-11-13T10:55:25.859335+0000","last_unstale":"2023-11-13T10:55:30.911772+0000","last_undegraded":"2023-11-13T10:55:30.911772+0000","last_fullsized":"2023-11-13T10:55:30.911772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:24.061923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898735+0000","last_change":"2023-11-13T10:55:25.847310+0000","last_active":"2023-11-13T10:55:30.898735+0000","last_peered":"2023-11-13T10:55:30.898735+0000","last_clean":"2023-11-13T10:55:30.898735+0000","last_became_active":"2023-11-13T10:55:25.846471+0000","last_became_peered":"2023-11-13T10:55:25.846471+0000","last_unstale":"2023-11-13T10:55:30.898735+0000","last_undegraded":"2023-11-13T10:55:30.898735+0000","last_fullsized":"2023-11-13T10:55:30.898735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:57.678774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894953+0000","last_change":"2023-11-13T10:55:26.865930+0000","last_active":"2023-11-13T10:55:30.894953+0000","last_peered":"2023-11-13T10:55:30.894953+0000","last_clean":"2023-11-13T10:55:30.894953+0000","last_became_active":"2023-11-13T10:55:26.865603+0000","last_became_peered":"2023-11-13T10:55:26.865603+0000","last_unstale":"2023-11-13T10:55:30.894953+0000","last_undegraded":"2023-11-13T10:55:30.894953+0000","last_fullsized":"2023-11-13T10:55:30.894953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:31.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884103+0000","last_change":"2023-11-13T10:55:30.922058+0000","last_active":"2023-11-13T10:55:31.884103+0000","last_peered":"2023-11-13T10:55:31.884103+0000","last_clean":"2023-11-13T10:55:31.884103+0000","last_became_active":"2023-11-13T10:55:30.921389+0000","last_became_peered":"2023-11-13T10:55:30.921389+0000","last_unstale":"2023-11-13T10:55:31.884103+0000","last_undegraded":"2023-11-13T10:55:31.884103+0000","last_fullsized":"2023-11-13T10:55:31.884103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:54.585715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909071+0000","last_change":"2023-11-13T10:55:25.859601+0000","last_active":"2023-11-13T10:55:30.909071+0000","last_peered":"2023-11-13T10:55:30.909071+0000","last_clean":"2023-11-13T10:55:30.909071+0000","last_became_active":"2023-11-13T10:55:25.859320+0000","last_became_peered":"2023-11-13T10:55:25.859320+0000","last_unstale":"2023-11-13T10:55:30.909071+0000","last_undegraded":"2023-11-13T10:55:30.909071+0000","last_fullsized":"2023-11-13T10:55:30.909071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.901962+0000","last_change":"2023-11-13T10:55:26.844138+0000","last_active":"2023-11-13T10:55:30.901962+0000","last_peered":"2023-11-13T10:55:30.901962+0000","last_clean":"2023-11-13T10:55:30.901962+0000","last_became_active":"2023-11-13T10:55:26.843850+0000","last_became_peered":"2023-11-13T10:55:26.843850+0000","last_unstale":"2023-11-13T10:55:30.901962+0000","last_undegraded":"2023-11-13T10:55:30.901962+0000","last_fullsized":"2023-11-13T10:55:30.901962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:43.105401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.907248+0000","last_change":"2023-11-13T10:55:30.929718+0000","last_active":"2023-11-13T10:55:31.907248+0000","last_peered":"2023-11-13T10:55:31.907248+0000","last_clean":"2023-11-13T10:55:31.907248+0000","last_became_active":"2023-11-13T10:55:30.929223+0000","last_became_peered":"2023-11-13T10:55:30.929223+0000","last_unstale":"2023-11-13T10:55:31.907248+0000","last_undegraded":"2023-11-13T10:55:31.907248+0000","last_fullsized":"2023-11-13T10:55:31.907248+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:28:22.239712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886249+0000","last_change":"2023-11-13T10:55:26.858768+0000","last_active":"2023-11-13T10:55:30.886249+0000","last_peered":"2023-11-13T10:55:30.886249+0000","last_clean":"2023-11-13T10:55:30.886249+0000","last_became_active":"2023-11-13T10:55:26.858493+0000","last_became_peered":"2023-11-13T10:55:26.858493+0000","last_unstale":"2023-11-13T10:55:30.886249+0000","last_undegraded":"2023-11-13T10:55:30.886249+0000","last_fullsized":"2023-11-13T10:55:30.886249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:25.074197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911186+0000","last_change":"2023-11-13T10:55:26.866587+0000","last_active":"2023-11-13T10:55:30.911186+0000","last_peered":"2023-11-13T10:55:30.911186+0000","last_clean":"2023-11-13T10:55:30.911186+0000","last_became_active":"2023-11-13T10:55:26.866277+0000","last_became_peered":"2023-11-13T10:55:26.866277+0000","last_unstale":"2023-11-13T10:55:30.911186+0000","last_undegraded":"2023-11-13T10:55:30.911186+0000","last_fullsized":"2023-11-13T10:55:30.911186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:49:42.713290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898545+0000","last_change":"2023-11-13T10:55:26.869354+0000","last_active":"2023-11-13T10:55:30.898545+0000","last_peered":"2023-11-13T10:55:30.898545+0000","last_clean":"2023-11-13T10:55:30.898545+0000","last_became_active":"2023-11-13T10:55:26.869040+0000","last_became_peered":"2023-11-13T10:55:26.869040+0000","last_unstale":"2023-11-13T10:55:30.898545+0000","last_undegraded":"2023-11-13T10:55:30.898545+0000","last_fullsized":"2023-11-13T10:55:30.898545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:50.524863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.908909+0000","last_change":"2023-11-13T10:55:26.861905+0000","last_active":"2023-11-13T10:55:30.908909+0000","last_peered":"2023-11-13T10:55:30.908909+0000","last_clean":"2023-11-13T10:55:30.908909+0000","last_became_active":"2023-11-13T10:55:26.861613+0000","last_became_peered":"2023-11-13T10:55:26.861613+0000","last_unstale":"2023-11-13T10:55:30.908909+0000","last_undegraded":"2023-11-13T10:55:30.908909+0000","last_fullsized":"2023-11-13T10:55:30.908909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:31.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913352+0000","last_change":"2023-11-13T10:55:26.873181+0000","last_active":"2023-11-13T10:55:30.913352+0000","last_peered":"2023-11-13T10:55:30.913352+0000","last_clean":"2023-11-13T10:55:30.913352+0000","last_became_active":"2023-11-13T10:55:26.872868+0000","last_became_peered":"2023-11-13T10:55:26.872868+0000","last_unstale":"2023-11-13T10:55:30.913352+0000","last_undegraded":"2023-11-13T10:55:30.913352+0000","last_fullsized":"2023-11-13T10:55:30.913352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:10.174343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.915906+0000","last_change":"2023-11-13T10:55:26.855441+0000","last_active":"2023-11-13T10:55:30.915906+0000","last_peered":"2023-11-13T10:55:30.915906+0000","last_clean":"2023-11-13T10:55:30.915906+0000","last_became_active":"2023-11-13T10:55:26.855101+0000","last_became_peered":"2023-11-13T10:55:26.855101+0000","last_unstale":"2023-11-13T10:55:30.915906+0000","last_undegraded":"2023-11-13T10:55:30.915906+0000","last_fullsized":"2023-11-13T10:55:30.915906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:53.364560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886903+0000","last_change":"2023-11-13T10:55:26.864741+0000","last_active":"2023-11-13T10:55:30.886903+0000","last_peered":"2023-11-13T10:55:30.886903+0000","last_clean":"2023-11-13T10:55:30.886903+0000","last_became_active":"2023-11-13T10:55:26.863408+0000","last_became_peered":"2023-11-13T10:55:26.863408+0000","last_unstale":"2023-11-13T10:55:30.886903+0000","last_undegraded":"2023-11-13T10:55:30.886903+0000","last_fullsized":"2023-11-13T10:55:30.886903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:12.028508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912752+0000","last_change":"2023-11-13T10:55:26.856400+0000","last_active":"2023-11-13T10:55:30.912752+0000","last_peered":"2023-11-13T10:55:30.912752+0000","last_clean":"2023-11-13T10:55:30.912752+0000","last_became_active":"2023-11-13T10:55:26.853838+0000","last_became_peered":"2023-11-13T10:55:26.853838+0000","last_unstale":"2023-11-13T10:55:30.912752+0000","last_undegraded":"2023-11-13T10:55:30.912752+0000","last_fullsized":"2023-11-13T10:55:30.912752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:45.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898736+0000","last_change":"2023-11-13T10:55:26.854049+0000","last_active":"2023-11-13T10:55:30.898736+0000","last_peered":"2023-11-13T10:55:30.898736+0000","last_clean":"2023-11-13T10:55:30.898736+0000","last_became_active":"2023-11-13T10:55:26.853767+0000","last_became_peered":"2023-11-13T10:55:26.853767+0000","last_unstale":"2023-11-13T10:55:30.898736+0000","last_undegraded":"2023-11-13T10:55:30.898736+0000","last_fullsized":"2023-11-13T10:55:30.898736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903782+0000","last_change":"2023-11-13T10:55:26.855717+0000","last_active":"2023-11-13T10:55:30.903782+0000","last_peered":"2023-11-13T10:55:30.903782+0000","last_clean":"2023-11-13T10:55:30.903782+0000","last_became_active":"2023-11-13T10:55:26.853617+0000","last_became_peered":"2023-11-13T10:55:26.853617+0000","last_unstale":"2023-11-13T10:55:30.903782+0000","last_undegraded":"2023-11-13T10:55:30.903782+0000","last_fullsized":"2023-11-13T10:55:30.903782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.895441+0000","last_change":"2023-11-13T10:55:26.857625+0000","last_active":"2023-11-13T10:55:30.895441+0000","last_peered":"2023-11-13T10:55:30.895441+0000","last_clean":"2023-11-13T10:55:30.895441+0000","last_became_active":"2023-11-13T10:55:26.857214+0000","last_became_peered":"2023-11-13T10:55:26.857214+0000","last_unstale":"2023-11-13T10:55:30.895441+0000","last_undegraded":"2023-11-13T10:55:30.895441+0000","last_fullsized":"2023-11-13T10:55:30.895441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:06.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894989+0000","last_change":"2023-11-13T10:55:26.870935+0000","last_active":"2023-11-13T10:55:30.894989+0000","last_peered":"2023-11-13T10:55:30.894989+0000","last_clean":"2023-11-13T10:55:30.894989+0000","last_became_active":"2023-11-13T10:55:26.870322+0000","last_became_peered":"2023-11-13T10:55:26.870322+0000","last_unstale":"2023-11-13T10:55:30.894989+0000","last_undegraded":"2023-11-13T10:55:30.894989+0000","last_fullsized":"2023-11-13T10:55:30.894989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:17.035738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900412+0000","last_change":"2023-11-13T10:55:26.867707+0000","last_active":"2023-11-13T10:55:30.900412+0000","last_peered":"2023-11-13T10:55:30.900412+0000","last_clean":"2023-11-13T10:55:30.900412+0000","last_became_active":"2023-11-13T10:55:26.864751+0000","last_became_peered":"2023-11-13T10:55:26.864751+0000","last_unstale":"2023-11-13T10:55:30.900412+0000","last_undegraded":"2023-11-13T10:55:30.900412+0000","last_fullsized":"2023-11-13T10:55:30.900412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:43.799944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898242+0000","last_change":"2023-11-13T10:55:26.855923+0000","last_active":"2023-11-13T10:55:30.898242+0000","last_peered":"2023-11-13T10:55:30.898242+0000","last_clean":"2023-11-13T10:55:30.898242+0000","last_became_active":"2023-11-13T10:55:26.855638+0000","last_became_peered":"2023-11-13T10:55:26.855638+0000","last_unstale":"2023-11-13T10:55:30.898242+0000","last_undegraded":"2023-11-13T10:55:30.898242+0000","last_fullsized":"2023-11-13T10:55:30.898242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:40:26.611005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.916348+0000","last_change":"2023-11-13T10:55:26.870707+0000","last_active":"2023-11-13T10:55:30.916348+0000","last_peered":"2023-11-13T10:55:30.916348+0000","last_clean":"2023-11-13T10:55:30.916348+0000","last_became_active":"2023-11-13T10:55:26.870417+0000","last_became_peered":"2023-11-13T10:55:26.870417+0000","last_unstale":"2023-11-13T10:55:30.916348+0000","last_undegraded":"2023-11-13T10:55:30.916348+0000","last_fullsized":"2023-11-13T10:55:30.916348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:24:16.048814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899902+0000","last_change":"2023-11-13T10:55:26.873122+0000","last_active":"2023-11-13T10:55:30.899902+0000","last_peered":"2023-11-13T10:55:30.899902+0000","last_clean":"2023-11-13T10:55:30.899902+0000","last_became_active":"2023-11-13T10:55:26.872823+0000","last_became_peered":"2023-11-13T10:55:26.872823+0000","last_unstale":"2023-11-13T10:55:30.899902+0000","last_undegraded":"2023-11-13T10:55:30.899902+0000","last_fullsized":"2023-11-13T10:55:30.899902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:09:58.900197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898638+0000","last_change":"2023-11-13T10:55:26.869223+0000","last_active":"2023-11-13T10:55:30.898638+0000","last_peered":"2023-11-13T10:55:30.898638+0000","last_clean":"2023-11-13T10:55:30.898638+0000","last_became_active":"2023-11-13T10:55:26.868919+0000","last_became_peered":"2023-11-13T10:55:26.868919+0000","last_unstale":"2023-11-13T10:55:30.898638+0000","last_undegraded":"2023-11-13T10:55:30.898638+0000","last_fullsized":"2023-11-13T10:55:30.898638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:57:13.609895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903525+0000","last_change":"2023-11-13T10:55:26.865259+0000","last_active":"2023-11-13T10:55:30.903525+0000","last_peered":"2023-11-13T10:55:30.903525+0000","last_clean":"2023-11-13T10:55:30.903525+0000","last_became_active":"2023-11-13T10:55:26.864675+0000","last_became_peered":"2023-11-13T10:55:26.864675+0000","last_unstale":"2023-11-13T10:55:30.903525+0000","last_undegraded":"2023-11-13T10:55:30.903525+0000","last_fullsized":"2023-11-13T10:55:30.903525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:17.035738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912854+0000","last_change":"2023-11-13T10:55:26.863002+0000","last_active":"2023-11-13T10:55:30.912854+0000","last_peered":"2023-11-13T10:55:30.912854+0000","last_clean":"2023-11-13T10:55:30.912854+0000","last_became_active":"2023-11-13T10:55:26.862257+0000","last_became_peered":"2023-11-13T10:55:26.862257+0000","last_unstale":"2023-11-13T10:55:30.912854+0000","last_undegraded":"2023-11-13T10:55:30.912854+0000","last_fullsized":"2023-11-13T10:55:30.912854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:17.901362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.914921+0000","last_change":"2023-11-13T10:55:26.872307+0000","last_active":"2023-11-13T10:55:30.914921+0000","last_peered":"2023-11-13T10:55:30.914921+0000","last_clean":"2023-11-13T10:55:30.914921+0000","last_became_active":"2023-11-13T10:55:26.872087+0000","last_became_peered":"2023-11-13T10:55:26.872087+0000","last_unstale":"2023-11-13T10:55:30.914921+0000","last_undegraded":"2023-11-13T10:55:30.914921+0000","last_fullsized":"2023-11-13T10:55:30.914921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:22.528830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900326+0000","last_change":"2023-11-13T10:55:26.872483+0000","last_active":"2023-11-13T10:55:30.900326+0000","last_peered":"2023-11-13T10:55:30.900326+0000","last_clean":"2023-11-13T10:55:30.900326+0000","last_became_active":"2023-11-13T10:55:26.872186+0000","last_became_peered":"2023-11-13T10:55:26.872186+0000","last_unstale":"2023-11-13T10:55:30.900326+0000","last_undegraded":"2023-11-13T10:55:30.900326+0000","last_fullsized":"2023-11-13T10:55:30.900326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:47.501708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897861+0000","last_change":"2023-11-13T10:55:26.862513+0000","last_active":"2023-11-13T10:55:30.897861+0000","last_peered":"2023-11-13T10:55:30.897861+0000","last_clean":"2023-11-13T10:55:30.897861+0000","last_became_active":"2023-11-13T10:55:26.862167+0000","last_became_peered":"2023-11-13T10:55:26.862167+0000","last_unstale":"2023-11-13T10:55:30.897861+0000","last_undegraded":"2023-11-13T10:55:30.897861+0000","last_fullsized":"2023-11-13T10:55:30.897861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902597+0000","last_change":"2023-11-13T10:55:26.861425+0000","last_active":"2023-11-13T10:55:30.902597+0000","last_peered":"2023-11-13T10:55:30.902597+0000","last_clean":"2023-11-13T10:55:30.902597+0000","last_became_active":"2023-11-13T10:55:26.861132+0000","last_became_peered":"2023-11-13T10:55:26.861132+0000","last_unstale":"2023-11-13T10:55:30.902597+0000","last_undegraded":"2023-11-13T10:55:30.902597+0000","last_fullsized":"2023-11-13T10:55:30.902597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:29.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896561+0000","last_change":"2023-11-13T10:55:26.858332+0000","last_active":"2023-11-13T10:55:30.896561+0000","last_peered":"2023-11-13T10:55:30.896561+0000","last_clean":"2023-11-13T10:55:30.896561+0000","last_became_active":"2023-11-13T10:55:26.858052+0000","last_became_peered":"2023-11-13T10:55:26.858052+0000","last_unstale":"2023-11-13T10:55:30.896561+0000","last_undegraded":"2023-11-13T10:55:30.896561+0000","last_fullsized":"2023-11-13T10:55:30.896561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903167+0000","last_change":"2023-11-13T10:55:26.863579+0000","last_active":"2023-11-13T10:55:30.903167+0000","last_peered":"2023-11-13T10:55:30.903167+0000","last_clean":"2023-11-13T10:55:30.903167+0000","last_became_active":"2023-11-13T10:55:26.863247+0000","last_became_peered":"2023-11-13T10:55:26.863247+0000","last_unstale":"2023-11-13T10:55:30.903167+0000","last_undegraded":"2023-11-13T10:55:30.903167+0000","last_fullsized":"2023-11-13T10:55:30.903167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.774455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898503+0000","last_change":"2023-11-13T10:55:26.856219+0000","last_active":"2023-11-13T10:55:30.898503+0000","last_peered":"2023-11-13T10:55:30.898503+0000","last_clean":"2023-11-13T10:55:30.898503+0000","last_became_active":"2023-11-13T10:55:26.855933+0000","last_became_peered":"2023-11-13T10:55:26.855933+0000","last_unstale":"2023-11-13T10:55:30.898503+0000","last_undegraded":"2023-11-13T10:55:30.898503+0000","last_fullsized":"2023-11-13T10:55:30.898503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887319+0000","last_change":"2023-11-13T10:55:26.853802+0000","last_active":"2023-11-13T10:55:30.887319+0000","last_peered":"2023-11-13T10:55:30.887319+0000","last_clean":"2023-11-13T10:55:30.887319+0000","last_became_active":"2023-11-13T10:55:26.853463+0000","last_became_peered":"2023-11-13T10:55:26.853463+0000","last_unstale":"2023-11-13T10:55:30.887319+0000","last_undegraded":"2023-11-13T10:55:30.887319+0000","last_fullsized":"2023-11-13T10:55:30.887319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:06.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902537+0000","last_change":"2023-11-13T10:55:26.865339+0000","last_active":"2023-11-13T10:55:30.902537+0000","last_peered":"2023-11-13T10:55:30.902537+0000","last_clean":"2023-11-13T10:55:30.902537+0000","last_became_active":"2023-11-13T10:55:26.865069+0000","last_became_peered":"2023-11-13T10:55:26.865069+0000","last_unstale":"2023-11-13T10:55:30.902537+0000","last_undegraded":"2023-11-13T10:55:30.902537+0000","last_fullsized":"2023-11-13T10:55:30.902537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:50.524863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887342+0000","last_change":"2023-11-13T10:55:26.857434+0000","last_active":"2023-11-13T10:55:30.887342+0000","last_peered":"2023-11-13T10:55:30.887342+0000","last_clean":"2023-11-13T10:55:30.887342+0000","last_became_active":"2023-11-13T10:55:26.857164+0000","last_became_peered":"2023-11-13T10:55:26.857164+0000","last_unstale":"2023-11-13T10:55:30.887342+0000","last_undegraded":"2023-11-13T10:55:30.887342+0000","last_fullsized":"2023-11-13T10:55:30.887342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909271+0000","last_change":"2023-11-13T10:55:26.845954+0000","last_active":"2023-11-13T10:55:30.909271+0000","last_peered":"2023-11-13T10:55:30.909271+0000","last_clean":"2023-11-13T10:55:30.909271+0000","last_became_active":"2023-11-13T10:55:26.845505+0000","last_became_peered":"2023-11-13T10:55:26.845505+0000","last_unstale":"2023-11-13T10:55:30.909271+0000","last_undegraded":"2023-11-13T10:55:30.909271+0000","last_fullsized":"2023-11-13T10:55:30.909271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:43.799944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911061+0000","last_change":"2023-11-13T10:55:26.857565+0000","last_active":"2023-11-13T10:55:30.911061+0000","last_peered":"2023-11-13T10:55:30.911061+0000","last_clean":"2023-11-13T10:55:30.911061+0000","last_became_active":"2023-11-13T10:55:26.857291+0000","last_became_peered":"2023-11-13T10:55:26.857291+0000","last_unstale":"2023-11-13T10:55:30.911061+0000","last_undegraded":"2023-11-13T10:55:30.911061+0000","last_fullsized":"2023-11-13T10:55:30.911061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:29.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902086+0000","last_change":"2023-11-13T10:55:26.858620+0000","last_active":"2023-11-13T10:55:30.902086+0000","last_peered":"2023-11-13T10:55:30.902086+0000","last_clean":"2023-11-13T10:55:30.902086+0000","last_became_active":"2023-11-13T10:55:26.858348+0000","last_became_peered":"2023-11-13T10:55:26.858348+0000","last_unstale":"2023-11-13T10:55:30.902086+0000","last_undegraded":"2023-11-13T10:55:30.902086+0000","last_fullsized":"2023-11-13T10:55:30.902086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:45.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.466}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]}]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":695602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]}]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":109655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5209999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":105724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.843}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0339999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0830000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":108983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]}]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":695580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.46}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T10:55:56.130 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-13T10:55:56.131 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-13T10:55:56.131 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-13T10:55:56.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-13T10:55:56.133 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-13T10:55:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.392+0000 7efc8758c640 1 Processor -- start 2023-11-13T10:55:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.393+0000 7efc8758c640 1 -- start start 2023-11-13T10:55:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.394+0000 7efc8758c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc80161060 0x7efc801621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.394+0000 7efc8758c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc8006f970 0x7efc80162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.394+0000 7efc8758c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc80162700 con 0x7efc8006a190 2023-11-13T10:55:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.394+0000 7efc8758c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc800b1840 con 0x7efc80161060 2023-11-13T10:55:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.394+0000 7efc8758c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc80071600 con 0x7efc8006f970 2023-11-13T10:55:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc86d8b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efc8006a190 0x7efc800b1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34472/0 (socket says 172.21.15.16:34472) 2023-11-13T10:55:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc86d8b640 1 -- 172.21.15.16:0/2274110956 learned_addr learned my addr 172.21.15.16:0/2274110956 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85d89640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc8006f970 0x7efc80162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc8658a640 1 --2- 172.21.15.16:0/2274110956 >> v2:172.21.15.184:3300/0 conn(0x7efc80161060 0x7efc801621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426042876 0 0) 0x7efc80162700 con 0x7efc8006a190 2023-11-13T10:55:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc6c003520 con 0x7efc8006a190 2023-11-13T10:55:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029670913 0 0) 0x7efc6c003520 con 0x7efc8006a190 2023-11-13T10:55:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc8006f970 msgr2=0x7efc80162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85588640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc8006f970 0x7efc80162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 >> v2:172.21.15.184:3300/0 conn(0x7efc80161060 msgr2=0x7efc801621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:56.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85588640 1 --2- 172.21.15.16:0/2274110956 >> v2:172.21.15.184:3300/0 conn(0x7efc80161060 0x7efc801621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800718e0 con 0x7efc8006a190 2023-11-13T10:55:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.395+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7efc780019f0 con 0x7efc8006a190 2023-11-13T10:55:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 >> v1:172.21.15.16:6789/0 conn(0x7efc8006a190 legacy=0x7efc800b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc85588640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc6c003970 0x7efc6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc85588640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc6c0047c0 0x7efc6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc85588640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c005180 0x7efc6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc800b1840 con 0x7efc6c005180 2023-11-13T10:55:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc80071600 con 0x7efc6c003970 2023-11-13T10:55:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c00bf40 con 0x7efc6c0047c0 2023-11-13T10:55:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc85d89640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc8006f970 0x7efc80162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc86d8b640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c005180 0x7efc6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc8658a640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc6c003970 0x7efc6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc86d8b640 1 -- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc6c0047c0 msgr2=0x7efc6c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc86d8b640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc6c0047c0 0x7efc6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc86d8b640 1 -- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc6c003970 msgr2=0x7efc6c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc86d8b640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc6c003970 0x7efc6c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.396+0000 7efc86d8b640 1 -- 172.21.15.16:0/2274110956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c003520 con 0x7efc6c005180 2023-11-13T10:55:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.397+0000 7efc86d8b640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c005180 0x7efc6c00b980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efc7800a790 tx=0x7efc6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f617773524df9e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78034630 con 0x7efc6c005180 2023-11-13T10:55:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7802f070 con 0x7efc6c005180 2023-11-13T10:55:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc85588640 1 -- 172.21.15.16:0/2274110956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780349f0 con 0x7efc6c005180 2023-11-13T10:55:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc8758c640 1 -- 172.21.15.16:0/2274110956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c005180 msgr2=0x7efc6c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc8758c640 1 --2- 172.21.15.16:0/2274110956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c005180 0x7efc6c00b980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efc7800a790 tx=0x7efc6c0057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc86d8b640 1 -- 172.21.15.16:0/2274110956 reap_dead start 2023-11-13T10:55:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc8758c640 1 -- 172.21.15.16:0/2274110956 shutdown_connections 2023-11-13T10:55:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc8758c640 1 -- 172.21.15.16:0/2274110956 >> 172.21.15.16:0/2274110956 conn(0x7efc800b04a0 msgr2=0x7efc800b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc8758c640 1 -- 172.21.15.16:0/2274110956 shutdown_connections 2023-11-13T10:55:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.398+0000 7efc8758c640 1 -- 172.21.15.16:0/2274110956 wait complete. 2023-11-13T10:55:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.400+0000 7efc8758c640 1 Processor -- start 2023-11-13T10:55:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.400+0000 7efc8758c640 1 -- start start 2023-11-13T10:55:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.401+0000 7efc8758c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc6c005180 0x7efc8015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.401+0000 7efc8758c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc80152650 0x7efc80153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.401+0000 7efc8758c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80154100 0x7efc801628e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.401+0000 7efc8758c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc801643e0 con 0x7efc80154100 2023-11-13T10:55:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.401+0000 7efc8758c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc80156ff0 con 0x7efc80152650 2023-11-13T10:55:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.401+0000 7efc8758c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc80154580 con 0x7efc6c005180 2023-11-13T10:55:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc86d8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80154100 0x7efc801628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc86d8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80154100 0x7efc801628e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55928/0 (socket says 172.21.15.16:55928) 2023-11-13T10:55:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc86d8b640 1 -- 172.21.15.16:0/4010418095 learned_addr learned my addr 172.21.15.16:0/4010418095 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc85d89640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc80152650 0x7efc80153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc8658a640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc6c005180 0x7efc8015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc86d8b640 1 -- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc6c005180 msgr2=0x7efc8015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc86d8b640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc6c005180 0x7efc8015ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc86d8b640 1 -- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc80152650 msgr2=0x7efc80153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc86d8b640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc80152650 0x7efc80153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc86d8b640 1 -- 172.21.15.16:0/4010418095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc80163e20 con 0x7efc80154100 2023-11-13T10:55:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc85d89640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc80152650 0x7efc80153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:55:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.402+0000 7efc86d8b640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80154100 0x7efc801628e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc7802cba0 tx=0x7efc78001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.403+0000 7efc76ffd640 1 -- 172.21.15.16:0/4010418095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78034840 con 0x7efc80154100 2023-11-13T10:55:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.403+0000 7efc76ffd640 1 -- 172.21.15.16:0/4010418095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc780349a0 con 0x7efc80154100 2023-11-13T10:55:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.403+0000 7efc76ffd640 1 -- 172.21.15.16:0/4010418095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78034d30 con 0x7efc80154100 2023-11-13T10:55:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.403+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc80164100 con 0x7efc80154100 2023-11-13T10:55:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.403+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc80152fc0 con 0x7efc80154100 2023-11-13T10:55:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.404+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc80156a70 con 0x7efc80154100 2023-11-13T10:55:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.405+0000 7efc76ffd640 1 -- 172.21.15.16:0/4010418095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc78031030 con 0x7efc80154100 2023-11-13T10:55:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.406+0000 7efc76ffd640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6406ae70 0x7efc6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.406+0000 7efc76ffd640 1 -- 172.21.15.16:0/4010418095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efc780abef0 con 0x7efc80154100 2023-11-13T10:55:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.407+0000 7efc8658a640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6406ae70 0x7efc6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.408+0000 7efc8658a640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6406ae70 0x7efc6406d330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009990 tx=0x7efc7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:56.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.410+0000 7efc76ffd640 1 -- 172.21.15.16:0/4010418095 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc80156a70 con 0x7efc80154100 2023-11-13T10:55:56.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.657+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efc80157130 con 0x7efc80154100 2023-11-13T10:55:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.666+0000 7efc76ffd640 1 -- 172.21.15.16:0/4010418095 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171039 (secure 0 0 0) 0x7efc80157130 con 0x7efc80154100 2023-11-13T10:55:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6406ae70 msgr2=0x7efc6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6406ae70 0x7efc6406d330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009990 tx=0x7efc7c008040 comp rx=0 tx=0).stop 2023-11-13T10:55:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80154100 msgr2=0x7efc801628e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80154100 0x7efc801628e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc7802cba0 tx=0x7efc78001cb0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 shutdown_connections 2023-11-13T10:55:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc6406ae70 0x7efc6406d330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80154100 0x7efc801628e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc80152650 0x7efc80153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 --2- 172.21.15.16:0/4010418095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc6c005180 0x7efc8015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 >> 172.21.15.16:0/4010418095 conn(0x7efc800b04a0 msgr2=0x7efc8006cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 shutdown_connections 2023-11-13T10:55:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.670+0000 7efc8758c640 1 -- 172.21.15.16:0/4010418095 wait complete. 2023-11-13T10:55:56.694 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1022916571},{"type":"v1","addr":"172.21.15.184:6833","nonce":1022916571}]},"active_addr":"172.21.15.184:6833/1022916571","active_change":"2023-11-13T10:55:05.941567+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":1782106738}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":2745855015}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":1171562292}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":2146196738}]}]} 2023-11-13T10:55:56.699 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-13T10:55:56.699 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-13T10:55:56.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.928+0000 7fe61359e640 1 Processor -- start 2023-11-13T10:55:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.928+0000 7fe61359e640 1 -- start start 2023-11-13T10:55:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.929+0000 7fe61359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61415b190 0x7fe61406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.930+0000 7fe61359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe614158c20 con 0x7fe61406a190 2023-11-13T10:55:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.930+0000 7fe61359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6140b1830 con 0x7fe6140cb190 2023-11-13T10:55:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.930+0000 7fe61359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61415b780 con 0x7fe61415b190 2023-11-13T10:55:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.930+0000 7fe612d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe61406a190 0x7fe6140b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34474/0 (socket says 172.21.15.16:34474) 2023-11-13T10:55:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.930+0000 7fe612d9d640 1 -- 172.21.15.16:0/1280734174 learned_addr learned my addr 172.21.15.16:0/1280734174 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.930+0000 7fe611d9b640 1 --2- 172.21.15.16:0/1280734174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61415b190 0x7fe61406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.930+0000 7fe61159a640 1 -- 172.21.15.16:0/1280734174 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154533721 0 0) 0x7fe614158c20 con 0x7fe61406a190 2023-11-13T10:55:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.931+0000 7fe61159a640 1 -- 172.21.15.16:0/1280734174 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5fc003520 con 0x7fe61406a190 2023-11-13T10:55:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.931+0000 7fe611d9b640 1 -- 172.21.15.16:0/1280734174 >> v1:172.21.15.184:6789/0 conn(0x7fe6140cb190 legacy=0x7fe6140716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.931+0000 7fe611d9b640 1 -- 172.21.15.16:0/1280734174 >> v1:172.21.15.16:6789/0 conn(0x7fe61406a190 legacy=0x7fe6140b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.931+0000 7fe611d9b640 1 -- 172.21.15.16:0/1280734174 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61415ba60 con 0x7fe61415b190 2023-11-13T10:55:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.932+0000 7fe611d9b640 1 --2- 172.21.15.16:0/1280734174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61415b190 0x7fe61406f970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe6080097d0 tx=0x7fe60802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1332c5c28814dab server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.933+0000 7fe61159a640 1 -- 172.21.15.16:0/1280734174 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608031800 con 0x7fe61415b190 2023-11-13T10:55:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.933+0000 7fe61159a640 1 -- 172.21.15.16:0/1280734174 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe608031960 con 0x7fe61415b190 2023-11-13T10:55:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.933+0000 7fe61159a640 1 -- 172.21.15.16:0/1280734174 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608031cf0 con 0x7fe61415b190 2023-11-13T10:55:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.934+0000 7fe61359e640 1 -- 172.21.15.16:0/1280734174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61415b190 msgr2=0x7fe61406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.934+0000 7fe61359e640 1 --2- 172.21.15.16:0/1280734174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61415b190 0x7fe61406f970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe6080097d0 tx=0x7fe60802b8a0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.934+0000 7fe61359e640 1 -- 172.21.15.16:0/1280734174 shutdown_connections 2023-11-13T10:55:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.934+0000 7fe61359e640 1 --2- 172.21.15.16:0/1280734174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe61415b190 0x7fe61406f970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.934+0000 7fe61359e640 1 -- 172.21.15.16:0/1280734174 >> 172.21.15.16:0/1280734174 conn(0x7fe6140b04a0 msgr2=0x7fe6140b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.934+0000 7fe61359e640 1 -- 172.21.15.16:0/1280734174 shutdown_connections 2023-11-13T10:55:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.934+0000 7fe61359e640 1 -- 172.21.15.16:0/1280734174 wait complete. 2023-11-13T10:55:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.935+0000 7fe61359e640 1 Processor -- start 2023-11-13T10:55:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.936+0000 7fe61359e640 1 -- start start 2023-11-13T10:55:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe61359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61406a190 0x7fe61415c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe61359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6140cb190 0x7fe61406cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe61359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61415b190 0x7fe614150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe61359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe614159c10 con 0x7fe61415b190 2023-11-13T10:55:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe61359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe614154230 con 0x7fe61406a190 2023-11-13T10:55:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe61359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6141592e0 con 0x7fe6140cb190 2023-11-13T10:55:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe612d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61415b190 0x7fe614150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe612d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61415b190 0x7fe614150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55938/0 (socket says 172.21.15.16:55938) 2023-11-13T10:55:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe612d9d640 1 -- 172.21.15.16:0/3083473900 learned_addr learned my addr 172.21.15.16:0/3083473900 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe61259c640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61406a190 0x7fe61415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe611d9b640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6140cb190 0x7fe61406cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe612d9d640 1 -- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6140cb190 msgr2=0x7fe61406cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe612d9d640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6140cb190 0x7fe61406cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe612d9d640 1 -- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61406a190 msgr2=0x7fe61415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:56.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe612d9d640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61406a190 0x7fe61415c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:56.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.937+0000 7fe612d9d640 1 -- 172.21.15.16:0/3083473900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608009400 con 0x7fe61415b190 2023-11-13T10:55:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.938+0000 7fe612d9d640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61415b190 0x7fe614150ab0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00cb40 tx=0x7fe60c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.938+0000 7fe602ffd640 1 -- 172.21.15.16:0/3083473900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00aca0 con 0x7fe61415b190 2023-11-13T10:55:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.938+0000 7fe602ffd640 1 -- 172.21.15.16:0/3083473900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c00ae00 con 0x7fe61415b190 2023-11-13T10:55:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.938+0000 7fe602ffd640 1 -- 172.21.15.16:0/3083473900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c012620 con 0x7fe61415b190 2023-11-13T10:55:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.938+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6141595c0 con 0x7fe61415b190 2023-11-13T10:55:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.938+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61415c7a0 con 0x7fe61415b190 2023-11-13T10:55:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.939+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe614159d80 con 0x7fe61415b190 2023-11-13T10:55:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.940+0000 7fe602ffd640 1 -- 172.21.15.16:0/3083473900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe60c007070 con 0x7fe61415b190 2023-11-13T10:55:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.942+0000 7fe602ffd640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe5f406ae40 0x7fe5f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.942+0000 7fe602ffd640 1 -- 172.21.15.16:0/3083473900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe60c0899f0 con 0x7fe61415b190 2023-11-13T10:55:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.942+0000 7fe61259c640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe5f406ae40 0x7fe5f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.943+0000 7fe61259c640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe5f406ae40 0x7fe5f406d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe604001640 tx=0x7fe604009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:56.947+0000 7fe602ffd640 1 -- 172.21.15.16:0/3083473900 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe60c054710 con 0x7fe61415b190 2023-11-13T10:55:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.144+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe61404e4d0 con 0x7fe61415b190 2023-11-13T10:55:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.146+0000 7fe602ffd640 1 -- 172.21.15.16:0/3083473900 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7fe60c017090 con 0x7fe61415b190 2023-11-13T10:55:57.150 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:57.150 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe5f406ae40 msgr2=0x7fe5f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe5f406ae40 0x7fe5f406d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe604001640 tx=0x7fe604009040 comp rx=0 tx=0).stop 2023-11-13T10:55:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61415b190 msgr2=0x7fe614150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61415b190 0x7fe614150ab0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00cb40 tx=0x7fe60c00a060 comp rx=0 tx=0).stop 2023-11-13T10:55:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 shutdown_connections 2023-11-13T10:55:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe5f406ae40 0x7fe5f406d300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61415b190 0x7fe614150ab0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6140cb190 0x7fe61406cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 --2- 172.21.15.16:0/3083473900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe61406a190 0x7fe61415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.148+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 >> 172.21.15.16:0/3083473900 conn(0x7fe6140b04a0 msgr2=0x7fe6140cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.149+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 shutdown_connections 2023-11-13T10:55:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.149+0000 7fe61359e640 1 -- 172.21.15.16:0/3083473900 wait complete. 2023-11-13T10:55:57.174 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-13T10:55:57.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:55:57.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.402+0000 7f4b9b2d1640 1 Processor -- start 2023-11-13T10:55:57.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.402+0000 7f4b9b2d1640 1 -- start start 2023-11-13T10:55:57.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.404+0000 7f4b9b2d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4b940cebe0 0x7f4b940cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.404+0000 7f4b9b2d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940ad800 0x7f4b940adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.404+0000 7f4b9b2d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9406e330 con 0x7f4b940cfc60 2023-11-13T10:55:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.404+0000 7f4b9b2d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b9406d2c0 con 0x7f4b940cebe0 2023-11-13T10:55:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.404+0000 7f4b9b2d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9406c2b0 con 0x7f4b940ad800 2023-11-13T10:55:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.405+0000 7f4b9a2cf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4b940cebe0 0x7f4b940cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.405+0000 7f4b9aad0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940ad800 0x7f4b940adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:57.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.405+0000 7f4b9a2cf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4b940cebe0 0x7f4b940cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38778/0 (socket says 172.21.15.16:38778) 2023-11-13T10:55:57.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.405+0000 7f4b9a2cf640 1 -- 172.21.15.16:0/1590483710 learned_addr learned my addr 172.21.15.16:0/1590483710 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.405+0000 7f4b9aad0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940ad800 0x7f4b940adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50322/0 (socket says 172.21.15.16:50322) 2023-11-13T10:55:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.405+0000 7f4b9a2cf640 1 -- 172.21.15.16:0/1590483710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940ad800 msgr2=0x7f4b940adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.405+0000 7f4b9a2cf640 1 --2- 172.21.15.16:0/1590483710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940ad800 0x7f4b940adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.406+0000 7f4b9a2cf640 1 -- 172.21.15.16:0/1590483710 >> v1:172.21.15.184:6789/0 conn(0x7f4b940cfc60 legacy=0x7f4b9406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.406+0000 7f4b9a2cf640 1 -- 172.21.15.16:0/1590483710 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b940cf500 con 0x7f4b940cebe0 2023-11-13T10:55:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.407+0000 7f4b9a2cf640 1 --2- 172.21.15.16:0/1590483710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b940cebe0 0x7f4b940cefc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009870 tx=0x7f4b9002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ce381cbe5378d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b992cd640 1 -- 172.21.15.16:0/1590483710 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90030840 con 0x7f4b940cebe0 2023-11-13T10:55:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b992cd640 1 -- 172.21.15.16:0/1590483710 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b900309a0 con 0x7f4b940cebe0 2023-11-13T10:55:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b992cd640 1 -- 172.21.15.16:0/1590483710 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90030d30 con 0x7f4b940cebe0 2023-11-13T10:55:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1590483710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b940cebe0 msgr2=0x7f4b940cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b9b2d1640 1 --2- 172.21.15.16:0/1590483710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b940cebe0 0x7f4b940cefc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009870 tx=0x7f4b9002b980 comp rx=0 tx=0).stop 2023-11-13T10:55:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1590483710 shutdown_connections 2023-11-13T10:55:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b9b2d1640 1 --2- 172.21.15.16:0/1590483710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940ad800 0x7f4b940adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b9b2d1640 1 --2- 172.21.15.16:0/1590483710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b940cebe0 0x7f4b940cefc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1590483710 >> 172.21.15.16:0/1590483710 conn(0x7f4b940b2520 msgr2=0x7f4b940b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.408+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1590483710 shutdown_connections 2023-11-13T10:55:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.409+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1590483710 wait complete. 2023-11-13T10:55:57.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.410+0000 7f4b9b2d1640 1 Processor -- start 2023-11-13T10:55:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.410+0000 7f4b9b2d1640 1 -- start start 2023-11-13T10:55:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.411+0000 7f4b9b2d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b940ad800 0x7f4b941f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.411+0000 7f4b9b2d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b940cebe0 0x7f4b940c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.411+0000 7f4b9b2d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940cfc60 0x7f4b9415d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.411+0000 7f4b9b2d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940b3910 con 0x7f4b940cebe0 2023-11-13T10:55:57.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.411+0000 7f4b9b2d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940c5ce0 con 0x7f4b940ad800 2023-11-13T10:55:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.411+0000 7f4b9b2d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940b35c0 con 0x7f4b940cfc60 2023-11-13T10:55:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b99ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b940cebe0 0x7f4b940c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:57.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b99ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b940cebe0 0x7f4b940c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55946/0 (socket says 172.21.15.16:55946) 2023-11-13T10:55:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b99ace640 1 -- 172.21.15.16:0/1779927329 learned_addr learned my addr 172.21.15.16:0/1779927329 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b9a2cf640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b940ad800 0x7f4b941f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b9aad0640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940cfc60 0x7f4b9415d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b99ace640 1 -- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940cfc60 msgr2=0x7f4b9415d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b99ace640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940cfc60 0x7f4b9415d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b99ace640 1 -- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b940ad800 msgr2=0x7f4b941f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b99ace640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b940ad800 0x7f4b941f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.412+0000 7f4b99ace640 1 -- 172.21.15.16:0/1779927329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b900094d0 con 0x7f4b940cebe0 2023-11-13T10:55:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.413+0000 7f4b99ace640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b940cebe0 0x7f4b940c57a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400c3c0 tx=0x7f4b8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.413+0000 7f4b8affd640 1 -- 172.21.15.16:0/1779927329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840052d0 con 0x7f4b940cebe0 2023-11-13T10:55:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.413+0000 7f4b8affd640 1 -- 172.21.15.16:0/1779927329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b84005450 con 0x7f4b940cebe0 2023-11-13T10:55:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.413+0000 7f4b8affd640 1 -- 172.21.15.16:0/1779927329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b84003830 con 0x7f4b940cebe0 2023-11-13T10:55:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.413+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9415d920 con 0x7f4b940cebe0 2023-11-13T10:55:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.413+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9415dee0 con 0x7f4b940cebe0 2023-11-13T10:55:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.414+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9404e4d0 con 0x7f4b940cebe0 2023-11-13T10:55:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.415+0000 7f4b8affd640 1 -- 172.21.15.16:0/1779927329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b84003990 con 0x7f4b940cebe0 2023-11-13T10:55:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.417+0000 7f4b8affd640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b7806ae40 0x7f4b7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.417+0000 7f4b8affd640 1 -- 172.21.15.16:0/1779927329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4b84088d10 con 0x7f4b940cebe0 2023-11-13T10:55:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.417+0000 7f4b9a2cf640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b7806ae40 0x7f4b7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.418+0000 7f4b9a2cf640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b7806ae40 0x7f4b7806d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b90036000 tx=0x7f4b90008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.421+0000 7f4b8affd640 1 -- 172.21.15.16:0/1779927329 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b8408e020 con 0x7f4b940cebe0 2023-11-13T10:55:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.617+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b9406e330 con 0x7f4b940cebe0 2023-11-13T10:55:57.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.619+0000 7f4b8affd640 1 -- 172.21.15.16:0/1779927329 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f4b9406e330 con 0x7f4b940cebe0 2023-11-13T10:55:57.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:55:57.624 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:55:57.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b7806ae40 msgr2=0x7f4b7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:57.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b7806ae40 0x7f4b7806d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b90036000 tx=0x7f4b90008030 comp rx=0 tx=0).stop 2023-11-13T10:55:57.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b940cebe0 msgr2=0x7f4b940c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:57.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b940cebe0 0x7f4b940c57a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400c3c0 tx=0x7f4b8400c880 comp rx=0 tx=0).stop 2023-11-13T10:55:57.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 shutdown_connections 2023-11-13T10:55:57.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b7806ae40 0x7f4b7806d300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b940cfc60 0x7f4b9415d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b940cebe0 0x7f4b940c57a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 --2- 172.21.15.16:0/1779927329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b940ad800 0x7f4b941f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:57.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 >> 172.21.15.16:0/1779927329 conn(0x7f4b940b2520 msgr2=0x7f4b9406e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:57.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 shutdown_connections 2023-11-13T10:55:57.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:57.622+0000 7f4b9b2d1640 1 -- 172.21.15.16:0/1779927329 wait complete. 2023-11-13T10:55:57.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-13T10:55:57.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-13T10:55:57.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-13T10:55:57.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-13T10:55:57.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-13T10:55:57.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-13T10:55:57.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-13T10:55:57.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-13T10:55:58.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.003+0000 7f29b61a9640 1 Processor -- start 2023-11-13T10:55:58.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.005+0000 7f3cd526d640 1 Processor -- start 2023-11-13T10:55:58.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.004+0000 7f29b61a9640 1 -- start start 2023-11-13T10:55:58.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.005+0000 7f3cd526d640 1 -- start start 2023-11-13T10:55:58.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.006+0000 7fcae1525640 1 Processor -- start 2023-11-13T10:55:58.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.006+0000 7fcae1525640 1 -- start start 2023-11-13T10:55:58.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.006+0000 7f29b61a9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f29a80e76d0 0x7f29a804c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.006+0000 7f29b61a9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29a8058980 0x7f29a80534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.006+0000 7f29b61a9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29a8056a70 0x7f29a8053a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.006+0000 7f29b61a9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a8057d50 con 0x7f29a8058980 2023-11-13T10:55:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.006+0000 7f29b61a9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a8006220 con 0x7f29a80e76d0 2023-11-13T10:55:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.006+0000 7f29b61a9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8006500 con 0x7f29a8056a70 2023-11-13T10:55:58.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.007+0000 7f29aeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29a8058980 0x7f29a80534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.008+0000 7f29aeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29a8058980 0x7f29a80534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55958/0 (socket says 172.21.15.16:55958) 2023-11-13T10:55:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.008+0000 7f29aeffd640 1 -- 172.21.15.16:0/1803210505 learned_addr learned my addr 172.21.15.16:0/1803210505 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.008+0000 7f29affff640 1 --2- 172.21.15.16:0/1803210505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29a8056a70 0x7f29a8053a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.008+0000 7f29af7fe640 1 --2- 172.21.15.16:0/1803210505 >> v2:172.21.15.184:3300/0 conn(0x7f29a80e76d0 0x7f29a804c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.008+0000 7fcae1525640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcad40e12d0 0x7fcad403a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.008+0000 7fcae1525640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcad400a420 0x7fcad4055d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.008+0000 7fcae1525640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad40558e0 con 0x7fcad400ab00 2023-11-13T10:55:58.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.008+0000 7fcae1525640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad40e16b0 con 0x7fcad400a420 2023-11-13T10:55:58.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.008+0000 7fcae1525640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad403a670 con 0x7fcad40e12d0 2023-11-13T10:55:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.009+0000 7fcadb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcad400a420 0x7fcad4055d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.009+0000 7fcadb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcad400a420 0x7fcad4055d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55964/0 (socket says 172.21.15.16:55964) 2023-11-13T10:55:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.009+0000 7fcadb7fe640 1 -- 172.21.15.16:0/4110692103 learned_addr learned my addr 172.21.15.16:0/4110692103 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7fcadb7fe640 1 -- 172.21.15.16:0/4110692103 >> v2:172.21.15.184:3300/0 conn(0x7fcad40e12d0 msgr2=0x7fcad403a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7f3cd526d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3cc8050170 0x7f3cc8038050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7fcadb7fe640 1 --2- 172.21.15.16:0/4110692103 >> v2:172.21.15.184:3300/0 conn(0x7fcad40e12d0 0x7fcad403a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7fcadb7fe640 1 -- 172.21.15.16:0/4110692103 >> v1:172.21.15.184:6789/0 conn(0x7fcad400ab00 legacy=0x7fcad4009d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7f3cd526d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc8054490 con 0x7f3cc8053c40 2023-11-13T10:55:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7fcadb7fe640 1 -- 172.21.15.16:0/4110692103 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad4054cd0 con 0x7fcad400a420 2023-11-13T10:55:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7f3cd526d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc80541b0 con 0x7f3cc80df1f0 2023-11-13T10:55:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7f3cd526d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc8005f00 con 0x7f3cc8050170 2023-11-13T10:55:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7f29aeffd640 1 -- 172.21.15.16:0/1803210505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29a8056a70 msgr2=0x7f29a8053a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7f29aeffd640 1 --2- 172.21.15.16:0/1803210505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29a8056a70 0x7f29a8053a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7f29aeffd640 1 -- 172.21.15.16:0/1803210505 >> v2:172.21.15.184:3300/0 conn(0x7f29a80e76d0 msgr2=0x7f29a804c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:58.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7f29aeffd640 1 --2- 172.21.15.16:0/1803210505 >> v2:172.21.15.184:3300/0 conn(0x7f29a80e76d0 0x7f29a804c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.010+0000 7f29aeffd640 1 -- 172.21.15.16:0/1803210505 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80067e0 con 0x7f29a8058980 2023-11-13T10:55:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.011+0000 7f3ccffff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f3cc80df1f0 0x7f3cc804fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43524/0 (socket says 172.21.15.16:43524) 2023-11-13T10:55:58.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.011+0000 7f3ccffff640 1 -- 172.21.15.16:0/1486388920 learned_addr learned my addr 172.21.15.16:0/1486388920 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:58.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.011+0000 7f3ccf7fe640 1 --2- 172.21.15.16:0/1486388920 >> v2:172.21.15.16:3300/0 conn(0x7f3cc8050170 0x7f3cc8038050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.012+0000 7f3ccf7fe640 1 -- 172.21.15.16:0/1486388920 >> v1:172.21.15.184:6789/0 conn(0x7f3cc80df1f0 legacy=0x7f3cc804fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.012+0000 7f3cceffd640 1 -- 172.21.15.16:0/1486388920 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356062898 0 0) 0x7f3cc80541b0 con 0x7f3cc80df1f0 2023-11-13T10:55:58.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.012+0000 7fcadb7fe640 1 --2- 172.21.15.16:0/4110692103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad400a420 0x7fcad4055d70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcad400fb40 tx=0x7fcad002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dffa6221deb24e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.012+0000 7f3ccf7fe640 1 -- 172.21.15.16:0/1486388920 >> v1:172.21.15.16:6789/0 conn(0x7f3cc8053c40 legacy=0x7f3cc80069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.012+0000 7f3ccf7fe640 1 -- 172.21.15.16:0/1486388920 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80061e0 con 0x7f3cc8050170 2023-11-13T10:55:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.013+0000 7fcad9ffb640 1 -- 172.21.15.16:0/4110692103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0030870 con 0x7fcad400a420 2023-11-13T10:55:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.013+0000 7f43c8c87640 1 Processor -- start 2023-11-13T10:55:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.013+0000 7fcad9ffb640 1 -- 172.21.15.16:0/4110692103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad00309d0 con 0x7fcad400a420 2023-11-13T10:55:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.013+0000 7fcad9ffb640 1 -- 172.21.15.16:0/4110692103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0030d60 con 0x7fcad400a420 2023-11-13T10:55:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.013+0000 7f29aeffd640 1 --2- 172.21.15.16:0/1803210505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a8058980 0x7f29a80534f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f29a800f7b0 tx=0x7f299c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85308ed359219627 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.013+0000 7fc015b47640 1 Processor -- start 2023-11-13T10:55:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.014+0000 7fc015b47640 1 -- start start 2023-11-13T10:55:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.014+0000 7fcae1525640 1 -- 172.21.15.16:0/4110692103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad400a420 msgr2=0x7fcad4055d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.014+0000 7fcae1525640 1 --2- 172.21.15.16:0/4110692103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad400a420 0x7fcad4055d70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcad400fb40 tx=0x7fcad002b9b0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.014+0000 7f1d71cf7640 1 Processor -- start 2023-11-13T10:55:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.013+0000 7f43c8c87640 1 -- start start 2023-11-13T10:55:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.014+0000 7f29ae7fc640 1 -- 172.21.15.16:0/1803210505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c031880 con 0x7f29a8058980 2023-11-13T10:55:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.014+0000 7f3ccf7fe640 1 --2- 172.21.15.16:0/1486388920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc8050170 0x7f3cc8038050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0097a0 tx=0x7f3cbc02d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a10cfc0232cf221 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.014+0000 7f1d71cf7640 1 -- start start 2023-11-13T10:55:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.015+0000 7f29ae7fc640 1 -- 172.21.15.16:0/1803210505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299c0319e0 con 0x7f29a8058980 2023-11-13T10:55:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.015+0000 7f29ae7fc640 1 -- 172.21.15.16:0/1803210505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c031d70 con 0x7f29a8058980 2023-11-13T10:55:58.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.015+0000 7f3cceffd640 1 -- 172.21.15.16:0/1486388920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc033870 con 0x7f3cc8050170 2023-11-13T10:55:58.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.015+0000 7f3cceffd640 1 -- 172.21.15.16:0/1486388920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc0339d0 con 0x7f3cc8050170 2023-11-13T10:55:58.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.015+0000 7f3cceffd640 1 -- 172.21.15.16:0/1486388920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc033d60 con 0x7f3cc8050170 2023-11-13T10:55:58.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f29b61a9640 1 -- 172.21.15.16:0/1803210505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a8058980 msgr2=0x7f29a80534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f29b61a9640 1 --2- 172.21.15.16:0/1803210505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a8058980 0x7f29a80534f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f29a800f7b0 tx=0x7f299c02b920 comp rx=0 tx=0).stop 2023-11-13T10:55:58.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7fc015b47640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc01006ac70 0x7fc010156a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7fc015b47640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc0100ad8c0 0x7fc01015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7fc015b47640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01006c5a0 con 0x7fc0100711d0 2023-11-13T10:55:58.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f1d71cf7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d6c0cf770 0x7f1d6c0d1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f1d71cf7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d6c15d630 0x7f1d6c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f1d71cf7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6c0ce9d0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f1d71cf7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6c0d0520 con 0x7f1d6c15d630 2023-11-13T10:55:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f1d71cf7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6c0d1b20 con 0x7f1d6c0cf770 2023-11-13T10:55:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f43c8c87640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f43c40cd9d0 0x7f43c406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f43c8c87640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c406f9b0 0x7f43c4164d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f43c8c87640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c40ceab0 con 0x7f43c4165240 2023-11-13T10:55:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f43c8c87640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c4161690 con 0x7f43c40cd9d0 2023-11-13T10:55:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f43c8c87640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41656d0 con 0x7f43c406f9b0 2023-11-13T10:55:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f29b61a9640 1 -- 172.21.15.16:0/1803210505 shutdown_connections 2023-11-13T10:55:58.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f29b61a9640 1 --2- 172.21.15.16:0/1803210505 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29a8056a70 0x7f29a8053a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f29b61a9640 1 --2- 172.21.15.16:0/1803210505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a8058980 0x7f29a80534f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7fc015b47640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01006fc00 con 0x7fc01006ac70 2023-11-13T10:55:58.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7fc015b47640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01006c7f0 con 0x7fc0100ad8c0 2023-11-13T10:55:58.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f29b61a9640 1 --2- 172.21.15.16:0/1803210505 >> v2:172.21.15.184:3300/0 conn(0x7f29a80e76d0 0x7f29a804c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f29b61a9640 1 -- 172.21.15.16:0/1803210505 >> 172.21.15.16:0/1803210505 conn(0x7f29a8058ea0 msgr2=0x7f29a8059280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f29b61a9640 1 -- 172.21.15.16:0/1803210505 shutdown_connections 2023-11-13T10:55:58.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7fcae1525640 1 -- 172.21.15.16:0/4110692103 shutdown_connections 2023-11-13T10:55:58.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7fcae1525640 1 --2- 172.21.15.16:0/4110692103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad400a420 0x7fcad4055d70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7fcae1525640 1 --2- 172.21.15.16:0/4110692103 >> v2:172.21.15.184:3300/0 conn(0x7fcad40e12d0 0x7fcad403a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7fcae1525640 1 -- 172.21.15.16:0/4110692103 >> 172.21.15.16:0/4110692103 conn(0x7fcad4056c90 msgr2=0x7fcad40570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.016+0000 7f29b61a9640 1 -- 172.21.15.16:0/1803210505 wait complete. 2023-11-13T10:55:58.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f43c1d9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c406f9b0 0x7f43c4164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f43c2d9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f43c4165240 0x7f43c40b1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43530/0 (socket says 172.21.15.16:43530) 2023-11-13T10:55:58.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7fcae1525640 1 -- 172.21.15.16:0/4110692103 shutdown_connections 2023-11-13T10:55:58.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f1d6b7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d6c0cf770 0x7f1d6c0d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f1d6b7fe640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d6c0cf770 0x7f1d6c0d1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38814/0 (socket says 172.21.15.16:38814) 2023-11-13T10:55:58.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f1d6b7fe640 1 -- 172.21.15.16:0/4093124022 learned_addr learned my addr 172.21.15.16:0/4093124022 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f1d6affd640 1 --1- 172.21.15.16:0/4093124022 >> v1:172.21.15.16:6789/0 conn(0x7f1d6c0cfb50 0x7f1d6c15cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34490/0 (socket says 172.21.15.16:34490) 2023-11-13T10:55:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f1d6a7fc640 1 -- 172.21.15.16:0/4093124022 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564195709 0 0) 0x7f1d6c0ce9d0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f1d6a7fc640 1 -- 172.21.15.16:0/4093124022 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d5c003520 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f1d6bfff640 1 --2- 172.21.15.16:0/4093124022 >> v2:172.21.15.16:3300/0 conn(0x7f1d6c15d630 0x7f1d6c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7fcae1525640 1 -- 172.21.15.16:0/4110692103 wait complete. 2023-11-13T10:55:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7f43c2d9d640 1 -- 172.21.15.16:0/1979685866 learned_addr learned my addr 172.21.15.16:0/1979685866 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7fc00effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc01006ac70 0x7fc010156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7fc00effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc01006ac70 0x7fc010156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55970/0 (socket says 172.21.15.16:55970) 2023-11-13T10:55:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7fc00effd640 1 -- 172.21.15.16:0/2404072227 learned_addr learned my addr 172.21.15.16:0/2404072227 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7fc00effd640 1 -- 172.21.15.16:0/2404072227 >> v2:172.21.15.184:3300/0 conn(0x7fc0100ad8c0 msgr2=0x7fc01015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7fc00effd640 1 --2- 172.21.15.16:0/2404072227 >> v2:172.21.15.184:3300/0 conn(0x7fc0100ad8c0 0x7fc01015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.017+0000 7fc00effd640 1 -- 172.21.15.16:0/2404072227 >> v1:172.21.15.184:6789/0 conn(0x7fc0100711d0 legacy=0x7fc010156370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-13T10:55:58.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.018+0000 7f43c259c640 1 --2- 172.21.15.16:0/1979685866 >> v2:172.21.15.184:3300/0 conn(0x7f43c40cd9d0 0x7f43c406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.018+0000 7fc00effd640 1 -- 172.21.15.16:0/2404072227 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01006cad0 con 0x7fc01006ac70 2023-11-13T10:55:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f43c1d9b640 1 -- 172.21.15.16:0/1979685866 >> v2:172.21.15.184:3300/0 conn(0x7f43c40cd9d0 msgr2=0x7f43c406d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:58.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f43c1d9b640 1 --2- 172.21.15.16:0/1979685866 >> v2:172.21.15.184:3300/0 conn(0x7f43c40cd9d0 0x7f43c406d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f29b61a9640 1 Processor -- start 2023-11-13T10:55:58.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f3cd526d640 1 -- 172.21.15.16:0/1486388920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc8050170 msgr2=0x7f3cc8038050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f3cd526d640 1 --2- 172.21.15.16:0/1486388920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc8050170 0x7f3cc8038050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0097a0 tx=0x7f3cbc02d990 comp rx=0 tx=0).stop 2023-11-13T10:55:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f29b61a9640 1 -- start start 2023-11-13T10:55:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.020+0000 7f3cd526d640 1 -- 172.21.15.16:0/1486388920 shutdown_connections 2023-11-13T10:55:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.020+0000 7f3cd526d640 1 --2- 172.21.15.16:0/1486388920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc8050170 0x7f3cc8038050 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0097a0 tx=0x7f3cbc02d990 comp rx=0 tx=0).stop 2023-11-13T10:55:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.020+0000 7f3cd526d640 1 -- 172.21.15.16:0/1486388920 >> 172.21.15.16:0/1486388920 conn(0x7f3cc8009f80 msgr2=0x7f3cc800a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.018+0000 7f1d6b7fe640 1 -- 172.21.15.16:0/4093124022 >> v2:172.21.15.16:3300/0 conn(0x7f1d6c15d630 msgr2=0x7f1d6c0cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:58.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.018+0000 7f1d6b7fe640 1 --2- 172.21.15.16:0/4093124022 >> v2:172.21.15.16:3300/0 conn(0x7f1d6c15d630 0x7f1d6c0cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.018+0000 7f1d6b7fe640 1 -- 172.21.15.16:0/4093124022 >> v1:172.21.15.16:6789/0 conn(0x7f1d6c0cfb50 legacy=0x7f1d6c15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.018+0000 7f1d6b7fe640 1 -- 172.21.15.16:0/4093124022 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c0b24f0 con 0x7f1d6c0cf770 2023-11-13T10:55:58.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.018+0000 7f1d6a7fc640 1 -- 172.21.15.16:0/4093124022 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 939103264 0 0) 0x7f1d5c003520 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f1d6b7fe640 1 --2- 172.21.15.16:0/4093124022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6c0cf770 0x7f1d6c0d1440 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c08b7d0 tx=0x7f1d6002b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e11aa047b1d3c5c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.020+0000 7f1d6a7fc640 1 -- 172.21.15.16:0/4093124022 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58001650 con 0x7f1d6c0cf770 2023-11-13T10:55:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.020+0000 7fc00effd640 1 --2- 172.21.15.16:0/2404072227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01006ac70 0x7fc010156a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc01006d8e0 tx=0x7fbff802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ab585bbdbfa8f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.020+0000 7f1d6a7fc640 1 -- 172.21.15.16:0/4093124022 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60031580 con 0x7f1d6c0cf770 2023-11-13T10:55:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.020+0000 7f1d6a7fc640 1 -- 172.21.15.16:0/4093124022 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60031950 con 0x7f1d6c0cf770 2023-11-13T10:55:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.020+0000 7fcae1525640 1 Processor -- start 2023-11-13T10:55:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f43c159a640 1 -- 172.21.15.16:0/1979685866 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039393517 0 0) 0x7f43c40ceab0 con 0x7f43c4165240 2023-11-13T10:55:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f43c1d9b640 1 -- 172.21.15.16:0/1979685866 >> v1:172.21.15.184:6789/0 conn(0x7f43c4165240 legacy=0x7f43c40b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.019+0000 7f43c1d9b640 1 -- 172.21.15.16:0/1979685866 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40ced90 con 0x7f43c406f9b0 2023-11-13T10:55:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.020+0000 7fc00e7fc640 1 -- 172.21.15.16:0/2404072227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff802ca80 con 0x7fc01006ac70 2023-11-13T10:55:58.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7f3cd526d640 1 -- 172.21.15.16:0/1486388920 shutdown_connections 2023-11-13T10:55:58.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7fc00e7fc640 1 -- 172.21.15.16:0/2404072227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff802cbe0 con 0x7fc01006ac70 2023-11-13T10:55:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7f3cd526d640 1 -- 172.21.15.16:0/1486388920 wait complete. 2023-11-13T10:55:58.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7fc00e7fc640 1 -- 172.21.15.16:0/2404072227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff802ca80 con 0x7fc01006ac70 2023-11-13T10:55:58.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7f1d71cf7640 1 -- 172.21.15.16:0/4093124022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6c0cf770 msgr2=0x7f1d6c0d1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7f1d71cf7640 1 --2- 172.21.15.16:0/4093124022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6c0cf770 0x7f1d6c0d1440 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c08b7d0 tx=0x7f1d6002b8a0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7fcae1525640 1 -- start start 2023-11-13T10:55:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7f1d71cf7640 1 -- 172.21.15.16:0/4093124022 shutdown_connections 2023-11-13T10:55:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7f1d71cf7640 1 --2- 172.21.15.16:0/4093124022 >> v2:172.21.15.16:3300/0 conn(0x7f1d6c15d630 0x7f1d6c0cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7f1d71cf7640 1 --2- 172.21.15.16:0/4093124022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6c0cf770 0x7f1d6c0d1440 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7f1d71cf7640 1 -- 172.21.15.16:0/4093124022 >> 172.21.15.16:0/4093124022 conn(0x7f1d6c0d2b30 msgr2=0x7f1d6c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.021+0000 7f43c1d9b640 1 --2- 172.21.15.16:0/1979685866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c406f9b0 0x7f43c4164d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f43b80097a0 tx=0x7f43b802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8a858cb061c5648 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f1d71cf7640 1 -- 172.21.15.16:0/4093124022 shutdown_connections 2023-11-13T10:55:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f1d71cf7640 1 -- 172.21.15.16:0/4093124022 wait complete. 2023-11-13T10:55:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7fc015b47640 1 -- 172.21.15.16:0/2404072227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01006ac70 msgr2=0x7fc010156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7fc015b47640 1 --2- 172.21.15.16:0/2404072227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01006ac70 0x7fc010156a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc01006d8e0 tx=0x7fbff802bee0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f43c159a640 1 -- 172.21.15.16:0/1979685866 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8031830 con 0x7f43c406f9b0 2023-11-13T10:55:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f29b61a9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a8056a70 0x7f29a80ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f43c159a640 1 -- 172.21.15.16:0/1979685866 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b8031990 con 0x7f43c406f9b0 2023-11-13T10:55:58.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f43c159a640 1 -- 172.21.15.16:0/1979685866 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8031d20 con 0x7f43c406f9b0 2023-11-13T10:55:58.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f29b61a9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29a8058980 0x7f29a81885b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f29b61a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80e76d0 0x7f29a8039bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f29b61a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8188af0 con 0x7f29a80e76d0 2023-11-13T10:55:58.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f29b61a9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a803be40 con 0x7f29a8056a70 2023-11-13T10:55:58.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.022+0000 7f29b61a9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a803c120 con 0x7f29a8058980 2023-11-13T10:55:58.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7fc015b47640 1 -- 172.21.15.16:0/2404072227 shutdown_connections 2023-11-13T10:55:58.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7fc015b47640 1 --2- 172.21.15.16:0/2404072227 >> v2:172.21.15.184:3300/0 conn(0x7fc0100ad8c0 0x7fc01015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7fc015b47640 1 --2- 172.21.15.16:0/2404072227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01006ac70 0x7fc010156a80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7fc015b47640 1 -- 172.21.15.16:0/2404072227 >> 172.21.15.16:0/2404072227 conn(0x7fc01006a7c0 msgr2=0x7fc0100b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7fc015b47640 1 -- 172.21.15.16:0/2404072227 shutdown_connections 2023-11-13T10:55:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7fc015b47640 1 -- 172.21.15.16:0/2404072227 wait complete. 2023-11-13T10:55:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f43c8c87640 1 -- 172.21.15.16:0/1979685866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c406f9b0 msgr2=0x7f43c4164d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f43c8c87640 1 --2- 172.21.15.16:0/1979685866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c406f9b0 0x7f43c4164d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f43b80097a0 tx=0x7f43b802b8d0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a8056a70 0x7f29a80ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f29affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80e76d0 0x7f29a8039bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f43c8c87640 1 -- 172.21.15.16:0/1979685866 shutdown_connections 2023-11-13T10:55:58.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f43c8c87640 1 --2- 172.21.15.16:0/1979685866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c406f9b0 0x7f43c4164d00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f43c8c87640 1 --2- 172.21.15.16:0/1979685866 >> v2:172.21.15.184:3300/0 conn(0x7f43c40cd9d0 0x7f43c406d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f43c8c87640 1 -- 172.21.15.16:0/1979685866 >> 172.21.15.16:0/1979685866 conn(0x7f43c406a190 msgr2=0x7f43c406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f29affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80e76d0 0x7f29a8039bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55986/0 (socket says 172.21.15.16:55986) 2023-11-13T10:55:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.023+0000 7f29affff640 1 -- 172.21.15.16:0/3485689902 learned_addr learned my addr 172.21.15.16:0/3485689902 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7f43c8c87640 1 -- 172.21.15.16:0/1979685866 shutdown_connections 2023-11-13T10:55:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7fcae1525640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad400a420 0x7fcad40dd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7fcae1525640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad400ab00 0x7fcad40e3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7fcae1525640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcad40e12d0 0x7fcad404c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7fcae1525640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad40e3d10 con 0x7fcad400a420 2023-11-13T10:55:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7fcae1525640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad404dd60 con 0x7fcad40e12d0 2023-11-13T10:55:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7fcae1525640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad404e040 con 0x7fcad400ab00 2023-11-13T10:55:58.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7f29aeffd640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29a8058980 0x7f29a81885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7f3cd526d640 1 Processor -- start 2023-11-13T10:55:58.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7f1d71cf7640 1 Processor -- start 2023-11-13T10:55:58.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7f1d71cf7640 1 -- start start 2023-11-13T10:55:58.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7f3cd526d640 1 -- start start 2023-11-13T10:55:58.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7f29affff640 1 -- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29a8058980 msgr2=0x7f29a81885b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7f29affff640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29a8058980 0x7f29a81885b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7f29affff640 1 -- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a8056a70 msgr2=0x7f29a80ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7f29affff640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29a8056a70 0x7f29a80ea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7f29affff640 1 -- 172.21.15.16:0/3485689902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c0094d0 con 0x7f29a80e76d0 2023-11-13T10:55:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7f813e8e0640 1 Processor -- start 2023-11-13T10:55:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.026+0000 7f813e8e0640 1 -- start start 2023-11-13T10:55:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.026+0000 7f29affff640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80e76d0 0x7f29a8039bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f29b004f280 tx=0x7f29b0073ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.026+0000 7fc015b47640 1 Processor -- start 2023-11-13T10:55:58.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.026+0000 7f299bfff640 1 -- 172.21.15.16:0/3485689902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b008f070 con 0x7f29a80e76d0 2023-11-13T10:55:58.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7fcadaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad400a420 0x7fcad40dd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7fcada7fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad400ab00 0x7fcad40e3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7fcada7fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad400ab00 0x7fcad40e3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50366/0 (socket says 172.21.15.16:50366) 2023-11-13T10:55:58.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.025+0000 7fcada7fc640 1 -- 172.21.15.16:0/3956112767 learned_addr learned my addr 172.21.15.16:0/3956112767 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.026+0000 7f299bfff640 1 -- 172.21.15.16:0/3485689902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0073c00 con 0x7f29a80e76d0 2023-11-13T10:55:58.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7fcadaffd640 1 -- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad400ab00 msgr2=0x7fcad40e3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7fcadaffd640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad400ab00 0x7fcad40e3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7fcadaffd640 1 -- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcad40e12d0 msgr2=0x7fcad404c2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-13T10:55:58.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7fcadaffd640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcad40e12d0 0x7fcad404c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7fcadaffd640 1 -- 172.21.15.16:0/3956112767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad00094d0 con 0x7fcad400a420 2023-11-13T10:55:58.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7fcada7fc640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad400ab00 0x7fcad40e3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:58.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.026+0000 7f29b61a9640 1 -- 172.21.15.16:0/3485689902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a803c400 con 0x7f29a80e76d0 2023-11-13T10:55:58.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7f29b61a9640 1 -- 172.21.15.16:0/3485689902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a803c9c0 con 0x7f29a80e76d0 2023-11-13T10:55:58.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7f299bfff640 1 -- 172.21.15.16:0/3485689902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0068590 con 0x7f29a80e76d0 2023-11-13T10:55:58.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.026+0000 7fc015b47640 1 -- start start 2023-11-13T10:55:58.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7fc015b47640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc01006ac70 0x7fc01014b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7fc015b47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0100711d0 0x7fc01006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7f1d71cf7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6c0cf770 0x7f1d6c0b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7f1d71cf7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d6c0cfb50 0x7f1d6c0c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7f1d71cf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c15d630 0x7f1d6c0b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7f1d71cf7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0b8590 con 0x7f1d6c15d630 2023-11-13T10:55:58.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7f1d71cf7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0b7a10 con 0x7f1d6c0cf770 2023-11-13T10:55:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7f1d71cf7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0b7cf0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f1d6b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6c0cf770 0x7f1d6c0b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f1d6affd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d6c0cfb50 0x7f1d6c0c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f1d6affd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d6c0cfb50 0x7f1d6c0c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50370/0 (socket says 172.21.15.16:50370) 2023-11-13T10:55:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f1d6affd640 1 -- 172.21.15.16:0/2492636474 learned_addr learned my addr 172.21.15.16:0/2492636474 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.024+0000 7f43c8c87640 1 -- 172.21.15.16:0/1979685866 wait complete. 2023-11-13T10:55:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.026+0000 7f43c8c87640 1 Processor -- start 2023-11-13T10:55:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.027+0000 7f43c8c87640 1 -- start start 2023-11-13T10:55:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f813e8e0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f813815cf20 0x7f813806cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f813e8e0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81380cecc0 0x7f8138070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f813e8e0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81380b2900 con 0x7f813806bb90 2023-11-13T10:55:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f813e8e0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813815d300 con 0x7f81380cecc0 2023-11-13T10:55:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f813e8e0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81380b1fd0 con 0x7f813815cf20 2023-11-13T10:55:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f812ffff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f813806bb90 0x7f81380ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43546/0 (socket says 172.21.15.16:43546) 2023-11-13T10:55:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f812ffff640 1 -- 172.21.15.16:0/3328517102 learned_addr learned my addr 172.21.15.16:0/3328517102 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f1d6bfff640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c15d630 0x7f1d6c0b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.028+0000 7f813ce7d640 1 --2- 172.21.15.16:0/3328517102 >> v2:172.21.15.16:3300/0 conn(0x7f81380cecc0 0x7f8138070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f1d6affd640 1 -- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6c0cf770 msgr2=0x7f1d6c0b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f1d6affd640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d6c0cf770 0x7f1d6c0b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f1d6affd640 1 -- 172.21.15.16:0/2492636474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c15d630 msgr2=0x7f1d6c0b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f1d6affd640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c15d630 0x7f1d6c0b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f1d6affd640 1 -- 172.21.15.16:0/2492636474 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d600094d0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7fc015b47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0100ad8c0 0x7fc010070520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7fc00f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc01006ac70 0x7fc01014b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7fc00f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc01006ac70 0x7fc01014b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50386/0 (socket says 172.21.15.16:50386) 2023-11-13T10:55:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f3cd526d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cc8053c40 0x7f3cc80cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f3cd526d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc80df1f0 0x7f3cc80c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f3cd526d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cc804d140 0x7f3cc804d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f3cd526d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80e12f0 con 0x7f3cc80df1f0 2023-11-13T10:55:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f3cd526d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80c44e0 con 0x7f3cc804d140 2023-11-13T10:55:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f3cd526d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc804e560 con 0x7f3cc8053c40 2023-11-13T10:55:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f813ce7d640 1 -- 172.21.15.16:0/3328517102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f813815cf20 msgr2=0x7f813806cab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-13T10:55:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f813ce7d640 1 --2- 172.21.15.16:0/3328517102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f813815cf20 0x7f813806cab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7fc00effd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0100711d0 0x7fc01006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.029+0000 7f43c8c87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c406f9b0 0x7f43c41470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f813ce7d640 1 -- 172.21.15.16:0/3328517102 >> v1:172.21.15.184:6789/0 conn(0x7f813806bb90 legacy=0x7f81380ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f1d6affd640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d6c0cfb50 0x7f1d6c0c8ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d58001360 tx=0x7f1d5800cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f43c8c87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43c40cd9d0 0x7f43c4163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f813ce7d640 1 -- 172.21.15.16:0/3328517102 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81380b22b0 con 0x7f81380cecc0 2023-11-13T10:55:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7fc00effd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0100711d0 0x7fc01006ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38866/0 (socket says 172.21.15.16:38866) 2023-11-13T10:55:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7fc00effd640 1 -- 172.21.15.16:0/705204057 learned_addr learned my addr 172.21.15.16:0/705204057 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f29b61a9640 1 -- 172.21.15.16:0/3485689902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2990000fc0 con 0x7f29a80e76d0 2023-11-13T10:55:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f3ccffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cc8053c40 0x7f3cc80cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f3ccffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cc8053c40 0x7f3cc80cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50396/0 (socket says 172.21.15.16:50396) 2023-11-13T10:55:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f3ccffff640 1 -- 172.21.15.16:0/4253860777 learned_addr learned my addr 172.21.15.16:0/4253860777 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01014bab0 con 0x7fc0100ad8c0 2023-11-13T10:55:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01014dc60 con 0x7fc0100711d0 2023-11-13T10:55:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100cbec0 con 0x7fc01006ac70 2023-11-13T10:55:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f43c8c87640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4165240 0x7f43c41444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f1d57fff640 1 -- 172.21.15.16:0/2492636474 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d580072d0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f43c1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43c40cd9d0 0x7f43c4163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f1d57fff640 1 -- 172.21.15.16:0/2492636474 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d58007430 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f1d57fff640 1 -- 172.21.15.16:0/2492636474 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d580072d0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f43c259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c406f9b0 0x7f43c41470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7fc00ffff640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0100ad8c0 0x7fc010070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f43c259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c406f9b0 0x7f43c41470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38882/0 (socket says 172.21.15.16:38882) 2023-11-13T10:55:58.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f43c259c640 1 -- 172.21.15.16:0/2866130719 learned_addr learned my addr 172.21.15.16:0/2866130719 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4164220 con 0x7f43c40cd9d0 2023-11-13T10:55:58.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.030+0000 7f299bfff640 1 -- 172.21.15.16:0/3485689902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f29b0072040 con 0x7f29a80e76d0 2023-11-13T10:55:58.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f3ccf7fe640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc80df1f0 0x7f3cc80c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4144a00 con 0x7f43c406f9b0 2023-11-13T10:55:58.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c415c9a0 con 0x7f43c4165240 2023-11-13T10:55:58.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f103fbf5640 1 Processor -- start 2023-11-13T10:55:58.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7fcadaffd640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad400a420 0x7fcad40dd650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcadc067e10 tx=0x7fcadc073ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f43c2d9d640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4165240 0x7f43c41444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7fc00ffff640 1 -- 172.21.15.16:0/705204057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc01006ac70 msgr2=0x7fc01014b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c0b7fd0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f3ccffff640 1 -- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cc804d140 msgr2=0x7f3cc804d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f3ccffff640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cc804d140 0x7f3cc804d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7fc00ffff640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc01006ac70 0x7fc01014b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7fc00ffff640 1 -- 172.21.15.16:0/705204057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0100711d0 msgr2=0x7fc01006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7fc00ffff640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0100711d0 0x7fc01006ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7fc00ffff640 1 -- 172.21.15.16:0/705204057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff8009b30 con 0x7fc0100ad8c0 2023-11-13T10:55:58.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f103fbf5640 1 -- start start 2023-11-13T10:55:58.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.032+0000 7f299bfff640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f299406ae40 0x7f299406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f3ccffff640 1 -- 172.21.15.16:0/4253860777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc80df1f0 msgr2=0x7f3cc80c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f3ccffff640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc80df1f0 0x7f3cc80c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.031+0000 7f3ccffff640 1 -- 172.21.15.16:0/4253860777 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc009400 con 0x7f3cc8053c40 2023-11-13T10:55:58.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.032+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c0c9010 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.032+0000 7f3ccffff640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cc8053c40 0x7f3cc80cd170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400c3f0 tx=0x7f3cc400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.032+0000 7f299bfff640 1 -- 172.21.15.16:0/3485689902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f29b01037b0 con 0x7f29a80e76d0 2023-11-13T10:55:58.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.032+0000 7fc00ffff640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0100ad8c0 0x7fc010070520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b390 tx=0x7fc00000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.032+0000 7f43c259c640 1 -- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4165240 msgr2=0x7f43c41444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7fc007fff640 1 -- 172.21.15.16:0/705204057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000118f0 con 0x7fc0100ad8c0 2023-11-13T10:55:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7fcac37fe640 1 -- 172.21.15.16:0/3956112767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc073c00 con 0x7fcad400a420 2023-11-13T10:55:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f43c259c640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4165240 0x7f43c41444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7fcac37fe640 1 -- 172.21.15.16:0/3956112767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc068410 con 0x7fcad400a420 2023-11-13T10:55:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7fcac37fe640 1 -- 172.21.15.16:0/3956112767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc072280 con 0x7fcad400a420 2023-11-13T10:55:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad404e320 con 0x7fcad400a420 2023-11-13T10:55:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.032+0000 7f299bfff640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f2994074400 0x7f2994076840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f43c259c640 1 -- 172.21.15.16:0/2866130719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43c40cd9d0 msgr2=0x7f43c4163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f43c259c640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43c40cd9d0 0x7f43c4163ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f43c259c640 1 -- 172.21.15.16:0/2866130719 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8009400 con 0x7f43c406f9b0 2023-11-13T10:55:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f29af7fe640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f299406ae40 0x7f299406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad40e0400 con 0x7fcad400a420 2023-11-13T10:55:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f299bfff640 1 -- 172.21.15.16:0/3485689902 --> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2994076f10 con 0x7f2994074400 2023-11-13T10:55:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f299bfff640 1 -- 172.21.15.16:0/3485689902 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b0105d90 con 0x7f29a80e76d0 2023-11-13T10:55:58.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d6c0cf1c0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f43c2d9d640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43c4165240 0x7f43c41444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f813ce7d640 1 --2- 172.21.15.16:0/3328517102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 0x7f8138070330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8130009d90 tx=0x7f813002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802c81f65360c220 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7fc007fff640 1 -- 172.21.15.16:0/705204057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc000011a70 con 0x7fc0100ad8c0 2023-11-13T10:55:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0100cc1a0 con 0x7fc0100ad8c0 2023-11-13T10:55:58.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7fc007fff640 1 -- 172.21.15.16:0/705204057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000011e20 con 0x7fc0100ad8c0 2023-11-13T10:55:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f43c1d9b640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43c40cd9d0 0x7f43c4163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:55:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.033+0000 7f3cccff9640 1 -- 172.21.15.16:0/4253860777 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40074e0 con 0x7f3cc8053c40 2023-11-13T10:55:58.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.034+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc80e1010 con 0x7f3cc8053c40 2023-11-13T10:55:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.034+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc804e1f0 con 0x7f3cc8053c40 2023-11-13T10:55:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.034+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01014ff20 con 0x7fc0100ad8c0 2023-11-13T10:55:58.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.034+0000 7f43c259c640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c406f9b0 0x7f43c41470f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43b400e9b0 tx=0x7f43b400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.034+0000 7f81377fe640 1 -- 172.21.15.16:0/3328517102 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813002cc80 con 0x7f81380cecc0 2023-11-13T10:55:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.034+0000 7f81377fe640 1 -- 172.21.15.16:0/3328517102 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813002cde0 con 0x7f81380cecc0 2023-11-13T10:55:58.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.034+0000 7f81377fe640 1 -- 172.21.15.16:0/3328517102 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130034660 con 0x7f81380cecc0 2023-11-13T10:55:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f29aeffd640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f2994074400 0x7f2994076840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f43b2ffd640 1 -- 172.21.15.16:0/2866130719 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4009670 con 0x7f43c406f9b0 2023-11-13T10:55:58.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f3cccff9640 1 -- 172.21.15.16:0/4253860777 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4007640 con 0x7f3cc8053c40 2023-11-13T10:55:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f3cccff9640 1 -- 172.21.15.16:0/4253860777 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4004a20 con 0x7f3cc8053c40 2023-11-13T10:55:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f103fbf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10380cfe20 0x7f10380b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f813e8e0640 1 -- 172.21.15.16:0/3328517102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 msgr2=0x7f8138070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f813e8e0640 1 --2- 172.21.15.16:0/3328517102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 0x7f8138070330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8130009d90 tx=0x7f813002bed0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3cb0000fc0 con 0x7f3cc8053c40 2023-11-13T10:55:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f813e8e0640 1 -- 172.21.15.16:0/3328517102 shutdown_connections 2023-11-13T10:55:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f813e8e0640 1 --2- 172.21.15.16:0/3328517102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 0x7f8138070330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f813e8e0640 1 --2- 172.21.15.16:0/3328517102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f813815cf20 0x7f813806cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.035+0000 7f813e8e0640 1 -- 172.21.15.16:0/3328517102 >> 172.21.15.16:0/3328517102 conn(0x7f813806a190 msgr2=0x7f813806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f103fbf5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10380d3c70 con 0x7f103815d0f0 2023-11-13T10:55:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f103fbf5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10380b45a0 con 0x7f10380d19d0 2023-11-13T10:55:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f103fbf5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10380c69d0 con 0x7f10380cfe20 2023-11-13T10:55:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f103d991640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f103815d0f0 0x7f10380d12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34494/0 (socket says 172.21.15.16:34494) 2023-11-13T10:55:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f103d991640 1 -- 172.21.15.16:0/3833085542 learned_addr learned my addr 172.21.15.16:0/3833085542 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f103d190640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f10380d19d0 0x7f10380b3950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43560/0 (socket says 172.21.15.16:43560) 2023-11-13T10:55:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcad4055570 con 0x7fcad400a420 2023-11-13T10:55:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f43b2ffd640 1 -- 172.21.15.16:0/2866130719 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b400c9a0 con 0x7f43c406f9b0 2023-11-13T10:55:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f43b2ffd640 1 -- 172.21.15.16:0/2866130719 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b400cd50 con 0x7f43c406f9b0 2023-11-13T10:55:58.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f29af7fe640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f299406ae40 0x7f299406d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f29a803e4d0 tx=0x7f29a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f29aeffd640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f2994074400 0x7f2994076840 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f299bfff640 1 -- 172.21.15.16:0/3485689902 <== osd.1 v2:172.21.15.16:6800/2377015218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f2994076f10 con 0x7f2994074400 2023-11-13T10:55:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc010150110 con 0x7fc0100ad8c0 2023-11-13T10:55:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7fc007fff640 1 -- 172.21.15.16:0/705204057 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc00001a400 con 0x7fc0100ad8c0 2023-11-13T10:55:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7fc007fff640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff006af10 0x7fbff006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7fc007fff640 1 -- 172.21.15.16:0/705204057 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc0000898d0 con 0x7fc0100ad8c0 2023-11-13T10:55:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7fc007fff640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7fbff0074550 0x7fbff0076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7fc007fff640 1 -- 172.21.15.16:0/705204057 --> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbff0077060 con 0x7fbff0074550 2023-11-13T10:55:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7fc007fff640 1 -- 172.21.15.16:0/705204057 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc00008e2a0 con 0x7fc0100ad8c0 2023-11-13T10:55:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7fc00effd640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7fbff0074550 0x7fbff0076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.036+0000 7f1d57fff640 1 -- 172.21.15.16:0/2492636474 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d580047e0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f1d57fff640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5006aea0 0x7f1d5006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7f1d57fff640 1 -- 172.21.15.16:0/2492636474 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1d58089030 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7f1d6b7fe640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5006aea0 0x7f1d5006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7f1d57fff640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f1d500744e0 0x7f1d50076920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7f1d57fff640 1 -- 172.21.15.16:0/2492636474 --> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d50076ff0 con 0x7f1d500744e0 2023-11-13T10:55:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7f1d57fff640 1 -- 172.21.15.16:0/2492636474 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d580893e0 con 0x7f1d6c0cfb50 2023-11-13T10:55:58.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c415cc40 con 0x7f43c406f9b0 2023-11-13T10:55:58.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c415d200 con 0x7f43c406f9b0 2023-11-13T10:55:58.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7f43b0ff9640 1 -- 172.21.15.16:0/2866130719 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f43c406fe10 con 0x7f43c406f9b0 2023-11-13T10:55:58.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851622122 0 0) 0x7f10380b45a0 con 0x7f10380d19d0 2023-11-13T10:55:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1024003520 con 0x7f10380d19d0 2023-11-13T10:55:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46204176 0 0) 0x7f1030001490 con 0x7f103815d0f0 2023-11-13T10:55:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103e192640 1 --2- 172.21.15.16:0/3833085542 >> v2:172.21.15.16:3300/0 conn(0x7f10380cfe20 0x7f10380b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10380b45a0 con 0x7f103815d0f0 2023-11-13T10:55:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1931297218 0 0) 0x7f1024003520 con 0x7f10380d19d0 2023-11-13T10:55:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 >> v2:172.21.15.16:3300/0 conn(0x7f10380cfe20 msgr2=0x7f10380b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103c98f640 1 --2- 172.21.15.16:0/3833085542 >> v2:172.21.15.16:3300/0 conn(0x7f10380cfe20 0x7f10380b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 >> v1:172.21.15.16:6789/0 conn(0x7f103815d0f0 legacy=0x7f10380d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10380c6cb0 con 0x7f10380d19d0 2023-11-13T10:55:58.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f1034001670 con 0x7f10380d19d0 2023-11-13T10:55:58.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 >> v1:172.21.15.184:6789/0 conn(0x7f10380d19d0 legacy=0x7f10380b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f103c98f640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 0x7f1024003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f103c98f640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10240047c0 0x7f1024004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f103c98f640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10240051d0 0x7f1024014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380c69d0 con 0x7f10240047c0 2023-11-13T10:55:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024014860 con 0x7f10240051d0 2023-11-13T10:55:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1024014a20 con 0x7f1024003b50 2023-11-13T10:55:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.038+0000 7f103d991640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 0x7f1024003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7f103d190640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10240047c0 0x7f1024004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.039+0000 7f103e192640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10240051d0 0x7f1024014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f103d991640 1 -- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10240051d0 msgr2=0x7f1024014190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f103d991640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10240051d0 0x7f1024014190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f103d991640 1 -- 172.21.15.16:0/3833085542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10240047c0 msgr2=0x7f1024004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f103d991640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10240047c0 0x7f1024004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f103d991640 1 -- 172.21.15.16:0/3833085542 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1024003520 con 0x7f1024003b50 2023-11-13T10:55:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f813e8e0640 1 -- 172.21.15.16:0/3328517102 shutdown_connections 2023-11-13T10:55:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.037+0000 7f813e8e0640 1 -- 172.21.15.16:0/3328517102 wait complete. 2023-11-13T10:55:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f813e8e0640 1 Processor -- start 2023-11-13T10:55:58.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7fc00f7fe640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff006af10 0x7fbff006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f1d6bfff640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f1d500744e0 0x7f1d50076920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.041+0000 7fcac37fe640 1 -- 172.21.15.16:0/3956112767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcadc0723e0 con 0x7fcad400a420 2023-11-13T10:55:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f103d190640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10240047c0 0x7f1024004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:55:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f3cccff9640 1 -- 172.21.15.16:0/4253860777 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3cc4004b80 con 0x7f3cc8053c40 2023-11-13T10:55:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.041+0000 7f103e192640 1 -- 172.21.15.16:0/3833085542 reap_dead start 2023-11-13T10:55:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.041+0000 7fc00effd640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7fbff0074550 0x7fbff0076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.040+0000 7f813e8e0640 1 -- start start 2023-11-13T10:55:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f1d6bfff640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f1d500744e0 0x7f1d50076920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f1d6b7fe640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5006aea0 0x7f1d5006d360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d60036000 tx=0x7f1d6003d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f813e8e0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f813806bb90 0x7f813814e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7fcac37fe640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcac806ae40 0x7fcac806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7fcac37fe640 1 -- 172.21.15.16:0/3956112767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcadc103870 con 0x7fcad400a420 2023-11-13T10:55:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f813e8e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 0x7f813815f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f813e8e0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f813815cf20 0x7f81380b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f813e8e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81381518e0 con 0x7f81380cecc0 2023-11-13T10:55:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f813e8e0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8138151bc0 con 0x7f813806bb90 2023-11-13T10:55:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f813e8e0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8138151ea0 con 0x7f813815cf20 2023-11-13T10:55:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7fc00f7fe640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff006af10 0x7fbff006d3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc008004ea0 tx=0x7fc008009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f3cccff9640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3cb406ae70 0x7f3cb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7f3cccff9640 1 -- 172.21.15.16:0/4253860777 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3cc4089750 con 0x7f3cc8053c40 2023-11-13T10:55:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7f812ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 0x7f813815f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7f813ce7d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f813815cf20 0x7f81380b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7f812ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 0x7f813815f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56062/0 (socket says 172.21.15.16:56062) 2023-11-13T10:55:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7f103d991640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 0x7f1024003f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f103000c320 tx=0x7f10240057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d7f89128a042178 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7f812ffff640 1 -- 172.21.15.16:0/4244994781 learned_addr learned my addr 172.21.15.16:0/4244994781 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7fcac37fe640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7fcac8074400 0x7fcac8076840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7f3cccff9640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f3cb4074430 0x7f3cb4076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7f3cccff9640 1 -- 172.21.15.16:0/4253860777 --> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cb4076f40 con 0x7f3cb4074430 2023-11-13T10:55:58.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7f3cccff9640 1 -- 172.21.15.16:0/4253860777 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cc408be10 con 0x7f3cc8053c40 2023-11-13T10:55:58.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7fcac37fe640 1 -- 172.21.15.16:0/3956112767 --> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcac8076f10 con 0x7fcac8074400 2023-11-13T10:55:58.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7fcac37fe640 1 -- 172.21.15.16:0/3956112767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcadc105e50 con 0x7fcad400a420 2023-11-13T10:55:58.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.043+0000 7fcada7fc640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcac806ae40 0x7fcac806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.042+0000 7f43b2ffd640 1 -- 172.21.15.16:0/2866130719 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43b4013470 con 0x7f43c406f9b0 2023-11-13T10:55:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f3ccf7fe640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3cb406ae70 0x7f3cb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f812ffff640 1 -- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f813815cf20 msgr2=0x7f81380b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f812ffff640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f813815cf20 0x7f81380b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f812ffff640 1 -- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f813806bb90 msgr2=0x7f813814e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f812ffff640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f813806bb90 0x7f813814e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f812ffff640 1 -- 172.21.15.16:0/4244994781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8130009a70 con 0x7f81380cecc0 2023-11-13T10:55:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103002cb90 con 0x7f1024003b50 2023-11-13T10:55:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103002ccf0 con 0x7f1024003b50 2023-11-13T10:55:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f103c98f640 1 -- 172.21.15.16:0/3833085542 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10300356b0 con 0x7f1024003b50 2023-11-13T10:55:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7fc007fff640 1 -- 172.21.15.16:0/705204057 <== osd.3 v2:172.21.15.16:6824/3562446965 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fbff0077060 con 0x7fbff0074550 2023-11-13T10:55:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f812ffff640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 0x7f813815f380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f812000c4a0 tx=0x7f812000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7fcadb7fe640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7fcac8074400 0x7fcac8076840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.044+0000 7f3cd4a6c640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f3cb4074430 0x7f3cb4076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f43b2ffd640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f43a406ae40 0x7f43a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7fcada7fc640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcac806ae40 0x7fcac806d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcad40e41f0 tx=0x7fcacc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f103fbf5640 1 -- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 msgr2=0x7f1024003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f103fbf5640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 0x7f1024003f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f103000c320 tx=0x7f10240057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f81357fa640 1 -- 172.21.15.16:0/4244994781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120005ce0 con 0x7f81380cecc0 2023-11-13T10:55:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f81357fa640 1 -- 172.21.15.16:0/4244994781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8120005e40 con 0x7f81380cecc0 2023-11-13T10:55:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f81357fa640 1 -- 172.21.15.16:0/4244994781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120007270 con 0x7f81380cecc0 2023-11-13T10:55:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8138152180 con 0x7f81380cecc0 2023-11-13T10:55:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f103fbf5640 1 -- 172.21.15.16:0/3833085542 shutdown_connections 2023-11-13T10:55:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f103fbf5640 1 --2- 172.21.15.16:0/3833085542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 0x7f1024003f50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f103fbf5640 1 -- 172.21.15.16:0/3833085542 >> 172.21.15.16:0/3833085542 conn(0x7f10380d2ce0 msgr2=0x7f10380d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f43b2ffd640 1 -- 172.21.15.16:0/2866130719 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f43b4089f50 con 0x7f43c406f9b0 2023-11-13T10:55:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f43c1d9b640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f43a406ae40 0x7f43a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f1d57fff640 1 -- 172.21.15.16:0/2492636474 <== osd.4 v2:172.21.15.184:6800/2936315873 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f1d50076ff0 con 0x7f1d500744e0 2023-11-13T10:55:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f103fbf5640 1 -- 172.21.15.16:0/3833085542 shutdown_connections 2023-11-13T10:55:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.046+0000 7f103fbf5640 1 -- 172.21.15.16:0/3833085542 wait complete. 2023-11-13T10:55:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.046+0000 7f3ccf7fe640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3cb406ae70 0x7f3cb406d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc038000 tx=0x7f3cbc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.046+0000 7f43b2ffd640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7f43a4074400 0x7f43a4076840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.046+0000 7f43b2ffd640 1 -- 172.21.15.16:0/2866130719 --> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43a4076f10 con 0x7f43a4074400 2023-11-13T10:55:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.046+0000 7f43b2ffd640 1 -- 172.21.15.16:0/2866130719 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43b408a3a0 con 0x7f43c406f9b0 2023-11-13T10:55:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.047+0000 7f43c1d9b640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f43a406ae40 0x7f43a406d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43b8036000 tx=0x7f43b803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.047+0000 7f43c2d9d640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7f43a4074400 0x7f43a4076840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.045+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813814ed70 con 0x7f81380cecc0 2023-11-13T10:55:58.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.047+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f813806cd00 con 0x7f81380cecc0 2023-11-13T10:55:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.046+0000 7fcadb7fe640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7fcac8074400 0x7fcac8076840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.047+0000 7fcac37fe640 1 -- 172.21.15.16:0/3956112767 <== osd.0 v2:172.21.15.16:6809/4203265327 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fcac8076f10 con 0x7fcac8074400 2023-11-13T10:55:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.047+0000 7f3cd4a6c640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f3cb4074430 0x7f3cb4076870 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.048+0000 7f43c2d9d640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7f43a4074400 0x7f43a4076840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.048+0000 7f103fbf5640 1 Processor -- start 2023-11-13T10:55:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.048+0000 7f103fbf5640 1 -- start start 2023-11-13T10:55:58.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.050+0000 7f3cccff9640 1 -- 172.21.15.16:0/4253860777 <== osd.2 v2:172.21.15.16:6808/3899450571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f3cb4076f40 con 0x7f3cb4074430 2023-11-13T10:55:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.050+0000 7f103fbf5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 0x7f10380b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.050+0000 7f103fbf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10380cc150 0x7f10380cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.050+0000 7f103fbf5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10381f2ad0 0x7f10381f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.050+0000 7f43b2ffd640 1 -- 172.21.15.16:0/2866130719 <== osd.5 v2:172.21.15.184:6808/1150854014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f43a4076f10 con 0x7f43a4074400 2023-11-13T10:55:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.051+0000 7f103fbf5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380b4460 con 0x7f10380cc150 2023-11-13T10:55:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.051+0000 7f103fbf5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380ccb10 con 0x7f10381f2ad0 2023-11-13T10:55:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.051+0000 7f103fbf5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103806df80 con 0x7f1024003b50 2023-11-13T10:55:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.051+0000 7f103d991640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 0x7f10380b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.051+0000 7f103d190640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10380cc150 0x7f10380cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.051+0000 7f103d991640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 0x7f10380b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50420/0 (socket says 172.21.15.16:50420) 2023-11-13T10:55:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.051+0000 7f103d991640 1 -- 172.21.15.16:0/3596597707 learned_addr learned my addr 172.21.15.16:0/3596597707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.051+0000 7f103e192640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10381f2ad0 0x7f10381f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.052+0000 7f103d190640 1 -- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 msgr2=0x7f10380b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.052+0000 7f103d190640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1024003b50 0x7f10380b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.052+0000 7f103d190640 1 -- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10381f2ad0 msgr2=0x7f10381f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.052+0000 7f103d190640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10381f2ad0 0x7f10381f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.052+0000 7f103d190640 1 -- 172.21.15.16:0/3596597707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103806d650 con 0x7f10380cc150 2023-11-13T10:55:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.053+0000 7f103d190640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10380cc150 0x7f10380cc5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1034001730 tx=0x7f1034001760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.054+0000 7f102e7fc640 1 -- 172.21.15.16:0/3596597707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034004e50 con 0x7f10380cc150 2023-11-13T10:55:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.051+0000 7f81357fa640 1 -- 172.21.15.16:0/4244994781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8120007490 con 0x7f81380cecc0 2023-11-13T10:55:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.053+0000 7f81357fa640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f811806af10 0x7f811806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.053+0000 7f81357fa640 1 -- 172.21.15.16:0/4244994781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8120088d80 con 0x7f81380cecc0 2023-11-13T10:55:58.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.053+0000 7f81357fa640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7f8118074550 0x7f8118076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.053+0000 7f81357fa640 1 -- 172.21.15.16:0/4244994781 --> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8118077060 con 0x7f8118074550 2023-11-13T10:55:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.053+0000 7f81357fa640 1 -- 172.21.15.16:0/4244994781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f812008b340 con 0x7f81380cecc0 2023-11-13T10:55:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.054+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103806d930 con 0x7f10380cc150 2023-11-13T10:55:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.054+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10381f3490 con 0x7f10380cc150 2023-11-13T10:55:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.054+0000 7f102e7fc640 1 -- 172.21.15.16:0/3596597707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103400f960 con 0x7f10380cc150 2023-11-13T10:55:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.054+0000 7f102e7fc640 1 -- 172.21.15.16:0/3596597707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400fcf0 con 0x7f10380cc150 2023-11-13T10:55:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.054+0000 7f813ce7d640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7f8118074550 0x7f8118076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.055+0000 7f8137fff640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f811806af10 0x7f811806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.056+0000 7f813ce7d640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7f8118074550 0x7f8118076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.058+0000 7f8137fff640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f811806af10 0x7f811806d3d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8128009650 tx=0x7f8128006ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.058+0000 7f81357fa640 1 -- 172.21.15.16:0/4244994781 <== osd.7 v2:172.21.15.184:6824/3047573298 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f8118077060 con 0x7f8118074550 2023-11-13T10:55:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.060+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1020000fc0 con 0x7f10380cc150 2023-11-13T10:55:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.060+0000 7f102e7fc640 1 -- 172.21.15.16:0/3596597707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f103401d460 con 0x7f10380cc150 2023-11-13T10:55:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.061+0000 7f102e7fc640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f101c06af10 0x7f101c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.062+0000 7f102e7fc640 1 -- 172.21.15.16:0/3596597707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1034089910 con 0x7f10380cc150 2023-11-13T10:55:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.062+0000 7f102e7fc640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f101c0744d0 0x7f101c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.062+0000 7f102e7fc640 1 -- 172.21.15.16:0/3596597707 --> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f101c076fe0 con 0x7f101c0744d0 2023-11-13T10:55:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.063+0000 7f102e7fc640 1 -- 172.21.15.16:0/3596597707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1034054760 con 0x7f10380cc150 2023-11-13T10:55:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.063+0000 7f103d991640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f101c06af10 0x7f101c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.063+0000 7f103e192640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f101c0744d0 0x7f101c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.066+0000 7f103d991640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f101c06af10 0x7f101c06d3d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f10300012e0 tx=0x7f1030039040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.066+0000 7f103e192640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f101c0744d0 0x7f101c076910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.068+0000 7f102e7fc640 1 -- 172.21.15.16:0/3596597707 <== osd.6 v2:172.21.15.184:6816/1720475067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f101c076fe0 con 0x7f101c0744d0 2023-11-13T10:55:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.071+0000 7f29b61a9640 1 -- 172.21.15.16:0/3485689902 --> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2990002cb0 con 0x7f2994074400 2023-11-13T10:55:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.071+0000 7f299bfff640 1 -- 172.21.15.16:0/3485689902 <== osd.1 v2:172.21.15.16:6800/2377015218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2990002cb0 con 0x7f2994074400 2023-11-13T10:55:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.075+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 --> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcad4055290 con 0x7fcac8074400 2023-11-13T10:55:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.075+0000 7f2999ffb640 1 -- 172.21.15.16:0/3485689902 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f2994074400 msgr2=0x7f2994076840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.075+0000 7f2999ffb640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f2994074400 0x7f2994076840 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.076+0000 7f2999ffb640 1 -- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f299406ae40 msgr2=0x7f299406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.076+0000 7f2999ffb640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f299406ae40 0x7f299406d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f29a803e4d0 tx=0x7f29a4008040 comp rx=0 tx=0).stop 2023-11-13T10:55:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.076+0000 7f2999ffb640 1 -- 172.21.15.16:0/3485689902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80e76d0 msgr2=0x7f29a8039bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.076+0000 7fcac37fe640 1 -- 172.21.15.16:0/3956112767 <== osd.0 v2:172.21.15.16:6809/4203265327 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcad0030b40 con 0x7fcac8074400 2023-11-13T10:55:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.076+0000 7f2999ffb640 1 --2- 172.21.15.16:0/3485689902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a80e76d0 0x7f29a8039bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f29b004f280 tx=0x7f29b0073ad0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.076+0000 7f29affff640 1 -- 172.21.15.16:0/3485689902 reap_dead start 2023-11-13T10:55:58.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.076+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7fcac8074400 msgr2=0x7fcac8076840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.076+0000 7fcae1525640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7fcac8074400 0x7fcac8076840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcac806ae40 msgr2=0x7fcac806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7f2999ffb640 1 -- 172.21.15.16:0/3485689902 shutdown_connections 2023-11-13T10:55:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7f2999ffb640 1 -- 172.21.15.16:0/3485689902 >> 172.21.15.16:0/3485689902 conn(0x7f29a8058ea0 msgr2=0x7f29a8038680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 --> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc0100cc6c0 con 0x7fbff0074550 2023-11-13T10:55:58.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7fcae1525640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcac806ae40 0x7fcac806d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcad40e41f0 tx=0x7fcacc009040 comp rx=0 tx=0).stop 2023-11-13T10:55:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7f2999ffb640 1 -- 172.21.15.16:0/3485689902 shutdown_connections 2023-11-13T10:55:58.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad400a420 msgr2=0x7fcad40dd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7fcae1525640 1 --2- 172.21.15.16:0/3956112767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad400a420 0x7fcad40dd650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcadc067e10 tx=0x7fcadc073ad0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7fcadb7fe640 1 -- 172.21.15.16:0/3956112767 reap_dead start 2023-11-13T10:55:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 shutdown_connections 2023-11-13T10:55:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 >> 172.21.15.16:0/3956112767 conn(0x7fcad4056c90 msgr2=0x7fcad403b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 shutdown_connections 2023-11-13T10:55:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.078+0000 7fcae1525640 1 -- 172.21.15.16:0/3956112767 wait complete. 2023-11-13T10:55:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.077+0000 7f2999ffb640 1 -- 172.21.15.16:0/3485689902 wait complete. 2023-11-13T10:55:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.078+0000 7fc007fff640 1 -- 172.21.15.16:0/705204057 <== osd.3 v2:172.21.15.16:6824/3562446965 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc0100cc6c0 con 0x7fbff0074550 2023-11-13T10:55:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.079+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7fbff0074550 msgr2=0x7fbff0076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.079+0000 7fc015b47640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7fbff0074550 0x7fbff0076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.079+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff006af10 msgr2=0x7fbff006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.079+0000 7fc015b47640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff006af10 0x7fbff006d3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc008004ea0 tx=0x7fc008009040 comp rx=0 tx=0).stop 2023-11-13T10:55:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.079+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0100ad8c0 msgr2=0x7fc010070520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.079+0000 7fc015b47640 1 --2- 172.21.15.16:0/705204057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0100ad8c0 0x7fc010070520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b390 tx=0x7fc00000b850 comp rx=0 tx=0).stop 2023-11-13T10:55:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.079+0000 7fc00ffff640 1 -- 172.21.15.16:0/705204057 reap_dead start 2023-11-13T10:55:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.080+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 shutdown_connections 2023-11-13T10:55:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.080+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 >> 172.21.15.16:0/705204057 conn(0x7fc01006a7c0 msgr2=0x7fc010156890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.082+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 shutdown_connections 2023-11-13T10:55:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.082+0000 7fc015b47640 1 -- 172.21.15.16:0/705204057 wait complete. 2023-11-13T10:55:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.081+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 --> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3cb0002cb0 con 0x7f3cb4074430 2023-11-13T10:55:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.083+0000 7f3cccff9640 1 -- 172.21.15.16:0/4253860777 <== osd.2 v2:172.21.15.16:6808/3899450571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3cb0002cb0 con 0x7f3cb4074430 2023-11-13T10:55:58.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.084+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f3cb4074430 msgr2=0x7f3cb4076870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.084+0000 7f3cd526d640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f3cb4074430 0x7f3cb4076870 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.085+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3cb406ae70 msgr2=0x7f3cb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.085+0000 7f3cd526d640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3cb406ae70 0x7f3cb406d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc038000 tx=0x7f3cbc008030 comp rx=0 tx=0).stop 2023-11-13T10:55:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.085+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cc8053c40 msgr2=0x7f3cc80cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.085+0000 7f3cd526d640 1 --2- 172.21.15.16:0/4253860777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cc8053c40 0x7f3cc80cd170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400c3f0 tx=0x7f3cc400c8b0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.085+0000 7f3cd4a6c640 1 -- 172.21.15.16:0/4253860777 reap_dead start 2023-11-13T10:55:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.085+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 shutdown_connections 2023-11-13T10:55:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.085+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 >> 172.21.15.16:0/4253860777 conn(0x7f3cc8009f80 msgr2=0x7f3cc80ddbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.085+0000 7f43b0ff9640 1 -- 172.21.15.16:0/2866130719 --> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f43c4071880 con 0x7f43a4074400 2023-11-13T10:55:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.086+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 shutdown_connections 2023-11-13T10:55:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.086+0000 7f3cd526d640 1 -- 172.21.15.16:0/4253860777 wait complete. 2023-11-13T10:55:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.087+0000 7f43b2ffd640 1 -- 172.21.15.16:0/2866130719 <== osd.5 v2:172.21.15.184:6808/1150854014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f43bc007170 con 0x7f43a4074400 2023-11-13T10:55:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.089+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 --> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d6c0b6040 con 0x7f1d500744e0 2023-11-13T10:55:58.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.089+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7f43a4074400 msgr2=0x7f43a4076840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.089+0000 7f43c8c87640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7f43a4074400 0x7f43a4076840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f43a406ae40 msgr2=0x7f43a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f43c8c87640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f43a406ae40 0x7f43a406d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f43b8036000 tx=0x7f43b803c040 comp rx=0 tx=0).stop 2023-11-13T10:55:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c406f9b0 msgr2=0x7f43c41470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f43c8c87640 1 --2- 172.21.15.16:0/2866130719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43c406f9b0 0x7f43c41470f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43b400e9b0 tx=0x7f43b400ee70 comp rx=0 tx=0).stop 2023-11-13T10:55:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f43c2d9d640 1 -- 172.21.15.16:0/2866130719 reap_dead start 2023-11-13T10:55:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 shutdown_connections 2023-11-13T10:55:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 --> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f813806ceb0 con 0x7f8118074550 2023-11-13T10:55:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f1d57fff640 1 -- 172.21.15.16:0/2492636474 <== osd.4 v2:172.21.15.184:6800/2936315873 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1d6c0b6040 con 0x7f1d500744e0 2023-11-13T10:55:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 >> 172.21.15.16:0/2866130719 conn(0x7f43c406a190 msgr2=0x7f43c40afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.090+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 shutdown_connections 2023-11-13T10:55:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.091+0000 7f43c8c87640 1 -- 172.21.15.16:0/2866130719 wait complete. 2023-11-13T10:55:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.091+0000 7f81357fa640 1 -- 172.21.15.16:0/4244994781 <== osd.7 v2:172.21.15.184:6824/3047573298 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f813806ceb0 con 0x7f8118074550 2023-11-13T10:55:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.091+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f1d500744e0 msgr2=0x7f1d50076920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.091+0000 7f1d71cf7640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f1d500744e0 0x7f1d50076920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5006aea0 msgr2=0x7f1d5006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f1d71cf7640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5006aea0 0x7f1d5006d360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d60036000 tx=0x7f1d6003d040 comp rx=0 tx=0).stop 2023-11-13T10:55:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d6c0cfb50 msgr2=0x7f1d6c0c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f1d71cf7640 1 --2- 172.21.15.16:0/2492636474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d6c0cfb50 0x7f1d6c0c8ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d58001360 tx=0x7f1d5800cbe0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f1d6bfff640 1 -- 172.21.15.16:0/2492636474 reap_dead start 2023-11-13T10:55:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 shutdown_connections 2023-11-13T10:55:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 >> 172.21.15.16:0/2492636474 conn(0x7f1d6c0d2b30 msgr2=0x7f1d6c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 shutdown_connections 2023-11-13T10:55:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f1d71cf7640 1 -- 172.21.15.16:0/2492636474 wait complete. 2023-11-13T10:55:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7f8118074550 msgr2=0x7f8118076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.092+0000 7f813e8e0640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7f8118074550 0x7f8118076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.093+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f811806af10 msgr2=0x7f811806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.093+0000 7f813e8e0640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f811806af10 0x7f811806d3d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8128009650 tx=0x7f8128006ba0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.093+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 msgr2=0x7f813815f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.093+0000 7f813e8e0640 1 --2- 172.21.15.16:0/4244994781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81380cecc0 0x7f813815f380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f812000c4a0 tx=0x7f812000c960 comp rx=0 tx=0).stop 2023-11-13T10:55:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.093+0000 7f813ce7d640 1 -- 172.21.15.16:0/4244994781 reap_dead start 2023-11-13T10:55:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.093+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 shutdown_connections 2023-11-13T10:55:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.093+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 >> 172.21.15.16:0/4244994781 conn(0x7f813806a190 msgr2=0x7f81380b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.093+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 shutdown_connections 2023-11-13T10:55:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.093+0000 7f813e8e0640 1 -- 172.21.15.16:0/4244994781 wait complete. 2023-11-13T10:55:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.098+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 --> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1020002cb0 con 0x7f101c0744d0 2023-11-13T10:55:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.099+0000 7f102e7fc640 1 -- 172.21.15.16:0/3596597707 <== osd.6 v2:172.21.15.184:6816/1720475067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1020002cb0 con 0x7f101c0744d0 2023-11-13T10:55:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.100+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f101c0744d0 msgr2=0x7f101c076910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.100+0000 7f103fbf5640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f101c0744d0 0x7f101c076910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.100+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f101c06af10 msgr2=0x7f101c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.100+0000 7f103fbf5640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f101c06af10 0x7f101c06d3d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f10300012e0 tx=0x7f1030039040 comp rx=0 tx=0).stop 2023-11-13T10:55:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.101+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10380cc150 msgr2=0x7f10380cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.101+0000 7f103fbf5640 1 --2- 172.21.15.16:0/3596597707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10380cc150 0x7f10380cc5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1034001730 tx=0x7f1034001760 comp rx=0 tx=0).stop 2023-11-13T10:55:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.101+0000 7f103e192640 1 -- 172.21.15.16:0/3596597707 reap_dead start 2023-11-13T10:55:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.101+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 shutdown_connections 2023-11-13T10:55:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.101+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 >> 172.21.15.16:0/3596597707 conn(0x7f10380d2ce0 msgr2=0x7f103806b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.102+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 shutdown_connections 2023-11-13T10:55:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:58.102+0000 7f103fbf5640 1 -- 172.21.15.16:0/3596597707 wait complete. 2023-11-13T10:55:58.771 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2023-11-13T10:55:58.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-13T10:55:58.773 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2023-11-13T10:55:58.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-13T10:55:58.777 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2023-11-13T10:55:58.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-13T10:55:58.778 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2023-11-13T10:55:58.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-13T10:55:58.779 INFO:teuthology.orchestra.run.smithi016.stdout:64424509452 2023-11-13T10:55:58.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-13T10:55:58.781 INFO:teuthology.orchestra.run.smithi016.stdout:64424509452 2023-11-13T10:55:58.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-13T10:55:58.788 INFO:teuthology.orchestra.run.smithi016.stdout:60129542156 2023-11-13T10:55:58.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-13T10:55:58.795 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2023-11-13T10:55:58.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-13T10:55:59.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.128+0000 7f6602bf0640 1 Processor -- start 2023-11-13T10:55:59.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.128+0000 7f6602bf0640 1 -- start start 2023-11-13T10:55:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.130+0000 7f6602bf0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f65fc0ce110 0x7f65fc06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.130+0000 7f6602bf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.130+0000 7f6602bf0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65fc071a70 con 0x7f65fc161170 2023-11-13T10:55:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.130+0000 7f6602bf0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65fc06e3b0 con 0x7f65fc0ce110 2023-11-13T10:55:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.130+0000 7f6602bf0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc06e160 con 0x7f65fc0710e0 2023-11-13T10:55:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.131+0000 7f65fbfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f65fc0ce110 0x7f65fc06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.131+0000 7f660098c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f65fc161170 0x7f65fc161550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43562/0 (socket says 172.21.15.16:43562) 2023-11-13T10:55:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.131+0000 7f660098c640 1 -- 172.21.15.16:0/1274388155 learned_addr learned my addr 172.21.15.16:0/1274388155 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.131+0000 7f660118d640 1 --2- 172.21.15.16:0/1274388155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.132+0000 7f660118d640 1 -- 172.21.15.16:0/1274388155 >> v2:172.21.15.184:3300/0 conn(0x7f65fc0ce110 msgr2=0x7f65fc06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.132+0000 7f660118d640 1 --2- 172.21.15.16:0/1274388155 >> v2:172.21.15.184:3300/0 conn(0x7f65fc0ce110 0x7f65fc06dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.132+0000 7f660118d640 1 -- 172.21.15.16:0/1274388155 >> v1:172.21.15.184:6789/0 conn(0x7f65fc161170 legacy=0x7f65fc161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.132+0000 7f660118d640 1 -- 172.21.15.16:0/1274388155 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc0b3ac0 con 0x7f65fc0710e0 2023-11-13T10:55:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.134+0000 7f660118d640 1 --2- 172.21.15.16:0/1274388155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc071530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f65f00097a0 tx=0x7f65f002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=679750545fd9e6c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.135+0000 7f65fb7fe640 1 -- 172.21.15.16:0/1274388155 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0031780 con 0x7f65fc0710e0 2023-11-13T10:55:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.135+0000 7f65fb7fe640 1 -- 172.21.15.16:0/1274388155 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f00318e0 con 0x7f65fc0710e0 2023-11-13T10:55:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.135+0000 7f65fb7fe640 1 -- 172.21.15.16:0/1274388155 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0031c70 con 0x7f65fc0710e0 2023-11-13T10:55:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.136+0000 7f6602bf0640 1 -- 172.21.15.16:0/1274388155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 msgr2=0x7f65fc071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.136+0000 7f6602bf0640 1 --2- 172.21.15.16:0/1274388155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc071530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f65f00097a0 tx=0x7f65f002b820 comp rx=0 tx=0).stop 2023-11-13T10:55:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.136+0000 7f6602bf0640 1 -- 172.21.15.16:0/1274388155 shutdown_connections 2023-11-13T10:55:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.136+0000 7f6602bf0640 1 --2- 172.21.15.16:0/1274388155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc071530 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.136+0000 7f6602bf0640 1 --2- 172.21.15.16:0/1274388155 >> v2:172.21.15.184:3300/0 conn(0x7f65fc0ce110 0x7f65fc06dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.136+0000 7f6602bf0640 1 -- 172.21.15.16:0/1274388155 >> 172.21.15.16:0/1274388155 conn(0x7f65fc0b0510 msgr2=0x7f65fc0b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.137+0000 7f6602bf0640 1 -- 172.21.15.16:0/1274388155 shutdown_connections 2023-11-13T10:55:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.137+0000 7f6602bf0640 1 -- 172.21.15.16:0/1274388155 wait complete. 2023-11-13T10:55:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.140+0000 7f6602bf0640 1 Processor -- start 2023-11-13T10:55:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.140+0000 7f6602bf0640 1 -- start start 2023-11-13T10:55:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.142+0000 7f6602bf0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.142+0000 7f6602bf0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65fc0ce110 0x7f65fc145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.143+0000 7f6602bf0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65fc161170 0x7f65fc1446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.143+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65fc0ce110 0x7f65fc145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.143+0000 7f6602bf0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc1469f0 con 0x7f65fc0ce110 2023-11-13T10:55:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.143+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65fc0ce110 0x7f65fc145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56072/0 (socket says 172.21.15.16:56072) 2023-11-13T10:55:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.143+0000 7f660098c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.143+0000 7f660118d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65fc161170 0x7f65fc1446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.143+0000 7f660098c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc0aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50432/0 (socket says 172.21.15.16:50432) 2023-11-13T10:55:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.143+0000 7f660098c640 1 -- 172.21.15.16:0/415523007 learned_addr learned my addr 172.21.15.16:0/415523007 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.144+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc145de0 con 0x7f65fc161170 2023-11-13T10:55:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.144+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc1460c0 con 0x7f65fc0710e0 2023-11-13T10:55:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.145+0000 7f65fbfff640 1 -- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 msgr2=0x7f65fc0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.145+0000 7f65fbfff640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.145+0000 7f65fbfff640 1 -- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65fc161170 msgr2=0x7f65fc1446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.145+0000 7f65fbfff640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65fc161170 0x7f65fc1446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.145+0000 7f65fbfff640 1 -- 172.21.15.16:0/415523007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f0009400 con 0x7f65fc0ce110 2023-11-13T10:55:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.145+0000 7f660098c640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:55:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.145+0000 7f65fbfff640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65fc0ce110 0x7f65fc145340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f65f4007d80 tx=0x7f65f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.145+0000 7fbd73fff640 1 Processor -- start 2023-11-13T10:55:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.146+0000 7fbd73fff640 1 -- start start 2023-11-13T10:55:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.146+0000 7f65f97fa640 1 -- 172.21.15.16:0/415523007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4014420 con 0x7f65fc0ce110 2023-11-13T10:55:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.146+0000 7f65f97fa640 1 -- 172.21.15.16:0/415523007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f4014580 con 0x7f65fc0ce110 2023-11-13T10:55:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.146+0000 7f65f97fa640 1 -- 172.21.15.16:0/415523007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4014910 con 0x7f65fc0ce110 2023-11-13T10:55:59.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.146+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65fc1463a0 con 0x7f65fc0ce110 2023-11-13T10:55:59.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.147+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65fc1586e0 con 0x7f65fc0ce110 2023-11-13T10:55:59.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.148+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65fc0675b0 con 0x7f65fc0ce110 2023-11-13T10:55:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.150+0000 7fbd73fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbd6c058990 0x7fbd6c053500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.150+0000 7fbd73fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6c057d60 con 0x7fbd6c056a80 2023-11-13T10:55:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.150+0000 7fbd73fff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6c006230 con 0x7fbd6c0e76e0 2023-11-13T10:55:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.150+0000 7fbd73fff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c006510 con 0x7fbd6c058990 2023-11-13T10:55:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.151+0000 7fbd737fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbd6c056a80 0x7fbd6c006ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34498/0 (socket says 172.21.15.16:34498) 2023-11-13T10:55:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.152+0000 7fbd737fe640 1 -- 172.21.15.16:0/4273369311 learned_addr learned my addr 172.21.15.16:0/4273369311 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.151+0000 7fbd727fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbd6c058990 0x7fbd6c053500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.152+0000 7fbd727fc640 1 -- 172.21.15.16:0/4273369311 >> v1:172.21.15.184:6789/0 conn(0x7fbd6c0e76e0 legacy=0x7fbd6c04ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.152+0000 7fbd727fc640 1 -- 172.21.15.16:0/4273369311 >> v1:172.21.15.16:6789/0 conn(0x7fbd6c056a80 legacy=0x7fbd6c006ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.152+0000 7fbd727fc640 1 -- 172.21.15.16:0/4273369311 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c0067f0 con 0x7fbd6c058990 2023-11-13T10:55:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.151+0000 7f29273a9640 1 Processor -- start 2023-11-13T10:55:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.151+0000 7f29273a9640 1 -- start start 2023-11-13T10:55:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.153+0000 7f29273a9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f292015d0f0 0x7f29200d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.153+0000 7f29273a9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29200d3c70 con 0x7f29200d0ac0 2023-11-13T10:55:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.153+0000 7f29273a9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29200c69d0 con 0x7f29200cfe20 2023-11-13T10:55:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.153+0000 7f29273a9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29200c6cb0 con 0x7f292015d0f0 2023-11-13T10:55:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.154+0000 7f2925145640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f292015d0f0 0x7f29200d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.154+0000 7f2924944640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f29200cfe20 0x7f29200b3950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43578/0 (socket says 172.21.15.16:43578) 2023-11-13T10:55:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.154+0000 7f2924944640 1 -- 172.21.15.16:0/1666430267 learned_addr learned my addr 172.21.15.16:0/1666430267 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7f02c0db6640 1 Processor -- start 2023-11-13T10:55:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.153+0000 7f65f97fa640 1 -- 172.21.15.16:0/415523007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f65f4014a70 con 0x7f65fc0ce110 2023-11-13T10:55:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.154+0000 7f65f97fa640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f65e806af40 0x7f65e806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.154+0000 7f65f97fa640 1 -- 172.21.15.16:0/415523007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f65f408ac00 con 0x7f65fc0ce110 2023-11-13T10:55:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7f02c0db6640 1 -- start start 2023-11-13T10:55:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7f291ffff640 1 -- 172.21.15.16:0/1666430267 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117298945 0 0) 0x7f29200c69d0 con 0x7f29200cfe20 2023-11-13T10:55:59.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.154+0000 7fbd727fc640 1 --2- 172.21.15.16:0/4273369311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd6c058990 0x7fbd6c053500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd640097a0 tx=0x7fbd6402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=948602a502199696 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7fbd71ffb640 1 -- 172.21.15.16:0/4273369311 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64031830 con 0x7fbd6c058990 2023-11-13T10:55:59.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7fbd71ffb640 1 -- 172.21.15.16:0/4273369311 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd64031990 con 0x7fbd6c058990 2023-11-13T10:55:59.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7fbd71ffb640 1 -- 172.21.15.16:0/4273369311 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64031d20 con 0x7fbd6c058990 2023-11-13T10:55:59.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.156+0000 7fbd73fff640 1 -- 172.21.15.16:0/4273369311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd6c058990 msgr2=0x7fbd6c053500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.156+0000 7fbd73fff640 1 --2- 172.21.15.16:0/4273369311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd6c058990 0x7fbd6c053500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd640097a0 tx=0x7fbd6402b8d0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.156+0000 7fbd73fff640 1 -- 172.21.15.16:0/4273369311 shutdown_connections 2023-11-13T10:55:59.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.156+0000 7fbd73fff640 1 --2- 172.21.15.16:0/4273369311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd6c058990 0x7fbd6c053500 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.156+0000 7fbd73fff640 1 -- 172.21.15.16:0/4273369311 >> 172.21.15.16:0/4273369311 conn(0x7fbd6c058eb0 msgr2=0x7fbd6c059290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7f660098c640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f65e806af40 0x7f65e806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7f291ffff640 1 -- 172.21.15.16:0/1666430267 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2910003520 con 0x7f29200cfe20 2023-11-13T10:55:59.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.156+0000 7fbd73fff640 1 -- 172.21.15.16:0/4273369311 shutdown_connections 2023-11-13T10:55:59.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.157+0000 7fbd73fff640 1 -- 172.21.15.16:0/4273369311 wait complete. 2023-11-13T10:55:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.157+0000 7f8b22b39640 1 Processor -- start 2023-11-13T10:55:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7f2925145640 1 -- 172.21.15.16:0/1666430267 >> v1:172.21.15.184:6789/0 conn(0x7f29200cfe20 legacy=0x7f29200b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7f2925145640 1 -- 172.21.15.16:0/1666430267 >> v1:172.21.15.16:6789/0 conn(0x7f29200d0ac0 legacy=0x7f29200b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.155+0000 7f2925145640 1 -- 172.21.15.16:0/1666430267 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200c6f90 con 0x7f292015d0f0 2023-11-13T10:55:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.157+0000 7f02c0db6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc15d0f0 0x7f02bc0d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.157+0000 7f02c0db6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02bc0d3c70 con 0x7f02bc0cfe20 2023-11-13T10:55:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.157+0000 7f02c0db6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02bc0c69d0 con 0x7f02bc0d0ac0 2023-11-13T10:55:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.157+0000 7f02c0db6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0c6cb0 con 0x7f02bc15d0f0 2023-11-13T10:55:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.157+0000 7f8b22b39640 1 -- start start 2023-11-13T10:55:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.158+0000 7f02bad9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f02bc0d0ac0 0x7f02bc0b4060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43584/0 (socket says 172.21.15.16:43584) 2023-11-13T10:55:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.158+0000 7f02bad9d640 1 -- 172.21.15.16:0/1228564755 learned_addr learned my addr 172.21.15.16:0/1228564755 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.158+0000 7f660098c640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f65e806af40 0x7f65e806d400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005880 tx=0x7f65ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.158+0000 7f02ba59c640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc15d0f0 0x7f02bc0d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f2925145640 1 --2- 172.21.15.16:0/1666430267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f292015d0f0 0x7f29200d12c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2914009d90 tx=0x7f291402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e8dc994232405e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167286570 0 0) 0x7f02bc0c69d0 con 0x7f02bc0d0ac0 2023-11-13T10:55:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f7605490640 1 Processor -- start 2023-11-13T10:55:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f7605490640 1 -- start start 2023-11-13T10:55:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a4003520 con 0x7f02bc0d0ac0 2023-11-13T10:55:59.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361236081 0 0) 0x7f02b0001110 con 0x7f02bc0cfe20 2023-11-13T10:55:59.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02bc0c69d0 con 0x7f02bc0cfe20 2023-11-13T10:55:59.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7fbd73fff640 1 Processor -- start 2023-11-13T10:55:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7fbd73fff640 1 -- start start 2023-11-13T10:55:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f291ffff640 1 -- 172.21.15.16:0/1666430267 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291402cc60 con 0x7f292015d0f0 2023-11-13T10:55:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f291ffff640 1 -- 172.21.15.16:0/1666430267 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291402cdc0 con 0x7f292015d0f0 2023-11-13T10:55:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.159+0000 7f291ffff640 1 -- 172.21.15.16:0/1666430267 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914034660 con 0x7f292015d0f0 2023-11-13T10:55:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3565695925 0 0) 0x7f02a4003520 con 0x7f02bc0d0ac0 2023-11-13T10:55:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc15d0f0 msgr2=0x7f02bc0d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f8b22b39640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b1c158e10 0x7f8b1c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f8b22b39640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8b1c070b30 0x7f8b1c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc15d0f0 0x7f02bc0d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 >> v1:172.21.15.16:6789/0 conn(0x7f02bc0cfe20 legacy=0x7f02bc0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0c6f90 con 0x7f02bc0d0ac0 2023-11-13T10:55:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f02b4001920 con 0x7f02bc0d0ac0 2023-11-13T10:55:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 >> v1:172.21.15.184:6789/0 conn(0x7f02bc0d0ac0 legacy=0x7f02bc0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a4003b00 0x7f02a4003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a40047c0 0x7f02a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 0x7f02a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0c6cb0 con 0x7f02a4005180 2023-11-13T10:55:59.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a400bef0 con 0x7f02a40047c0 2023-11-13T10:55:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a400c0b0 con 0x7f02a4003b00 2023-11-13T10:55:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f8b22b39640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1c071080 con 0x7f8b1c0b10e0 2023-11-13T10:55:59.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f8b22b39640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c071360 con 0x7f8b1c158e10 2023-11-13T10:55:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.160+0000 7f8b22b39640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c071640 con 0x7f8b1c070b30 2023-11-13T10:55:59.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f29273a9640 1 -- 172.21.15.16:0/1666430267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f292015d0f0 msgr2=0x7f29200d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f29273a9640 1 --2- 172.21.15.16:0/1666430267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f292015d0f0 0x7f29200d12c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2914009d90 tx=0x7f291402beb0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f02ba59c640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc15d0f0 0x7f02bc0d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:55:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f29273a9640 1 -- 172.21.15.16:0/1666430267 shutdown_connections 2023-11-13T10:55:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f29273a9640 1 --2- 172.21.15.16:0/1666430267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f292015d0f0 0x7f29200d12c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f29273a9640 1 -- 172.21.15.16:0/1666430267 >> 172.21.15.16:0/1666430267 conn(0x7f29200d2ce0 msgr2=0x7f29200d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f8b208d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b1c158e10 0x7f8b1c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f29273a9640 1 -- 172.21.15.16:0/1666430267 shutdown_connections 2023-11-13T10:55:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f02bad9d640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 0x7f02a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f8b1bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8b1c070b30 0x7f8b1c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f8b1bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8b1c070b30 0x7f8b1c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:38988/0 (socket says 172.21.15.16:38988) 2023-11-13T10:55:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f8b1bfff640 1 -- 172.21.15.16:0/4111895570 learned_addr learned my addr 172.21.15.16:0/4111895570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f02ba59c640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a4003b00 0x7f02a4003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f65f97fa640 1 -- 172.21.15.16:0/415523007 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65f40559c0 con 0x7f65fc0ce110 2023-11-13T10:55:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f02b9d9b640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a40047c0 0x7f02a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c056a80 0x7fbd6c0e1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd6c058990 0x7fbd6c051520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f7605490640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f760015d0f0 0x7f76000d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f7605490640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000d3c70 con 0x7f76000d0ac0 2023-11-13T10:55:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f7605490640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000c69d0 con 0x7f76000cfe20 2023-11-13T10:55:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f7605490640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76000c6cb0 con 0x7f760015d0f0 2023-11-13T10:55:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f02bad9d640 1 -- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a4003b00 msgr2=0x7f02a4003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.161+0000 7f29273a9640 1 -- 172.21.15.16:0/1666430267 wait complete. 2023-11-13T10:55:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f02bad9d640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02a4003b00 0x7f02a4003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f02bad9d640 1 -- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a40047c0 msgr2=0x7f02a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f02bad9d640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02a40047c0 0x7f02a4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f02bad9d640 1 -- 172.21.15.16:0/1228564755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0c6f90 con 0x7f02a4005180 2023-11-13T10:55:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f75f67fc640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f76000cfe20 0x7f76000b3950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43594/0 (socket says 172.21.15.16:43594) 2023-11-13T10:55:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f75f67fc640 1 -- 172.21.15.16:0/2919510545 learned_addr learned my addr 172.21.15.16:0/2919510545 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f8b1bfff640 1 -- 172.21.15.16:0/4111895570 >> v2:172.21.15.16:3300/0 conn(0x7f8b1c158e10 msgr2=0x7f8b1c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f8b1bfff640 1 --2- 172.21.15.16:0/4111895570 >> v2:172.21.15.16:3300/0 conn(0x7f8b1c158e10 0x7f8b1c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f8b1bfff640 1 -- 172.21.15.16:0/4111895570 >> v1:172.21.15.184:6789/0 conn(0x7f8b1c0b10e0 legacy=0x7f8b1c15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.162+0000 7f8b1bfff640 1 -- 172.21.15.16:0/4111895570 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c0b1f40 con 0x7f8b1c070b30 2023-11-13T10:55:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7f75feffd640 1 --2- 172.21.15.16:0/2919510545 >> v2:172.21.15.184:3300/0 conn(0x7f760015d0f0 0x7f76000d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7fbd72ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c056a80 0x7fbd6c0e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7fbd72ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c056a80 0x7fbd6c0e1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56104/0 (socket says 172.21.15.16:56104) 2023-11-13T10:55:59.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7f75fe7fc640 1 -- 172.21.15.16:0/2919510545 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279458917 0 0) 0x7f76000c69d0 con 0x7f76000cfe20 2023-11-13T10:55:59.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7f75fe7fc640 1 -- 172.21.15.16:0/2919510545 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75ec003520 con 0x7f76000cfe20 2023-11-13T10:55:59.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd6c0e76e0 0x7fbd6c04dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7fbd73fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0e1dd0 con 0x7fbd6c056a80 2023-11-13T10:55:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7fbd73fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c051000 con 0x7fbd6c0e76e0 2023-11-13T10:55:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7fbd73fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0e7cd0 con 0x7fbd6c058990 2023-11-13T10:55:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.163+0000 7fbd72ffd640 1 -- 172.21.15.16:0/2258464690 learned_addr learned my addr 172.21.15.16:0/2258464690 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7f75fe7fc640 1 -- 172.21.15.16:0/2919510545 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206550368 0 0) 0x7f75f8001110 con 0x7f76000d0ac0 2023-11-13T10:55:59.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7f75fe7fc640 1 -- 172.21.15.16:0/2919510545 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76000c69d0 con 0x7f76000d0ac0 2023-11-13T10:55:59.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7fbd72ffd640 1 -- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd6c058990 msgr2=0x7fbd6c051520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7fbd72ffd640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd6c058990 0x7fbd6c051520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7fbd72ffd640 1 -- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd6c0e76e0 msgr2=0x7fbd6c04dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7fbd72ffd640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd6c0e76e0 0x7fbd6c04dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7fbd72ffd640 1 -- 172.21.15.16:0/2258464690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009400 con 0x7fbd6c056a80 2023-11-13T10:55:59.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7fbd72ffd640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c056a80 0x7fbd6c0e1890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00c400 tx=0x7fbd5c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7f02bad9d640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 0x7f02a400b980 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0d0f70 tx=0x7f02a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40e7960d44e2cdd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4034630 con 0x7f02a4005180 2023-11-13T10:55:59.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b402f070 con 0x7f02a4005180 2023-11-13T10:55:59.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7f8b1bfff640 1 --2- 172.21.15.16:0/4111895570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 0x7f8b1c06c330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c072280 tx=0x7f8b0802bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d5d2c1c6a8217fb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7f75feffd640 1 -- 172.21.15.16:0/2919510545 >> v1:172.21.15.184:6789/0 conn(0x7f76000cfe20 legacy=0x7f76000b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7f75fe7fc640 1 -- 172.21.15.16:0/2919510545 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3448202729 0 0) 0x7f75ec003520 con 0x7f76000cfe20 2023-11-13T10:55:59.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7f75feffd640 1 -- 172.21.15.16:0/2919510545 >> v1:172.21.15.16:6789/0 conn(0x7f76000d0ac0 legacy=0x7f76000b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7fa3c2bc7640 1 Processor -- start 2023-11-13T10:55:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/2258464690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c0052d0 con 0x7fbd6c056a80 2023-11-13T10:55:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7f02b959a640 1 -- 172.21.15.16:0/1228564755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4034a10 con 0x7f02a4005180 2023-11-13T10:55:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/2258464690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c005430 con 0x7fbd6c056a80 2023-11-13T10:55:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/2258464690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c0052d0 con 0x7fbd6c056a80 2023-11-13T10:55:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c0e7fb0 con 0x7fbd6c056a80 2023-11-13T10:55:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.165+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c0e8570 con 0x7fbd6c056a80 2023-11-13T10:55:59.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7fa3c2bc7640 1 -- start start 2023-11-13T10:55:59.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f29273a9640 1 Processor -- start 2023-11-13T10:55:59.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f02c0db6640 1 -- 172.21.15.16:0/1228564755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 msgr2=0x7f02a400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f02c0db6640 1 --2- 172.21.15.16:0/1228564755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 0x7f02a400b980 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0d0f70 tx=0x7f02a40057f0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.164+0000 7f75feffd640 1 -- 172.21.15.16:0/2919510545 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000c6f90 con 0x7f760015d0f0 2023-11-13T10:55:59.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f02bad9d640 1 -- 172.21.15.16:0/1228564755 reap_dead start 2023-11-13T10:55:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f29273a9640 1 -- start start 2023-11-13T10:55:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f8b1b7fe640 1 -- 172.21.15.16:0/4111895570 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0802c9f0 con 0x7f8b1c070b30 2023-11-13T10:55:59.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f02c0db6640 1 -- 172.21.15.16:0/1228564755 shutdown_connections 2023-11-13T10:55:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f02c0db6640 1 -- 172.21.15.16:0/1228564755 >> 172.21.15.16:0/1228564755 conn(0x7f02bc0d2ce0 msgr2=0x7f02bc0d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f8b1b7fe640 1 -- 172.21.15.16:0/4111895570 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0802cb50 con 0x7f8b1c070b30 2023-11-13T10:55:59.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.166+0000 7f8b1b7fe640 1 -- 172.21.15.16:0/4111895570 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08034440 con 0x7f8b1c070b30 2023-11-13T10:55:59.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.167+0000 7f75feffd640 1 --2- 172.21.15.16:0/2919510545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f760015d0f0 0x7f76000d12c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009720 tx=0x7f75e802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e441f09fb93672d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.167+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c0076a0 con 0x7fbd6c056a80 2023-11-13T10:55:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.167+0000 7f75fe7fc640 1 -- 172.21.15.16:0/2919510545 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f00019f0 con 0x7f760015d0f0 2023-11-13T10:55:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.167+0000 7fa3c2bc7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc159ae0 0x7fa3bc0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.167+0000 7fa3c2bc7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa3bc0adbc0 0x7fa3bc0ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7f75fe7fc640 1 -- 172.21.15.16:0/2919510545 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e80315c0 con 0x7f760015d0f0 2023-11-13T10:55:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7f75fe7fc640 1 -- 172.21.15.16:0/2919510545 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8031930 con 0x7f760015d0f0 2023-11-13T10:55:59.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7f02c0db6640 1 -- 172.21.15.16:0/1228564755 shutdown_connections 2023-11-13T10:55:59.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7fa3c2bc7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc159ec0 con 0x7fa3bc15d680 2023-11-13T10:55:59.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7fa3c2bc7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc15dc20 con 0x7fa3bc0adbc0 2023-11-13T10:55:59.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7fa3c2bc7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc15df00 con 0x7fa3bc159ae0 2023-11-13T10:55:59.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.167+0000 7f8b22b39640 1 -- 172.21.15.16:0/4111895570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 msgr2=0x7f8b1c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7fa3c1164640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa3bc15d680 0x7fa3bc06ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34532/0 (socket says 172.21.15.16:34532) 2023-11-13T10:55:59.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7fa3c1164640 1 -- 172.21.15.16:0/2557773217 learned_addr learned my addr 172.21.15.16:0/2557773217 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:55:59.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.167+0000 7f8b22b39640 1 --2- 172.21.15.16:0/4111895570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 0x7f8b1c06c330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c072280 tx=0x7f8b0802bc40 comp rx=0 tx=0).stop 2023-11-13T10:55:59.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7f8b22b39640 1 -- 172.21.15.16:0/4111895570 shutdown_connections 2023-11-13T10:55:59.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7f8b22b39640 1 --2- 172.21.15.16:0/4111895570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 0x7f8b1c06c330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7f8b22b39640 1 --2- 172.21.15.16:0/4111895570 >> v2:172.21.15.16:3300/0 conn(0x7f8b1c158e10 0x7f8b1c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7f8b22b39640 1 -- 172.21.15.16:0/4111895570 >> 172.21.15.16:0/4111895570 conn(0x7f8b1c0b05f0 msgr2=0x7f8b1c0b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7f02c0db6640 1 -- 172.21.15.16:0/1228564755 wait complete. 2023-11-13T10:55:59.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.169+0000 7f7605490640 1 -- 172.21.15.16:0/2919510545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f760015d0f0 msgr2=0x7f76000d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.169+0000 7f7605490640 1 --2- 172.21.15.16:0/2919510545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f760015d0f0 0x7f76000d12c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009720 tx=0x7f75e802b860 comp rx=0 tx=0).stop 2023-11-13T10:55:59.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.169+0000 7f8b22b39640 1 -- 172.21.15.16:0/4111895570 shutdown_connections 2023-11-13T10:55:59.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.169+0000 7f29273a9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29200cfe20 0x7f29200b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.169+0000 7f29273a9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200d0ac0 0x7f29200c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.169+0000 7f29273a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292015d0f0 0x7f29200b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7f29273a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200cc0a0 con 0x7f292015d0f0 2023-11-13T10:55:59.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7f29273a9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200b6d20 con 0x7f29200cfe20 2023-11-13T10:55:59.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7f2925946640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292015d0f0 0x7f29200b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7f2925145640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29200cfe20 0x7f29200b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7f29273a9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200cc280 con 0x7f29200d0ac0 2023-11-13T10:55:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7f2925145640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29200cfe20 0x7f29200b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39048/0 (socket says 172.21.15.16:39048) 2023-11-13T10:55:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7f2925145640 1 -- 172.21.15.16:0/2649169631 learned_addr learned my addr 172.21.15.16:0/2649169631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7fa3bbfff640 1 --2- 172.21.15.16:0/2557773217 >> v2:172.21.15.184:3300/0 conn(0x7fa3bc0adbc0 0x7fa3bc0ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.168+0000 7fa3c0963640 1 --2- 172.21.15.16:0/2557773217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc159ae0 0x7fa3bc0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7fa3bbfff640 1 -- 172.21.15.16:0/2557773217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc159ae0 msgr2=0x7fa3bc0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7fa3bbfff640 1 --2- 172.21.15.16:0/2557773217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc159ae0 0x7fa3bc0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7fa3bbfff640 1 -- 172.21.15.16:0/2557773217 >> v1:172.21.15.16:6789/0 conn(0x7fa3bc15d680 legacy=0x7fa3bc06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7fa3bbfff640 1 -- 172.21.15.16:0/2557773217 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc15e1e0 con 0x7fa3bc0adbc0 2023-11-13T10:55:59.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7f2924944640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200d0ac0 0x7f29200c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.171+0000 7f7605490640 1 -- 172.21.15.16:0/2919510545 shutdown_connections 2023-11-13T10:55:59.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.171+0000 7f7605490640 1 --2- 172.21.15.16:0/2919510545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f760015d0f0 0x7f76000d12c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.171+0000 7f7605490640 1 -- 172.21.15.16:0/2919510545 >> 172.21.15.16:0/2919510545 conn(0x7f76000d2ce0 msgr2=0x7f76000d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.171+0000 7f7605490640 1 -- 172.21.15.16:0/2919510545 shutdown_connections 2023-11-13T10:55:59.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.171+0000 7f7605490640 1 -- 172.21.15.16:0/2919510545 wait complete. 2023-11-13T10:55:59.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.171+0000 7f2924944640 1 -- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29200cfe20 msgr2=0x7f29200b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.171+0000 7f2924944640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29200cfe20 0x7f29200b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.171+0000 7f2924944640 1 -- 172.21.15.16:0/2649169631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292015d0f0 msgr2=0x7f29200b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.170+0000 7f8b22b39640 1 -- 172.21.15.16:0/4111895570 wait complete. 2023-11-13T10:55:59.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.172+0000 7f2924944640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292015d0f0 0x7f29200b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.172+0000 7f2924944640 1 -- 172.21.15.16:0/2649169631 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2914009a70 con 0x7f29200d0ac0 2023-11-13T10:55:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.172+0000 7f2925946640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292015d0f0 0x7f29200b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:55:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.172+0000 7f02c0db6640 1 Processor -- start 2023-11-13T10:55:59.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.172+0000 7fa3bbfff640 1 --2- 172.21.15.16:0/2557773217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3bc0adbc0 0x7fa3bc0ce870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00f230 tx=0x7fa3ac034620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ce4750cc1f0e2a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.172+0000 7f2924944640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200d0ac0 0x7f29200c9db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f290c001eb0 tx=0x7f290c00c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.173+0000 7f8b22b39640 1 Processor -- start 2023-11-13T10:55:59.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.173+0000 7f02c0db6640 1 -- start start 2023-11-13T10:55:59.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.173+0000 7f8b22b39640 1 -- start start 2023-11-13T10:55:59.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.173+0000 7f291dffb640 1 -- 172.21.15.16:0/2649169631 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c0074e0 con 0x7f29200d0ac0 2023-11-13T10:55:59.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7f7605490640 1 Processor -- start 2023-11-13T10:55:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7fa3bb7fe640 1 -- 172.21.15.16:0/2557773217 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac034ea0 con 0x7fa3bc0adbc0 2023-11-13T10:55:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7fa3bb7fe640 1 -- 172.21.15.16:0/2557773217 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac037050 con 0x7fa3bc0adbc0 2023-11-13T10:55:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7fa3bb7fe640 1 -- 172.21.15.16:0/2557773217 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00b740 con 0x7fa3bc0adbc0 2023-11-13T10:55:59.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7f7605490640 1 -- start start 2023-11-13T10:55:59.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7f291dffb640 1 -- 172.21.15.16:0/2649169631 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c007640 con 0x7f29200d0ac0 2023-11-13T10:55:59.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7f29273a9640 1 -- 172.21.15.16:0/2649169631 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29200cc560 con 0x7f29200d0ac0 2023-11-13T10:55:59.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.173+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/2258464690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd5c0035f0 con 0x7fbd6c056a80 2023-11-13T10:55:59.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7fbd6b7fe640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbd5406aea0 0x7fbd5406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/2258464690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd5c089280 con 0x7fbd6c056a80 2023-11-13T10:55:59.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.175+0000 7fbd727fc640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbd5406aea0 0x7fbd5406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7f291dffb640 1 -- 172.21.15.16:0/2649169631 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c0049e0 con 0x7f29200d0ac0 2023-11-13T10:55:59.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.174+0000 7f29273a9640 1 -- 172.21.15.16:0/2649169631 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29200b7250 con 0x7f29200d0ac0 2023-11-13T10:55:59.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.176+0000 7f02c0db6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 0x7f02bc0b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.176+0000 7f02c0db6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02bc1f2750 0x7f02bc15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.176+0000 7f02c0db6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc144260 0x7f02bc157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.176+0000 7f02c0db6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0cb7d0 con 0x7f02a4005180 2023-11-13T10:55:59.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.176+0000 7f02c0db6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc15ae10 con 0x7f02bc1f2750 2023-11-13T10:55:59.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.176+0000 7f290b7fe640 1 -- 172.21.15.16:0/2649169631 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f8005250 con 0x7f29200d0ac0 2023-11-13T10:55:59.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.176+0000 7f02c0db6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0caea0 con 0x7f02bc144260 2023-11-13T10:55:59.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.176+0000 7f8b22b39640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 0x7f8b1c1423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.176+0000 7fbd727fc640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbd5406aea0 0x7fbd5406d360 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd64036000 tx=0x7fbd64030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.175+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/2557773217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3bc0adbc0 msgr2=0x7fa3bc0ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.175+0000 7fa3c2bc7640 1 --2- 172.21.15.16:0/2557773217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3bc0adbc0 0x7fa3bc0ce870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00f230 tx=0x7fa3ac034620 comp rx=0 tx=0).stop 2023-11-13T10:55:59.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f8b22b39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0b10e0 0x7f8b1c15a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f02b9d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02bc1f2750 0x7f02bc15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f02bad9d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc144260 0x7f02bc157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/2557773217 shutdown_connections 2023-11-13T10:55:59.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7fa3c2bc7640 1 --2- 172.21.15.16:0/2557773217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3bc0adbc0 0x7fa3bc0ce870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7fa3c2bc7640 1 --2- 172.21.15.16:0/2557773217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc159ae0 0x7fa3bc0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f02b9d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02bc1f2750 0x7f02bc15a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39064/0 (socket says 172.21.15.16:39064) 2023-11-13T10:55:59.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f02b9d9b640 1 -- 172.21.15.16:0/3830319991 learned_addr learned my addr 172.21.15.16:0/3830319991 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f02ba59c640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 0x7f02bc0b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f8b22b39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c158e10 0x7f8b1c155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/2557773217 >> 172.21.15.16:0/2557773217 conn(0x7fa3bc06a190 msgr2=0x7fa3bc06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.175+0000 7f7605490640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76000cfe20 0x7f760009d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f7605490640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76000d0ac0 0x7f76000b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f7605490640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760015d0f0 0x7f76000b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f7605490640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76001f3160 con 0x7f760015d0f0 2023-11-13T10:55:59.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f7605490640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760009e350 con 0x7f76000cfe20 2023-11-13T10:55:59.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f7605490640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760009dab0 con 0x7f76000d0ac0 2023-11-13T10:55:59.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f8b208d5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 0x7f8b1c1423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f75feffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76000cfe20 0x7f760009d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/2557773217 shutdown_connections 2023-11-13T10:55:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f75feffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76000cfe20 0x7f760009d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39092/0 (socket says 172.21.15.16:39092) 2023-11-13T10:55:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f75feffd640 1 -- 172.21.15.16:0/967374450 learned_addr learned my addr 172.21.15.16:0/967374450 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f8b208d5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 0x7f8b1c1423d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39076/0 (socket says 172.21.15.16:39076) 2023-11-13T10:55:59.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.177+0000 7f8b1bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0b10e0 0x7f8b1c15a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f8b22b39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c15b100 con 0x7f8b1c158e10 2023-11-13T10:55:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f8b22b39640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c15b3e0 con 0x7f8b1c070b30 2023-11-13T10:55:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f8b22b39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c15b6c0 con 0x7f8b1c0b10e0 2023-11-13T10:55:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f8b1bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0b10e0 0x7f8b1c15a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50502/0 (socket says 172.21.15.16:50502) 2023-11-13T10:55:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f8b1bfff640 1 -- 172.21.15.16:0/783248106 learned_addr learned my addr 172.21.15.16:0/783248106 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f8b210d6640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c158e10 0x7f8b1c155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f75f67fc640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76000d0ac0 0x7f76000b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f02b9d9b640 1 -- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc144260 msgr2=0x7f02bc157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f291dffb640 1 -- 172.21.15.16:0/2649169631 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f290c004b40 con 0x7f29200d0ac0 2023-11-13T10:55:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f02b9d9b640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc144260 0x7f02bc157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f02b9d9b640 1 -- 172.21.15.16:0/3830319991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 msgr2=0x7f02bc0b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f02b9d9b640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 0x7f02bc0b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7f02b9d9b640 1 -- 172.21.15.16:0/3830319991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0cb180 con 0x7f02bc1f2750 2023-11-13T10:55:59.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f75ff7fe640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760015d0f0 0x7f76000b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.178+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/2557773217 wait complete. 2023-11-13T10:55:59.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f8b210d6640 1 -- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0b10e0 msgr2=0x7f8b1c15a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f8b210d6640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0b10e0 0x7f8b1c15a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f8b210d6640 1 -- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 msgr2=0x7f8b1c1423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f8b210d6640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 0x7f8b1c1423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f8b210d6640 1 -- 172.21.15.16:0/783248106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b080094d0 con 0x7f8b1c158e10 2023-11-13T10:55:59.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7fb0a86cf640 1 Processor -- start 2023-11-13T10:55:59.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.180+0000 7f291dffb640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f290406ae40 0x7f290406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7fb0a86cf640 1 -- start start 2023-11-13T10:55:59.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.180+0000 7f291dffb640 1 -- 172.21.15.16:0/2649169631 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f290c089b80 con 0x7f29200d0ac0 2023-11-13T10:55:59.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.180+0000 7f8b210d6640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c158e10 0x7f8b1c155260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c700 tx=0x7f8b1400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f75feffd640 1 -- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76000d0ac0 msgr2=0x7f76000b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f75feffd640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76000d0ac0 0x7f76000b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f75feffd640 1 -- 172.21.15.16:0/967374450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760015d0f0 msgr2=0x7f76000b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f75feffd640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760015d0f0 0x7f76000b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.179+0000 7f75feffd640 1 -- 172.21.15.16:0/967374450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8009400 con 0x7f76000cfe20 2023-11-13T10:55:59.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.180+0000 7f75feffd640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76000cfe20 0x7f760009d060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75e8034040 tx=0x7f75e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.180+0000 7f02b9d9b640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02bc1f2750 0x7f02bc15a5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e6d0 tx=0x7f02b000eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.180+0000 7f8b197fa640 1 -- 172.21.15.16:0/783248106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14006d50 con 0x7f8b1c158e10 2023-11-13T10:55:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f75f7fff640 1 -- 172.21.15.16:0/967374450 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8003820 con 0x7f76000cfe20 2023-11-13T10:55:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f75f7fff640 1 -- 172.21.15.16:0/967374450 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e8040070 con 0x7f76000cfe20 2023-11-13T10:55:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f75f7fff640 1 -- 172.21.15.16:0/967374450 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8003bc0 con 0x7f76000cfe20 2023-11-13T10:55:59.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/2258464690 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd5c054050 con 0x7fbd6c056a80 2023-11-13T10:55:59.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f2925145640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f290406ae40 0x7f290406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f8b197fa640 1 -- 172.21.15.16:0/783248106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14003970 con 0x7f8b1c158e10 2023-11-13T10:55:59.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1c15b9a0 con 0x7f8b1c158e10 2023-11-13T10:55:59.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f8b197fa640 1 -- 172.21.15.16:0/783248106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14003d20 con 0x7f8b1c158e10 2023-11-13T10:55:59.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f02aaffd640 1 -- 172.21.15.16:0/3830319991 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00050d0 con 0x7f02bc1f2750 2023-11-13T10:55:59.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7fb0a86cf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb0a015d0f0 0x7fb0a00d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7fb0a86cf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0a00cfe20 0x7fb0a00d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7fb0a86cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0a00d0ac0 0x7fb0a00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7fb0a86cf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a00d3c70 con 0x7fb0a00cfe20 2023-11-13T10:55:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7fb0a86cf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a00c69d0 con 0x7fb0a015d0f0 2023-11-13T10:55:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7fb0a86cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00c6cb0 con 0x7fb0a00d0ac0 2023-11-13T10:55:59.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7fa3c2bc7640 1 Processor -- start 2023-11-13T10:55:59.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f02c0db6640 1 -- 172.21.15.16:0/3830319991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc0cb460 con 0x7f02bc1f2750 2023-11-13T10:55:59.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f02c0db6640 1 -- 172.21.15.16:0/3830319991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc1f2bd0 con 0x7f02bc1f2750 2023-11-13T10:55:59.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7f02aaffd640 1 -- 172.21.15.16:0/3830319991 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b0005230 con 0x7f02bc1f2750 2023-11-13T10:55:59.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7f02aaffd640 1 -- 172.21.15.16:0/3830319991 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000c6c0 con 0x7f02bc1f2750 2023-11-13T10:55:59.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760009dd90 con 0x7f76000cfe20 2023-11-13T10:55:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7fb0a5c6a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0a00cfe20 0x7fb0a00d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7fa3c2bc7640 1 -- start start 2023-11-13T10:55:59.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000cb4b0 con 0x7f76000cfe20 2023-11-13T10:55:59.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7fb0a5c6a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0a00cfe20 0x7fb0a00d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56148/0 (socket says 172.21.15.16:56148) 2023-11-13T10:55:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7fb0a5c6a640 1 -- 172.21.15.16:0/1033213844 learned_addr learned my addr 172.21.15.16:0/1033213844 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.182+0000 7f2925145640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f290406ae40 0x7f290406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2914000f80 tx=0x7f2914030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.181+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1c15c1a0 con 0x7f8b1c158e10 2023-11-13T10:55:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.183+0000 7fb0a5c6a640 1 -- 172.21.15.16:0/1033213844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0a00d0ac0 msgr2=0x7fb0a00b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:55:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.183+0000 7fb0a5c6a640 1 --2- 172.21.15.16:0/1033213844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0a00d0ac0 0x7fb0a00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.183+0000 7fb0a5c6a640 1 -- 172.21.15.16:0/1033213844 >> v2:172.21.15.184:3300/0 conn(0x7fb0a015d0f0 msgr2=0x7fb0a00d12c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-13T10:55:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.183+0000 7fb0a5c6a640 1 --2- 172.21.15.16:0/1033213844 >> v2:172.21.15.184:3300/0 conn(0x7fb0a015d0f0 0x7fb0a00d12c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.183+0000 7fb0a5c6a640 1 -- 172.21.15.16:0/1033213844 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00c6f90 con 0x7fb0a00cfe20 2023-11-13T10:55:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.184+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76000cb6d0 con 0x7f76000cfe20 2023-11-13T10:55:59.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.184+0000 7f02a8ff9640 1 -- 172.21.15.16:0/3830319991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0294005250 con 0x7f02bc1f2750 2023-11-13T10:55:59.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.184+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1c067090 con 0x7f8b1c158e10 2023-11-13T10:55:59.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.184+0000 7fb0a5c6a640 1 --2- 172.21.15.16:0/1033213844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a00cfe20 0x7fb0a00d1800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb0a008d4c0 tx=0x7fb09c02b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54093d0384e5ca4f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.185+0000 7fb0a5469640 1 -- 172.21.15.16:0/1033213844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c030890 con 0x7fb0a00cfe20 2023-11-13T10:55:59.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.185+0000 7fb0a5469640 1 -- 172.21.15.16:0/1033213844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c0309f0 con 0x7fb0a00cfe20 2023-11-13T10:55:59.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.185+0000 7fb0a5469640 1 -- 172.21.15.16:0/1033213844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c030dc0 con 0x7fb0a00cfe20 2023-11-13T10:55:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.185+0000 7fa3c2bc7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc0adbc0 0x7fa3bc148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.186+0000 7fa3c2bc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3bc159ae0 0x7fa3bc15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.186+0000 7f8b197fa640 1 -- 172.21.15.16:0/783248106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b140050d0 con 0x7f8b1c158e10 2023-11-13T10:55:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.186+0000 7f75f7fff640 1 -- 172.21.15.16:0/967374450 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f75e8003d20 con 0x7f76000cfe20 2023-11-13T10:55:59.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.186+0000 7fa3c2bc7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3bc15d680 0x7fa3bc153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.186+0000 7fa3c2bc7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0b3370 con 0x7fa3bc159ae0 2023-11-13T10:55:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.187+0000 7f02aaffd640 1 -- 172.21.15.16:0/3830319991 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02b000c820 con 0x7f02bc1f2750 2023-11-13T10:55:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.187+0000 7f8b197fa640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8b0006ae40 0x7f8b0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.187+0000 7f8b197fa640 1 -- 172.21.15.16:0/783248106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8b14088e70 con 0x7f8b1c158e10 2023-11-13T10:55:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.187+0000 7f75f7fff640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f75e006aec0 0x7f75e006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.186+0000 7fb0a86cf640 1 -- 172.21.15.16:0/1033213844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a00cfe20 msgr2=0x7fb0a00d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.186+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/1033213844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a00cfe20 0x7fb0a00d1800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb0a008d4c0 tx=0x7fb09c02b9d0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7fb0a86cf640 1 -- 172.21.15.16:0/1033213844 shutdown_connections 2023-11-13T10:55:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/1033213844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0a00d0ac0 0x7fb0a00b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/1033213844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a00cfe20 0x7fb0a00d1800 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/1033213844 >> v2:172.21.15.184:3300/0 conn(0x7fb0a015d0f0 0x7fb0a00d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7fb0a86cf640 1 -- 172.21.15.16:0/1033213844 >> 172.21.15.16:0/1033213844 conn(0x7fb0a00d2ce0 msgr2=0x7fb0a00d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7f75f7fff640 1 -- 172.21.15.16:0/967374450 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f75e8042070 con 0x7f76000cfe20 2023-11-13T10:55:59.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7f8b208d5640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8b0006ae40 0x7f8b0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7f02aaffd640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f029c06ae70 0x7f029c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7f02aaffd640 1 -- 172.21.15.16:0/3830319991 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f02b0089800 con 0x7f02bc1f2750 2023-11-13T10:55:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7f75f67fc640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f75e006aec0 0x7f75e006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.187+0000 7fa3bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3bc159ae0 0x7fa3bc15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.187+0000 7fa3bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3bc159ae0 0x7fa3bc15bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56154/0 (socket says 172.21.15.16:56154) 2023-11-13T10:55:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.187+0000 7fa3bbfff640 1 -- 172.21.15.16:0/3004397931 learned_addr learned my addr 172.21.15.16:0/3004397931 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7fa3c0963640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc0adbc0 0x7fa3bc148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.188+0000 7fa3c1164640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3bc15d680 0x7fa3bc153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.189+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc15b4f0 con 0x7fa3bc15d680 2023-11-13T10:55:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.189+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc15b7d0 con 0x7fa3bc0adbc0 2023-11-13T10:55:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.189+0000 7fb0a86cf640 1 -- 172.21.15.16:0/1033213844 shutdown_connections 2023-11-13T10:55:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.189+0000 7fb0a86cf640 1 -- 172.21.15.16:0/1033213844 wait complete. 2023-11-13T10:55:59.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.190+0000 7f8b208d5640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8b0006ae40 0x7f8b0006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0ce070 tx=0x7f8b10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.190+0000 7f02ba59c640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f029c06ae70 0x7f029c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.190+0000 7fa3bbfff640 1 -- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc0adbc0 msgr2=0x7fa3bc148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.190+0000 7fa3bbfff640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc0adbc0 0x7fa3bc148a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.190+0000 7fa3bbfff640 1 -- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3bc15d680 msgr2=0x7fa3bc153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.190+0000 7fa3bbfff640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3bc15d680 0x7fa3bc153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.190+0000 7f75f67fc640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f75e006aec0 0x7f75e006d380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f75f0007df0 tx=0x7f75f0007d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.190+0000 7fa3bbfff640 1 -- 172.21.15.16:0/3004397931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac00f050 con 0x7fa3bc159ae0 2023-11-13T10:55:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.190+0000 7fa3bbfff640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3bc159ae0 0x7fa3bc15bfd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00f230 tx=0x7fa3ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.191+0000 7f02ba59c640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f029c06ae70 0x7f029c06d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02bc08d4c0 tx=0x7f02ac009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.192+0000 7fb0a86cf640 1 Processor -- start 2023-11-13T10:55:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.193+0000 7fb0a86cf640 1 -- start start 2023-11-13T10:55:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.194+0000 7fb0a86cf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0a00cfe20 0x7fb0a0143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.194+0000 7fb0a86cf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a00d0ac0 0x7fb0a00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.194+0000 7fb0a86cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a015d0f0 0x7fb0a0159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.194+0000 7fb0a86cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a01580c0 con 0x7fb0a015d0f0 2023-11-13T10:55:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.194+0000 7fb0a86cf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a01f2b30 con 0x7fb0a00d0ac0 2023-11-13T10:55:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.194+0000 7fb0a86cf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00ca230 con 0x7fb0a00cfe20 2023-11-13T10:55:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a6c6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a015d0f0 0x7fb0a0159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a6c6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a015d0f0 0x7fb0a0159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56156/0 (socket says 172.21.15.16:56156) 2023-11-13T10:55:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a6c6c640 1 -- 172.21.15.16:0/4253688697 learned_addr learned my addr 172.21.15.16:0/4253688697 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:55:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a5c6a640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a00d0ac0 0x7fb0a00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a646b640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0a00cfe20 0x7fb0a0143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a6c6c640 1 -- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0a00cfe20 msgr2=0x7fb0a0143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a6c6c640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0a00cfe20 0x7fb0a0143fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a6c6c640 1 -- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a00d0ac0 msgr2=0x7fb0a00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a6c6c640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a00d0ac0 0x7fb0a00b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.195+0000 7fb0a6c6c640 1 -- 172.21.15.16:0/4253688697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c0094d0 con 0x7fb0a015d0f0 2023-11-13T10:55:59.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.196+0000 7fa3b97fa640 1 -- 172.21.15.16:0/3004397931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac047860 con 0x7fa3bc159ae0 2023-11-13T10:55:59.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.196+0000 7fa3b97fa640 1 -- 172.21.15.16:0/3004397931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac0479c0 con 0x7fa3bc159ae0 2023-11-13T10:55:59.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.196+0000 7fa3b97fa640 1 -- 172.21.15.16:0/3004397931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac047d70 con 0x7fa3bc159ae0 2023-11-13T10:55:59.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.196+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc15bab0 con 0x7fa3bc159ae0 2023-11-13T10:55:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.196+0000 7f8b197fa640 1 -- 172.21.15.16:0/783248106 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b14053c50 con 0x7f8b1c158e10 2023-11-13T10:55:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.196+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc15b1e0 con 0x7fa3bc159ae0 2023-11-13T10:55:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.196+0000 7fb0a6c6c640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a015d0f0 0x7fb0a0159800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb09800b9e0 tx=0x7fb09800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.198+0000 7f291dffb640 1 -- 172.21.15.16:0/2649169631 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f290c0549d0 con 0x7f29200d0ac0 2023-11-13T10:55:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.198+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc067090 con 0x7fa3bc159ae0 2023-11-13T10:55:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.199+0000 7fb092ffd640 1 -- 172.21.15.16:0/4253688697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09800cb20 con 0x7fb0a015d0f0 2023-11-13T10:55:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.199+0000 7fb092ffd640 1 -- 172.21.15.16:0/4253688697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09800cc80 con 0x7fb0a015d0f0 2023-11-13T10:55:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.199+0000 7fb092ffd640 1 -- 172.21.15.16:0/4253688697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098014440 con 0x7fb0a015d0f0 2023-11-13T10:55:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.199+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a00caa90 con 0x7fb0a015d0f0 2023-11-13T10:55:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.199+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a0157de0 con 0x7fb0a015d0f0 2023-11-13T10:55:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.200+0000 7fa3b97fa640 1 -- 172.21.15.16:0/3004397931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3ac045400 con 0x7fa3bc159ae0 2023-11-13T10:55:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.201+0000 7fa3b97fa640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa39c06af10 0x7fa39c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.201+0000 7fa3b97fa640 1 -- 172.21.15.16:0/3004397931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa3ac0b4640 con 0x7fa3bc159ae0 2023-11-13T10:55:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.202+0000 7fa3c0963640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa39c06af10 0x7fa39c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.203+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a00cac20 con 0x7fb0a015d0f0 2023-11-13T10:55:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.203+0000 7fb092ffd640 1 -- 172.21.15.16:0/4253688697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb09801d3b0 con 0x7fb0a015d0f0 2023-11-13T10:55:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.203+0000 7fa3c0963640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa39c06af10 0x7fa39c06d3d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009b40 tx=0x7fa3b4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.204+0000 7f02aaffd640 1 -- 172.21.15.16:0/3830319991 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02b0054650 con 0x7f02bc1f2750 2023-11-13T10:55:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.204+0000 7fb092ffd640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb08406ae40 0x7fb08406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:55:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.205+0000 7fb0a646b640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb08406ae40 0x7fb08406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:55:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.206+0000 7fb0a646b640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb08406ae40 0x7fb08406d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb094006fd0 tx=0x7fb094008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:55:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.209+0000 7fb092ffd640 1 -- 172.21.15.16:0/4253688697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb098058290 con 0x7fb0a015d0f0 2023-11-13T10:55:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.210+0000 7fa3b97fa640 1 -- 172.21.15.16:0/3004397931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3ac07f490 con 0x7fa3bc159ae0 2023-11-13T10:55:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.217+0000 7f75f7fff640 1 -- 172.21.15.16:0/967374450 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75e8077f50 con 0x7f76000cfe20 2023-11-13T10:55:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.219+0000 7fb092ffd640 1 -- 172.21.15.16:0/4253688697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb098089e60 con 0x7fb0a015d0f0 2023-11-13T10:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.591+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f65fc15f5c0 con 0x7f65fc0ce110 2023-11-13T10:55:59.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.592+0000 7f65f97fa640 1 -- 172.21.15.16:0/415523007 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f65f4058fe0 con 0x7f65fc0ce110 2023-11-13T10:55:59.598 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2023-11-13T10:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.596+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f65e806af40 msgr2=0x7f65e806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.596+0000 7f6602bf0640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f65e806af40 0x7f65e806d400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005880 tx=0x7f65ec009040 comp rx=0 tx=0).stop 2023-11-13T10:55:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.596+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65fc0ce110 msgr2=0x7f65fc145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.596+0000 7f6602bf0640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65fc0ce110 0x7f65fc145340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f65f4007d80 tx=0x7f65f400e4b0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.597+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 shutdown_connections 2023-11-13T10:55:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.597+0000 7f6602bf0640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f65e806af40 0x7f65e806d400 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.597+0000 7f6602bf0640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f65fc161170 0x7f65fc1446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.597+0000 7f6602bf0640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65fc0ce110 0x7f65fc145340 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.597+0000 7f6602bf0640 1 --2- 172.21.15.16:0/415523007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65fc0710e0 0x7f65fc0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.597+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 >> 172.21.15.16:0/415523007 conn(0x7f65fc0b0510 msgr2=0x7f65fc15e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.598+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 shutdown_connections 2023-11-13T10:55:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.598+0000 7f6602bf0640 1 -- 172.21.15.16:0/415523007 wait complete. 2023-11-13T10:55:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.599+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbd6c0e8760 con 0x7fbd6c056a80 2023-11-13T10:55:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.599+0000 7fbd6b7fe640 1 -- 172.21.15.16:0/2258464690 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbd5c057670 con 0x7fbd6c056a80 2023-11-13T10:55:59.606 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2023-11-13T10:55:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.603+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbd5406aea0 msgr2=0x7fbd5406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.603+0000 7fbd73fff640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbd5406aea0 0x7fbd5406d360 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd64036000 tx=0x7fbd64030000 comp rx=0 tx=0).stop 2023-11-13T10:55:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.604+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c056a80 msgr2=0x7fbd6c0e1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.604+0000 7fbd73fff640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c056a80 0x7fbd6c0e1890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00c400 tx=0x7fbd5c00c8c0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.604+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 shutdown_connections 2023-11-13T10:55:59.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.604+0000 7fbd73fff640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbd5406aea0 0x7fbd5406d360 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.604+0000 7fbd73fff640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd6c0e76e0 0x7fbd6c04dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.604+0000 7fbd73fff640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd6c058990 0x7fbd6c051520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.604+0000 7fbd73fff640 1 --2- 172.21.15.16:0/2258464690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6c056a80 0x7fbd6c0e1890 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.604+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 >> 172.21.15.16:0/2258464690 conn(0x7fbd6c058eb0 msgr2=0x7fbd6c0534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.607+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 shutdown_connections 2023-11-13T10:55:59.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.607+0000 7fbd73fff640 1 -- 172.21.15.16:0/2258464690 wait complete. 2023-11-13T10:55:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.614+0000 7f290b7fe640 1 -- 172.21.15.16:0/2649169631 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f28f80057d0 con 0x7f29200d0ac0 2023-11-13T10:55:59.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.615+0000 7f291dffb640 1 -- 172.21.15.16:0/2649169631 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f290c057ff0 con 0x7f29200d0ac0 2023-11-13T10:55:59.621 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2023-11-13T10:55:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.618+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8b1c06ff00 con 0x7f8b1c158e10 2023-11-13T10:55:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.618+0000 7f8b197fa640 1 -- 172.21.15.16:0/783248106 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b14057270 con 0x7f8b1c158e10 2023-11-13T10:55:59.622 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2023-11-13T10:55:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f290b7fe640 1 -- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f290406ae40 msgr2=0x7f290406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f290b7fe640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f290406ae40 0x7f290406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2914000f80 tx=0x7f2914030000 comp rx=0 tx=0).stop 2023-11-13T10:55:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f290b7fe640 1 -- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200d0ac0 msgr2=0x7f29200c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f290b7fe640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200d0ac0 0x7f29200c9db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f290c001eb0 tx=0x7f290c00c870 comp rx=0 tx=0).stop 2023-11-13T10:55:59.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8b0006ae40 msgr2=0x7f8b0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f8b22b39640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8b0006ae40 0x7f8b0006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0ce070 tx=0x7f8b10008040 comp rx=0 tx=0).stop 2023-11-13T10:55:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c158e10 msgr2=0x7f8b1c155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f8b22b39640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c158e10 0x7f8b1c155260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c700 tx=0x7f8b1400cbc0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 shutdown_connections 2023-11-13T10:55:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f8b22b39640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8b0006ae40 0x7f8b0006d300 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0ce070 tx=0x7f8b10008040 comp rx=0 tx=0).stop 2023-11-13T10:55:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f8b22b39640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c158e10 0x7f8b1c155260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f8b22b39640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b1c0b10e0 0x7f8b1c15a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.622+0000 7f8b22b39640 1 --2- 172.21.15.16:0/783248106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1c070b30 0x7f8b1c1423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 >> 172.21.15.16:0/783248106 conn(0x7f8b1c0b05f0 msgr2=0x7f8b1c0afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 shutdown_connections 2023-11-13T10:55:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f290b7fe640 1 -- 172.21.15.16:0/2649169631 shutdown_connections 2023-11-13T10:55:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f290b7fe640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f290406ae40 0x7f290406d300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f290b7fe640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f292015d0f0 0x7f29200b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f290b7fe640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29200d0ac0 0x7f29200c9db0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f290b7fe640 1 --2- 172.21.15.16:0/2649169631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29200cfe20 0x7f29200b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f290b7fe640 1 -- 172.21.15.16:0/2649169631 >> 172.21.15.16:0/2649169631 conn(0x7f29200d2ce0 msgr2=0x7f292006aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f290b7fe640 1 -- 172.21.15.16:0/2649169631 shutdown_connections 2023-11-13T10:55:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f8b22b39640 1 -- 172.21.15.16:0/783248106 wait complete. 2023-11-13T10:55:59.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.623+0000 7f290b7fe640 1 -- 172.21.15.16:0/2649169631 wait complete. 2023-11-13T10:55:59.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.1 2023-11-13T10:55:59.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.632+0000 7f02a8ff9640 1 -- 172.21.15.16:0/3830319991 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f02940057d0 con 0x7f02bc1f2750 2023-11-13T10:55:59.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.0 2023-11-13T10:55:59.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.632+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa3bc15b910 con 0x7fa3bc159ae0 2023-11-13T10:55:59.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.633+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f760015cc30 con 0x7f76000cfe20 2023-11-13T10:55:59.644 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2023-11-13T10:55:59.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.633+0000 7f02aaffd640 1 -- 172.21.15.16:0/3830319991 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02b0057c70 con 0x7f02bc1f2750 2023-11-13T10:55:59.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.633+0000 7f75f7fff640 1 -- 172.21.15.16:0/967374450 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f75e80780f0 con 0x7f76000cfe20 2023-11-13T10:55:59.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.633+0000 7fa3b97fa640 1 -- 172.21.15.16:0/3004397931 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3bc15b910 con 0x7fa3bc159ae0 2023-11-13T10:55:59.646 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2023-11-13T10:55:59.651 INFO:teuthology.orchestra.run.smithi016.stdout:60129542154 2023-11-13T10:55:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f75e006aec0 msgr2=0x7f75e006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f75e006aec0 0x7f75e006d380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f75f0007df0 tx=0x7f75f0007d80 comp rx=0 tx=0).stop 2023-11-13T10:55:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76000cfe20 msgr2=0x7f760009d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76000cfe20 0x7f760009d060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75e8034040 tx=0x7f75e80046c0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 shutdown_connections 2023-11-13T10:55:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f75e006aec0 0x7f75e006d380 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f760015d0f0 0x7f76000b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76000d0ac0 0x7f76000b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 --2- 172.21.15.16:0/967374450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76000cfe20 0x7f760009d060 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 >> 172.21.15.16:0/967374450 conn(0x7f76000d2ce0 msgr2=0x7f760006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 shutdown_connections 2023-11-13T10:55:59.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f7605490640 1 -- 172.21.15.16:0/967374450 wait complete. 2023-11-13T10:55:59.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f02a8ff9640 1 -- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f029c06ae70 msgr2=0x7f029c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f02a8ff9640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f029c06ae70 0x7f029c06d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02bc08d4c0 tx=0x7f02ac009190 comp rx=0 tx=0).stop 2023-11-13T10:55:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f02a8ff9640 1 -- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02bc1f2750 msgr2=0x7f02bc15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.636+0000 7f02a8ff9640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02bc1f2750 0x7f02bc15a5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e6d0 tx=0x7f02b000eb90 comp rx=0 tx=0).stop 2023-11-13T10:55:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7f02a8ff9640 1 -- 172.21.15.16:0/3830319991 shutdown_connections 2023-11-13T10:55:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7f02a8ff9640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f029c06ae70 0x7f029c06d330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7f02a8ff9640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02bc144260 0x7f02bc157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7f02a8ff9640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02bc1f2750 0x7f02bc15a5a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7f02a8ff9640 1 --2- 172.21.15.16:0/3830319991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a4005180 0x7f02bc0b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7f02a8ff9640 1 -- 172.21.15.16:0/3830319991 >> 172.21.15.16:0/3830319991 conn(0x7f02bc0d2ce0 msgr2=0x7f02bc0c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7f02a8ff9640 1 -- 172.21.15.16:0/3830319991 shutdown_connections 2023-11-13T10:55:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7f02a8ff9640 1 -- 172.21.15.16:0/3830319991 wait complete. 2023-11-13T10:55:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa39c06af10 msgr2=0x7fa39c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7fa3c2bc7640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa39c06af10 0x7fa39c06d3d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009b40 tx=0x7fa3b4009190 comp rx=0 tx=0).stop 2023-11-13T10:55:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3bc159ae0 msgr2=0x7fa3bc15bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.637+0000 7fa3c2bc7640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3bc159ae0 0x7fa3bc15bfd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00f230 tx=0x7fa3ac004a40 comp rx=0 tx=0).stop 2023-11-13T10:55:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.638+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 shutdown_connections 2023-11-13T10:55:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.638+0000 7fa3c2bc7640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa39c06af10 0x7fa39c06d3d0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009b40 tx=0x7fa3b4009190 comp rx=0 tx=0).stop 2023-11-13T10:55:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.638+0000 7fa3c2bc7640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3bc15d680 0x7fa3bc153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.638+0000 7fa3c2bc7640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3bc159ae0 0x7fa3bc15bfd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.638+0000 7fa3c2bc7640 1 --2- 172.21.15.16:0/3004397931 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3bc0adbc0 0x7fa3bc148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.638+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 >> 172.21.15.16:0/3004397931 conn(0x7fa3bc06a190 msgr2=0x7fa3bc1570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.638+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 shutdown_connections 2023-11-13T10:55:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.638+0000 7fa3c2bc7640 1 -- 172.21.15.16:0/3004397931 wait complete. 2023-11-13T10:55:59.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.645+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb0a0067090 con 0x7fb0a015d0f0 2023-11-13T10:55:59.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.646+0000 7fb092ffd640 1 -- 172.21.15.16:0/4253688697 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb098019020 con 0x7fb0a015d0f0 2023-11-13T10:55:59.697 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2023-11-13T10:55:59.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.5 2023-11-13T10:55:59.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb08406ae40 msgr2=0x7fb08406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb08406ae40 0x7fb08406d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb094006fd0 tx=0x7fb094008040 comp rx=0 tx=0).stop 2023-11-13T10:55:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a015d0f0 msgr2=0x7fb0a0159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:55:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a015d0f0 0x7fb0a0159800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb09800b9e0 tx=0x7fb09800bea0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.2 2023-11-13T10:55:59.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 shutdown_connections 2023-11-13T10:55:59.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb08406ae40 0x7fb08406d300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a015d0f0 0x7fb0a0159800 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0a00d0ac0 0x7fb0a00b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 --2- 172.21.15.16:0/4253688697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0a00cfe20 0x7fb0a0143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:55:59.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 >> 172.21.15.16:0/4253688697 conn(0x7fb0a00d2ce0 msgr2=0x7fb0a00b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:55:59.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 shutdown_connections 2023-11-13T10:55:59.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:55:59.650+0000 7fb0a86cf640 1 -- 172.21.15.16:0/4253688697 wait complete. 2023-11-13T10:55:59.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.3 2023-11-13T10:55:59.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.4 2023-11-13T10:55:59.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542154 for osd.7 2023-11-13T10:55:59.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.6 2023-11-13T10:56:00.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-13T10:56:00.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-13T10:56:00.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-13T10:56:00.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-13T10:56:00.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-13T10:56:00.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-13T10:56:00.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-13T10:56:00.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-13T10:56:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.901+0000 7f481b1e9640 1 Processor -- start 2023-11-13T10:56:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.902+0000 7f481b1e9640 1 -- start start 2023-11-13T10:56:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.902+0000 7f7bc45c9640 1 Processor -- start 2023-11-13T10:56:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.904+0000 7f481b1e9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f480c039890 0x7f480c0dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.904+0000 7f481b1e9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f480c054310 0x7f480c0525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.904+0000 7f481b1e9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f480c0e03c0 con 0x7f480c052b00 2023-11-13T10:56:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.904+0000 7f481b1e9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480c0076a0 con 0x7f480c039890 2023-11-13T10:56:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.904+0000 7f481b1e9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480c0e6d60 con 0x7f480c054310 2023-11-13T10:56:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.903+0000 7f7bc45c9640 1 -- start start 2023-11-13T10:56:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.905+0000 7f7bc45c9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bb404fa50 0x7f7bb404fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.905+0000 7f7bc45c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb4054230 0x7f7bb4038990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.905+0000 7f7bc45c9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb4006860 con 0x7f7bb40df3e0 2023-11-13T10:56:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.905+0000 7f7bc45c9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb4007730 con 0x7f7bb404fa50 2023-11-13T10:56:00.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.905+0000 7f7bc45c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4050370 con 0x7f7bb4054230 2023-11-13T10:56:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.905+0000 7f4818f85640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f480c039890 0x7f480c0dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.906+0000 7f7bc2365640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bb404fa50 0x7f7bb404fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.906+0000 7f7bc2365640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bb404fa50 0x7f7bb404fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39148/0 (socket says 172.21.15.16:39148) 2023-11-13T10:56:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.906+0000 7f7bc2365640 1 -- 172.21.15.16:0/2650694838 learned_addr learned my addr 172.21.15.16:0/2650694838 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.906+0000 7f4818f85640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f480c039890 0x7f480c0dfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56162/0 (socket says 172.21.15.16:56162) 2023-11-13T10:56:00.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.906+0000 7f4819786640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f480c052b00 0x7f480c04fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34536/0 (socket says 172.21.15.16:34536) 2023-11-13T10:56:00.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.906+0000 7f4819786640 1 -- 172.21.15.16:0/598888418 learned_addr learned my addr 172.21.15.16:0/598888418 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:00.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.906+0000 7f4813fff640 1 --2- 172.21.15.16:0/598888418 >> v2:172.21.15.184:3300/0 conn(0x7f480c054310 0x7f480c0525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.907+0000 7f7bc2365640 1 -- 172.21.15.16:0/2650694838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb4054230 msgr2=0x7f7bb4038990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.907+0000 7f7bc2365640 1 --2- 172.21.15.16:0/2650694838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb4054230 0x7f7bb4038990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.907+0000 7f7bc2365640 1 -- 172.21.15.16:0/2650694838 >> v1:172.21.15.184:6789/0 conn(0x7f7bb40df3e0 legacy=0x7f7bb4038280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-13T10:56:00.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.908+0000 7f7bc2365640 1 -- 172.21.15.16:0/2650694838 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4006ab0 con 0x7f7bb404fa50 2023-11-13T10:56:00.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.907+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197585586 0 0) 0x7f480c0e03c0 con 0x7f480c052b00 2023-11-13T10:56:00.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.907+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4804003520 con 0x7f480c052b00 2023-11-13T10:56:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.908+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3183103 0 0) 0x7f4804003520 con 0x7f480c052b00 2023-11-13T10:56:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.908+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 >> v2:172.21.15.184:3300/0 conn(0x7f480c054310 msgr2=0x7f480c0525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:00.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.908+0000 7f48137fe640 1 --2- 172.21.15.16:0/598888418 >> v2:172.21.15.184:3300/0 conn(0x7f480c054310 0x7f480c0525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.908+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 >> v2:172.21.15.16:3300/0 conn(0x7f480c039890 msgr2=0x7f480c0dfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:00.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.908+0000 7f48137fe640 1 --2- 172.21.15.16:0/598888418 >> v2:172.21.15.16:3300/0 conn(0x7f480c039890 0x7f480c0dfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.908+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c039d90 con 0x7f480c052b00 2023-11-13T10:56:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.909+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f481404e9f0 con 0x7f480c052b00 2023-11-13T10:56:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.909+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 >> v1:172.21.15.16:6789/0 conn(0x7f480c052b00 legacy=0x7f480c04fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.909+0000 7f48137fe640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4804003970 0x7f4804003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.909+0000 7f48137fe640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4804004350 0x7f4804004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.909+0000 7f48137fe640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 0x7f480400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.909+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c0e6d60 con 0x7f4804005020 2023-11-13T10:56:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.909+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480400bfa0 con 0x7f4804003970 2023-11-13T10:56:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.909+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480400c160 con 0x7f4804004350 2023-11-13T10:56:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.909+0000 7f4818f85640 1 --2- 172.21.15.16:0/598888418 >> v2:172.21.15.16:3300/0 conn(0x7f480c039890 0x7f480c0dfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:56:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.910+0000 7f4819786640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 0x7f480400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.911+0000 7f4818f85640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4804003970 0x7f4804003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.911+0000 7f4813fff640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4804004350 0x7f4804004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.911+0000 7f4819786640 1 -- 172.21.15.16:0/598888418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4804004350 msgr2=0x7f4804004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:00.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.911+0000 7f4819786640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4804004350 0x7f4804004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.911+0000 7f4819786640 1 -- 172.21.15.16:0/598888418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4804003970 msgr2=0x7f4804003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.911+0000 7f4819786640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4804003970 0x7f4804003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.911+0000 7f4819786640 1 -- 172.21.15.16:0/598888418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4804003520 con 0x7f4804005020 2023-11-13T10:56:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.912+0000 7f7bc2365640 1 --2- 172.21.15.16:0/2650694838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb404fa50 0x7f7bb404fe30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009870 tx=0x7f7bb802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be0e92c8e556193e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.913+0000 7f7bc1363640 1 -- 172.21.15.16:0/2650694838 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8031640 con 0x7f7bb404fa50 2023-11-13T10:56:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.913+0000 7f7bc1363640 1 -- 172.21.15.16:0/2650694838 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb80317a0 con 0x7f7bb404fa50 2023-11-13T10:56:00.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.913+0000 7f4819786640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 0x7f480400b980 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f481404fdd0 tx=0x7f48040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ad73dade6ff5df server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:00.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.913+0000 7f4819786640 1 -- 172.21.15.16:0/598888418 reap_dead start 2023-11-13T10:56:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.913+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140ad620 con 0x7f4804005020 2023-11-13T10:56:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.913+0000 7f7bc1363640 1 -- 172.21.15.16:0/2650694838 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8031b30 con 0x7f7bb404fa50 2023-11-13T10:56:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.913+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48140a9070 con 0x7f4804005020 2023-11-13T10:56:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.913+0000 7f48137fe640 1 -- 172.21.15.16:0/598888418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140ad980 con 0x7f4804005020 2023-11-13T10:56:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.915+0000 7f7bc45c9640 1 -- 172.21.15.16:0/2650694838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb404fa50 msgr2=0x7f7bb404fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.915+0000 7f7bc45c9640 1 --2- 172.21.15.16:0/2650694838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb404fa50 0x7f7bb404fe30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009870 tx=0x7f7bb802b8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.914+0000 7f481b1e9640 1 -- 172.21.15.16:0/598888418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 msgr2=0x7f480400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.914+0000 7f481b1e9640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 0x7f480400b980 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f481404fdd0 tx=0x7f48040057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.915+0000 7f7bc45c9640 1 -- 172.21.15.16:0/2650694838 shutdown_connections 2023-11-13T10:56:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.915+0000 7f7bc45c9640 1 --2- 172.21.15.16:0/2650694838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb4054230 0x7f7bb4038990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.915+0000 7f481b1e9640 1 -- 172.21.15.16:0/598888418 shutdown_connections 2023-11-13T10:56:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.915+0000 7f7bc45c9640 1 --2- 172.21.15.16:0/2650694838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb404fa50 0x7f7bb404fe30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.915+0000 7f7bc45c9640 1 -- 172.21.15.16:0/2650694838 >> 172.21.15.16:0/2650694838 conn(0x7f7bb4009f70 msgr2=0x7f7bb400a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.916+0000 7f481b1e9640 1 --2- 172.21.15.16:0/598888418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 0x7f480400b980 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.916+0000 7f7bc45c9640 1 -- 172.21.15.16:0/2650694838 shutdown_connections 2023-11-13T10:56:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.916+0000 7f7bc45c9640 1 -- 172.21.15.16:0/2650694838 wait complete. 2023-11-13T10:56:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.916+0000 7f481b1e9640 1 -- 172.21.15.16:0/598888418 >> 172.21.15.16:0/598888418 conn(0x7f480c009f70 msgr2=0x7f480c00a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.916+0000 7f481b1e9640 1 -- 172.21.15.16:0/598888418 shutdown_connections 2023-11-13T10:56:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.917+0000 7f481b1e9640 1 -- 172.21.15.16:0/598888418 wait complete. 2023-11-13T10:56:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.921+0000 7f481b1e9640 1 Processor -- start 2023-11-13T10:56:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.922+0000 7f481b1e9640 1 -- start start 2023-11-13T10:56:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.922+0000 7f7bc45c9640 1 Processor -- start 2023-11-13T10:56:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.923+0000 7f481b1e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 0x7f480c0d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.923+0000 7f7bc45c9640 1 -- start start 2023-11-13T10:56:00.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.924+0000 7f481b1e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f480c0d5fa0 0x7f480c04dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.924+0000 7f7bc45c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb404fa50 0x7f7bb40d2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.924+0000 7f7bc45c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4054230 0x7f7bb40e1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.924+0000 7f7bc45c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40df3e0 0x7f7bb40e1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.924+0000 7f7bc45c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40d2dc0 con 0x7f7bb4054230 2023-11-13T10:56:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.924+0000 7f7bc45c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40e04f0 con 0x7f7bb40df3e0 2023-11-13T10:56:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.924+0000 7f7bc45c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40e07d0 con 0x7f7bb404fa50 2023-11-13T10:56:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.925+0000 7f481b1e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f480c04e2d0 0x7f480c0d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:00.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.925+0000 7f481b1e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c0d2760 con 0x7f4804005020 2023-11-13T10:56:00.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.925+0000 7f481b1e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c0d8700 con 0x7f480c0d5fa0 2023-11-13T10:56:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.925+0000 7f7bc2365640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb404fa50 0x7f7bb40d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.925+0000 7f4813fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f480c0d5fa0 0x7f480c04dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.925+0000 7f4813fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f480c0d5fa0 0x7f480c04dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39158/0 (socket says 172.21.15.16:39158) 2023-11-13T10:56:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.926+0000 7f4819786640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f480c04e2d0 0x7f480c0d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.926+0000 7f7bc2365640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb404fa50 0x7f7bb40d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50564/0 (socket says 172.21.15.16:50564) 2023-11-13T10:56:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.926+0000 7f4813fff640 1 -- 172.21.15.16:0/1616038315 learned_addr learned my addr 172.21.15.16:0/1616038315 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.926+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c04e750 con 0x7f480c04e2d0 2023-11-13T10:56:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.926+0000 7f7bc1b64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4054230 0x7f7bb40e1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.926+0000 7f7bc1b64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4054230 0x7f7bb40e1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56196/0 (socket says 172.21.15.16:56196) 2023-11-13T10:56:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.926+0000 7f7bc1b64640 1 -- 172.21.15.16:0/1136506102 learned_addr learned my addr 172.21.15.16:0/1136506102 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.926+0000 7f7bc2b66640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40df3e0 0x7f7bb40e1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.926+0000 7f4818f85640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 0x7f480c0d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.927+0000 7f4819786640 1 -- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f480c0d5fa0 msgr2=0x7f480c04dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.927+0000 7f4819786640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f480c0d5fa0 0x7f480c04dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.927+0000 7f4819786640 1 -- 172.21.15.16:0/1616038315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 msgr2=0x7f480c0d5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.927+0000 7f4819786640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 0x7f480c0d5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.927+0000 7f4819786640 1 -- 172.21.15.16:0/1616038315 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c0d6d10 con 0x7f480c04e2d0 2023-11-13T10:56:00.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.928+0000 7f7bc2b66640 1 -- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb404fa50 msgr2=0x7f7bb40d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:00.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.928+0000 7f7bc2b66640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb404fa50 0x7f7bb40d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.928+0000 7f7bc2b66640 1 -- 172.21.15.16:0/1136506102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4054230 msgr2=0x7f7bb40e1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:00.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.928+0000 7f7bc2b66640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4054230 0x7f7bb40e1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:00.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.928+0000 7f7bc2b66640 1 -- 172.21.15.16:0/1136506102 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80094d0 con 0x7f7bb40df3e0 2023-11-13T10:56:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.928+0000 7f4819786640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f480c04e2d0 0x7f480c0d18a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4814068d00 tx=0x7f4814069e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.929+0000 7f7bc2b66640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40df3e0 0x7f7bb40e1ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc067e10 tx=0x7f7bbc073ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.929+0000 7f48117fa640 1 -- 172.21.15.16:0/1616038315 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140ae8f0 con 0x7f480c04e2d0 2023-11-13T10:56:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.929+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480c0d6470 con 0x7f480c04e2d0 2023-11-13T10:56:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.929+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480c0d6a30 con 0x7f480c04e2d0 2023-11-13T10:56:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.929+0000 7f7bb2ffd640 1 -- 172.21.15.16:0/1136506102 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc073c00 con 0x7f7bb40df3e0 2023-11-13T10:56:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.929+0000 7f7bb2ffd640 1 -- 172.21.15.16:0/1136506102 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc068410 con 0x7f7bb40df3e0 2023-11-13T10:56:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.929+0000 7f7bb2ffd640 1 -- 172.21.15.16:0/1136506102 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc072280 con 0x7f7bb40df3e0 2023-11-13T10:56:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.929+0000 7f48117fa640 1 -- 172.21.15.16:0/1616038315 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48140aea50 con 0x7f480c04e2d0 2023-11-13T10:56:01.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.931+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb40e0ab0 con 0x7f7bb40df3e0 2023-11-13T10:56:01.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.929+0000 7f48117fa640 1 -- 172.21.15.16:0/1616038315 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140aee00 con 0x7f480c04e2d0 2023-11-13T10:56:01.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.931+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb40e1070 con 0x7f7bb40df3e0 2023-11-13T10:56:01.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.934+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb404f8c0 con 0x7f7bb40df3e0 2023-11-13T10:56:01.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.933+0000 7f48117fa640 1 -- 172.21.15.16:0/1616038315 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f48140ab070 con 0x7f480c04e2d0 2023-11-13T10:56:01.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.935+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f4005250 con 0x7f480c04e2d0 2023-11-13T10:56:01.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.936+0000 7f48117fa640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47f806ae40 0x7f47f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.936+0000 7f48117fa640 1 -- 172.21.15.16:0/1616038315 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f481406b050 con 0x7f480c04e2d0 2023-11-13T10:56:01.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.937+0000 7f4818f85640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47f806ae40 0x7f47f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.937+0000 7f7bb2ffd640 1 -- 172.21.15.16:0/1136506102 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bbc0723e0 con 0x7f7bb40df3e0 2023-11-13T10:56:01.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.938+0000 7f4818f85640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47f806ae40 0x7f47f806d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f480800da90 tx=0x7f4808008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.939+0000 7f7bb2ffd640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ba006ae90 0x7f7ba006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.939+0000 7f7bb2ffd640 1 -- 172.21.15.16:0/1136506102 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7bbc1034b0 con 0x7f7bb40df3e0 2023-11-13T10:56:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.940+0000 7f7bc2365640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ba006ae90 0x7f7ba006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.941+0000 7f7bc2365640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ba006ae90 0x7f7ba006d350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8036000 tx=0x7f7bb803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.968+0000 7f48117fa640 1 -- 172.21.15.16:0/1616038315 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48140f01a0 con 0x7f480c04e2d0 2023-11-13T10:56:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.971+0000 7f7bb2ffd640 1 -- 172.21.15.16:0/1136506102 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bbc0ce400 con 0x7f7bb40df3e0 2023-11-13T10:56:01.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.977+0000 7ffab272c640 1 Processor -- start 2023-11-13T10:56:01.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.977+0000 7ffab272c640 1 -- start start 2023-11-13T10:56:01.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.979+0000 7ffab272c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffaac15d150 0x7ffaac0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.979+0000 7ffab272c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac0cfeb0 0x7ffaac0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.979+0000 7ffab272c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac0cebd0 con 0x7ffaac161520 2023-11-13T10:56:01.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.979+0000 7ffab272c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac0d3ac0 con 0x7ffaac15d150 2023-11-13T10:56:01.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.979+0000 7ffab272c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac0cdfc0 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.980+0000 7ffab0cc9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac0cfeb0 0x7ffaac0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.980+0000 7ffab0cc9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac0cfeb0 0x7ffaac0d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50578/0 (socket says 172.21.15.16:50578) 2023-11-13T10:56:01.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.980+0000 7ffab0cc9640 1 -- 172.21.15.16:0/3431962065 learned_addr learned my addr 172.21.15.16:0/3431962065 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:01.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.981+0000 7ffaabfff640 1 --2- 172.21.15.16:0/3431962065 >> v2:172.21.15.16:3300/0 conn(0x7ffaac15d150 0x7ffaac0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.981+0000 7ffab0cc9640 1 -- 172.21.15.16:0/3431962065 >> v2:172.21.15.16:3300/0 conn(0x7ffaac15d150 msgr2=0x7ffaac0cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.981+0000 7ffab0cc9640 1 --2- 172.21.15.16:0/3431962065 >> v2:172.21.15.16:3300/0 conn(0x7ffaac15d150 0x7ffaac0cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.981+0000 7ffab0cc9640 1 -- 172.21.15.16:0/3431962065 >> v1:172.21.15.16:6789/0 conn(0x7ffaac161520 legacy=0x7ffaac0cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.982+0000 7ffab0cc9640 1 -- 172.21.15.16:0/3431962065 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0ce2a0 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.984+0000 7ffab0cc9640 1 --2- 172.21.15.16:0/3431962065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac0cfeb0 0x7ffaac0d0aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009870 tx=0x7ffa9c02b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c261690a0a9814 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.984+0000 7f7b013bb640 1 Processor -- start 2023-11-13T10:56:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.984+0000 7f7b013bb640 1 -- start start 2023-11-13T10:56:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.984+0000 7ffaaaffd640 1 -- 172.21.15.16:0/3431962065 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c031870 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.985+0000 7ffaaaffd640 1 -- 172.21.15.16:0/3431962065 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c0319d0 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.985+0000 7ffaaaffd640 1 -- 172.21.15.16:0/3431962065 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c031d60 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.986+0000 7ffab272c640 1 -- 172.21.15.16:0/3431962065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac0cfeb0 msgr2=0x7ffaac0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.986+0000 7ffab272c640 1 --2- 172.21.15.16:0/3431962065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac0cfeb0 0x7ffaac0d0aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009870 tx=0x7ffa9c02b910 comp rx=0 tx=0).stop 2023-11-13T10:56:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.986+0000 7ffab272c640 1 -- 172.21.15.16:0/3431962065 shutdown_connections 2023-11-13T10:56:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.986+0000 7ffab272c640 1 --2- 172.21.15.16:0/3431962065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac0cfeb0 0x7ffaac0d0aa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.986+0000 7ffab272c640 1 --2- 172.21.15.16:0/3431962065 >> v2:172.21.15.16:3300/0 conn(0x7ffaac15d150 0x7ffaac0cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.986+0000 7ffab272c640 1 -- 172.21.15.16:0/3431962065 >> 172.21.15.16:0/3431962065 conn(0x7ffaac09c030 msgr2=0x7ffaac0d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.987+0000 7ffab272c640 1 -- 172.21.15.16:0/3431962065 shutdown_connections 2023-11-13T10:56:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.987+0000 7ffab272c640 1 -- 172.21.15.16:0/3431962065 wait complete. 2023-11-13T10:56:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.987+0000 7f7b013bb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc165a10 0x7f7afc0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.987+0000 7f7b013bb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7afc0b2840 0x7f7afc164e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.987+0000 7f7b013bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc06dc10 con 0x7f7afc0cd940 2023-11-13T10:56:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.987+0000 7f7b013bb640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc071e50 con 0x7f7afc0b2840 2023-11-13T10:56:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.987+0000 7f7b013bb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc06fc20 con 0x7f7afc165a10 2023-11-13T10:56:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.988+0000 7f7afa7fc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc165a10 0x7f7afc0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.988+0000 7f7afaffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7afc0cd940 0x7f7afc071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34562/0 (socket says 172.21.15.16:34562) 2023-11-13T10:56:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.988+0000 7f7afaffd640 1 -- 172.21.15.16:0/2369040221 learned_addr learned my addr 172.21.15.16:0/2369040221 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.989+0000 7f7af9ffb640 1 -- 172.21.15.16:0/2369040221 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488501266 0 0) 0x7f7afc06dc10 con 0x7f7afc0cd940 2023-11-13T10:56:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.990+0000 7f7af9ffb640 1 -- 172.21.15.16:0/2369040221 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae4003520 con 0x7f7afc0cd940 2023-11-13T10:56:01.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.991+0000 7ffab272c640 1 Processor -- start 2023-11-13T10:56:01.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.991+0000 7ffab272c640 1 -- start start 2023-11-13T10:56:01.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.990+0000 7f7afb7fe640 1 --2- 172.21.15.16:0/2369040221 >> v2:172.21.15.184:3300/0 conn(0x7f7afc0b2840 0x7f7afc164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.990+0000 7f7afa7fc640 1 -- 172.21.15.16:0/2369040221 >> v2:172.21.15.184:3300/0 conn(0x7f7afc0b2840 msgr2=0x7f7afc164e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.990+0000 7f7afa7fc640 1 --2- 172.21.15.16:0/2369040221 >> v2:172.21.15.184:3300/0 conn(0x7f7afc0b2840 0x7f7afc164e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.990+0000 7f7afa7fc640 1 -- 172.21.15.16:0/2369040221 >> v1:172.21.15.16:6789/0 conn(0x7f7afc0cd940 legacy=0x7f7afc071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.990+0000 7f7afa7fc640 1 -- 172.21.15.16:0/2369040221 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc06ff00 con 0x7f7afc165a10 2023-11-13T10:56:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.992+0000 7f7afa7fc640 1 --2- 172.21.15.16:0/2369040221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc165a10 0x7f7afc0b2300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0097a0 tx=0x7f7aec02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87a5761ce147106b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.993+0000 7f7af9ffb640 1 -- 172.21.15.16:0/2369040221 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec031830 con 0x7f7afc165a10 2023-11-13T10:56:01.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.993+0000 7ffab272c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaac0cfeb0 0x7ffaac0cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.994+0000 7f7af9ffb640 1 -- 172.21.15.16:0/2369040221 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec031990 con 0x7f7afc165a10 2023-11-13T10:56:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.994+0000 7f7af9ffb640 1 -- 172.21.15.16:0/2369040221 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec031d60 con 0x7f7afc165a10 2023-11-13T10:56:01.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.994+0000 7ffab272c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac15d150 0x7ffaac15a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.994+0000 7ffab272c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaac161520 0x7ffaac09ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.995+0000 7ffab272c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac15ac40 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.995+0000 7ffab272c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac0b6a20 con 0x7ffaac161520 2023-11-13T10:56:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.995+0000 7f7b013bb640 1 -- 172.21.15.16:0/2369040221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc165a10 msgr2=0x7f7afc0b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.995+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac15d150 0x7ffaac15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.995+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac15d150 0x7ffaac15a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50598/0 (socket says 172.21.15.16:50598) 2023-11-13T10:56:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.995+0000 7ffaab7fe640 1 -- 172.21.15.16:0/3331556174 learned_addr learned my addr 172.21.15.16:0/3331556174 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.995+0000 7f7b013bb640 1 --2- 172.21.15.16:0/2369040221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc165a10 0x7f7afc0b2300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0097a0 tx=0x7f7aec02b8d0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.995+0000 7ffaabfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaac0cfeb0 0x7ffaac0cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.995+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac15b680 con 0x7ffaac15d150 2023-11-13T10:56:01.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.996+0000 7ffab0cc9640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaac161520 0x7ffaac09ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.996+0000 7f7b013bb640 1 -- 172.21.15.16:0/2369040221 shutdown_connections 2023-11-13T10:56:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.996+0000 7f7b013bb640 1 --2- 172.21.15.16:0/2369040221 >> v2:172.21.15.184:3300/0 conn(0x7f7afc0b2840 0x7f7afc164e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.996+0000 7f7b013bb640 1 --2- 172.21.15.16:0/2369040221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc165a10 0x7f7afc0b2300 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.996+0000 7f7b013bb640 1 -- 172.21.15.16:0/2369040221 >> 172.21.15.16:0/2369040221 conn(0x7f7afc06a190 msgr2=0x7f7afc06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.997+0000 7f7b013bb640 1 -- 172.21.15.16:0/2369040221 shutdown_connections 2023-11-13T10:56:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.997+0000 7f7b013bb640 1 -- 172.21.15.16:0/2369040221 wait complete. 2023-11-13T10:56:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.998+0000 7ffaabfff640 1 -- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac15d150 msgr2=0x7ffaac15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.998+0000 7ffaabfff640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac15d150 0x7ffaac15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.998+0000 7ffaabfff640 1 -- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaac161520 msgr2=0x7ffaac09ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.998+0000 7ffaabfff640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaac161520 0x7ffaac09ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.998+0000 7ffaabfff640 1 -- 172.21.15.16:0/3331556174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c0094d0 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.998+0000 7ffaab7fe640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac15d150 0x7ffaac15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:56:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.999+0000 7ffab0cc9640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaac161520 0x7ffaac09ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:56:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:00.999+0000 7ffaabfff640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaac0cfeb0 0x7ffaac0cc720 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400cd90 tx=0x7ffaa400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.000+0000 7f7b013bb640 1 Processor -- start 2023-11-13T10:56:01.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.000+0000 7ffaa8ff9640 1 -- 172.21.15.16:0/3331556174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4003b90 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.000+0000 7ffaa8ff9640 1 -- 172.21.15.16:0/3331556174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa4003cf0 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.000+0000 7f7b013bb640 1 -- start start 2023-11-13T10:56:01.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.000+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac0ccdb0 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.001+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac0c7c60 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.003+0000 7f7b013bb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc0cd940 0x7f7afc15cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.001+0000 7ffaa8ff9640 1 -- 172.21.15.16:0/3331556174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4004240 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.003+0000 7f7b013bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7afc165a10 0x7f7afc14ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.003+0000 7f7b013bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7afc15b130 0x7f7afc0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.004+0000 7f7afa7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7afc165a10 0x7f7afc14ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.004+0000 7f7afa7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7afc165a10 0x7f7afc14ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39188/0 (socket says 172.21.15.16:39188) 2023-11-13T10:56:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.004+0000 7f7afaffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc0cd940 0x7f7afc15cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.004+0000 7f7afaffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc0cd940 0x7f7afc15cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50610/0 (socket says 172.21.15.16:50610) 2023-11-13T10:56:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.004+0000 7f7b013bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc15b6a0 con 0x7f7afc15b130 2023-11-13T10:56:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.004+0000 7f7b013bb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1fb020 con 0x7f7afc165a10 2023-11-13T10:56:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.004+0000 7f7b013bb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1fa260 con 0x7f7afc0cd940 2023-11-13T10:56:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.005+0000 7f7afb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7afc15b130 0x7f7afc0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.005+0000 7f7afa7fc640 1 -- 172.21.15.16:0/2035955787 learned_addr learned my addr 172.21.15.16:0/2035955787 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.005+0000 7f7afb7fe640 1 -- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc0cd940 msgr2=0x7f7afc15cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.005+0000 7f7afb7fe640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc0cd940 0x7f7afc15cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.005+0000 7f7afb7fe640 1 -- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7afc165a10 msgr2=0x7f7afc14ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.005+0000 7f7afb7fe640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7afc165a10 0x7f7afc14ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.005+0000 7f7afb7fe640 1 -- 172.21.15.16:0/2035955787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009400 con 0x7f7afc15b130 2023-11-13T10:56:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.006+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c005250 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.006+0000 7f7afaffd640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc0cd940 0x7f7afc15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:56:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.006+0000 7f7afb7fe640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7afc15b130 0x7f7afc0aee10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7af000c590 tx=0x7f7af000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.007+0000 7f7ae37fe640 1 -- 172.21.15.16:0/2035955787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0004220 con 0x7f7afc15b130 2023-11-13T10:56:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.007+0000 7ffaa8ff9640 1 -- 172.21.15.16:0/3331556174 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffaa40043a0 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.007+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc1fa540 con 0x7f7afc15b130 2023-11-13T10:56:01.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.007+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc1fab00 con 0x7f7afc15b130 2023-11-13T10:56:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.008+0000 7f7ae37fe640 1 -- 172.21.15.16:0/2035955787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af0004380 con 0x7f7afc15b130 2023-11-13T10:56:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.008+0000 7f7ae37fe640 1 -- 172.21.15.16:0/2035955787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0003830 con 0x7f7afc15b130 2023-11-13T10:56:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.012+0000 7f7ae37fe640 1 -- 172.21.15.16:0/2035955787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7af0003990 con 0x7f7afc15b130 2023-11-13T10:56:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.013+0000 7ffaa8ff9640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffa9806ae40 0x7ffa9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.014+0000 7f7ae37fe640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ae806ae40 0x7f7ae806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.014+0000 7f7ae37fe640 1 -- 172.21.15.16:0/2035955787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7af00894e0 con 0x7f7afc15b130 2023-11-13T10:56:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.015+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc005250 con 0x7f7afc15b130 2023-11-13T10:56:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.015+0000 7f7afaffd640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ae806ae40 0x7f7ae806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.016+0000 7ffaa8ff9640 1 -- 172.21.15.16:0/3331556174 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffaa4012070 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.016+0000 7ffaab7fe640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffa9806ae40 0x7ffa9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.017+0000 7f7afaffd640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ae806ae40 0x7f7ae806d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7af4002230 tx=0x7f7af40013f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.021+0000 7ffaab7fe640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffa9806ae40 0x7ffa9806d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffaac15df10 tx=0x7ffaa0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.022+0000 7ffaa8ff9640 1 -- 172.21.15.16:0/3331556174 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffaa4054fb0 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.032+0000 7f7ae37fe640 1 -- 172.21.15.16:0/2035955787 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7af00542c0 con 0x7f7afc15b130 2023-11-13T10:56:01.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.055+0000 7f7084dd0640 1 Processor -- start 2023-11-13T10:56:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.055+0000 7f7084dd0640 1 -- start start 2023-11-13T10:56:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.058+0000 7f7084dd0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f707803a590 0x7f70780385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.058+0000 7f7084dd0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f707804f500 0x7f7078006330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.058+0000 7f7084dd0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707804f990 con 0x7f70780e1050 2023-11-13T10:56:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.058+0000 7f7084dd0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7078051120 con 0x7f707803a590 2023-11-13T10:56:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.058+0000 7f7084dd0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7078005e50 con 0x7f707804f500 2023-11-13T10:56:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.058+0000 7f707effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f707803a590 0x7f70780385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.059+0000 7f707f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f70780e1050 0x7f7078039e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34574/0 (socket says 172.21.15.16:34574) 2023-11-13T10:56:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.059+0000 7f707f7fe640 1 -- 172.21.15.16:0/8598081 learned_addr learned my addr 172.21.15.16:0/8598081 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.059+0000 7f707effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f707803a590 0x7f70780385f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56224/0 (socket says 172.21.15.16:56224) 2023-11-13T10:56:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.059+0000 7f707ffff640 1 --2- 172.21.15.16:0/8598081 >> v2:172.21.15.184:3300/0 conn(0x7f707804f500 0x7f7078006330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.060+0000 7faf8ac32640 1 Processor -- start 2023-11-13T10:56:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.060+0000 7faf8ac32640 1 -- start start 2023-11-13T10:56:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.060+0000 7fe8edab4640 1 Processor -- start 2023-11-13T10:56:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.060+0000 7fe8edab4640 1 -- start start 2023-11-13T10:56:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.061+0000 7f53c7f1d640 1 Processor -- start 2023-11-13T10:56:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.061+0000 7f707effd640 1 -- 172.21.15.16:0/8598081 >> v2:172.21.15.184:3300/0 conn(0x7f707804f500 msgr2=0x7f7078006330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.061+0000 7f707effd640 1 --2- 172.21.15.16:0/8598081 >> v2:172.21.15.184:3300/0 conn(0x7f707804f500 0x7f7078006330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.061+0000 7f707effd640 1 -- 172.21.15.16:0/8598081 >> v1:172.21.15.16:6789/0 conn(0x7f70780e1050 legacy=0x7f7078039e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.060+0000 7f707e7fc640 1 -- 172.21.15.16:0/8598081 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467322872 0 0) 0x7f707804f990 con 0x7f70780e1050 2023-11-13T10:56:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.061+0000 7f53c7f1d640 1 -- start start 2023-11-13T10:56:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7faf8ac32640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf7c0e02a0 0x7faf7c0e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7faf8ac32640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faf7c0e40d0 0x7faf7c006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7faf8ac32640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7c009480 con 0x7faf7c0dfec0 2023-11-13T10:56:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7faf8ac32640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7c0e44e0 con 0x7faf7c0e02a0 2023-11-13T10:56:01.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7faf8ac32640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7c056560 con 0x7faf7c0e40d0 2023-11-13T10:56:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7fe8edab4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8e8158fc0 0x7fe8e80b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7fe8edab4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e80b1ab0 con 0x7fe8e80b37b0 2023-11-13T10:56:01.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7fe8edab4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e806fa10 con 0x7fe8e806a190 2023-11-13T10:56:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7f707effd640 1 -- 172.21.15.16:0/8598081 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707804fbe0 con 0x7f707803a590 2023-11-13T10:56:01.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.063+0000 7f707ffff640 1 --2- 172.21.15.16:0/8598081 >> v2:172.21.15.184:3300/0 conn(0x7f707804f500 0x7f7078006330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:56:01.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.063+0000 7faf8a431640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faf7c0e40d0 0x7faf7c006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.063+0000 7faf8a431640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faf7c0e40d0 0x7faf7c006a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39216/0 (socket says 172.21.15.16:39216) 2023-11-13T10:56:01.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.063+0000 7faf8a431640 1 -- 172.21.15.16:0/85624262 learned_addr learned my addr 172.21.15.16:0/85624262 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:01.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.063+0000 7f707effd640 1 --2- 172.21.15.16:0/8598081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f707803a590 0x7f70780385f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f70800690c0 tx=0x7f70800a60c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778fe94b33fcd536 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7faf8a431640 1 -- 172.21.15.16:0/85624262 >> v2:172.21.15.16:3300/0 conn(0x7faf7c0e02a0 msgr2=0x7faf7c0e3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.062+0000 7fe8edab4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e806fcf0 con 0x7fe8e8158fc0 2023-11-13T10:56:01.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7fe8ecab2640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fe8e806a190 0x7fe8e806a570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43616/0 (socket says 172.21.15.16:43616) 2023-11-13T10:56:01.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7fe8ecab2640 1 -- 172.21.15.16:0/2083632986 learned_addr learned my addr 172.21.15.16:0/2083632986 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:01.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7fe8ed2b3640 1 --2- 172.21.15.16:0/2083632986 >> v2:172.21.15.16:3300/0 conn(0x7fe8e8158fc0 0x7fe8e80b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7faf8942f640 1 --2- 172.21.15.16:0/85624262 >> v2:172.21.15.16:3300/0 conn(0x7faf7c0e02a0 0x7faf7c0e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7f707e7fc640 1 -- 172.21.15.16:0/8598081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800ae440 con 0x7f707803a590 2023-11-13T10:56:01.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7f707e7fc640 1 -- 172.21.15.16:0/8598081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70800ae5a0 con 0x7f707803a590 2023-11-13T10:56:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7f53c7f1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b8005e50 0x7f53b8006290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7f53c7f1d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b80542f0 con 0x7f53b80eacc0 2023-11-13T10:56:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7f53c7f1d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b80536e0 con 0x7f53b80067d0 2023-11-13T10:56:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7f53c7f1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80539c0 con 0x7f53b8005e50 2023-11-13T10:56:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7faf8a431640 1 --2- 172.21.15.16:0/85624262 >> v2:172.21.15.16:3300/0 conn(0x7faf7c0e02a0 0x7faf7c0e3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7faf8a431640 1 -- 172.21.15.16:0/85624262 >> v1:172.21.15.184:6789/0 conn(0x7faf7c0dfec0 legacy=0x7faf7c0085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.064+0000 7faf8a431640 1 -- 172.21.15.16:0/85624262 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c056840 con 0x7faf7c0e40d0 2023-11-13T10:56:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7faf8942f640 1 --2- 172.21.15.16:0/85624262 >> v2:172.21.15.16:3300/0 conn(0x7faf7c0e02a0 0x7faf7c0e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:01.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7fe8e77fe640 1 -- 172.21.15.16:0/2083632986 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299837528 0 0) 0x7fe8e806fa10 con 0x7fe8e806a190 2023-11-13T10:56:01.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7fe8e77fe640 1 -- 172.21.15.16:0/2083632986 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d0003520 con 0x7fe8e806a190 2023-11-13T10:56:01.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f53c54b8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b8005e50 0x7f53b8006290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f53c64ba640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f53b80067d0 0x7f53b8006c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43640/0 (socket says 172.21.15.16:43640) 2023-11-13T10:56:01.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f53c64ba640 1 -- 172.21.15.16:0/3150826822 learned_addr learned my addr 172.21.15.16:0/3150826822 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:01.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f53c5cb9640 1 --1- 172.21.15.16:0/3150826822 >> v1:172.21.15.16:6789/0 conn(0x7f53b80eacc0 0x7f53b804fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34586/0 (socket says 172.21.15.16:34586) 2023-11-13T10:56:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7fe8ed2b3640 1 -- 172.21.15.16:0/2083632986 >> v1:172.21.15.184:6789/0 conn(0x7fe8e806a190 legacy=0x7fe8e806a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7fe8ed2b3640 1 -- 172.21.15.16:0/2083632986 >> v1:172.21.15.16:6789/0 conn(0x7fe8e80b37b0 legacy=0x7fe8e806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7fe8ed2b3640 1 -- 172.21.15.16:0/2083632986 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e806ffd0 con 0x7fe8e8158fc0 2023-11-13T10:56:01.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f7084dd0640 1 -- 172.21.15.16:0/8598081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f707803a590 msgr2=0x7f70780385f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f7084dd0640 1 --2- 172.21.15.16:0/8598081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f707803a590 0x7f70780385f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f70800690c0 tx=0x7f70800a60c0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f7084dd0640 1 -- 172.21.15.16:0/8598081 shutdown_connections 2023-11-13T10:56:01.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f7084dd0640 1 --2- 172.21.15.16:0/8598081 >> v2:172.21.15.184:3300/0 conn(0x7f707804f500 0x7f7078006330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f7084dd0640 1 --2- 172.21.15.16:0/8598081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f707803a590 0x7f70780385f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.065+0000 7f7084dd0640 1 -- 172.21.15.16:0/8598081 >> 172.21.15.16:0/8598081 conn(0x7f7078009f70 msgr2=0x7f707800a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.066+0000 7f53c4cb7640 1 -- 172.21.15.16:0/3150826822 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664598004 0 0) 0x7f53b80542f0 con 0x7f53b80eacc0 2023-11-13T10:56:01.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.066+0000 7f7084dd0640 1 -- 172.21.15.16:0/8598081 shutdown_connections 2023-11-13T10:56:01.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.066+0000 7f53c54b8640 1 -- 172.21.15.16:0/3150826822 >> v1:172.21.15.184:6789/0 conn(0x7f53b80067d0 legacy=0x7f53b8006c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7f53c54b8640 1 -- 172.21.15.16:0/3150826822 >> v1:172.21.15.16:6789/0 conn(0x7f53b80eacc0 legacy=0x7f53b804fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7fe8ed2b3640 1 --2- 172.21.15.16:0/2083632986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e8158fc0 0x7fe8e80b1fb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009c80 tx=0x7fe8e002cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d179ec4df130307 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7f7084dd0640 1 -- 172.21.15.16:0/8598081 wait complete. 2023-11-13T10:56:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7faf8a431640 1 --2- 172.21.15.16:0/85624262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c0e40d0 0x7faf7c006a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf84067e10 tx=0x7faf840a6200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c0ab0f773bae1a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7f53c54b8640 1 -- 172.21.15.16:0/3150826822 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8053ca0 con 0x7f53b8005e50 2023-11-13T10:56:01.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7fe8e77fe640 1 -- 172.21.15.16:0/2083632986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e002dd70 con 0x7fe8e8158fc0 2023-11-13T10:56:01.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7fe8e77fe640 1 -- 172.21.15.16:0/2083632986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0035440 con 0x7fe8e8158fc0 2023-11-13T10:56:01.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7fe8e77fe640 1 -- 172.21.15.16:0/2083632986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00357d0 con 0x7fe8e8158fc0 2023-11-13T10:56:01.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7faf88c2e640 1 -- 172.21.15.16:0/85624262 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf840ae3f0 con 0x7faf7c0e40d0 2023-11-13T10:56:01.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7faf88c2e640 1 -- 172.21.15.16:0/85624262 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf840ae590 con 0x7faf7c0e40d0 2023-11-13T10:56:01.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.067+0000 7faf88c2e640 1 -- 172.21.15.16:0/85624262 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf840ae9c0 con 0x7faf7c0e40d0 2023-11-13T10:56:01.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.068+0000 7fe8edab4640 1 -- 172.21.15.16:0/2083632986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e8158fc0 msgr2=0x7fe8e80b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.068+0000 7fe8edab4640 1 --2- 172.21.15.16:0/2083632986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e8158fc0 0x7fe8e80b1fb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009c80 tx=0x7fe8e002cfc0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.068+0000 7faf8ac32640 1 -- 172.21.15.16:0/85624262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c0e40d0 msgr2=0x7faf7c006a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.068+0000 7faf8ac32640 1 --2- 172.21.15.16:0/85624262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c0e40d0 0x7faf7c006a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf84067e10 tx=0x7faf840a6200 comp rx=0 tx=0).stop 2023-11-13T10:56:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7fe8edab4640 1 -- 172.21.15.16:0/2083632986 shutdown_connections 2023-11-13T10:56:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7fe8edab4640 1 --2- 172.21.15.16:0/2083632986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e8158fc0 0x7fe8e80b1fb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7fe8edab4640 1 -- 172.21.15.16:0/2083632986 >> 172.21.15.16:0/2083632986 conn(0x7fe8e80b04a0 msgr2=0x7fe8e80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7f7084dd0640 1 Processor -- start 2023-11-13T10:56:01.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7fe8edab4640 1 -- 172.21.15.16:0/2083632986 shutdown_connections 2023-11-13T10:56:01.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7fe8edab4640 1 -- 172.21.15.16:0/2083632986 wait complete. 2023-11-13T10:56:01.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7faf8ac32640 1 -- 172.21.15.16:0/85624262 shutdown_connections 2023-11-13T10:56:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7faf8ac32640 1 --2- 172.21.15.16:0/85624262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c0e40d0 0x7faf7c006a80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7faf8ac32640 1 --2- 172.21.15.16:0/85624262 >> v2:172.21.15.16:3300/0 conn(0x7faf7c0e02a0 0x7faf7c0e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7faf8ac32640 1 -- 172.21.15.16:0/85624262 >> 172.21.15.16:0/85624262 conn(0x7faf7c01ef10 msgr2=0x7faf7c055c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7f53c54b8640 1 --2- 172.21.15.16:0/3150826822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b8005e50 0x7f53b8006290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009750 tx=0x7f53bc02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19e74a3cd7d908db server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7faf8ac32640 1 -- 172.21.15.16:0/85624262 shutdown_connections 2023-11-13T10:56:01.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7faf8ac32640 1 -- 172.21.15.16:0/85624262 wait complete. 2023-11-13T10:56:01.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.070+0000 7f53c4cb7640 1 -- 172.21.15.16:0/3150826822 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0316f0 con 0x7f53b8005e50 2023-11-13T10:56:01.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.070+0000 7f53c4cb7640 1 -- 172.21.15.16:0/3150826822 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc031850 con 0x7f53b8005e50 2023-11-13T10:56:01.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.070+0000 7f53c4cb7640 1 -- 172.21.15.16:0/3150826822 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc031be0 con 0x7f53b8005e50 2023-11-13T10:56:01.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.071+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3150826822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b8005e50 msgr2=0x7f53b8006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.069+0000 7f7084dd0640 1 -- start start 2023-11-13T10:56:01.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.071+0000 7f53c7f1d640 1 --2- 172.21.15.16:0/3150826822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b8005e50 0x7f53b8006290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009750 tx=0x7f53bc02b790 comp rx=0 tx=0).stop 2023-11-13T10:56:01.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.071+0000 7f7084dd0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f707803a590 0x7f70780ef330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.071+0000 7f7084dd0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f707804f500 0x7f70780e04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.071+0000 7f7084dd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70780e1050 0x7f707804e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.071+0000 7f7084dd0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780ef930 con 0x7f70780e1050 2023-11-13T10:56:01.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.071+0000 7f7084dd0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780efc10 con 0x7f707804f500 2023-11-13T10:56:01.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.071+0000 7f7084dd0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780efef0 con 0x7f707803a590 2023-11-13T10:56:01.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.072+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3150826822 shutdown_connections 2023-11-13T10:56:01.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.072+0000 7f53c7f1d640 1 --2- 172.21.15.16:0/3150826822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b8005e50 0x7f53b8006290 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.072+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3150826822 >> 172.21.15.16:0/3150826822 conn(0x7f53b8009f70 msgr2=0x7f53b800a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.072+0000 7fe8edab4640 1 Processor -- start 2023-11-13T10:56:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.072+0000 7f707f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f707803a590 0x7f70780ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.072+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3150826822 shutdown_connections 2023-11-13T10:56:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.072+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3150826822 wait complete. 2023-11-13T10:56:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.072+0000 7faf8ac32640 1 Processor -- start 2023-11-13T10:56:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.072+0000 7fe8edab4640 1 -- start start 2023-11-13T10:56:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.073+0000 7f707ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70780e1050 0x7f707804e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.073+0000 7f707ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70780e1050 0x7f707804e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56248/0 (socket says 172.21.15.16:56248) 2023-11-13T10:56:01.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.073+0000 7f707ffff640 1 -- 172.21.15.16:0/2166078017 learned_addr learned my addr 172.21.15.16:0/2166078017 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.073+0000 7faf8ac32640 1 -- start start 2023-11-13T10:56:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.074+0000 7f707ffff640 1 -- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f707803a590 msgr2=0x7f70780ef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.074+0000 7fe8edab4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e806a190 0x7fe8e81e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.074+0000 7fe8edab4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b37b0 0x7fe8e81e6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.074+0000 7fe8edab4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e8158fc0 0x7fe8e81ff840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.074+0000 7fe8edab4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81e6b60 con 0x7fe8e80b37b0 2023-11-13T10:56:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7fe8edab4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e82014d0 con 0x7fe8e8158fc0 2023-11-13T10:56:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.074+0000 7f707ffff640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f707803a590 0x7f70780ef330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7f707effd640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f707804f500 0x7f70780e04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b37b0 0x7fe8e81e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7fe8edab4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8200ba0 con 0x7fe8e806a190 2023-11-13T10:56:01.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b37b0 0x7fe8e81e6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56260/0 (socket says 172.21.15.16:56260) 2023-11-13T10:56:01.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7fe8e7fff640 1 -- 172.21.15.16:0/2553566850 learned_addr learned my addr 172.21.15.16:0/2553566850 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:01.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7fe8ecab2640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e806a190 0x7fe8e81e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7fe8ed2b3640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e8158fc0 0x7fe8e81ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7f707ffff640 1 -- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f707804f500 msgr2=0x7f70780e04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7f707ffff640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f707804f500 0x7f70780e04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7f53c7f1d640 1 Processor -- start 2023-11-13T10:56:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.075+0000 7f707ffff640 1 -- 172.21.15.16:0/2166078017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c009400 con 0x7f70780e1050 2023-11-13T10:56:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7fe8e7fff640 1 -- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e806a190 msgr2=0x7fe8e81e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7f707effd640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f707804f500 0x7f70780e04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7faf8ac32640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf7c0dfec0 0x7faf7c051f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7faf8ac32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0e40d0 0x7faf7c0d1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7f53c7f1d640 1 -- start start 2023-11-13T10:56:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7f707ffff640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70780e1050 0x7f707804e730 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f706c00e930 tx=0x7f706c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7f706bfff640 1 -- 172.21.15.16:0/2166078017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c004020 con 0x7f70780e1050 2023-11-13T10:56:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70780f01d0 con 0x7f70780e1050 2023-11-13T10:56:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7faf8ac32640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c0d61b0 0x7faf7c0d7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7faf8ac32640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c0524b0 con 0x7faf7c0e40d0 2023-11-13T10:56:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7faf8ac32640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c050c80 con 0x7faf7c0d61b0 2023-11-13T10:56:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7faf8ac32640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c0d70e0 con 0x7faf7c0dfec0 2023-11-13T10:56:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7fe8e7fff640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e806a190 0x7fe8e81e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7fe8e7fff640 1 -- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e8158fc0 msgr2=0x7fe8e81ff840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7fe8e7fff640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e8158fc0 0x7fe8e81ff840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.076+0000 7fe8e7fff640 1 -- 172.21.15.16:0/2553566850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e000d040 con 0x7fe8e80b37b0 2023-11-13T10:56:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707804ec70 con 0x7f70780e1050 2023-11-13T10:56:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7fe8e7fff640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b37b0 0x7fe8e81e6300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c410 tx=0x7fe8dc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7faf8942f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0e40d0 0x7faf7c0d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7faf89c30640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf7c0dfec0 0x7faf7c051f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7f706bfff640 1 -- 172.21.15.16:0/2166078017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c0041a0 con 0x7f70780e1050 2023-11-13T10:56:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7f706bfff640 1 -- 172.21.15.16:0/2166078017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c00c940 con 0x7f70780e1050 2023-11-13T10:56:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7fe8e57fa640 1 -- 172.21.15.16:0/2553566850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0035f0 con 0x7fe8e80b37b0 2023-11-13T10:56:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7fe8e57fa640 1 -- 172.21.15.16:0/2553566850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc003750 con 0x7fe8e80b37b0 2023-11-13T10:56:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7faf89c30640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf7c0dfec0 0x7faf7c051f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50638/0 (socket says 172.21.15.16:50638) 2023-11-13T10:56:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7faf89c30640 1 -- 172.21.15.16:0/544070905 learned_addr learned my addr 172.21.15.16:0/544070905 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7faf8a431640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c0d61b0 0x7faf7c0d7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.077+0000 7fe8e57fa640 1 -- 172.21.15.16:0/2553566850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc003b00 con 0x7fe8e80b37b0 2023-11-13T10:56:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.078+0000 7faf8942f640 1 -- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf7c0dfec0 msgr2=0x7faf7c051f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.078+0000 7faf8942f640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf7c0dfec0 0x7faf7c051f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.079+0000 7faf8942f640 1 -- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c0d61b0 msgr2=0x7faf7c0d7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.079+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7060005250 con 0x7f70780e1050 2023-11-13T10:56:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.079+0000 7faf8942f640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c0d61b0 0x7faf7c0d7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.079+0000 7faf8942f640 1 -- 172.21.15.16:0/544070905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c050ed0 con 0x7faf7c0e40d0 2023-11-13T10:56:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.080+0000 7f53c7f1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8005e50 0x7f53b804e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.080+0000 7f53c7f1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53b80067d0 0x7f53b80e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.080+0000 7f53c7f1d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b80eacc0 0x7f53b80e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.080+0000 7f53c7f1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b804bb50 con 0x7f53b8005e50 2023-11-13T10:56:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.080+0000 7f53c7f1d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80e56f0 con 0x7f53b80067d0 2023-11-13T10:56:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.080+0000 7f53c7f1d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80e4820 con 0x7f53b80eacc0 2023-11-13T10:56:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.080+0000 7faf8942f640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0e40d0 0x7faf7c0d1190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf7400c7f0 tx=0x7faf7400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.081+0000 7faf7a7fc640 1 -- 172.21.15.16:0/544070905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74004220 con 0x7faf7c0e40d0 2023-11-13T10:56:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.079+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e8200e80 con 0x7fe8e80b37b0 2023-11-13T10:56:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.079+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e81e6840 con 0x7fe8e80b37b0 2023-11-13T10:56:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.081+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c0511b0 con 0x7faf7c0e40d0 2023-11-13T10:56:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.081+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c051770 con 0x7faf7c0e40d0 2023-11-13T10:56:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.082+0000 7faf7a7fc640 1 -- 172.21.15.16:0/544070905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf74004380 con 0x7faf7c0e40d0 2023-11-13T10:56:01.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.082+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e8071130 con 0x7fe8e80b37b0 2023-11-13T10:56:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.083+0000 7faf7a7fc640 1 -- 172.21.15.16:0/544070905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74004220 con 0x7faf7c0e40d0 2023-11-13T10:56:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.081+0000 7f53c54b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53b80067d0 0x7f53b80e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.081+0000 7f53c54b8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53b80067d0 0x7f53b80e2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39246/0 (socket says 172.21.15.16:39246) 2023-11-13T10:56:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.081+0000 7f53c54b8640 1 -- 172.21.15.16:0/3712624179 learned_addr learned my addr 172.21.15.16:0/3712624179 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.081+0000 7f53c64ba640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b80eacc0 0x7f53b80e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.082+0000 7f53c64ba640 1 -- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53b80067d0 msgr2=0x7f53b80e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.082+0000 7f53c64ba640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53b80067d0 0x7f53b80e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.082+0000 7f53c64ba640 1 -- 172.21.15.16:0/3712624179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8005e50 msgr2=0x7f53b804e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.083+0000 7f53c5cb9640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8005e50 0x7f53b804e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.083+0000 7f53c64ba640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8005e50 0x7f53b804e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.083+0000 7f53c64ba640 1 -- 172.21.15.16:0/3712624179 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc009400 con 0x7f53b80eacc0 2023-11-13T10:56:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.083+0000 7f53c54b8640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53b80067d0 0x7f53b80e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:56:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.082+0000 7f706bfff640 1 -- 172.21.15.16:0/2166078017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f706c00caa0 con 0x7f70780e1050 2023-11-13T10:56:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.083+0000 7f706bfff640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f706406ae70 0x7f706406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.084+0000 7f706bfff640 1 -- 172.21.15.16:0/2166078017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f706c089840 con 0x7f70780e1050 2023-11-13T10:56:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.084+0000 7f53c5cb9640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8005e50 0x7f53b804e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.084+0000 7f707f7fe640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f706406ae70 0x7f706406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.084+0000 7fe8e57fa640 1 -- 172.21.15.16:0/2553566850 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe8dc003c60 con 0x7fe8e80b37b0 2023-11-13T10:56:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.085+0000 7f707f7fe640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f706406ae70 0x7f706406d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7074000f60 tx=0x7f7074009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.085+0000 7f53c64ba640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b80eacc0 0x7f53b80e2f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53c002cd20 tx=0x7f53c0072700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.086+0000 7f53b67fc640 1 -- 172.21.15.16:0/3712624179 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00680c0 con 0x7f53b80eacc0 2023-11-13T10:56:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.086+0000 7f53b67fc640 1 -- 172.21.15.16:0/3712624179 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c008e070 con 0x7f53b80eacc0 2023-11-13T10:56:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.086+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c005250 con 0x7faf7c0e40d0 2023-11-13T10:56:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.087+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b804e730 con 0x7f53b80eacc0 2023-11-13T10:56:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.087+0000 7f53b67fc640 1 -- 172.21.15.16:0/3712624179 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0068460 con 0x7f53b80eacc0 2023-11-13T10:56:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.087+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b80e4dc0 con 0x7f53b80eacc0 2023-11-13T10:56:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.086+0000 7fe8e57fa640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8d406ae90 0x7fe8d406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.086+0000 7fe8e57fa640 1 -- 172.21.15.16:0/2553566850 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe8dc088fe0 con 0x7fe8e80b37b0 2023-11-13T10:56:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.087+0000 7fe8ecab2640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8d406ae90 0x7fe8d406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.088+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b8005350 con 0x7f53b80eacc0 2023-11-13T10:56:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.089+0000 7fe8ecab2640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8d406ae90 0x7fe8d406d350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80049d0 tx=0x7fe8d800b250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.091+0000 7f53b67fc640 1 -- 172.21.15.16:0/3712624179 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f53c0072a50 con 0x7f53b80eacc0 2023-11-13T10:56:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.092+0000 7f53b67fc640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53a406ae40 0x7f53a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.092+0000 7f53b67fc640 1 -- 172.21.15.16:0/3712624179 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f53c006b070 con 0x7f53b80eacc0 2023-11-13T10:56:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.093+0000 7f53c5cb9640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53a406ae40 0x7f53a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.094+0000 7f706bfff640 1 -- 172.21.15.16:0/2166078017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f706c054560 con 0x7f70780e1050 2023-11-13T10:56:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.096+0000 7fe8e57fa640 1 -- 172.21.15.16:0/2553566850 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8dc053d90 con 0x7fe8e80b37b0 2023-11-13T10:56:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.095+0000 7faf7a7fc640 1 -- 172.21.15.16:0/544070905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf74007830 con 0x7faf7c0e40d0 2023-11-13T10:56:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.097+0000 7faf7a7fc640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf6806af30 0x7faf6806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.098+0000 7faf7a7fc640 1 -- 172.21.15.16:0/544070905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faf74024e20 con 0x7faf7c0e40d0 2023-11-13T10:56:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.098+0000 7faf89c30640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf6806af30 0x7faf6806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.097+0000 7f53c5cb9640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53a406ae40 0x7f53a406d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53b0000f60 tx=0x7f53b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.099+0000 7faf89c30640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf6806af30 0x7faf6806d3f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faf7c050180 tx=0x7faf80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.103+0000 7f53b67fc640 1 -- 172.21.15.16:0/3712624179 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53c00cfba0 con 0x7f53b80eacc0 2023-11-13T10:56:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.105+0000 7faf7a7fc640 1 -- 172.21.15.16:0/544070905 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf74054d80 con 0x7faf7c0e40d0 2023-11-13T10:56:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.368+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f47f40057d0 con 0x7f480c04e2d0 2023-11-13T10:56:01.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.369+0000 7f48117fa640 1 -- 172.21.15.16:0/1616038315 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f48140c5020 con 0x7f480c04e2d0 2023-11-13T10:56:01.377 INFO:teuthology.orchestra.run.smithi016.stdout:64424509452 2023-11-13T10:56:01.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.373+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47f806ae40 msgr2=0x7f47f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.373+0000 7f481b1e9640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47f806ae40 0x7f47f806d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f480800da90 tx=0x7f4808008040 comp rx=0 tx=0).stop 2023-11-13T10:56:01.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.374+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f480c04e2d0 msgr2=0x7f480c0d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.374+0000 7f481b1e9640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f480c04e2d0 0x7f480c0d18a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4814068d00 tx=0x7f4814069e90 comp rx=0 tx=0).stop 2023-11-13T10:56:01.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.374+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 shutdown_connections 2023-11-13T10:56:01.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.374+0000 7f481b1e9640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47f806ae40 0x7f47f806d300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.375+0000 7f481b1e9640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f480c04e2d0 0x7f480c0d18a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.375+0000 7f481b1e9640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f480c0d5fa0 0x7f480c04dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.375+0000 7f481b1e9640 1 --2- 172.21.15.16:0/1616038315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4804005020 0x7f480c0d5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.375+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 >> 172.21.15.16:0/1616038315 conn(0x7f480c009f70 msgr2=0x7f480c0e5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.376+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 shutdown_connections 2023-11-13T10:56:01.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.376+0000 7f481b1e9640 1 -- 172.21.15.16:0/1616038315 wait complete. 2023-11-13T10:56:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.399+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7bb4007050 con 0x7f7bb40df3e0 2023-11-13T10:56:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.403+0000 7f7bb2ffd640 1 -- 172.21.15.16:0/1136506102 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7bb4007050 con 0x7f7bb40df3e0 2023-11-13T10:56:01.407 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2023-11-13T10:56:01.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509452 for osd.0 2023-11-13T10:56:01.409 DEBUG:teuthology.parallel:result is None 2023-11-13T10:56:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ba006ae90 msgr2=0x7f7ba006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ba006ae90 0x7f7ba006d350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8036000 tx=0x7f7bb803c040 comp rx=0 tx=0).stop 2023-11-13T10:56:01.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40df3e0 msgr2=0x7f7bb40e1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40df3e0 0x7f7bb40e1ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc067e10 tx=0x7f7bbc073ad0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 shutdown_connections 2023-11-13T10:56:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ba006ae90 0x7f7ba006d350 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40df3e0 0x7f7bb40e1ea0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4054230 0x7f7bb40e1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 --2- 172.21.15.16:0/1136506102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb404fa50 0x7f7bb40d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.407+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 >> 172.21.15.16:0/1136506102 conn(0x7f7bb4009f70 msgr2=0x7f7bb403a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.412+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 shutdown_connections 2023-11-13T10:56:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.412+0000 7f7bc45c9640 1 -- 172.21.15.16:0/1136506102 wait complete. 2023-11-13T10:56:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.436+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffa8c0057d0 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.1 2023-11-13T10:56:01.442 INFO:teuthology.orchestra.run.smithi016.stdout:64424509452 2023-11-13T10:56:01.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.437+0000 7ffaa8ff9640 1 -- 172.21.15.16:0/3331556174 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffaa4003e60 con 0x7ffaac0cfeb0 2023-11-13T10:56:01.444 DEBUG:teuthology.parallel:result is None 2023-11-13T10:56:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffa9806ae40 msgr2=0x7ffa9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffa9806ae40 0x7ffa9806d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffaac15df10 tx=0x7ffaa0009040 comp rx=0 tx=0).stop 2023-11-13T10:56:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaac0cfeb0 msgr2=0x7ffaac0cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaac0cfeb0 0x7ffaac0cc720 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400cd90 tx=0x7ffaa400a660 comp rx=0 tx=0).stop 2023-11-13T10:56:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 shutdown_connections 2023-11-13T10:56:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffa9806ae40 0x7ffa9806d300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaac161520 0x7ffaac09ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaac15d150 0x7ffaac15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 --2- 172.21.15.16:0/3331556174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaac0cfeb0 0x7ffaac0cc720 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400cd90 tx=0x7ffaa400a660 comp rx=0 tx=0).stop 2023-11-13T10:56:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 >> 172.21.15.16:0/3331556174 conn(0x7ffaac09c030 msgr2=0x7ffaac06bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 shutdown_connections 2023-11-13T10:56:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.441+0000 7ffab272c640 1 -- 172.21.15.16:0/3331556174 wait complete. 2023-11-13T10:56:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.445+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7adc0050c0 con 0x7f7afc15b130 2023-11-13T10:56:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.446+0000 7f7ae37fe640 1 -- 172.21.15.16:0/2035955787 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7af001e070 con 0x7f7afc15b130 2023-11-13T10:56:01.454 INFO:teuthology.orchestra.run.smithi016.stdout:64424509453 2023-11-13T10:56:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ae806ae40 msgr2=0x7f7ae806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ae806ae40 0x7f7ae806d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7af4002230 tx=0x7f7af40013f0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7afc15b130 msgr2=0x7f7afc0aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7afc15b130 0x7f7afc0aee10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7af000c590 tx=0x7f7af000ca50 comp rx=0 tx=0).stop 2023-11-13T10:56:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 shutdown_connections 2023-11-13T10:56:01.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ae806ae40 0x7f7ae806d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7afc15b130 0x7f7afc0aee10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7afc165a10 0x7f7afc14ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 --2- 172.21.15.16:0/2035955787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7afc0cd940 0x7f7afc15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 >> 172.21.15.16:0/2035955787 conn(0x7f7afc06a190 msgr2=0x7f7afc15ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.448+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 shutdown_connections 2023-11-13T10:56:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.449+0000 7f7b013bb640 1 -- 172.21.15.16:0/2035955787 wait complete. 2023-11-13T10:56:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.452+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7060005500 con 0x7f70780e1050 2023-11-13T10:56:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.453+0000 7f706bfff640 1 -- 172.21.15.16:0/2166078017 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f706c057b80 con 0x7f70780e1050 2023-11-13T10:56:01.459 INFO:teuthology.orchestra.run.smithi016.stdout:64424509452 2023-11-13T10:56:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.455+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f706406ae70 msgr2=0x7f706406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.455+0000 7f7084dd0640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f706406ae70 0x7f706406d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7074000f60 tx=0x7f7074009040 comp rx=0 tx=0).stop 2023-11-13T10:56:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.455+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70780e1050 msgr2=0x7f707804e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.455+0000 7f7084dd0640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70780e1050 0x7f707804e730 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f706c00e930 tx=0x7f706c00edf0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.456+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 shutdown_connections 2023-11-13T10:56:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.456+0000 7f7084dd0640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f706406ae70 0x7f706406d330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.456+0000 7f7084dd0640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70780e1050 0x7f707804e730 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.456+0000 7f7084dd0640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f707804f500 0x7f70780e04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.456+0000 7f7084dd0640 1 --2- 172.21.15.16:0/2166078017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f707803a590 0x7f70780ef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.456+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 >> 172.21.15.16:0/2166078017 conn(0x7f7078009f70 msgr2=0x7f70780347f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.456+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 shutdown_connections 2023-11-13T10:56:01.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.456+0000 7f7084dd0640 1 -- 172.21.15.16:0/2166078017 wait complete. 2023-11-13T10:56:01.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509453 for osd.5 2023-11-13T10:56:01.475 DEBUG:teuthology.parallel:result is None 2023-11-13T10:56:01.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509452 for osd.2 2023-11-13T10:56:01.476 DEBUG:teuthology.parallel:result is None 2023-11-13T10:56:01.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509452 for osd.6 2023-11-13T10:56:01.477 DEBUG:teuthology.parallel:result is None 2023-11-13T10:56:01.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.490+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faf6c0050c0 con 0x7faf7c0e40d0 2023-11-13T10:56:01.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.491+0000 7faf7a7fc640 1 -- 172.21.15.16:0/544070905 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf74015090 con 0x7faf7c0e40d0 2023-11-13T10:56:01.496 INFO:teuthology.orchestra.run.smithi016.stdout:64424509453 2023-11-13T10:56:01.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf6806af30 msgr2=0x7faf6806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf6806af30 0x7faf6806d3f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faf7c050180 tx=0x7faf80009040 comp rx=0 tx=0).stop 2023-11-13T10:56:01.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0e40d0 msgr2=0x7faf7c0d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0e40d0 0x7faf7c0d1190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf7400c7f0 tx=0x7faf7400ccb0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 shutdown_connections 2023-11-13T10:56:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf6806af30 0x7faf6806d3f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf7c0d61b0 0x7faf7c0d7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0e40d0 0x7faf7c0d1190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 --2- 172.21.15.16:0/544070905 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf7c0dfec0 0x7faf7c051f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 >> 172.21.15.16:0/544070905 conn(0x7faf7c01ef10 msgr2=0x7faf7c00c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 shutdown_connections 2023-11-13T10:56:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.493+0000 7faf8ac32640 1 -- 172.21.15.16:0/544070905 wait complete. 2023-11-13T10:56:01.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509453 for osd.4 2023-11-13T10:56:01.518 DEBUG:teuthology.parallel:result is None 2023-11-13T10:56:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.531+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe8e806cb50 con 0x7fe8e80b37b0 2023-11-13T10:56:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.532+0000 7fe8e57fa640 1 -- 172.21.15.16:0/2553566850 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe8dc0573b0 con 0x7fe8e80b37b0 2023-11-13T10:56:01.536 INFO:teuthology.orchestra.run.smithi016.stdout:60129542156 2023-11-13T10:56:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8d406ae90 msgr2=0x7fe8d406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8d406ae90 0x7fe8d406d350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80049d0 tx=0x7fe8d800b250 comp rx=0 tx=0).stop 2023-11-13T10:56:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b37b0 msgr2=0x7fe8e81e6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b37b0 0x7fe8e81e6300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c410 tx=0x7fe8dc00c8d0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 shutdown_connections 2023-11-13T10:56:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8d406ae90 0x7fe8d406d350 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e8158fc0 0x7fe8e81ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8e80b37b0 0x7fe8e81e6300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 --2- 172.21.15.16:0/2553566850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e806a190 0x7fe8e81e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 >> 172.21.15.16:0/2553566850 conn(0x7fe8e80b04a0 msgr2=0x7fe8e80af780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 shutdown_connections 2023-11-13T10:56:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.535+0000 7fe8edab4640 1 -- 172.21.15.16:0/2553566850 wait complete. 2023-11-13T10:56:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.536+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f53b80e5830 con 0x7f53b80eacc0 2023-11-13T10:56:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.537+0000 7f53b67fc640 1 -- 172.21.15.16:0/3712624179 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f53c0099020 con 0x7f53b80eacc0 2023-11-13T10:56:01.543 INFO:teuthology.orchestra.run.smithi016.stdout:64424509452 2023-11-13T10:56:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53a406ae40 msgr2=0x7f53a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53a406ae40 0x7f53a406d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53b0000f60 tx=0x7f53b0009040 comp rx=0 tx=0).stop 2023-11-13T10:56:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b80eacc0 msgr2=0x7f53b80e2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b80eacc0 0x7f53b80e2f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53c002cd20 tx=0x7f53c0072700 comp rx=0 tx=0).stop 2023-11-13T10:56:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 shutdown_connections 2023-11-13T10:56:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53a406ae40 0x7f53a406d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53b80eacc0 0x7f53b80e2f50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53b80067d0 0x7f53b80e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 --2- 172.21.15.16:0/3712624179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53b8005e50 0x7f53b804e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 >> 172.21.15.16:0/3712624179 conn(0x7f53b8009f70 msgr2=0x7f53b8052c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 shutdown_connections 2023-11-13T10:56:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.539+0000 7f53c7f1d640 1 -- 172.21.15.16:0/3712624179 wait complete. 2023-11-13T10:56:01.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.7 2023-11-13T10:56:01.557 DEBUG:teuthology.parallel:result is None 2023-11-13T10:56:01.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509452 for osd.3 2023-11-13T10:56:01.559 DEBUG:teuthology.parallel:result is None 2023-11-13T10:56:01.559 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-13T10:56:01.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T10:56:01.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.820+0000 7f6a7f59e640 1 Processor -- start 2023-11-13T10:56:01.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.820+0000 7f6a7f59e640 1 -- start start 2023-11-13T10:56:01.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.821+0000 7f6a7f59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6a800cb190 0x7f6a8015b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.821+0000 7f6a7f59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a80158c20 con 0x7f6a800716b0 2023-11-13T10:56:01.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.821+0000 7f6a7f59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a800b1830 con 0x7f6a8006a190 2023-11-13T10:56:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.821+0000 7f6a7f59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8015b780 con 0x7f6a800cb190 2023-11-13T10:56:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7dd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a800716b0 0x7f6a8006f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34604/0 (socket says 172.21.15.16:34604) 2023-11-13T10:56:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7dd9b640 1 -- 172.21.15.16:0/1671500236 learned_addr learned my addr 172.21.15.16:0/1671500236 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7e59c640 1 --2- 172.21.15.16:0/1671500236 >> v2:172.21.15.184:3300/0 conn(0x7f6a800cb190 0x7f6a8015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773088122 0 0) 0x7f6a80158c20 con 0x7f6a800716b0 2023-11-13T10:56:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a68003520 con 0x7f6a800716b0 2023-11-13T10:56:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1155092040 0 0) 0x7f6a68003520 con 0x7f6a800716b0 2023-11-13T10:56:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 >> v2:172.21.15.184:3300/0 conn(0x7f6a800cb190 msgr2=0x7f6a8015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7d59a640 1 --2- 172.21.15.16:0/1671500236 >> v2:172.21.15.184:3300/0 conn(0x7f6a800cb190 0x7f6a8015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 >> v1:172.21.15.184:6789/0 conn(0x7f6a8006a190 legacy=0x7f6a800b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.822+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8015ba60 con 0x7f6a800716b0 2023-11-13T10:56:01.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f6a74001670 con 0x7f6a800716b0 2023-11-13T10:56:01.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 >> v1:172.21.15.16:6789/0 conn(0x7f6a800716b0 legacy=0x7f6a8006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7d59a640 1 --2- 172.21.15.16:0/1671500236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a68003970 0x7f6a68003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7d59a640 1 --2- 172.21.15.16:0/1671500236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a680047c0 0x7f6a68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7d59a640 1 --2- 172.21.15.16:0/1671500236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a68005180 0x7f6a6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8015b780 con 0x7f6a68003970 2023-11-13T10:56:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6800c000 con 0x7f6a68005180 2023-11-13T10:56:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6800c1c0 con 0x7f6a680047c0 2023-11-13T10:56:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7e59c640 1 --2- 172.21.15.16:0/1671500236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a68003970 0x7f6a68003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7e59c640 1 -- 172.21.15.16:0/1671500236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a680047c0 msgr2=0x7f6a68004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7e59c640 1 --2- 172.21.15.16:0/1671500236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a680047c0 0x7f6a68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7e59c640 1 -- 172.21.15.16:0/1671500236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a68005180 msgr2=0x7f6a6800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7e59c640 1 --2- 172.21.15.16:0/1671500236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a68005180 0x7f6a6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7e59c640 1 -- 172.21.15.16:0/1671500236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a68003520 con 0x7f6a68003970 2023-11-13T10:56:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.823+0000 7f6a7ed9d640 1 -- 172.21.15.16:0/1671500236 reap_dead start 2023-11-13T10:56:01.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.824+0000 7f6a7e59c640 1 --2- 172.21.15.16:0/1671500236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a68003970 0x7f6a68003d70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009ac0 tx=0x7f6a680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badd66d74fa2cda9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.824+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7002c710 con 0x7f6a68003970 2023-11-13T10:56:01.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.825+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7002c870 con 0x7f6a68003970 2023-11-13T10:56:01.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.825+0000 7f6a7d59a640 1 -- 172.21.15.16:0/1671500236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7002cc00 con 0x7f6a68003970 2023-11-13T10:56:01.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.825+0000 7f6a7f59e640 1 -- 172.21.15.16:0/1671500236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a68003970 msgr2=0x7f6a68003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.825+0000 7f6a7f59e640 1 --2- 172.21.15.16:0/1671500236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a68003970 0x7f6a68003d70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009ac0 tx=0x7f6a680057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.825+0000 7f6a7f59e640 1 -- 172.21.15.16:0/1671500236 shutdown_connections 2023-11-13T10:56:01.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.825+0000 7f6a7f59e640 1 --2- 172.21.15.16:0/1671500236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a68003970 0x7f6a68003d70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.825+0000 7f6a7f59e640 1 -- 172.21.15.16:0/1671500236 >> 172.21.15.16:0/1671500236 conn(0x7f6a800b04a0 msgr2=0x7f6a800b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:01.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.825+0000 7f6a7f59e640 1 -- 172.21.15.16:0/1671500236 shutdown_connections 2023-11-13T10:56:01.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.825+0000 7f6a7f59e640 1 -- 172.21.15.16:0/1671500236 wait complete. 2023-11-13T10:56:01.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.827+0000 7f6a7f59e640 1 Processor -- start 2023-11-13T10:56:01.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.827+0000 7f6a7f59e640 1 -- start start 2023-11-13T10:56:01.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7f59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a68003970 0x7f6a801524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7f59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80158ff0 0x7f6a80150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a8015d6c0 0x7f6a8015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7f59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8015e090 con 0x7f6a8015d6c0 2023-11-13T10:56:01.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7f59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80152b30 con 0x7f6a80158ff0 2023-11-13T10:56:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7f59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80152e10 con 0x7f6a68003970 2023-11-13T10:56:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a8015d6c0 0x7f6a8015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a8015d6c0 0x7f6a8015db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56296/0 (socket says 172.21.15.16:56296) 2023-11-13T10:56:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7ed9d640 1 -- 172.21.15.16:0/3326419734 learned_addr learned my addr 172.21.15.16:0/3326419734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.828+0000 7f6a7e59c640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a68003970 0x7f6a801524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a7dd9b640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80158ff0 0x7f6a80150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a7ed9d640 1 -- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a68003970 msgr2=0x7f6a801524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a7ed9d640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a68003970 0x7f6a801524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a7ed9d640 1 -- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80158ff0 msgr2=0x7f6a80150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a7ed9d640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80158ff0 0x7f6a80150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a7ed9d640 1 -- 172.21.15.16:0/3326419734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a801530f0 con 0x7f6a8015d6c0 2023-11-13T10:56:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a7ed9d640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a8015d6c0 0x7f6a8015db50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6a8006dca0 tx=0x7f6a7800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a6effd640 1 -- 172.21.15.16:0/3326419734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a780070d0 con 0x7f6a8015d6c0 2023-11-13T10:56:01.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a6effd640 1 -- 172.21.15.16:0/3326419734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a78007230 con 0x7f6a8015d6c0 2023-11-13T10:56:01.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a6effd640 1 -- 172.21.15.16:0/3326419734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a780075e0 con 0x7f6a8015d6c0 2023-11-13T10:56:01.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.829+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a801533d0 con 0x7f6a8015d6c0 2023-11-13T10:56:01.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.830+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a80159470 con 0x7f6a8015d6c0 2023-11-13T10:56:01.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.830+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8004e4d0 con 0x7f6a8015d6c0 2023-11-13T10:56:01.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.831+0000 7f6a6effd640 1 -- 172.21.15.16:0/3326419734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a78007740 con 0x7f6a8015d6c0 2023-11-13T10:56:01.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.832+0000 7f6a6effd640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6a6006ae40 0x7f6a6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:01.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.832+0000 7f6a6effd640 1 -- 172.21.15.16:0/3326419734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6a78088e00 con 0x7f6a8015d6c0 2023-11-13T10:56:01.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.833+0000 7f6a7e59c640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6a6006ae40 0x7f6a6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:01.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.834+0000 7f6a7e59c640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6a6006ae40 0x7f6a6006d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a70036000 tx=0x7f6a70030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:01.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:01.836+0000 7f6a6effd640 1 -- 172.21.15.16:0/3326419734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a78053bb0 con 0x7f6a8015d6c0 2023-11-13T10:56:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.022+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a800cb570 con 0x7f6a6006ae40 2023-11-13T10:56:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.052+0000 7f6a6effd640 1 -- 172.21.15.16:0/3326419734 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345067 (secure 0 0 0) 0x7f6a70034440 con 0x7f6a6006ae40 2023-11-13T10:56:02.058 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T10:56:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6a6006ae40 msgr2=0x7f6a6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6a6006ae40 0x7f6a6006d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a70036000 tx=0x7f6a70030000 comp rx=0 tx=0).stop 2023-11-13T10:56:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a8015d6c0 msgr2=0x7f6a8015db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a8015d6c0 0x7f6a8015db50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6a8006dca0 tx=0x7f6a7800c890 comp rx=0 tx=0).stop 2023-11-13T10:56:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 shutdown_connections 2023-11-13T10:56:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6a6006ae40 0x7f6a6006d300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a8015d6c0 0x7f6a8015db50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a80158ff0 0x7f6a80150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 --2- 172.21.15.16:0/3326419734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a68003970 0x7f6a801524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 >> 172.21.15.16:0/3326419734 conn(0x7f6a800b04a0 msgr2=0x7f6a800b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 shutdown_connections 2023-11-13T10:56:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.061+0000 7f6a7f59e640 1 -- 172.21.15.16:0/3326419734 wait complete. 2023-11-13T10:56:02.087 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-11-13T10:56:01.981184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232004,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742716,"statfs":{"total":773094113280,"available":772856541184,"internally_reserved":0,"allocated":3416064,"data_stored":2031150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005625"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894617+0000","last_change":"2023-11-13T10:55:25.848955+0000","last_active":"2023-11-13T10:55:30.894617+0000","last_peered":"2023-11-13T10:55:30.894617+0000","last_clean":"2023-11-13T10:55:30.894617+0000","last_became_active":"2023-11-13T10:55:25.848644+0000","last_became_peered":"2023-11-13T10:55:25.848644+0000","last_unstale":"2023-11-13T10:55:30.894617+0000","last_undegraded":"2023-11-13T10:55:30.894617+0000","last_fullsized":"2023-11-13T10:55:30.894617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:40.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911163+0000","last_change":"2023-11-13T10:55:26.862834+0000","last_active":"2023-11-13T10:55:30.911163+0000","last_peered":"2023-11-13T10:55:30.911163+0000","last_clean":"2023-11-13T10:55:30.911163+0000","last_became_active":"2023-11-13T10:55:26.861881+0000","last_became_peered":"2023-11-13T10:55:26.861881+0000","last_unstale":"2023-11-13T10:55:30.911163+0000","last_undegraded":"2023-11-13T10:55:30.911163+0000","last_fullsized":"2023-11-13T10:55:30.911163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:47.501708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906481+0000","last_change":"2023-11-13T10:55:30.929671+0000","last_active":"2023-11-13T10:55:31.906481+0000","last_peered":"2023-11-13T10:55:31.906481+0000","last_clean":"2023-11-13T10:55:31.906481+0000","last_became_active":"2023-11-13T10:55:30.929229+0000","last_became_peered":"2023-11-13T10:55:30.929229+0000","last_unstale":"2023-11-13T10:55:31.906481+0000","last_undegraded":"2023-11-13T10:55:31.906481+0000","last_fullsized":"2023-11-13T10:55:31.906481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:46.695309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911269+0000","last_change":"2023-11-13T10:55:25.855701+0000","last_active":"2023-11-13T10:55:30.911269+0000","last_peered":"2023-11-13T10:55:30.911269+0000","last_clean":"2023-11-13T10:55:30.911269+0000","last_became_active":"2023-11-13T10:55:25.855360+0000","last_became_peered":"2023-11-13T10:55:25.855360+0000","last_unstale":"2023-11-13T10:55:30.911269+0000","last_undegraded":"2023-11-13T10:55:30.911269+0000","last_fullsized":"2023-11-13T10:55:30.911269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:52.352286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912877+0000","last_change":"2023-11-13T10:55:26.848124+0000","last_active":"2023-11-13T10:55:30.912877+0000","last_peered":"2023-11-13T10:55:30.912877+0000","last_clean":"2023-11-13T10:55:30.912877+0000","last_became_active":"2023-11-13T10:55:26.847478+0000","last_became_peered":"2023-11-13T10:55:26.847478+0000","last_unstale":"2023-11-13T10:55:30.912877+0000","last_undegraded":"2023-11-13T10:55:30.912877+0000","last_fullsized":"2023-11-13T10:55:30.912877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:06.034946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.903358+0000","last_change":"2023-11-13T10:55:30.918537+0000","last_active":"2023-11-13T10:55:31.903358+0000","last_peered":"2023-11-13T10:55:31.903358+0000","last_clean":"2023-11-13T10:55:31.903358+0000","last_became_active":"2023-11-13T10:55:30.918092+0000","last_became_peered":"2023-11-13T10:55:30.918092+0000","last_unstale":"2023-11-13T10:55:31.903358+0000","last_undegraded":"2023-11-13T10:55:31.903358+0000","last_fullsized":"2023-11-13T10:55:31.903358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:50:02.751900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898103+0000","last_change":"2023-11-13T10:55:25.846685+0000","last_active":"2023-11-13T10:55:30.898103+0000","last_peered":"2023-11-13T10:55:30.898103+0000","last_clean":"2023-11-13T10:55:30.898103+0000","last_became_active":"2023-11-13T10:55:25.846370+0000","last_became_peered":"2023-11-13T10:55:25.846370+0000","last_unstale":"2023-11-13T10:55:30.898103+0000","last_undegraded":"2023-11-13T10:55:30.898103+0000","last_fullsized":"2023-11-13T10:55:30.898103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896163+0000","last_change":"2023-11-13T10:55:26.867122+0000","last_active":"2023-11-13T10:55:30.896163+0000","last_peered":"2023-11-13T10:55:30.896163+0000","last_clean":"2023-11-13T10:55:30.896163+0000","last_became_active":"2023-11-13T10:55:26.866669+0000","last_became_peered":"2023-11-13T10:55:26.866669+0000","last_unstale":"2023-11-13T10:55:30.896163+0000","last_undegraded":"2023-11-13T10:55:30.896163+0000","last_fullsized":"2023-11-13T10:55:30.896163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:25.074197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904843+0000","last_change":"2023-11-13T10:55:30.918484+0000","last_active":"2023-11-13T10:55:31.904843+0000","last_peered":"2023-11-13T10:55:31.904843+0000","last_clean":"2023-11-13T10:55:31.904843+0000","last_became_active":"2023-11-13T10:55:30.917997+0000","last_became_peered":"2023-11-13T10:55:30.917997+0000","last_unstale":"2023-11-13T10:55:31.904843+0000","last_undegraded":"2023-11-13T10:55:31.904843+0000","last_fullsized":"2023-11-13T10:55:31.904843+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:17.817768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900603+0000","last_change":"2023-11-13T10:55:25.851246+0000","last_active":"2023-11-13T10:55:30.900603+0000","last_peered":"2023-11-13T10:55:30.900603+0000","last_clean":"2023-11-13T10:55:30.900603+0000","last_became_active":"2023-11-13T10:55:25.850865+0000","last_became_peered":"2023-11-13T10:55:25.850865+0000","last_unstale":"2023-11-13T10:55:30.900603+0000","last_undegraded":"2023-11-13T10:55:30.900603+0000","last_fullsized":"2023-11-13T10:55:30.900603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:30.550401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898058+0000","last_change":"2023-11-13T10:55:26.866532+0000","last_active":"2023-11-13T10:55:30.898058+0000","last_peered":"2023-11-13T10:55:30.898058+0000","last_clean":"2023-11-13T10:55:30.898058+0000","last_became_active":"2023-11-13T10:55:26.866199+0000","last_became_peered":"2023-11-13T10:55:26.866199+0000","last_unstale":"2023-11-13T10:55:30.898058+0000","last_undegraded":"2023-11-13T10:55:30.898058+0000","last_fullsized":"2023-11-13T10:55:30.898058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:18.260379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905023+0000","last_change":"2023-11-13T10:55:30.919396+0000","last_active":"2023-11-13T10:55:31.905023+0000","last_peered":"2023-11-13T10:55:31.905023+0000","last_clean":"2023-11-13T10:55:31.905023+0000","last_became_active":"2023-11-13T10:55:30.918942+0000","last_became_peered":"2023-11-13T10:55:30.918942+0000","last_unstale":"2023-11-13T10:55:31.905023+0000","last_undegraded":"2023-11-13T10:55:31.905023+0000","last_fullsized":"2023-11-13T10:55:31.905023+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:51.562560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898635+0000","last_change":"2023-11-13T10:55:26.867788+0000","last_active":"2023-11-13T10:55:30.898635+0000","last_peered":"2023-11-13T10:55:30.898635+0000","last_clean":"2023-11-13T10:55:30.898635+0000","last_became_active":"2023-11-13T10:55:26.865720+0000","last_became_peered":"2023-11-13T10:55:26.865720+0000","last_unstale":"2023-11-13T10:55:30.898635+0000","last_undegraded":"2023-11-13T10:55:30.898635+0000","last_fullsized":"2023-11-13T10:55:30.898635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:04:53.206295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899232+0000","last_change":"2023-11-13T10:55:25.848578+0000","last_active":"2023-11-13T10:55:30.899232+0000","last_peered":"2023-11-13T10:55:30.899232+0000","last_clean":"2023-11-13T10:55:30.899232+0000","last_became_active":"2023-11-13T10:55:25.848218+0000","last_became_peered":"2023-11-13T10:55:25.848218+0000","last_unstale":"2023-11-13T10:55:30.899232+0000","last_undegraded":"2023-11-13T10:55:30.899232+0000","last_fullsized":"2023-11-13T10:55:30.899232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.882851+0000","last_change":"2023-11-13T10:55:30.927208+0000","last_active":"2023-11-13T10:55:31.882851+0000","last_peered":"2023-11-13T10:55:31.882851+0000","last_clean":"2023-11-13T10:55:31.882851+0000","last_became_active":"2023-11-13T10:55:30.926587+0000","last_became_peered":"2023-11-13T10:55:30.926587+0000","last_unstale":"2023-11-13T10:55:31.882851+0000","last_undegraded":"2023-11-13T10:55:31.882851+0000","last_fullsized":"2023-11-13T10:55:31.882851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:58:30.075199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911307+0000","last_change":"2023-11-13T10:55:26.863647+0000","last_active":"2023-11-13T10:55:30.911307+0000","last_peered":"2023-11-13T10:55:30.911307+0000","last_clean":"2023-11-13T10:55:30.911307+0000","last_became_active":"2023-11-13T10:55:26.863171+0000","last_became_peered":"2023-11-13T10:55:26.863171+0000","last_unstale":"2023-11-13T10:55:30.911307+0000","last_undegraded":"2023-11-13T10:55:30.911307+0000","last_fullsized":"2023-11-13T10:55:30.911307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:44:59.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913133+0000","last_change":"2023-11-13T10:55:25.861244+0000","last_active":"2023-11-13T10:55:30.913133+0000","last_peered":"2023-11-13T10:55:30.913133+0000","last_clean":"2023-11-13T10:55:30.913133+0000","last_became_active":"2023-11-13T10:55:25.860949+0000","last_became_peered":"2023-11-13T10:55:25.860949+0000","last_unstale":"2023-11-13T10:55:30.913133+0000","last_undegraded":"2023-11-13T10:55:30.913133+0000","last_fullsized":"2023-11-13T10:55:30.913133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:05.431553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.902438+0000","last_change":"2023-11-13T10:55:30.922731+0000","last_active":"2023-11-13T10:55:31.902438+0000","last_peered":"2023-11-13T10:55:31.902438+0000","last_clean":"2023-11-13T10:55:31.902438+0000","last_became_active":"2023-11-13T10:55:30.922328+0000","last_became_peered":"2023-11-13T10:55:30.922328+0000","last_unstale":"2023-11-13T10:55:31.902438+0000","last_undegraded":"2023-11-13T10:55:31.902438+0000","last_fullsized":"2023-11-13T10:55:31.902438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:57:17.670747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897411+0000","last_change":"2023-11-13T10:55:26.857718+0000","last_active":"2023-11-13T10:55:30.897411+0000","last_peered":"2023-11-13T10:55:30.897411+0000","last_clean":"2023-11-13T10:55:30.897411+0000","last_became_active":"2023-11-13T10:55:26.852435+0000","last_became_peered":"2023-11-13T10:55:26.852435+0000","last_unstale":"2023-11-13T10:55:30.897411+0000","last_undegraded":"2023-11-13T10:55:30.897411+0000","last_fullsized":"2023-11-13T10:55:30.897411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:45.182794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892276+0000","last_change":"2023-11-13T10:55:25.858975+0000","last_active":"2023-11-13T10:55:30.892276+0000","last_peered":"2023-11-13T10:55:30.892276+0000","last_clean":"2023-11-13T10:55:30.892276+0000","last_became_active":"2023-11-13T10:55:25.858642+0000","last_became_peered":"2023-11-13T10:55:25.858642+0000","last_unstale":"2023-11-13T10:55:30.892276+0000","last_undegraded":"2023-11-13T10:55:30.892276+0000","last_fullsized":"2023-11-13T10:55:30.892276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.908730+0000","last_change":"2023-11-13T10:55:30.931892+0000","last_active":"2023-11-13T10:55:31.908730+0000","last_peered":"2023-11-13T10:55:31.908730+0000","last_clean":"2023-11-13T10:55:31.908730+0000","last_became_active":"2023-11-13T10:55:30.931357+0000","last_became_peered":"2023-11-13T10:55:30.931357+0000","last_unstale":"2023-11-13T10:55:31.908730+0000","last_undegraded":"2023-11-13T10:55:31.908730+0000","last_fullsized":"2023-11-13T10:55:31.908730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:07:10.060960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897798+0000","last_change":"2023-11-13T10:55:26.855031+0000","last_active":"2023-11-13T10:55:30.897798+0000","last_peered":"2023-11-13T10:55:30.897798+0000","last_clean":"2023-11-13T10:55:30.897798+0000","last_became_active":"2023-11-13T10:55:26.854761+0000","last_became_peered":"2023-11-13T10:55:26.854761+0000","last_unstale":"2023-11-13T10:55:30.897798+0000","last_undegraded":"2023-11-13T10:55:30.897798+0000","last_fullsized":"2023-11-13T10:55:30.897798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:27.863825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898689+0000","last_change":"2023-11-13T10:55:25.848611+0000","last_active":"2023-11-13T10:55:30.898689+0000","last_peered":"2023-11-13T10:55:30.898689+0000","last_clean":"2023-11-13T10:55:30.898689+0000","last_became_active":"2023-11-13T10:55:25.848277+0000","last_became_peered":"2023-11-13T10:55:25.848277+0000","last_unstale":"2023-11-13T10:55:30.898689+0000","last_undegraded":"2023-11-13T10:55:30.898689+0000","last_fullsized":"2023-11-13T10:55:30.898689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905715+0000","last_change":"2023-11-13T10:55:30.920124+0000","last_active":"2023-11-13T10:55:31.905715+0000","last_peered":"2023-11-13T10:55:31.905715+0000","last_clean":"2023-11-13T10:55:31.905715+0000","last_became_active":"2023-11-13T10:55:30.919731+0000","last_became_peered":"2023-11-13T10:55:30.919731+0000","last_unstale":"2023-11-13T10:55:31.905715+0000","last_undegraded":"2023-11-13T10:55:31.905715+0000","last_fullsized":"2023-11-13T10:55:31.905715+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899898+0000","last_change":"2023-11-13T10:55:26.860831+0000","last_active":"2023-11-13T10:55:30.899898+0000","last_peered":"2023-11-13T10:55:30.899898+0000","last_clean":"2023-11-13T10:55:30.899898+0000","last_became_active":"2023-11-13T10:55:26.860527+0000","last_became_peered":"2023-11-13T10:55:26.860527+0000","last_unstale":"2023-11-13T10:55:30.899898+0000","last_undegraded":"2023-11-13T10:55:30.899898+0000","last_fullsized":"2023-11-13T10:55:30.899898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.774455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899998+0000","last_change":"2023-11-13T10:55:25.846361+0000","last_active":"2023-11-13T10:55:30.899998+0000","last_peered":"2023-11-13T10:55:30.899998+0000","last_clean":"2023-11-13T10:55:30.899998+0000","last_became_active":"2023-11-13T10:55:25.846031+0000","last_became_peered":"2023-11-13T10:55:25.846031+0000","last_unstale":"2023-11-13T10:55:30.899998+0000","last_undegraded":"2023-11-13T10:55:30.899998+0000","last_fullsized":"2023-11-13T10:55:30.899998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:44.868236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904780+0000","last_change":"2023-11-13T10:55:30.910764+0000","last_active":"2023-11-13T10:55:31.904780+0000","last_peered":"2023-11-13T10:55:31.904780+0000","last_clean":"2023-11-13T10:55:31.904780+0000","last_became_active":"2023-11-13T10:55:30.910363+0000","last_became_peered":"2023-11-13T10:55:30.910363+0000","last_unstale":"2023-11-13T10:55:31.904780+0000","last_undegraded":"2023-11-13T10:55:31.904780+0000","last_fullsized":"2023-11-13T10:55:31.904780+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:33.673828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.893835+0000","last_change":"2023-11-13T10:55:26.859968+0000","last_active":"2023-11-13T10:55:30.893835+0000","last_peered":"2023-11-13T10:55:30.893835+0000","last_clean":"2023-11-13T10:55:30.893835+0000","last_became_active":"2023-11-13T10:55:26.859344+0000","last_became_peered":"2023-11-13T10:55:26.859344+0000","last_unstale":"2023-11-13T10:55:30.893835+0000","last_undegraded":"2023-11-13T10:55:30.893835+0000","last_fullsized":"2023-11-13T10:55:30.893835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.107404+0000","last_change":"2023-11-13T10:55:25.857948+0000","last_active":"2023-11-13T10:55:31.107404+0000","last_peered":"2023-11-13T10:55:31.107404+0000","last_clean":"2023-11-13T10:55:31.107404+0000","last_became_active":"2023-11-13T10:55:25.857670+0000","last_became_peered":"2023-11-13T10:55:25.857670+0000","last_unstale":"2023-11-13T10:55:31.107404+0000","last_undegraded":"2023-11-13T10:55:31.107404+0000","last_fullsized":"2023-11-13T10:55:31.107404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:11.016234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883636+0000","last_change":"2023-11-13T10:55:30.918361+0000","last_active":"2023-11-13T10:55:31.883636+0000","last_peered":"2023-11-13T10:55:31.883636+0000","last_clean":"2023-11-13T10:55:31.883636+0000","last_became_active":"2023-11-13T10:55:30.912409+0000","last_became_peered":"2023-11-13T10:55:30.912409+0000","last_unstale":"2023-11-13T10:55:31.883636+0000","last_undegraded":"2023-11-13T10:55:31.883636+0000","last_fullsized":"2023-11-13T10:55:31.883636+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:05.903305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913447+0000","last_change":"2023-11-13T10:55:26.864360+0000","last_active":"2023-11-13T10:55:30.913447+0000","last_peered":"2023-11-13T10:55:30.913447+0000","last_clean":"2023-11-13T10:55:30.913447+0000","last_became_active":"2023-11-13T10:55:26.863969+0000","last_became_peered":"2023-11-13T10:55:26.863969+0000","last_unstale":"2023-11-13T10:55:30.913447+0000","last_undegraded":"2023-11-13T10:55:30.913447+0000","last_fullsized":"2023-11-13T10:55:30.913447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:45.182794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898332+0000","last_change":"2023-11-13T10:55:25.850034+0000","last_active":"2023-11-13T10:55:30.898332+0000","last_peered":"2023-11-13T10:55:30.898332+0000","last_clean":"2023-11-13T10:55:30.898332+0000","last_became_active":"2023-11-13T10:55:25.849670+0000","last_became_peered":"2023-11-13T10:55:25.849670+0000","last_unstale":"2023-11-13T10:55:30.898332+0000","last_undegraded":"2023-11-13T10:55:30.898332+0000","last_fullsized":"2023-11-13T10:55:30.898332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:42.787670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905007+0000","last_change":"2023-11-13T10:55:30.921622+0000","last_active":"2023-11-13T10:55:31.905007+0000","last_peered":"2023-11-13T10:55:31.905007+0000","last_clean":"2023-11-13T10:55:31.905007+0000","last_became_active":"2023-11-13T10:55:30.921092+0000","last_became_peered":"2023-11-13T10:55:30.921092+0000","last_unstale":"2023-11-13T10:55:31.905007+0000","last_undegraded":"2023-11-13T10:55:31.905007+0000","last_fullsized":"2023-11-13T10:55:31.905007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:49.941362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886149+0000","last_change":"2023-11-13T10:55:26.858625+0000","last_active":"2023-11-13T10:55:30.886149+0000","last_peered":"2023-11-13T10:55:30.886149+0000","last_clean":"2023-11-13T10:55:30.886149+0000","last_became_active":"2023-11-13T10:55:26.858318+0000","last_became_peered":"2023-11-13T10:55:26.858318+0000","last_unstale":"2023-11-13T10:55:30.886149+0000","last_undegraded":"2023-11-13T10:55:30.886149+0000","last_fullsized":"2023-11-13T10:55:30.886149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:43.105401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909064+0000","last_change":"2023-11-13T10:55:25.862368+0000","last_active":"2023-11-13T10:55:30.909064+0000","last_peered":"2023-11-13T10:55:30.909064+0000","last_clean":"2023-11-13T10:55:30.909064+0000","last_became_active":"2023-11-13T10:55:25.862060+0000","last_became_peered":"2023-11-13T10:55:25.862060+0000","last_unstale":"2023-11-13T10:55:30.909064+0000","last_undegraded":"2023-11-13T10:55:30.909064+0000","last_fullsized":"2023-11-13T10:55:30.909064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:09.162069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904580+0000","last_change":"2023-11-13T10:55:30.923556+0000","last_active":"2023-11-13T10:55:31.904580+0000","last_peered":"2023-11-13T10:55:31.904580+0000","last_clean":"2023-11-13T10:55:31.904580+0000","last_became_active":"2023-11-13T10:55:30.923070+0000","last_became_peered":"2023-11-13T10:55:30.923070+0000","last_unstale":"2023-11-13T10:55:31.904580+0000","last_undegraded":"2023-11-13T10:55:31.904580+0000","last_fullsized":"2023-11-13T10:55:31.904580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:51.562560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902612+0000","last_change":"2023-11-13T10:55:26.864760+0000","last_active":"2023-11-13T10:55:30.902612+0000","last_peered":"2023-11-13T10:55:30.902612+0000","last_clean":"2023-11-13T10:55:30.902612+0000","last_became_active":"2023-11-13T10:55:26.864419+0000","last_became_peered":"2023-11-13T10:55:26.864419+0000","last_unstale":"2023-11-13T10:55:30.902612+0000","last_undegraded":"2023-11-13T10:55:30.902612+0000","last_fullsized":"2023-11-13T10:55:30.902612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:01.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898908+0000","last_change":"2023-11-13T10:55:25.856907+0000","last_active":"2023-11-13T10:55:30.898908+0000","last_peered":"2023-11-13T10:55:30.898908+0000","last_clean":"2023-11-13T10:55:30.898908+0000","last_became_active":"2023-11-13T10:55:25.856545+0000","last_became_peered":"2023-11-13T10:55:25.856545+0000","last_unstale":"2023-11-13T10:55:30.898908+0000","last_undegraded":"2023-11-13T10:55:30.898908+0000","last_fullsized":"2023-11-13T10:55:30.898908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:28.600702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906072+0000","last_change":"2023-11-13T10:55:30.921637+0000","last_active":"2023-11-13T10:55:31.906072+0000","last_peered":"2023-11-13T10:55:31.906072+0000","last_clean":"2023-11-13T10:55:31.906072+0000","last_became_active":"2023-11-13T10:55:30.921177+0000","last_became_peered":"2023-11-13T10:55:30.921177+0000","last_unstale":"2023-11-13T10:55:31.906072+0000","last_undegraded":"2023-11-13T10:55:31.906072+0000","last_fullsized":"2023-11-13T10:55:31.906072+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903258+0000","last_change":"2023-11-13T10:55:26.851557+0000","last_active":"2023-11-13T10:55:30.903258+0000","last_peered":"2023-11-13T10:55:30.903258+0000","last_clean":"2023-11-13T10:55:30.903258+0000","last_became_active":"2023-11-13T10:55:26.850792+0000","last_became_peered":"2023-11-13T10:55:26.850792+0000","last_unstale":"2023-11-13T10:55:30.903258+0000","last_undegraded":"2023-11-13T10:55:30.903258+0000","last_fullsized":"2023-11-13T10:55:30.903258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911327+0000","last_change":"2023-11-13T10:55:25.855880+0000","last_active":"2023-11-13T10:55:30.911327+0000","last_peered":"2023-11-13T10:55:30.911327+0000","last_clean":"2023-11-13T10:55:30.911327+0000","last_became_active":"2023-11-13T10:55:25.855543+0000","last_became_peered":"2023-11-13T10:55:25.855543+0000","last_unstale":"2023-11-13T10:55:30.911327+0000","last_undegraded":"2023-11-13T10:55:30.911327+0000","last_fullsized":"2023-11-13T10:55:30.911327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:02:28.405745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.907032+0000","last_change":"2023-11-13T10:55:30.925168+0000","last_active":"2023-11-13T10:55:31.907032+0000","last_peered":"2023-11-13T10:55:31.907032+0000","last_clean":"2023-11-13T10:55:31.907032+0000","last_became_active":"2023-11-13T10:55:30.924675+0000","last_became_peered":"2023-11-13T10:55:30.924675+0000","last_unstale":"2023-11-13T10:55:31.907032+0000","last_undegraded":"2023-11-13T10:55:31.907032+0000","last_fullsized":"2023-11-13T10:55:31.907032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:25.376399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911697+0000","last_change":"2023-11-13T10:55:26.863984+0000","last_active":"2023-11-13T10:55:30.911697+0000","last_peered":"2023-11-13T10:55:30.911697+0000","last_clean":"2023-11-13T10:55:30.911697+0000","last_became_active":"2023-11-13T10:55:26.862398+0000","last_became_peered":"2023-11-13T10:55:26.862398+0000","last_unstale":"2023-11-13T10:55:30.911697+0000","last_undegraded":"2023-11-13T10:55:30.911697+0000","last_fullsized":"2023-11-13T10:55:30.911697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898919+0000","last_change":"2023-11-13T10:55:25.857004+0000","last_active":"2023-11-13T10:55:30.898919+0000","last_peered":"2023-11-13T10:55:30.898919+0000","last_clean":"2023-11-13T10:55:30.898919+0000","last_became_active":"2023-11-13T10:55:25.856718+0000","last_became_peered":"2023-11-13T10:55:25.856718+0000","last_unstale":"2023-11-13T10:55:30.898919+0000","last_undegraded":"2023-11-13T10:55:30.898919+0000","last_fullsized":"2023-11-13T10:55:30.898919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:28.167168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888398+0000","last_change":"2023-11-13T10:55:30.923781+0000","last_active":"2023-11-13T10:55:31.888398+0000","last_peered":"2023-11-13T10:55:31.888398+0000","last_clean":"2023-11-13T10:55:31.888398+0000","last_became_active":"2023-11-13T10:55:30.922714+0000","last_became_peered":"2023-11-13T10:55:30.922714+0000","last_unstale":"2023-11-13T10:55:31.888398+0000","last_undegraded":"2023-11-13T10:55:31.888398+0000","last_fullsized":"2023-11-13T10:55:31.888398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:05:31.401669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899007+0000","last_change":"2023-11-13T10:55:26.862991+0000","last_active":"2023-11-13T10:55:30.899007+0000","last_peered":"2023-11-13T10:55:30.899007+0000","last_clean":"2023-11-13T10:55:30.899007+0000","last_became_active":"2023-11-13T10:55:26.861100+0000","last_became_peered":"2023-11-13T10:55:26.861100+0000","last_unstale":"2023-11-13T10:55:30.899007+0000","last_undegraded":"2023-11-13T10:55:30.899007+0000","last_fullsized":"2023-11-13T10:55:30.899007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899324+0000","last_change":"2023-11-13T10:55:25.848422+0000","last_active":"2023-11-13T10:55:30.899324+0000","last_peered":"2023-11-13T10:55:30.899324+0000","last_clean":"2023-11-13T10:55:30.899324+0000","last_became_active":"2023-11-13T10:55:25.847809+0000","last_became_peered":"2023-11-13T10:55:25.847809+0000","last_unstale":"2023-11-13T10:55:30.899324+0000","last_undegraded":"2023-11-13T10:55:30.899324+0000","last_fullsized":"2023-11-13T10:55:30.899324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892816+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T10:55:30.892816+0000","last_peered":"2023-11-13T10:55:30.892816+0000","last_clean":"2023-11-13T10:55:30.892816+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T10:55:30.892816+0000","last_undegraded":"2023-11-13T10:55:30.892816+0000","last_fullsized":"2023-11-13T10:55:30.892816+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883757+0000","last_change":"2023-11-13T10:55:30.907496+0000","last_active":"2023-11-13T10:55:31.883757+0000","last_peered":"2023-11-13T10:55:31.883757+0000","last_clean":"2023-11-13T10:55:31.883757+0000","last_became_active":"2023-11-13T10:55:30.905911+0000","last_became_peered":"2023-11-13T10:55:30.905911+0000","last_unstale":"2023-11-13T10:55:31.883757+0000","last_undegraded":"2023-11-13T10:55:31.883757+0000","last_fullsized":"2023-11-13T10:55:31.883757+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:26:30.660465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898140+0000","last_change":"2023-11-13T10:55:26.865638+0000","last_active":"2023-11-13T10:55:30.898140+0000","last_peered":"2023-11-13T10:55:30.898140+0000","last_clean":"2023-11-13T10:55:30.898140+0000","last_became_active":"2023-11-13T10:55:26.864005+0000","last_became_peered":"2023-11-13T10:55:26.864005+0000","last_unstale":"2023-11-13T10:55:30.898140+0000","last_undegraded":"2023-11-13T10:55:30.898140+0000","last_fullsized":"2023-11-13T10:55:30.898140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:18:03.109671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.914054+0000","last_change":"2023-11-13T10:55:25.860502+0000","last_active":"2023-11-13T10:55:30.914054+0000","last_peered":"2023-11-13T10:55:30.914054+0000","last_clean":"2023-11-13T10:55:30.914054+0000","last_became_active":"2023-11-13T10:55:25.860157+0000","last_became_peered":"2023-11-13T10:55:25.860157+0000","last_unstale":"2023-11-13T10:55:30.914054+0000","last_undegraded":"2023-11-13T10:55:30.914054+0000","last_fullsized":"2023-11-13T10:55:30.914054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:42.093127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905171+0000","last_change":"2023-11-13T10:55:30.909998+0000","last_active":"2023-11-13T10:55:31.905171+0000","last_peered":"2023-11-13T10:55:31.905171+0000","last_clean":"2023-11-13T10:55:31.905171+0000","last_became_active":"2023-11-13T10:55:30.909623+0000","last_became_peered":"2023-11-13T10:55:30.909623+0000","last_unstale":"2023-11-13T10:55:31.905171+0000","last_undegraded":"2023-11-13T10:55:31.905171+0000","last_fullsized":"2023-11-13T10:55:31.905171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:47.860796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900512+0000","last_change":"2023-11-13T10:55:26.867419+0000","last_active":"2023-11-13T10:55:30.900512+0000","last_peered":"2023-11-13T10:55:30.900512+0000","last_clean":"2023-11-13T10:55:30.900512+0000","last_became_active":"2023-11-13T10:55:26.864802+0000","last_became_peered":"2023-11-13T10:55:26.864802+0000","last_unstale":"2023-11-13T10:55:30.900512+0000","last_undegraded":"2023-11-13T10:55:30.900512+0000","last_fullsized":"2023-11-13T10:55:30.900512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909289+0000","last_change":"2023-11-13T10:55:25.852321+0000","last_active":"2023-11-13T10:55:30.909289+0000","last_peered":"2023-11-13T10:55:30.909289+0000","last_clean":"2023-11-13T10:55:30.909289+0000","last_became_active":"2023-11-13T10:55:25.850605+0000","last_became_peered":"2023-11-13T10:55:25.850605+0000","last_unstale":"2023-11-13T10:55:30.909289+0000","last_undegraded":"2023-11-13T10:55:30.909289+0000","last_fullsized":"2023-11-13T10:55:30.909289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.887690+0000","last_change":"2023-11-13T10:55:30.913188+0000","last_active":"2023-11-13T10:55:31.887690+0000","last_peered":"2023-11-13T10:55:31.887690+0000","last_clean":"2023-11-13T10:55:31.887690+0000","last_became_active":"2023-11-13T10:55:30.912748+0000","last_became_peered":"2023-11-13T10:55:30.912748+0000","last_unstale":"2023-11-13T10:55:31.887690+0000","last_undegraded":"2023-11-13T10:55:31.887690+0000","last_fullsized":"2023-11-13T10:55:31.887690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:20.801658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902130+0000","last_change":"2023-11-13T10:55:26.867097+0000","last_active":"2023-11-13T10:55:30.902130+0000","last_peered":"2023-11-13T10:55:30.902130+0000","last_clean":"2023-11-13T10:55:30.902130+0000","last_became_active":"2023-11-13T10:55:26.866770+0000","last_became_peered":"2023-11-13T10:55:26.866770+0000","last_unstale":"2023-11-13T10:55:30.902130+0000","last_undegraded":"2023-11-13T10:55:30.902130+0000","last_fullsized":"2023-11-13T10:55:30.902130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:06.034946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900035+0000","last_change":"2023-11-13T10:55:25.855464+0000","last_active":"2023-11-13T10:55:30.900035+0000","last_peered":"2023-11-13T10:55:30.900035+0000","last_clean":"2023-11-13T10:55:30.900035+0000","last_became_active":"2023-11-13T10:55:25.853955+0000","last_became_peered":"2023-11-13T10:55:25.853955+0000","last_unstale":"2023-11-13T10:55:30.900035+0000","last_undegraded":"2023-11-13T10:55:30.900035+0000","last_fullsized":"2023-11-13T10:55:30.900035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.881810+0000","last_change":"2023-11-13T10:55:30.915984+0000","last_active":"2023-11-13T10:55:31.881810+0000","last_peered":"2023-11-13T10:55:31.881810+0000","last_clean":"2023-11-13T10:55:31.881810+0000","last_became_active":"2023-11-13T10:55:30.915438+0000","last_became_peered":"2023-11-13T10:55:30.915438+0000","last_unstale":"2023-11-13T10:55:31.881810+0000","last_undegraded":"2023-11-13T10:55:31.881810+0000","last_fullsized":"2023-11-13T10:55:31.881810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:06:39.428736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887086+0000","last_change":"2023-11-13T10:55:26.874069+0000","last_active":"2023-11-13T10:55:30.887086+0000","last_peered":"2023-11-13T10:55:30.887086+0000","last_clean":"2023-11-13T10:55:30.887086+0000","last_became_active":"2023-11-13T10:55:26.873537+0000","last_became_peered":"2023-11-13T10:55:26.873537+0000","last_unstale":"2023-11-13T10:55:30.887086+0000","last_undegraded":"2023-11-13T10:55:30.887086+0000","last_fullsized":"2023-11-13T10:55:30.887086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:22.528830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902191+0000","last_change":"2023-11-13T10:55:25.853256+0000","last_active":"2023-11-13T10:55:30.902191+0000","last_peered":"2023-11-13T10:55:30.902191+0000","last_clean":"2023-11-13T10:55:30.902191+0000","last_became_active":"2023-11-13T10:55:25.852947+0000","last_became_peered":"2023-11-13T10:55:25.852947+0000","last_unstale":"2023-11-13T10:55:30.902191+0000","last_undegraded":"2023-11-13T10:55:30.902191+0000","last_fullsized":"2023-11-13T10:55:30.902191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:15.762181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904654+0000","last_change":"2023-11-13T10:55:30.912617+0000","last_active":"2023-11-13T10:55:31.904654+0000","last_peered":"2023-11-13T10:55:31.904654+0000","last_clean":"2023-11-13T10:55:31.904654+0000","last_became_active":"2023-11-13T10:55:30.912019+0000","last_became_peered":"2023-11-13T10:55:30.912019+0000","last_unstale":"2023-11-13T10:55:31.904654+0000","last_undegraded":"2023-11-13T10:55:31.904654+0000","last_fullsized":"2023-11-13T10:55:31.904654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:16.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897711+0000","last_change":"2023-11-13T10:55:26.868126+0000","last_active":"2023-11-13T10:55:30.897711+0000","last_peered":"2023-11-13T10:55:30.897711+0000","last_clean":"2023-11-13T10:55:30.897711+0000","last_became_active":"2023-11-13T10:55:26.867810+0000","last_became_peered":"2023-11-13T10:55:26.867810+0000","last_unstale":"2023-11-13T10:55:30.897711+0000","last_undegraded":"2023-11-13T10:55:30.897711+0000","last_fullsized":"2023-11-13T10:55:30.897711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:16.740806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897637+0000","last_change":"2023-11-13T10:55:25.851131+0000","last_active":"2023-11-13T10:55:30.897637+0000","last_peered":"2023-11-13T10:55:30.897637+0000","last_clean":"2023-11-13T10:55:30.897637+0000","last_became_active":"2023-11-13T10:55:25.850827+0000","last_became_peered":"2023-11-13T10:55:25.850827+0000","last_unstale":"2023-11-13T10:55:30.897637+0000","last_undegraded":"2023-11-13T10:55:30.897637+0000","last_fullsized":"2023-11-13T10:55:30.897637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:05.431553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888952+0000","last_change":"2023-11-13T10:55:30.924096+0000","last_active":"2023-11-13T10:55:31.888952+0000","last_peered":"2023-11-13T10:55:31.888952+0000","last_clean":"2023-11-13T10:55:31.888952+0000","last_became_active":"2023-11-13T10:55:30.923375+0000","last_became_peered":"2023-11-13T10:55:30.923375+0000","last_unstale":"2023-11-13T10:55:31.888952+0000","last_undegraded":"2023-11-13T10:55:31.888952+0000","last_fullsized":"2023-11-13T10:55:31.888952+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:18:07.170524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892700+0000","last_change":"2023-11-13T10:55:26.852339+0000","last_active":"2023-11-13T10:55:30.892700+0000","last_peered":"2023-11-13T10:55:30.892700+0000","last_clean":"2023-11-13T10:55:30.892700+0000","last_became_active":"2023-11-13T10:55:26.852036+0000","last_became_peered":"2023-11-13T10:55:26.852036+0000","last_unstale":"2023-11-13T10:55:30.892700+0000","last_undegraded":"2023-11-13T10:55:30.892700+0000","last_fullsized":"2023-11-13T10:55:30.892700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:41:12.172895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898608+0000","last_change":"2023-11-13T10:55:25.854134+0000","last_active":"2023-11-13T10:55:30.898608+0000","last_peered":"2023-11-13T10:55:30.898608+0000","last_clean":"2023-11-13T10:55:30.898608+0000","last_became_active":"2023-11-13T10:55:25.853195+0000","last_became_peered":"2023-11-13T10:55:25.853195+0000","last_unstale":"2023-11-13T10:55:30.898608+0000","last_undegraded":"2023-11-13T10:55:30.898608+0000","last_fullsized":"2023-11-13T10:55:30.898608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:09.162069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888654+0000","last_change":"2023-11-13T10:55:30.914433+0000","last_active":"2023-11-13T10:55:31.888654+0000","last_peered":"2023-11-13T10:55:31.888654+0000","last_clean":"2023-11-13T10:55:31.888654+0000","last_became_active":"2023-11-13T10:55:30.913947+0000","last_became_peered":"2023-11-13T10:55:30.913947+0000","last_unstale":"2023-11-13T10:55:31.888654+0000","last_undegraded":"2023-11-13T10:55:31.888654+0000","last_fullsized":"2023-11-13T10:55:31.888654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:22.321232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899143+0000","last_change":"2023-11-13T10:55:26.866642+0000","last_active":"2023-11-13T10:55:30.899143+0000","last_peered":"2023-11-13T10:55:30.899143+0000","last_clean":"2023-11-13T10:55:30.899143+0000","last_became_active":"2023-11-13T10:55:26.866294+0000","last_became_peered":"2023-11-13T10:55:26.866294+0000","last_unstale":"2023-11-13T10:55:30.899143+0000","last_undegraded":"2023-11-13T10:55:30.899143+0000","last_fullsized":"2023-11-13T10:55:30.899143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:01.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898826+0000","last_change":"2023-11-13T10:55:25.848632+0000","last_active":"2023-11-13T10:55:30.898826+0000","last_peered":"2023-11-13T10:55:30.898826+0000","last_clean":"2023-11-13T10:55:30.898826+0000","last_became_active":"2023-11-13T10:55:25.848324+0000","last_became_peered":"2023-11-13T10:55:25.848324+0000","last_unstale":"2023-11-13T10:55:30.898826+0000","last_undegraded":"2023-11-13T10:55:30.898826+0000","last_fullsized":"2023-11-13T10:55:30.898826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:30.550401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905683+0000","last_change":"2023-11-13T10:55:30.931468+0000","last_active":"2023-11-13T10:55:31.905683+0000","last_peered":"2023-11-13T10:55:31.905683+0000","last_clean":"2023-11-13T10:55:31.905683+0000","last_became_active":"2023-11-13T10:55:30.930565+0000","last_became_peered":"2023-11-13T10:55:30.930565+0000","last_unstale":"2023-11-13T10:55:31.905683+0000","last_undegraded":"2023-11-13T10:55:31.905683+0000","last_fullsized":"2023-11-13T10:55:31.905683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:47.860796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912956+0000","last_change":"2023-11-13T10:55:26.866047+0000","last_active":"2023-11-13T10:55:30.912956+0000","last_peered":"2023-11-13T10:55:30.912956+0000","last_clean":"2023-11-13T10:55:30.912956+0000","last_became_active":"2023-11-13T10:55:26.865641+0000","last_became_peered":"2023-11-13T10:55:26.865641+0000","last_unstale":"2023-11-13T10:55:30.912956+0000","last_undegraded":"2023-11-13T10:55:30.912956+0000","last_fullsized":"2023-11-13T10:55:30.912956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:12.028508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911415+0000","last_change":"2023-11-13T10:55:25.860552+0000","last_active":"2023-11-13T10:55:30.911415+0000","last_peered":"2023-11-13T10:55:30.911415+0000","last_clean":"2023-11-13T10:55:30.911415+0000","last_became_active":"2023-11-13T10:55:25.859833+0000","last_became_peered":"2023-11-13T10:55:25.859833+0000","last_unstale":"2023-11-13T10:55:30.911415+0000","last_undegraded":"2023-11-13T10:55:30.911415+0000","last_fullsized":"2023-11-13T10:55:30.911415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:03:57.732657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883273+0000","last_change":"2023-11-13T10:55:30.926972+0000","last_active":"2023-11-13T10:55:31.883273+0000","last_peered":"2023-11-13T10:55:31.883273+0000","last_clean":"2023-11-13T10:55:31.883273+0000","last_became_active":"2023-11-13T10:55:30.926026+0000","last_became_peered":"2023-11-13T10:55:30.926026+0000","last_unstale":"2023-11-13T10:55:31.883273+0000","last_undegraded":"2023-11-13T10:55:31.883273+0000","last_fullsized":"2023-11-13T10:55:31.883273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:41:10.341680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897975+0000","last_change":"2023-11-13T10:55:26.860060+0000","last_active":"2023-11-13T10:55:30.897975+0000","last_peered":"2023-11-13T10:55:30.897975+0000","last_clean":"2023-11-13T10:55:30.897975+0000","last_became_active":"2023-11-13T10:55:26.859793+0000","last_became_peered":"2023-11-13T10:55:26.859793+0000","last_unstale":"2023-11-13T10:55:30.897975+0000","last_undegraded":"2023-11-13T10:55:30.897975+0000","last_fullsized":"2023-11-13T10:55:30.897975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:13.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:45.351786+0000","last_change":"2023-11-13T10:55:25.857501+0000","last_active":"2023-11-13T10:55:45.351786+0000","last_peered":"2023-11-13T10:55:45.351786+0000","last_clean":"2023-11-13T10:55:45.351786+0000","last_became_active":"2023-11-13T10:55:25.853910+0000","last_became_peered":"2023-11-13T10:55:25.853910+0000","last_unstale":"2023-11-13T10:55:45.351786+0000","last_undegraded":"2023-11-13T10:55:45.351786+0000","last_fullsized":"2023-11-13T10:55:45.351786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:41:11.160621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883442+0000","last_change":"2023-11-13T10:55:30.921872+0000","last_active":"2023-11-13T10:55:31.883442+0000","last_peered":"2023-11-13T10:55:31.883442+0000","last_clean":"2023-11-13T10:55:31.883442+0000","last_became_active":"2023-11-13T10:55:30.921430+0000","last_became_peered":"2023-11-13T10:55:30.921430+0000","last_unstale":"2023-11-13T10:55:31.883442+0000","last_undegraded":"2023-11-13T10:55:31.883442+0000","last_fullsized":"2023-11-13T10:55:31.883442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:25.376399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899011+0000","last_change":"2023-11-13T10:55:26.870526+0000","last_active":"2023-11-13T10:55:30.899011+0000","last_peered":"2023-11-13T10:55:30.899011+0000","last_clean":"2023-11-13T10:55:30.899011+0000","last_became_active":"2023-11-13T10:55:26.870044+0000","last_became_peered":"2023-11-13T10:55:26.870044+0000","last_unstale":"2023-11-13T10:55:30.899011+0000","last_undegraded":"2023-11-13T10:55:30.899011+0000","last_fullsized":"2023-11-13T10:55:30.899011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:05:27.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.915525+0000","last_change":"2023-11-13T10:55:25.863136+0000","last_active":"2023-11-13T10:55:30.915525+0000","last_peered":"2023-11-13T10:55:30.915525+0000","last_clean":"2023-11-13T10:55:30.915525+0000","last_became_active":"2023-11-13T10:55:25.862731+0000","last_became_peered":"2023-11-13T10:55:25.862731+0000","last_unstale":"2023-11-13T10:55:30.915525+0000","last_undegraded":"2023-11-13T10:55:30.915525+0000","last_fullsized":"2023-11-13T10:55:30.915525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:50.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905039+0000","last_change":"2023-11-13T10:55:30.920930+0000","last_active":"2023-11-13T10:55:31.905039+0000","last_peered":"2023-11-13T10:55:31.905039+0000","last_clean":"2023-11-13T10:55:31.905039+0000","last_became_active":"2023-11-13T10:55:30.920498+0000","last_became_peered":"2023-11-13T10:55:30.920498+0000","last_unstale":"2023-11-13T10:55:31.905039+0000","last_undegraded":"2023-11-13T10:55:31.905039+0000","last_fullsized":"2023-11-13T10:55:31.905039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:49.941362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912962+0000","last_change":"2023-11-13T10:55:26.860086+0000","last_active":"2023-11-13T10:55:30.912962+0000","last_peered":"2023-11-13T10:55:30.912962+0000","last_clean":"2023-11-13T10:55:30.912962+0000","last_became_active":"2023-11-13T10:55:26.859816+0000","last_became_peered":"2023-11-13T10:55:26.859816+0000","last_unstale":"2023-11-13T10:55:30.912962+0000","last_undegraded":"2023-11-13T10:55:30.912962+0000","last_fullsized":"2023-11-13T10:55:30.912962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902678+0000","last_change":"2023-11-13T10:55:25.846097+0000","last_active":"2023-11-13T10:55:30.902678+0000","last_peered":"2023-11-13T10:55:30.902678+0000","last_clean":"2023-11-13T10:55:30.902678+0000","last_became_active":"2023-11-13T10:55:25.845770+0000","last_became_peered":"2023-11-13T10:55:25.845770+0000","last_unstale":"2023-11-13T10:55:30.902678+0000","last_undegraded":"2023-11-13T10:55:30.902678+0000","last_fullsized":"2023-11-13T10:55:30.902678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:16.023464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906043+0000","last_change":"2023-11-13T10:55:30.925552+0000","last_active":"2023-11-13T10:55:31.906043+0000","last_peered":"2023-11-13T10:55:31.906043+0000","last_clean":"2023-11-13T10:55:31.906043+0000","last_became_active":"2023-11-13T10:55:30.925120+0000","last_became_peered":"2023-11-13T10:55:30.925120+0000","last_unstale":"2023-11-13T10:55:31.906043+0000","last_undegraded":"2023-11-13T10:55:31.906043+0000","last_fullsized":"2023-11-13T10:55:31.906043+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:54:40.287372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886993+0000","last_change":"2023-11-13T10:55:26.876001+0000","last_active":"2023-11-13T10:55:30.886993+0000","last_peered":"2023-11-13T10:55:30.886993+0000","last_clean":"2023-11-13T10:55:30.886993+0000","last_became_active":"2023-11-13T10:55:26.875523+0000","last_became_peered":"2023-11-13T10:55:26.875523+0000","last_unstale":"2023-11-13T10:55:30.886993+0000","last_undegraded":"2023-11-13T10:55:30.886993+0000","last_fullsized":"2023-11-13T10:55:30.886993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:17.901362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896671+0000","last_change":"2023-11-13T10:55:25.848415+0000","last_active":"2023-11-13T10:55:30.896671+0000","last_peered":"2023-11-13T10:55:30.896671+0000","last_clean":"2023-11-13T10:55:30.896671+0000","last_became_active":"2023-11-13T10:55:25.848102+0000","last_became_peered":"2023-11-13T10:55:25.848102+0000","last_unstale":"2023-11-13T10:55:30.896671+0000","last_undegraded":"2023-11-13T10:55:30.896671+0000","last_fullsized":"2023-11-13T10:55:30.896671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T10:56:29.442219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883284+0000","last_change":"2023-11-13T10:55:30.918365+0000","last_active":"2023-11-13T10:55:31.883284+0000","last_peered":"2023-11-13T10:55:31.883284+0000","last_clean":"2023-11-13T10:55:31.883284+0000","last_became_active":"2023-11-13T10:55:30.917240+0000","last_became_peered":"2023-11-13T10:55:30.917240+0000","last_unstale":"2023-11-13T10:55:31.883284+0000","last_undegraded":"2023-11-13T10:55:31.883284+0000","last_fullsized":"2023-11-13T10:55:31.883284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884505+0000","last_change":"2023-11-13T10:55:30.924912+0000","last_active":"2023-11-13T10:55:31.884505+0000","last_peered":"2023-11-13T10:55:31.884505+0000","last_clean":"2023-11-13T10:55:31.884505+0000","last_became_active":"2023-11-13T10:55:30.919446+0000","last_became_peered":"2023-11-13T10:55:30.919446+0000","last_unstale":"2023-11-13T10:55:31.884505+0000","last_undegraded":"2023-11-13T10:55:31.884505+0000","last_fullsized":"2023-11-13T10:55:31.884505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:31.924677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892009+0000","last_change":"2023-11-13T10:55:26.852233+0000","last_active":"2023-11-13T10:55:30.892009+0000","last_peered":"2023-11-13T10:55:30.892009+0000","last_clean":"2023-11-13T10:55:30.892009+0000","last_became_active":"2023-11-13T10:55:26.849611+0000","last_became_peered":"2023-11-13T10:55:26.849611+0000","last_unstale":"2023-11-13T10:55:30.892009+0000","last_undegraded":"2023-11-13T10:55:30.892009+0000","last_fullsized":"2023-11-13T10:55:30.892009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:53.364560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898832+0000","last_change":"2023-11-13T10:55:25.856192+0000","last_active":"2023-11-13T10:55:30.898832+0000","last_peered":"2023-11-13T10:55:30.898832+0000","last_clean":"2023-11-13T10:55:30.898832+0000","last_became_active":"2023-11-13T10:55:25.852741+0000","last_became_peered":"2023-11-13T10:55:25.852741+0000","last_unstale":"2023-11-13T10:55:30.898832+0000","last_undegraded":"2023-11-13T10:55:30.898832+0000","last_fullsized":"2023-11-13T10:55:30.898832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:15.762181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884097+0000","last_change":"2023-11-13T10:55:30.910075+0000","last_active":"2023-11-13T10:55:31.884097+0000","last_peered":"2023-11-13T10:55:31.884097+0000","last_clean":"2023-11-13T10:55:31.884097+0000","last_became_active":"2023-11-13T10:55:30.909468+0000","last_became_peered":"2023-11-13T10:55:30.909468+0000","last_unstale":"2023-11-13T10:55:31.884097+0000","last_undegraded":"2023-11-13T10:55:31.884097+0000","last_fullsized":"2023-11-13T10:55:31.884097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:04:21.045625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903339+0000","last_change":"2023-11-13T10:55:26.856127+0000","last_active":"2023-11-13T10:55:30.903339+0000","last_peered":"2023-11-13T10:55:30.903339+0000","last_clean":"2023-11-13T10:55:30.903339+0000","last_became_active":"2023-11-13T10:55:26.855805+0000","last_became_peered":"2023-11-13T10:55:26.855805+0000","last_unstale":"2023-11-13T10:55:30.903339+0000","last_undegraded":"2023-11-13T10:55:30.903339+0000","last_fullsized":"2023-11-13T10:55:30.903339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:13.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911772+0000","last_change":"2023-11-13T10:55:25.859736+0000","last_active":"2023-11-13T10:55:30.911772+0000","last_peered":"2023-11-13T10:55:30.911772+0000","last_clean":"2023-11-13T10:55:30.911772+0000","last_became_active":"2023-11-13T10:55:25.859335+0000","last_became_peered":"2023-11-13T10:55:25.859335+0000","last_unstale":"2023-11-13T10:55:30.911772+0000","last_undegraded":"2023-11-13T10:55:30.911772+0000","last_fullsized":"2023-11-13T10:55:30.911772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:24.061923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898735+0000","last_change":"2023-11-13T10:55:25.847310+0000","last_active":"2023-11-13T10:55:30.898735+0000","last_peered":"2023-11-13T10:55:30.898735+0000","last_clean":"2023-11-13T10:55:30.898735+0000","last_became_active":"2023-11-13T10:55:25.846471+0000","last_became_peered":"2023-11-13T10:55:25.846471+0000","last_unstale":"2023-11-13T10:55:30.898735+0000","last_undegraded":"2023-11-13T10:55:30.898735+0000","last_fullsized":"2023-11-13T10:55:30.898735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:57.678774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894953+0000","last_change":"2023-11-13T10:55:26.865930+0000","last_active":"2023-11-13T10:55:30.894953+0000","last_peered":"2023-11-13T10:55:30.894953+0000","last_clean":"2023-11-13T10:55:30.894953+0000","last_became_active":"2023-11-13T10:55:26.865603+0000","last_became_peered":"2023-11-13T10:55:26.865603+0000","last_unstale":"2023-11-13T10:55:30.894953+0000","last_undegraded":"2023-11-13T10:55:30.894953+0000","last_fullsized":"2023-11-13T10:55:30.894953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:31.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884103+0000","last_change":"2023-11-13T10:55:30.922058+0000","last_active":"2023-11-13T10:55:31.884103+0000","last_peered":"2023-11-13T10:55:31.884103+0000","last_clean":"2023-11-13T10:55:31.884103+0000","last_became_active":"2023-11-13T10:55:30.921389+0000","last_became_peered":"2023-11-13T10:55:30.921389+0000","last_unstale":"2023-11-13T10:55:31.884103+0000","last_undegraded":"2023-11-13T10:55:31.884103+0000","last_fullsized":"2023-11-13T10:55:31.884103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:54.585715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909071+0000","last_change":"2023-11-13T10:55:25.859601+0000","last_active":"2023-11-13T10:55:30.909071+0000","last_peered":"2023-11-13T10:55:30.909071+0000","last_clean":"2023-11-13T10:55:30.909071+0000","last_became_active":"2023-11-13T10:55:25.859320+0000","last_became_peered":"2023-11-13T10:55:25.859320+0000","last_unstale":"2023-11-13T10:55:30.909071+0000","last_undegraded":"2023-11-13T10:55:30.909071+0000","last_fullsized":"2023-11-13T10:55:30.909071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.901962+0000","last_change":"2023-11-13T10:55:26.844138+0000","last_active":"2023-11-13T10:55:30.901962+0000","last_peered":"2023-11-13T10:55:30.901962+0000","last_clean":"2023-11-13T10:55:30.901962+0000","last_became_active":"2023-11-13T10:55:26.843850+0000","last_became_peered":"2023-11-13T10:55:26.843850+0000","last_unstale":"2023-11-13T10:55:30.901962+0000","last_undegraded":"2023-11-13T10:55:30.901962+0000","last_fullsized":"2023-11-13T10:55:30.901962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:43.105401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.907248+0000","last_change":"2023-11-13T10:55:30.929718+0000","last_active":"2023-11-13T10:55:31.907248+0000","last_peered":"2023-11-13T10:55:31.907248+0000","last_clean":"2023-11-13T10:55:31.907248+0000","last_became_active":"2023-11-13T10:55:30.929223+0000","last_became_peered":"2023-11-13T10:55:30.929223+0000","last_unstale":"2023-11-13T10:55:31.907248+0000","last_undegraded":"2023-11-13T10:55:31.907248+0000","last_fullsized":"2023-11-13T10:55:31.907248+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:28:22.239712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886249+0000","last_change":"2023-11-13T10:55:26.858768+0000","last_active":"2023-11-13T10:55:30.886249+0000","last_peered":"2023-11-13T10:55:30.886249+0000","last_clean":"2023-11-13T10:55:30.886249+0000","last_became_active":"2023-11-13T10:55:26.858493+0000","last_became_peered":"2023-11-13T10:55:26.858493+0000","last_unstale":"2023-11-13T10:55:30.886249+0000","last_undegraded":"2023-11-13T10:55:30.886249+0000","last_fullsized":"2023-11-13T10:55:30.886249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:25.074197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911186+0000","last_change":"2023-11-13T10:55:26.866587+0000","last_active":"2023-11-13T10:55:30.911186+0000","last_peered":"2023-11-13T10:55:30.911186+0000","last_clean":"2023-11-13T10:55:30.911186+0000","last_became_active":"2023-11-13T10:55:26.866277+0000","last_became_peered":"2023-11-13T10:55:26.866277+0000","last_unstale":"2023-11-13T10:55:30.911186+0000","last_undegraded":"2023-11-13T10:55:30.911186+0000","last_fullsized":"2023-11-13T10:55:30.911186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:49:42.713290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898545+0000","last_change":"2023-11-13T10:55:26.869354+0000","last_active":"2023-11-13T10:55:30.898545+0000","last_peered":"2023-11-13T10:55:30.898545+0000","last_clean":"2023-11-13T10:55:30.898545+0000","last_became_active":"2023-11-13T10:55:26.869040+0000","last_became_peered":"2023-11-13T10:55:26.869040+0000","last_unstale":"2023-11-13T10:55:30.898545+0000","last_undegraded":"2023-11-13T10:55:30.898545+0000","last_fullsized":"2023-11-13T10:55:30.898545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:50.524863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.908909+0000","last_change":"2023-11-13T10:55:26.861905+0000","last_active":"2023-11-13T10:55:30.908909+0000","last_peered":"2023-11-13T10:55:30.908909+0000","last_clean":"2023-11-13T10:55:30.908909+0000","last_became_active":"2023-11-13T10:55:26.861613+0000","last_became_peered":"2023-11-13T10:55:26.861613+0000","last_unstale":"2023-11-13T10:55:30.908909+0000","last_undegraded":"2023-11-13T10:55:30.908909+0000","last_fullsized":"2023-11-13T10:55:30.908909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:31.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913352+0000","last_change":"2023-11-13T10:55:26.873181+0000","last_active":"2023-11-13T10:55:30.913352+0000","last_peered":"2023-11-13T10:55:30.913352+0000","last_clean":"2023-11-13T10:55:30.913352+0000","last_became_active":"2023-11-13T10:55:26.872868+0000","last_became_peered":"2023-11-13T10:55:26.872868+0000","last_unstale":"2023-11-13T10:55:30.913352+0000","last_undegraded":"2023-11-13T10:55:30.913352+0000","last_fullsized":"2023-11-13T10:55:30.913352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:10.174343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.915906+0000","last_change":"2023-11-13T10:55:26.855441+0000","last_active":"2023-11-13T10:55:30.915906+0000","last_peered":"2023-11-13T10:55:30.915906+0000","last_clean":"2023-11-13T10:55:30.915906+0000","last_became_active":"2023-11-13T10:55:26.855101+0000","last_became_peered":"2023-11-13T10:55:26.855101+0000","last_unstale":"2023-11-13T10:55:30.915906+0000","last_undegraded":"2023-11-13T10:55:30.915906+0000","last_fullsized":"2023-11-13T10:55:30.915906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:53.364560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886903+0000","last_change":"2023-11-13T10:55:26.864741+0000","last_active":"2023-11-13T10:55:30.886903+0000","last_peered":"2023-11-13T10:55:30.886903+0000","last_clean":"2023-11-13T10:55:30.886903+0000","last_became_active":"2023-11-13T10:55:26.863408+0000","last_became_peered":"2023-11-13T10:55:26.863408+0000","last_unstale":"2023-11-13T10:55:30.886903+0000","last_undegraded":"2023-11-13T10:55:30.886903+0000","last_fullsized":"2023-11-13T10:55:30.886903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:12.028508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912752+0000","last_change":"2023-11-13T10:55:26.856400+0000","last_active":"2023-11-13T10:55:30.912752+0000","last_peered":"2023-11-13T10:55:30.912752+0000","last_clean":"2023-11-13T10:55:30.912752+0000","last_became_active":"2023-11-13T10:55:26.853838+0000","last_became_peered":"2023-11-13T10:55:26.853838+0000","last_unstale":"2023-11-13T10:55:30.912752+0000","last_undegraded":"2023-11-13T10:55:30.912752+0000","last_fullsized":"2023-11-13T10:55:30.912752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:45.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898736+0000","last_change":"2023-11-13T10:55:26.854049+0000","last_active":"2023-11-13T10:55:30.898736+0000","last_peered":"2023-11-13T10:55:30.898736+0000","last_clean":"2023-11-13T10:55:30.898736+0000","last_became_active":"2023-11-13T10:55:26.853767+0000","last_became_peered":"2023-11-13T10:55:26.853767+0000","last_unstale":"2023-11-13T10:55:30.898736+0000","last_undegraded":"2023-11-13T10:55:30.898736+0000","last_fullsized":"2023-11-13T10:55:30.898736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903782+0000","last_change":"2023-11-13T10:55:26.855717+0000","last_active":"2023-11-13T10:55:30.903782+0000","last_peered":"2023-11-13T10:55:30.903782+0000","last_clean":"2023-11-13T10:55:30.903782+0000","last_became_active":"2023-11-13T10:55:26.853617+0000","last_became_peered":"2023-11-13T10:55:26.853617+0000","last_unstale":"2023-11-13T10:55:30.903782+0000","last_undegraded":"2023-11-13T10:55:30.903782+0000","last_fullsized":"2023-11-13T10:55:30.903782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.895441+0000","last_change":"2023-11-13T10:55:26.857625+0000","last_active":"2023-11-13T10:55:30.895441+0000","last_peered":"2023-11-13T10:55:30.895441+0000","last_clean":"2023-11-13T10:55:30.895441+0000","last_became_active":"2023-11-13T10:55:26.857214+0000","last_became_peered":"2023-11-13T10:55:26.857214+0000","last_unstale":"2023-11-13T10:55:30.895441+0000","last_undegraded":"2023-11-13T10:55:30.895441+0000","last_fullsized":"2023-11-13T10:55:30.895441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:06.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894989+0000","last_change":"2023-11-13T10:55:26.870935+0000","last_active":"2023-11-13T10:55:30.894989+0000","last_peered":"2023-11-13T10:55:30.894989+0000","last_clean":"2023-11-13T10:55:30.894989+0000","last_became_active":"2023-11-13T10:55:26.870322+0000","last_became_peered":"2023-11-13T10:55:26.870322+0000","last_unstale":"2023-11-13T10:55:30.894989+0000","last_undegraded":"2023-11-13T10:55:30.894989+0000","last_fullsized":"2023-11-13T10:55:30.894989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:17.035738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900412+0000","last_change":"2023-11-13T10:55:26.867707+0000","last_active":"2023-11-13T10:55:30.900412+0000","last_peered":"2023-11-13T10:55:30.900412+0000","last_clean":"2023-11-13T10:55:30.900412+0000","last_became_active":"2023-11-13T10:55:26.864751+0000","last_became_peered":"2023-11-13T10:55:26.864751+0000","last_unstale":"2023-11-13T10:55:30.900412+0000","last_undegraded":"2023-11-13T10:55:30.900412+0000","last_fullsized":"2023-11-13T10:55:30.900412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:43.799944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898242+0000","last_change":"2023-11-13T10:55:26.855923+0000","last_active":"2023-11-13T10:55:30.898242+0000","last_peered":"2023-11-13T10:55:30.898242+0000","last_clean":"2023-11-13T10:55:30.898242+0000","last_became_active":"2023-11-13T10:55:26.855638+0000","last_became_peered":"2023-11-13T10:55:26.855638+0000","last_unstale":"2023-11-13T10:55:30.898242+0000","last_undegraded":"2023-11-13T10:55:30.898242+0000","last_fullsized":"2023-11-13T10:55:30.898242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:40:26.611005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.916348+0000","last_change":"2023-11-13T10:55:26.870707+0000","last_active":"2023-11-13T10:55:30.916348+0000","last_peered":"2023-11-13T10:55:30.916348+0000","last_clean":"2023-11-13T10:55:30.916348+0000","last_became_active":"2023-11-13T10:55:26.870417+0000","last_became_peered":"2023-11-13T10:55:26.870417+0000","last_unstale":"2023-11-13T10:55:30.916348+0000","last_undegraded":"2023-11-13T10:55:30.916348+0000","last_fullsized":"2023-11-13T10:55:30.916348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:24:16.048814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899902+0000","last_change":"2023-11-13T10:55:26.873122+0000","last_active":"2023-11-13T10:55:30.899902+0000","last_peered":"2023-11-13T10:55:30.899902+0000","last_clean":"2023-11-13T10:55:30.899902+0000","last_became_active":"2023-11-13T10:55:26.872823+0000","last_became_peered":"2023-11-13T10:55:26.872823+0000","last_unstale":"2023-11-13T10:55:30.899902+0000","last_undegraded":"2023-11-13T10:55:30.899902+0000","last_fullsized":"2023-11-13T10:55:30.899902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:09:58.900197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898638+0000","last_change":"2023-11-13T10:55:26.869223+0000","last_active":"2023-11-13T10:55:30.898638+0000","last_peered":"2023-11-13T10:55:30.898638+0000","last_clean":"2023-11-13T10:55:30.898638+0000","last_became_active":"2023-11-13T10:55:26.868919+0000","last_became_peered":"2023-11-13T10:55:26.868919+0000","last_unstale":"2023-11-13T10:55:30.898638+0000","last_undegraded":"2023-11-13T10:55:30.898638+0000","last_fullsized":"2023-11-13T10:55:30.898638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:57:13.609895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903525+0000","last_change":"2023-11-13T10:55:26.865259+0000","last_active":"2023-11-13T10:55:30.903525+0000","last_peered":"2023-11-13T10:55:30.903525+0000","last_clean":"2023-11-13T10:55:30.903525+0000","last_became_active":"2023-11-13T10:55:26.864675+0000","last_became_peered":"2023-11-13T10:55:26.864675+0000","last_unstale":"2023-11-13T10:55:30.903525+0000","last_undegraded":"2023-11-13T10:55:30.903525+0000","last_fullsized":"2023-11-13T10:55:30.903525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:17.035738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912854+0000","last_change":"2023-11-13T10:55:26.863002+0000","last_active":"2023-11-13T10:55:30.912854+0000","last_peered":"2023-11-13T10:55:30.912854+0000","last_clean":"2023-11-13T10:55:30.912854+0000","last_became_active":"2023-11-13T10:55:26.862257+0000","last_became_peered":"2023-11-13T10:55:26.862257+0000","last_unstale":"2023-11-13T10:55:30.912854+0000","last_undegraded":"2023-11-13T10:55:30.912854+0000","last_fullsized":"2023-11-13T10:55:30.912854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:17.901362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.914921+0000","last_change":"2023-11-13T10:55:26.872307+0000","last_active":"2023-11-13T10:55:30.914921+0000","last_peered":"2023-11-13T10:55:30.914921+0000","last_clean":"2023-11-13T10:55:30.914921+0000","last_became_active":"2023-11-13T10:55:26.872087+0000","last_became_peered":"2023-11-13T10:55:26.872087+0000","last_unstale":"2023-11-13T10:55:30.914921+0000","last_undegraded":"2023-11-13T10:55:30.914921+0000","last_fullsized":"2023-11-13T10:55:30.914921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:22.528830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900326+0000","last_change":"2023-11-13T10:55:26.872483+0000","last_active":"2023-11-13T10:55:30.900326+0000","last_peered":"2023-11-13T10:55:30.900326+0000","last_clean":"2023-11-13T10:55:30.900326+0000","last_became_active":"2023-11-13T10:55:26.872186+0000","last_became_peered":"2023-11-13T10:55:26.872186+0000","last_unstale":"2023-11-13T10:55:30.900326+0000","last_undegraded":"2023-11-13T10:55:30.900326+0000","last_fullsized":"2023-11-13T10:55:30.900326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:47.501708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897861+0000","last_change":"2023-11-13T10:55:26.862513+0000","last_active":"2023-11-13T10:55:30.897861+0000","last_peered":"2023-11-13T10:55:30.897861+0000","last_clean":"2023-11-13T10:55:30.897861+0000","last_became_active":"2023-11-13T10:55:26.862167+0000","last_became_peered":"2023-11-13T10:55:26.862167+0000","last_unstale":"2023-11-13T10:55:30.897861+0000","last_undegraded":"2023-11-13T10:55:30.897861+0000","last_fullsized":"2023-11-13T10:55:30.897861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902597+0000","last_change":"2023-11-13T10:55:26.861425+0000","last_active":"2023-11-13T10:55:30.902597+0000","last_peered":"2023-11-13T10:55:30.902597+0000","last_clean":"2023-11-13T10:55:30.902597+0000","last_became_active":"2023-11-13T10:55:26.861132+0000","last_became_peered":"2023-11-13T10:55:26.861132+0000","last_unstale":"2023-11-13T10:55:30.902597+0000","last_undegraded":"2023-11-13T10:55:30.902597+0000","last_fullsized":"2023-11-13T10:55:30.902597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:29.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896561+0000","last_change":"2023-11-13T10:55:26.858332+0000","last_active":"2023-11-13T10:55:30.896561+0000","last_peered":"2023-11-13T10:55:30.896561+0000","last_clean":"2023-11-13T10:55:30.896561+0000","last_became_active":"2023-11-13T10:55:26.858052+0000","last_became_peered":"2023-11-13T10:55:26.858052+0000","last_unstale":"2023-11-13T10:55:30.896561+0000","last_undegraded":"2023-11-13T10:55:30.896561+0000","last_fullsized":"2023-11-13T10:55:30.896561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903167+0000","last_change":"2023-11-13T10:55:26.863579+0000","last_active":"2023-11-13T10:55:30.903167+0000","last_peered":"2023-11-13T10:55:30.903167+0000","last_clean":"2023-11-13T10:55:30.903167+0000","last_became_active":"2023-11-13T10:55:26.863247+0000","last_became_peered":"2023-11-13T10:55:26.863247+0000","last_unstale":"2023-11-13T10:55:30.903167+0000","last_undegraded":"2023-11-13T10:55:30.903167+0000","last_fullsized":"2023-11-13T10:55:30.903167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.774455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898503+0000","last_change":"2023-11-13T10:55:26.856219+0000","last_active":"2023-11-13T10:55:30.898503+0000","last_peered":"2023-11-13T10:55:30.898503+0000","last_clean":"2023-11-13T10:55:30.898503+0000","last_became_active":"2023-11-13T10:55:26.855933+0000","last_became_peered":"2023-11-13T10:55:26.855933+0000","last_unstale":"2023-11-13T10:55:30.898503+0000","last_undegraded":"2023-11-13T10:55:30.898503+0000","last_fullsized":"2023-11-13T10:55:30.898503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887319+0000","last_change":"2023-11-13T10:55:26.853802+0000","last_active":"2023-11-13T10:55:30.887319+0000","last_peered":"2023-11-13T10:55:30.887319+0000","last_clean":"2023-11-13T10:55:30.887319+0000","last_became_active":"2023-11-13T10:55:26.853463+0000","last_became_peered":"2023-11-13T10:55:26.853463+0000","last_unstale":"2023-11-13T10:55:30.887319+0000","last_undegraded":"2023-11-13T10:55:30.887319+0000","last_fullsized":"2023-11-13T10:55:30.887319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:06.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902537+0000","last_change":"2023-11-13T10:55:26.865339+0000","last_active":"2023-11-13T10:55:30.902537+0000","last_peered":"2023-11-13T10:55:30.902537+0000","last_clean":"2023-11-13T10:55:30.902537+0000","last_became_active":"2023-11-13T10:55:26.865069+0000","last_became_peered":"2023-11-13T10:55:26.865069+0000","last_unstale":"2023-11-13T10:55:30.902537+0000","last_undegraded":"2023-11-13T10:55:30.902537+0000","last_fullsized":"2023-11-13T10:55:30.902537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:50.524863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887342+0000","last_change":"2023-11-13T10:55:26.857434+0000","last_active":"2023-11-13T10:55:30.887342+0000","last_peered":"2023-11-13T10:55:30.887342+0000","last_clean":"2023-11-13T10:55:30.887342+0000","last_became_active":"2023-11-13T10:55:26.857164+0000","last_became_peered":"2023-11-13T10:55:26.857164+0000","last_unstale":"2023-11-13T10:55:30.887342+0000","last_undegraded":"2023-11-13T10:55:30.887342+0000","last_fullsized":"2023-11-13T10:55:30.887342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909271+0000","last_change":"2023-11-13T10:55:26.845954+0000","last_active":"2023-11-13T10:55:30.909271+0000","last_peered":"2023-11-13T10:55:30.909271+0000","last_clean":"2023-11-13T10:55:30.909271+0000","last_became_active":"2023-11-13T10:55:26.845505+0000","last_became_peered":"2023-11-13T10:55:26.845505+0000","last_unstale":"2023-11-13T10:55:30.909271+0000","last_undegraded":"2023-11-13T10:55:30.909271+0000","last_fullsized":"2023-11-13T10:55:30.909271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:43.799944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911061+0000","last_change":"2023-11-13T10:55:26.857565+0000","last_active":"2023-11-13T10:55:30.911061+0000","last_peered":"2023-11-13T10:55:30.911061+0000","last_clean":"2023-11-13T10:55:30.911061+0000","last_became_active":"2023-11-13T10:55:26.857291+0000","last_became_peered":"2023-11-13T10:55:26.857291+0000","last_unstale":"2023-11-13T10:55:30.911061+0000","last_undegraded":"2023-11-13T10:55:30.911061+0000","last_fullsized":"2023-11-13T10:55:30.911061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:29.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902086+0000","last_change":"2023-11-13T10:55:26.858620+0000","last_active":"2023-11-13T10:55:30.902086+0000","last_peered":"2023-11-13T10:55:30.902086+0000","last_clean":"2023-11-13T10:55:30.902086+0000","last_became_active":"2023-11-13T10:55:26.858348+0000","last_became_peered":"2023-11-13T10:55:26.858348+0000","last_unstale":"2023-11-13T10:55:30.902086+0000","last_undegraded":"2023-11-13T10:55:30.902086+0000","last_fullsized":"2023-11-13T10:55:30.902086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:45.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]}]},{"osd":5,"up_from":15,"seq":64424509453,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":695602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]}]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":109655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.627}]}]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":105724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":108983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":695580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2570000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3039999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7849999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T10:56:02.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T10:56:02.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.356+0000 7fd1cc4b4640 1 Processor -- start 2023-11-13T10:56:02.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.356+0000 7fd1cc4b4640 1 -- start start 2023-11-13T10:56:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.357+0000 7fd1cc4b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd1c4155410 0x7fd1c40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.357+0000 7fd1cc4b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4071b50 0x7fd1c40cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.357+0000 7fd1cc4b4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c415b500 con 0x7fd1c40ce990 2023-11-13T10:56:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.357+0000 7fd1cc4b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c415b7e0 con 0x7fd1c4155410 2023-11-13T10:56:02.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.357+0000 7fd1cc4b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c415bac0 con 0x7fd1c4071b50 2023-11-13T10:56:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.358+0000 7fd1ca250640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd1c4155410 0x7fd1c40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.358+0000 7fd1ca250640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd1c4155410 0x7fd1c40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39306/0 (socket says 172.21.15.16:39306) 2023-11-13T10:56:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.358+0000 7fd1c9a4f640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fd1c40ce990 0x7fd1c4071440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:43660/0 (socket says 172.21.15.16:43660) 2023-11-13T10:56:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.358+0000 7fd1ca250640 1 -- 172.21.15.16:0/4167383743 learned_addr learned my addr 172.21.15.16:0/4167383743 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.358+0000 7fd1caa51640 1 --2- 172.21.15.16:0/4167383743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4071b50 0x7fd1c40cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.359+0000 7fd1ca250640 1 -- 172.21.15.16:0/4167383743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4071b50 msgr2=0x7fd1c40cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.359+0000 7fd1ca250640 1 --2- 172.21.15.16:0/4167383743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4071b50 0x7fd1c40cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.359+0000 7fd1ca250640 1 -- 172.21.15.16:0/4167383743 >> v1:172.21.15.184:6789/0 conn(0x7fd1c40ce990 legacy=0x7fd1c4071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.359+0000 7fd1ca250640 1 -- 172.21.15.16:0/4167383743 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c415bda0 con 0x7fd1c4155410 2023-11-13T10:56:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.360+0000 7fd1ca250640 1 --2- 172.21.15.16:0/4167383743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c4155410 0x7fd1c40ccc60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009870 tx=0x7fd1c002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede9fa7680d0cfee server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.360+0000 7fd1c924e640 1 -- 172.21.15.16:0/4167383743 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0030840 con 0x7fd1c4155410 2023-11-13T10:56:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.360+0000 7fd1c924e640 1 -- 172.21.15.16:0/4167383743 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c00309a0 con 0x7fd1c4155410 2023-11-13T10:56:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.360+0000 7fd1c924e640 1 -- 172.21.15.16:0/4167383743 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0030d30 con 0x7fd1c4155410 2023-11-13T10:56:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.361+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/4167383743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c4155410 msgr2=0x7fd1c40ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.361+0000 7fd1cc4b4640 1 --2- 172.21.15.16:0/4167383743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c4155410 0x7fd1c40ccc60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009870 tx=0x7fd1c002b980 comp rx=0 tx=0).stop 2023-11-13T10:56:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.361+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/4167383743 shutdown_connections 2023-11-13T10:56:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.361+0000 7fd1cc4b4640 1 --2- 172.21.15.16:0/4167383743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4071b50 0x7fd1c40cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.361+0000 7fd1cc4b4640 1 --2- 172.21.15.16:0/4167383743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c4155410 0x7fd1c40ccc60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.361+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/4167383743 >> 172.21.15.16:0/4167383743 conn(0x7fd1c406a190 msgr2=0x7fd1c406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.361+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/4167383743 shutdown_connections 2023-11-13T10:56:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.361+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/4167383743 wait complete. 2023-11-13T10:56:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.363+0000 7fd1cc4b4640 1 Processor -- start 2023-11-13T10:56:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.363+0000 7fd1cc4b4640 1 -- start start 2023-11-13T10:56:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.364+0000 7fd1cc4b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c4071b50 0x7fd1c414d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.364+0000 7fd1cc4b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c40ce990 0x7fd1c41527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.364+0000 7fd1cc4b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4155410 0x7fd1c4156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.364+0000 7fd1cc4b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c4152ce0 con 0x7fd1c40ce990 2023-11-13T10:56:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.364+0000 7fd1cc4b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c4156b70 con 0x7fd1c4071b50 2023-11-13T10:56:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.364+0000 7fd1cc4b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c4152f10 con 0x7fd1c4155410 2023-11-13T10:56:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1c9a4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c40ce990 0x7fd1c41527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1c9a4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c40ce990 0x7fd1c41527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56304/0 (socket says 172.21.15.16:56304) 2023-11-13T10:56:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1c9a4f640 1 -- 172.21.15.16:0/3175293747 learned_addr learned my addr 172.21.15.16:0/3175293747 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1caa51640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4155410 0x7fd1c4156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1ca250640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c4071b50 0x7fd1c414d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1c9a4f640 1 -- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4155410 msgr2=0x7fd1c4156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1c9a4f640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4155410 0x7fd1c4156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1c9a4f640 1 -- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c4071b50 msgr2=0x7fd1c414d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1c9a4f640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c4071b50 0x7fd1c414d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1c9a4f640 1 -- 172.21.15.16:0/3175293747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c00094d0 con 0x7fd1c40ce990 2023-11-13T10:56:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.365+0000 7fd1c9a4f640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c40ce990 0x7fd1c41527a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c4d0 tx=0x7fd1b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.366+0000 7fd1baffd640 1 -- 172.21.15.16:0/3175293747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4007480 con 0x7fd1c40ce990 2023-11-13T10:56:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.366+0000 7fd1baffd640 1 -- 172.21.15.16:0/3175293747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b40075e0 con 0x7fd1c40ce990 2023-11-13T10:56:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.366+0000 7fd1baffd640 1 -- 172.21.15.16:0/3175293747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40049e0 con 0x7fd1c40ce990 2023-11-13T10:56:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.366+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c4155b50 con 0x7fd1c40ce990 2023-11-13T10:56:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.366+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c4156110 con 0x7fd1c40ce990 2023-11-13T10:56:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.367+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c4067090 con 0x7fd1c40ce990 2023-11-13T10:56:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.368+0000 7fd1baffd640 1 -- 172.21.15.16:0/3175293747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1b4004b40 con 0x7fd1c40ce990 2023-11-13T10:56:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.369+0000 7fd1baffd640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae90 0x7fd1a806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.369+0000 7fd1baffd640 1 -- 172.21.15.16:0/3175293747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd1b4088dc0 con 0x7fd1c40ce990 2023-11-13T10:56:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.370+0000 7fd1ca250640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae90 0x7fd1a806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.371+0000 7fd1ca250640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae90 0x7fd1a806d350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0036000 tx=0x7fd1c003b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.373+0000 7fd1baffd640 1 -- 172.21.15.16:0/3175293747 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1b4053c10 con 0x7fd1c40ce990 2023-11-13T10:56:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.557+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1c415c080 con 0x7fd1a806ae90 2023-11-13T10:56:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.583+0000 7fd1baffd640 1 -- 172.21.15.16:0/3175293747 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345067 (secure 0 0 0) 0x7fd1c0030b90 con 0x7fd1a806ae90 2023-11-13T10:56:02.597 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T10:56:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae90 msgr2=0x7fd1a806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae90 0x7fd1a806d350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0036000 tx=0x7fd1c003b040 comp rx=0 tx=0).stop 2023-11-13T10:56:02.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c40ce990 msgr2=0x7fd1c41527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c40ce990 0x7fd1c41527a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c4d0 tx=0x7fd1b400c990 comp rx=0 tx=0).stop 2023-11-13T10:56:02.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 shutdown_connections 2023-11-13T10:56:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae90 0x7fd1a806d350 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c4155410 0x7fd1c4156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c40ce990 0x7fd1c41527a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 --2- 172.21.15.16:0/3175293747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c4071b50 0x7fd1c414d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 >> 172.21.15.16:0/3175293747 conn(0x7fd1c406a190 msgr2=0x7fd1c4154720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:02.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 shutdown_connections 2023-11-13T10:56:02.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.591+0000 7fd1cc4b4640 1 -- 172.21.15.16:0/3175293747 wait complete. 2023-11-13T10:56:02.616 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-11-13T10:56:01.981184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232004,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742716,"statfs":{"total":773094113280,"available":772856541184,"internally_reserved":0,"allocated":3416064,"data_stored":2031150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005625"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894617+0000","last_change":"2023-11-13T10:55:25.848955+0000","last_active":"2023-11-13T10:55:30.894617+0000","last_peered":"2023-11-13T10:55:30.894617+0000","last_clean":"2023-11-13T10:55:30.894617+0000","last_became_active":"2023-11-13T10:55:25.848644+0000","last_became_peered":"2023-11-13T10:55:25.848644+0000","last_unstale":"2023-11-13T10:55:30.894617+0000","last_undegraded":"2023-11-13T10:55:30.894617+0000","last_fullsized":"2023-11-13T10:55:30.894617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:40.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911163+0000","last_change":"2023-11-13T10:55:26.862834+0000","last_active":"2023-11-13T10:55:30.911163+0000","last_peered":"2023-11-13T10:55:30.911163+0000","last_clean":"2023-11-13T10:55:30.911163+0000","last_became_active":"2023-11-13T10:55:26.861881+0000","last_became_peered":"2023-11-13T10:55:26.861881+0000","last_unstale":"2023-11-13T10:55:30.911163+0000","last_undegraded":"2023-11-13T10:55:30.911163+0000","last_fullsized":"2023-11-13T10:55:30.911163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:47.501708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906481+0000","last_change":"2023-11-13T10:55:30.929671+0000","last_active":"2023-11-13T10:55:31.906481+0000","last_peered":"2023-11-13T10:55:31.906481+0000","last_clean":"2023-11-13T10:55:31.906481+0000","last_became_active":"2023-11-13T10:55:30.929229+0000","last_became_peered":"2023-11-13T10:55:30.929229+0000","last_unstale":"2023-11-13T10:55:31.906481+0000","last_undegraded":"2023-11-13T10:55:31.906481+0000","last_fullsized":"2023-11-13T10:55:31.906481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:46.695309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911269+0000","last_change":"2023-11-13T10:55:25.855701+0000","last_active":"2023-11-13T10:55:30.911269+0000","last_peered":"2023-11-13T10:55:30.911269+0000","last_clean":"2023-11-13T10:55:30.911269+0000","last_became_active":"2023-11-13T10:55:25.855360+0000","last_became_peered":"2023-11-13T10:55:25.855360+0000","last_unstale":"2023-11-13T10:55:30.911269+0000","last_undegraded":"2023-11-13T10:55:30.911269+0000","last_fullsized":"2023-11-13T10:55:30.911269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:52.352286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912877+0000","last_change":"2023-11-13T10:55:26.848124+0000","last_active":"2023-11-13T10:55:30.912877+0000","last_peered":"2023-11-13T10:55:30.912877+0000","last_clean":"2023-11-13T10:55:30.912877+0000","last_became_active":"2023-11-13T10:55:26.847478+0000","last_became_peered":"2023-11-13T10:55:26.847478+0000","last_unstale":"2023-11-13T10:55:30.912877+0000","last_undegraded":"2023-11-13T10:55:30.912877+0000","last_fullsized":"2023-11-13T10:55:30.912877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:06.034946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.903358+0000","last_change":"2023-11-13T10:55:30.918537+0000","last_active":"2023-11-13T10:55:31.903358+0000","last_peered":"2023-11-13T10:55:31.903358+0000","last_clean":"2023-11-13T10:55:31.903358+0000","last_became_active":"2023-11-13T10:55:30.918092+0000","last_became_peered":"2023-11-13T10:55:30.918092+0000","last_unstale":"2023-11-13T10:55:31.903358+0000","last_undegraded":"2023-11-13T10:55:31.903358+0000","last_fullsized":"2023-11-13T10:55:31.903358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:50:02.751900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898103+0000","last_change":"2023-11-13T10:55:25.846685+0000","last_active":"2023-11-13T10:55:30.898103+0000","last_peered":"2023-11-13T10:55:30.898103+0000","last_clean":"2023-11-13T10:55:30.898103+0000","last_became_active":"2023-11-13T10:55:25.846370+0000","last_became_peered":"2023-11-13T10:55:25.846370+0000","last_unstale":"2023-11-13T10:55:30.898103+0000","last_undegraded":"2023-11-13T10:55:30.898103+0000","last_fullsized":"2023-11-13T10:55:30.898103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896163+0000","last_change":"2023-11-13T10:55:26.867122+0000","last_active":"2023-11-13T10:55:30.896163+0000","last_peered":"2023-11-13T10:55:30.896163+0000","last_clean":"2023-11-13T10:55:30.896163+0000","last_became_active":"2023-11-13T10:55:26.866669+0000","last_became_peered":"2023-11-13T10:55:26.866669+0000","last_unstale":"2023-11-13T10:55:30.896163+0000","last_undegraded":"2023-11-13T10:55:30.896163+0000","last_fullsized":"2023-11-13T10:55:30.896163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:25.074197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904843+0000","last_change":"2023-11-13T10:55:30.918484+0000","last_active":"2023-11-13T10:55:31.904843+0000","last_peered":"2023-11-13T10:55:31.904843+0000","last_clean":"2023-11-13T10:55:31.904843+0000","last_became_active":"2023-11-13T10:55:30.917997+0000","last_became_peered":"2023-11-13T10:55:30.917997+0000","last_unstale":"2023-11-13T10:55:31.904843+0000","last_undegraded":"2023-11-13T10:55:31.904843+0000","last_fullsized":"2023-11-13T10:55:31.904843+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:17.817768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900603+0000","last_change":"2023-11-13T10:55:25.851246+0000","last_active":"2023-11-13T10:55:30.900603+0000","last_peered":"2023-11-13T10:55:30.900603+0000","last_clean":"2023-11-13T10:55:30.900603+0000","last_became_active":"2023-11-13T10:55:25.850865+0000","last_became_peered":"2023-11-13T10:55:25.850865+0000","last_unstale":"2023-11-13T10:55:30.900603+0000","last_undegraded":"2023-11-13T10:55:30.900603+0000","last_fullsized":"2023-11-13T10:55:30.900603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:30.550401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898058+0000","last_change":"2023-11-13T10:55:26.866532+0000","last_active":"2023-11-13T10:55:30.898058+0000","last_peered":"2023-11-13T10:55:30.898058+0000","last_clean":"2023-11-13T10:55:30.898058+0000","last_became_active":"2023-11-13T10:55:26.866199+0000","last_became_peered":"2023-11-13T10:55:26.866199+0000","last_unstale":"2023-11-13T10:55:30.898058+0000","last_undegraded":"2023-11-13T10:55:30.898058+0000","last_fullsized":"2023-11-13T10:55:30.898058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:18.260379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905023+0000","last_change":"2023-11-13T10:55:30.919396+0000","last_active":"2023-11-13T10:55:31.905023+0000","last_peered":"2023-11-13T10:55:31.905023+0000","last_clean":"2023-11-13T10:55:31.905023+0000","last_became_active":"2023-11-13T10:55:30.918942+0000","last_became_peered":"2023-11-13T10:55:30.918942+0000","last_unstale":"2023-11-13T10:55:31.905023+0000","last_undegraded":"2023-11-13T10:55:31.905023+0000","last_fullsized":"2023-11-13T10:55:31.905023+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:51.562560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898635+0000","last_change":"2023-11-13T10:55:26.867788+0000","last_active":"2023-11-13T10:55:30.898635+0000","last_peered":"2023-11-13T10:55:30.898635+0000","last_clean":"2023-11-13T10:55:30.898635+0000","last_became_active":"2023-11-13T10:55:26.865720+0000","last_became_peered":"2023-11-13T10:55:26.865720+0000","last_unstale":"2023-11-13T10:55:30.898635+0000","last_undegraded":"2023-11-13T10:55:30.898635+0000","last_fullsized":"2023-11-13T10:55:30.898635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:04:53.206295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899232+0000","last_change":"2023-11-13T10:55:25.848578+0000","last_active":"2023-11-13T10:55:30.899232+0000","last_peered":"2023-11-13T10:55:30.899232+0000","last_clean":"2023-11-13T10:55:30.899232+0000","last_became_active":"2023-11-13T10:55:25.848218+0000","last_became_peered":"2023-11-13T10:55:25.848218+0000","last_unstale":"2023-11-13T10:55:30.899232+0000","last_undegraded":"2023-11-13T10:55:30.899232+0000","last_fullsized":"2023-11-13T10:55:30.899232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.882851+0000","last_change":"2023-11-13T10:55:30.927208+0000","last_active":"2023-11-13T10:55:31.882851+0000","last_peered":"2023-11-13T10:55:31.882851+0000","last_clean":"2023-11-13T10:55:31.882851+0000","last_became_active":"2023-11-13T10:55:30.926587+0000","last_became_peered":"2023-11-13T10:55:30.926587+0000","last_unstale":"2023-11-13T10:55:31.882851+0000","last_undegraded":"2023-11-13T10:55:31.882851+0000","last_fullsized":"2023-11-13T10:55:31.882851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:58:30.075199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911307+0000","last_change":"2023-11-13T10:55:26.863647+0000","last_active":"2023-11-13T10:55:30.911307+0000","last_peered":"2023-11-13T10:55:30.911307+0000","last_clean":"2023-11-13T10:55:30.911307+0000","last_became_active":"2023-11-13T10:55:26.863171+0000","last_became_peered":"2023-11-13T10:55:26.863171+0000","last_unstale":"2023-11-13T10:55:30.911307+0000","last_undegraded":"2023-11-13T10:55:30.911307+0000","last_fullsized":"2023-11-13T10:55:30.911307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:44:59.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913133+0000","last_change":"2023-11-13T10:55:25.861244+0000","last_active":"2023-11-13T10:55:30.913133+0000","last_peered":"2023-11-13T10:55:30.913133+0000","last_clean":"2023-11-13T10:55:30.913133+0000","last_became_active":"2023-11-13T10:55:25.860949+0000","last_became_peered":"2023-11-13T10:55:25.860949+0000","last_unstale":"2023-11-13T10:55:30.913133+0000","last_undegraded":"2023-11-13T10:55:30.913133+0000","last_fullsized":"2023-11-13T10:55:30.913133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:05.431553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.902438+0000","last_change":"2023-11-13T10:55:30.922731+0000","last_active":"2023-11-13T10:55:31.902438+0000","last_peered":"2023-11-13T10:55:31.902438+0000","last_clean":"2023-11-13T10:55:31.902438+0000","last_became_active":"2023-11-13T10:55:30.922328+0000","last_became_peered":"2023-11-13T10:55:30.922328+0000","last_unstale":"2023-11-13T10:55:31.902438+0000","last_undegraded":"2023-11-13T10:55:31.902438+0000","last_fullsized":"2023-11-13T10:55:31.902438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:57:17.670747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897411+0000","last_change":"2023-11-13T10:55:26.857718+0000","last_active":"2023-11-13T10:55:30.897411+0000","last_peered":"2023-11-13T10:55:30.897411+0000","last_clean":"2023-11-13T10:55:30.897411+0000","last_became_active":"2023-11-13T10:55:26.852435+0000","last_became_peered":"2023-11-13T10:55:26.852435+0000","last_unstale":"2023-11-13T10:55:30.897411+0000","last_undegraded":"2023-11-13T10:55:30.897411+0000","last_fullsized":"2023-11-13T10:55:30.897411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:45.182794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892276+0000","last_change":"2023-11-13T10:55:25.858975+0000","last_active":"2023-11-13T10:55:30.892276+0000","last_peered":"2023-11-13T10:55:30.892276+0000","last_clean":"2023-11-13T10:55:30.892276+0000","last_became_active":"2023-11-13T10:55:25.858642+0000","last_became_peered":"2023-11-13T10:55:25.858642+0000","last_unstale":"2023-11-13T10:55:30.892276+0000","last_undegraded":"2023-11-13T10:55:30.892276+0000","last_fullsized":"2023-11-13T10:55:30.892276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.908730+0000","last_change":"2023-11-13T10:55:30.931892+0000","last_active":"2023-11-13T10:55:31.908730+0000","last_peered":"2023-11-13T10:55:31.908730+0000","last_clean":"2023-11-13T10:55:31.908730+0000","last_became_active":"2023-11-13T10:55:30.931357+0000","last_became_peered":"2023-11-13T10:55:30.931357+0000","last_unstale":"2023-11-13T10:55:31.908730+0000","last_undegraded":"2023-11-13T10:55:31.908730+0000","last_fullsized":"2023-11-13T10:55:31.908730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:07:10.060960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897798+0000","last_change":"2023-11-13T10:55:26.855031+0000","last_active":"2023-11-13T10:55:30.897798+0000","last_peered":"2023-11-13T10:55:30.897798+0000","last_clean":"2023-11-13T10:55:30.897798+0000","last_became_active":"2023-11-13T10:55:26.854761+0000","last_became_peered":"2023-11-13T10:55:26.854761+0000","last_unstale":"2023-11-13T10:55:30.897798+0000","last_undegraded":"2023-11-13T10:55:30.897798+0000","last_fullsized":"2023-11-13T10:55:30.897798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:27.863825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898689+0000","last_change":"2023-11-13T10:55:25.848611+0000","last_active":"2023-11-13T10:55:30.898689+0000","last_peered":"2023-11-13T10:55:30.898689+0000","last_clean":"2023-11-13T10:55:30.898689+0000","last_became_active":"2023-11-13T10:55:25.848277+0000","last_became_peered":"2023-11-13T10:55:25.848277+0000","last_unstale":"2023-11-13T10:55:30.898689+0000","last_undegraded":"2023-11-13T10:55:30.898689+0000","last_fullsized":"2023-11-13T10:55:30.898689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905715+0000","last_change":"2023-11-13T10:55:30.920124+0000","last_active":"2023-11-13T10:55:31.905715+0000","last_peered":"2023-11-13T10:55:31.905715+0000","last_clean":"2023-11-13T10:55:31.905715+0000","last_became_active":"2023-11-13T10:55:30.919731+0000","last_became_peered":"2023-11-13T10:55:30.919731+0000","last_unstale":"2023-11-13T10:55:31.905715+0000","last_undegraded":"2023-11-13T10:55:31.905715+0000","last_fullsized":"2023-11-13T10:55:31.905715+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899898+0000","last_change":"2023-11-13T10:55:26.860831+0000","last_active":"2023-11-13T10:55:30.899898+0000","last_peered":"2023-11-13T10:55:30.899898+0000","last_clean":"2023-11-13T10:55:30.899898+0000","last_became_active":"2023-11-13T10:55:26.860527+0000","last_became_peered":"2023-11-13T10:55:26.860527+0000","last_unstale":"2023-11-13T10:55:30.899898+0000","last_undegraded":"2023-11-13T10:55:30.899898+0000","last_fullsized":"2023-11-13T10:55:30.899898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.774455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899998+0000","last_change":"2023-11-13T10:55:25.846361+0000","last_active":"2023-11-13T10:55:30.899998+0000","last_peered":"2023-11-13T10:55:30.899998+0000","last_clean":"2023-11-13T10:55:30.899998+0000","last_became_active":"2023-11-13T10:55:25.846031+0000","last_became_peered":"2023-11-13T10:55:25.846031+0000","last_unstale":"2023-11-13T10:55:30.899998+0000","last_undegraded":"2023-11-13T10:55:30.899998+0000","last_fullsized":"2023-11-13T10:55:30.899998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:44.868236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904780+0000","last_change":"2023-11-13T10:55:30.910764+0000","last_active":"2023-11-13T10:55:31.904780+0000","last_peered":"2023-11-13T10:55:31.904780+0000","last_clean":"2023-11-13T10:55:31.904780+0000","last_became_active":"2023-11-13T10:55:30.910363+0000","last_became_peered":"2023-11-13T10:55:30.910363+0000","last_unstale":"2023-11-13T10:55:31.904780+0000","last_undegraded":"2023-11-13T10:55:31.904780+0000","last_fullsized":"2023-11-13T10:55:31.904780+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:33.673828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.893835+0000","last_change":"2023-11-13T10:55:26.859968+0000","last_active":"2023-11-13T10:55:30.893835+0000","last_peered":"2023-11-13T10:55:30.893835+0000","last_clean":"2023-11-13T10:55:30.893835+0000","last_became_active":"2023-11-13T10:55:26.859344+0000","last_became_peered":"2023-11-13T10:55:26.859344+0000","last_unstale":"2023-11-13T10:55:30.893835+0000","last_undegraded":"2023-11-13T10:55:30.893835+0000","last_fullsized":"2023-11-13T10:55:30.893835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.107404+0000","last_change":"2023-11-13T10:55:25.857948+0000","last_active":"2023-11-13T10:55:31.107404+0000","last_peered":"2023-11-13T10:55:31.107404+0000","last_clean":"2023-11-13T10:55:31.107404+0000","last_became_active":"2023-11-13T10:55:25.857670+0000","last_became_peered":"2023-11-13T10:55:25.857670+0000","last_unstale":"2023-11-13T10:55:31.107404+0000","last_undegraded":"2023-11-13T10:55:31.107404+0000","last_fullsized":"2023-11-13T10:55:31.107404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:11.016234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883636+0000","last_change":"2023-11-13T10:55:30.918361+0000","last_active":"2023-11-13T10:55:31.883636+0000","last_peered":"2023-11-13T10:55:31.883636+0000","last_clean":"2023-11-13T10:55:31.883636+0000","last_became_active":"2023-11-13T10:55:30.912409+0000","last_became_peered":"2023-11-13T10:55:30.912409+0000","last_unstale":"2023-11-13T10:55:31.883636+0000","last_undegraded":"2023-11-13T10:55:31.883636+0000","last_fullsized":"2023-11-13T10:55:31.883636+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:05.903305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913447+0000","last_change":"2023-11-13T10:55:26.864360+0000","last_active":"2023-11-13T10:55:30.913447+0000","last_peered":"2023-11-13T10:55:30.913447+0000","last_clean":"2023-11-13T10:55:30.913447+0000","last_became_active":"2023-11-13T10:55:26.863969+0000","last_became_peered":"2023-11-13T10:55:26.863969+0000","last_unstale":"2023-11-13T10:55:30.913447+0000","last_undegraded":"2023-11-13T10:55:30.913447+0000","last_fullsized":"2023-11-13T10:55:30.913447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:45.182794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898332+0000","last_change":"2023-11-13T10:55:25.850034+0000","last_active":"2023-11-13T10:55:30.898332+0000","last_peered":"2023-11-13T10:55:30.898332+0000","last_clean":"2023-11-13T10:55:30.898332+0000","last_became_active":"2023-11-13T10:55:25.849670+0000","last_became_peered":"2023-11-13T10:55:25.849670+0000","last_unstale":"2023-11-13T10:55:30.898332+0000","last_undegraded":"2023-11-13T10:55:30.898332+0000","last_fullsized":"2023-11-13T10:55:30.898332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:42.787670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905007+0000","last_change":"2023-11-13T10:55:30.921622+0000","last_active":"2023-11-13T10:55:31.905007+0000","last_peered":"2023-11-13T10:55:31.905007+0000","last_clean":"2023-11-13T10:55:31.905007+0000","last_became_active":"2023-11-13T10:55:30.921092+0000","last_became_peered":"2023-11-13T10:55:30.921092+0000","last_unstale":"2023-11-13T10:55:31.905007+0000","last_undegraded":"2023-11-13T10:55:31.905007+0000","last_fullsized":"2023-11-13T10:55:31.905007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:49.941362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886149+0000","last_change":"2023-11-13T10:55:26.858625+0000","last_active":"2023-11-13T10:55:30.886149+0000","last_peered":"2023-11-13T10:55:30.886149+0000","last_clean":"2023-11-13T10:55:30.886149+0000","last_became_active":"2023-11-13T10:55:26.858318+0000","last_became_peered":"2023-11-13T10:55:26.858318+0000","last_unstale":"2023-11-13T10:55:30.886149+0000","last_undegraded":"2023-11-13T10:55:30.886149+0000","last_fullsized":"2023-11-13T10:55:30.886149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:43.105401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909064+0000","last_change":"2023-11-13T10:55:25.862368+0000","last_active":"2023-11-13T10:55:30.909064+0000","last_peered":"2023-11-13T10:55:30.909064+0000","last_clean":"2023-11-13T10:55:30.909064+0000","last_became_active":"2023-11-13T10:55:25.862060+0000","last_became_peered":"2023-11-13T10:55:25.862060+0000","last_unstale":"2023-11-13T10:55:30.909064+0000","last_undegraded":"2023-11-13T10:55:30.909064+0000","last_fullsized":"2023-11-13T10:55:30.909064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:09.162069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904580+0000","last_change":"2023-11-13T10:55:30.923556+0000","last_active":"2023-11-13T10:55:31.904580+0000","last_peered":"2023-11-13T10:55:31.904580+0000","last_clean":"2023-11-13T10:55:31.904580+0000","last_became_active":"2023-11-13T10:55:30.923070+0000","last_became_peered":"2023-11-13T10:55:30.923070+0000","last_unstale":"2023-11-13T10:55:31.904580+0000","last_undegraded":"2023-11-13T10:55:31.904580+0000","last_fullsized":"2023-11-13T10:55:31.904580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:51.562560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902612+0000","last_change":"2023-11-13T10:55:26.864760+0000","last_active":"2023-11-13T10:55:30.902612+0000","last_peered":"2023-11-13T10:55:30.902612+0000","last_clean":"2023-11-13T10:55:30.902612+0000","last_became_active":"2023-11-13T10:55:26.864419+0000","last_became_peered":"2023-11-13T10:55:26.864419+0000","last_unstale":"2023-11-13T10:55:30.902612+0000","last_undegraded":"2023-11-13T10:55:30.902612+0000","last_fullsized":"2023-11-13T10:55:30.902612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:01.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898908+0000","last_change":"2023-11-13T10:55:25.856907+0000","last_active":"2023-11-13T10:55:30.898908+0000","last_peered":"2023-11-13T10:55:30.898908+0000","last_clean":"2023-11-13T10:55:30.898908+0000","last_became_active":"2023-11-13T10:55:25.856545+0000","last_became_peered":"2023-11-13T10:55:25.856545+0000","last_unstale":"2023-11-13T10:55:30.898908+0000","last_undegraded":"2023-11-13T10:55:30.898908+0000","last_fullsized":"2023-11-13T10:55:30.898908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:28.600702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906072+0000","last_change":"2023-11-13T10:55:30.921637+0000","last_active":"2023-11-13T10:55:31.906072+0000","last_peered":"2023-11-13T10:55:31.906072+0000","last_clean":"2023-11-13T10:55:31.906072+0000","last_became_active":"2023-11-13T10:55:30.921177+0000","last_became_peered":"2023-11-13T10:55:30.921177+0000","last_unstale":"2023-11-13T10:55:31.906072+0000","last_undegraded":"2023-11-13T10:55:31.906072+0000","last_fullsized":"2023-11-13T10:55:31.906072+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903258+0000","last_change":"2023-11-13T10:55:26.851557+0000","last_active":"2023-11-13T10:55:30.903258+0000","last_peered":"2023-11-13T10:55:30.903258+0000","last_clean":"2023-11-13T10:55:30.903258+0000","last_became_active":"2023-11-13T10:55:26.850792+0000","last_became_peered":"2023-11-13T10:55:26.850792+0000","last_unstale":"2023-11-13T10:55:30.903258+0000","last_undegraded":"2023-11-13T10:55:30.903258+0000","last_fullsized":"2023-11-13T10:55:30.903258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911327+0000","last_change":"2023-11-13T10:55:25.855880+0000","last_active":"2023-11-13T10:55:30.911327+0000","last_peered":"2023-11-13T10:55:30.911327+0000","last_clean":"2023-11-13T10:55:30.911327+0000","last_became_active":"2023-11-13T10:55:25.855543+0000","last_became_peered":"2023-11-13T10:55:25.855543+0000","last_unstale":"2023-11-13T10:55:30.911327+0000","last_undegraded":"2023-11-13T10:55:30.911327+0000","last_fullsized":"2023-11-13T10:55:30.911327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:02:28.405745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.907032+0000","last_change":"2023-11-13T10:55:30.925168+0000","last_active":"2023-11-13T10:55:31.907032+0000","last_peered":"2023-11-13T10:55:31.907032+0000","last_clean":"2023-11-13T10:55:31.907032+0000","last_became_active":"2023-11-13T10:55:30.924675+0000","last_became_peered":"2023-11-13T10:55:30.924675+0000","last_unstale":"2023-11-13T10:55:31.907032+0000","last_undegraded":"2023-11-13T10:55:31.907032+0000","last_fullsized":"2023-11-13T10:55:31.907032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:25.376399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911697+0000","last_change":"2023-11-13T10:55:26.863984+0000","last_active":"2023-11-13T10:55:30.911697+0000","last_peered":"2023-11-13T10:55:30.911697+0000","last_clean":"2023-11-13T10:55:30.911697+0000","last_became_active":"2023-11-13T10:55:26.862398+0000","last_became_peered":"2023-11-13T10:55:26.862398+0000","last_unstale":"2023-11-13T10:55:30.911697+0000","last_undegraded":"2023-11-13T10:55:30.911697+0000","last_fullsized":"2023-11-13T10:55:30.911697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898919+0000","last_change":"2023-11-13T10:55:25.857004+0000","last_active":"2023-11-13T10:55:30.898919+0000","last_peered":"2023-11-13T10:55:30.898919+0000","last_clean":"2023-11-13T10:55:30.898919+0000","last_became_active":"2023-11-13T10:55:25.856718+0000","last_became_peered":"2023-11-13T10:55:25.856718+0000","last_unstale":"2023-11-13T10:55:30.898919+0000","last_undegraded":"2023-11-13T10:55:30.898919+0000","last_fullsized":"2023-11-13T10:55:30.898919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:28.167168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888398+0000","last_change":"2023-11-13T10:55:30.923781+0000","last_active":"2023-11-13T10:55:31.888398+0000","last_peered":"2023-11-13T10:55:31.888398+0000","last_clean":"2023-11-13T10:55:31.888398+0000","last_became_active":"2023-11-13T10:55:30.922714+0000","last_became_peered":"2023-11-13T10:55:30.922714+0000","last_unstale":"2023-11-13T10:55:31.888398+0000","last_undegraded":"2023-11-13T10:55:31.888398+0000","last_fullsized":"2023-11-13T10:55:31.888398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:05:31.401669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899007+0000","last_change":"2023-11-13T10:55:26.862991+0000","last_active":"2023-11-13T10:55:30.899007+0000","last_peered":"2023-11-13T10:55:30.899007+0000","last_clean":"2023-11-13T10:55:30.899007+0000","last_became_active":"2023-11-13T10:55:26.861100+0000","last_became_peered":"2023-11-13T10:55:26.861100+0000","last_unstale":"2023-11-13T10:55:30.899007+0000","last_undegraded":"2023-11-13T10:55:30.899007+0000","last_fullsized":"2023-11-13T10:55:30.899007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899324+0000","last_change":"2023-11-13T10:55:25.848422+0000","last_active":"2023-11-13T10:55:30.899324+0000","last_peered":"2023-11-13T10:55:30.899324+0000","last_clean":"2023-11-13T10:55:30.899324+0000","last_became_active":"2023-11-13T10:55:25.847809+0000","last_became_peered":"2023-11-13T10:55:25.847809+0000","last_unstale":"2023-11-13T10:55:30.899324+0000","last_undegraded":"2023-11-13T10:55:30.899324+0000","last_fullsized":"2023-11-13T10:55:30.899324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892816+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T10:55:30.892816+0000","last_peered":"2023-11-13T10:55:30.892816+0000","last_clean":"2023-11-13T10:55:30.892816+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T10:55:30.892816+0000","last_undegraded":"2023-11-13T10:55:30.892816+0000","last_fullsized":"2023-11-13T10:55:30.892816+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883757+0000","last_change":"2023-11-13T10:55:30.907496+0000","last_active":"2023-11-13T10:55:31.883757+0000","last_peered":"2023-11-13T10:55:31.883757+0000","last_clean":"2023-11-13T10:55:31.883757+0000","last_became_active":"2023-11-13T10:55:30.905911+0000","last_became_peered":"2023-11-13T10:55:30.905911+0000","last_unstale":"2023-11-13T10:55:31.883757+0000","last_undegraded":"2023-11-13T10:55:31.883757+0000","last_fullsized":"2023-11-13T10:55:31.883757+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:26:30.660465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898140+0000","last_change":"2023-11-13T10:55:26.865638+0000","last_active":"2023-11-13T10:55:30.898140+0000","last_peered":"2023-11-13T10:55:30.898140+0000","last_clean":"2023-11-13T10:55:30.898140+0000","last_became_active":"2023-11-13T10:55:26.864005+0000","last_became_peered":"2023-11-13T10:55:26.864005+0000","last_unstale":"2023-11-13T10:55:30.898140+0000","last_undegraded":"2023-11-13T10:55:30.898140+0000","last_fullsized":"2023-11-13T10:55:30.898140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:18:03.109671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.914054+0000","last_change":"2023-11-13T10:55:25.860502+0000","last_active":"2023-11-13T10:55:30.914054+0000","last_peered":"2023-11-13T10:55:30.914054+0000","last_clean":"2023-11-13T10:55:30.914054+0000","last_became_active":"2023-11-13T10:55:25.860157+0000","last_became_peered":"2023-11-13T10:55:25.860157+0000","last_unstale":"2023-11-13T10:55:30.914054+0000","last_undegraded":"2023-11-13T10:55:30.914054+0000","last_fullsized":"2023-11-13T10:55:30.914054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:42.093127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905171+0000","last_change":"2023-11-13T10:55:30.909998+0000","last_active":"2023-11-13T10:55:31.905171+0000","last_peered":"2023-11-13T10:55:31.905171+0000","last_clean":"2023-11-13T10:55:31.905171+0000","last_became_active":"2023-11-13T10:55:30.909623+0000","last_became_peered":"2023-11-13T10:55:30.909623+0000","last_unstale":"2023-11-13T10:55:31.905171+0000","last_undegraded":"2023-11-13T10:55:31.905171+0000","last_fullsized":"2023-11-13T10:55:31.905171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:47.860796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900512+0000","last_change":"2023-11-13T10:55:26.867419+0000","last_active":"2023-11-13T10:55:30.900512+0000","last_peered":"2023-11-13T10:55:30.900512+0000","last_clean":"2023-11-13T10:55:30.900512+0000","last_became_active":"2023-11-13T10:55:26.864802+0000","last_became_peered":"2023-11-13T10:55:26.864802+0000","last_unstale":"2023-11-13T10:55:30.900512+0000","last_undegraded":"2023-11-13T10:55:30.900512+0000","last_fullsized":"2023-11-13T10:55:30.900512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909289+0000","last_change":"2023-11-13T10:55:25.852321+0000","last_active":"2023-11-13T10:55:30.909289+0000","last_peered":"2023-11-13T10:55:30.909289+0000","last_clean":"2023-11-13T10:55:30.909289+0000","last_became_active":"2023-11-13T10:55:25.850605+0000","last_became_peered":"2023-11-13T10:55:25.850605+0000","last_unstale":"2023-11-13T10:55:30.909289+0000","last_undegraded":"2023-11-13T10:55:30.909289+0000","last_fullsized":"2023-11-13T10:55:30.909289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:44.170520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.887690+0000","last_change":"2023-11-13T10:55:30.913188+0000","last_active":"2023-11-13T10:55:31.887690+0000","last_peered":"2023-11-13T10:55:31.887690+0000","last_clean":"2023-11-13T10:55:31.887690+0000","last_became_active":"2023-11-13T10:55:30.912748+0000","last_became_peered":"2023-11-13T10:55:30.912748+0000","last_unstale":"2023-11-13T10:55:31.887690+0000","last_undegraded":"2023-11-13T10:55:31.887690+0000","last_fullsized":"2023-11-13T10:55:31.887690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:20.801658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902130+0000","last_change":"2023-11-13T10:55:26.867097+0000","last_active":"2023-11-13T10:55:30.902130+0000","last_peered":"2023-11-13T10:55:30.902130+0000","last_clean":"2023-11-13T10:55:30.902130+0000","last_became_active":"2023-11-13T10:55:26.866770+0000","last_became_peered":"2023-11-13T10:55:26.866770+0000","last_unstale":"2023-11-13T10:55:30.902130+0000","last_undegraded":"2023-11-13T10:55:30.902130+0000","last_fullsized":"2023-11-13T10:55:30.902130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:06.034946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900035+0000","last_change":"2023-11-13T10:55:25.855464+0000","last_active":"2023-11-13T10:55:30.900035+0000","last_peered":"2023-11-13T10:55:30.900035+0000","last_clean":"2023-11-13T10:55:30.900035+0000","last_became_active":"2023-11-13T10:55:25.853955+0000","last_became_peered":"2023-11-13T10:55:25.853955+0000","last_unstale":"2023-11-13T10:55:30.900035+0000","last_undegraded":"2023-11-13T10:55:30.900035+0000","last_fullsized":"2023-11-13T10:55:30.900035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.881810+0000","last_change":"2023-11-13T10:55:30.915984+0000","last_active":"2023-11-13T10:55:31.881810+0000","last_peered":"2023-11-13T10:55:31.881810+0000","last_clean":"2023-11-13T10:55:31.881810+0000","last_became_active":"2023-11-13T10:55:30.915438+0000","last_became_peered":"2023-11-13T10:55:30.915438+0000","last_unstale":"2023-11-13T10:55:31.881810+0000","last_undegraded":"2023-11-13T10:55:31.881810+0000","last_fullsized":"2023-11-13T10:55:31.881810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:06:39.428736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887086+0000","last_change":"2023-11-13T10:55:26.874069+0000","last_active":"2023-11-13T10:55:30.887086+0000","last_peered":"2023-11-13T10:55:30.887086+0000","last_clean":"2023-11-13T10:55:30.887086+0000","last_became_active":"2023-11-13T10:55:26.873537+0000","last_became_peered":"2023-11-13T10:55:26.873537+0000","last_unstale":"2023-11-13T10:55:30.887086+0000","last_undegraded":"2023-11-13T10:55:30.887086+0000","last_fullsized":"2023-11-13T10:55:30.887086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:22.528830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902191+0000","last_change":"2023-11-13T10:55:25.853256+0000","last_active":"2023-11-13T10:55:30.902191+0000","last_peered":"2023-11-13T10:55:30.902191+0000","last_clean":"2023-11-13T10:55:30.902191+0000","last_became_active":"2023-11-13T10:55:25.852947+0000","last_became_peered":"2023-11-13T10:55:25.852947+0000","last_unstale":"2023-11-13T10:55:30.902191+0000","last_undegraded":"2023-11-13T10:55:30.902191+0000","last_fullsized":"2023-11-13T10:55:30.902191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:15.762181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.904654+0000","last_change":"2023-11-13T10:55:30.912617+0000","last_active":"2023-11-13T10:55:31.904654+0000","last_peered":"2023-11-13T10:55:31.904654+0000","last_clean":"2023-11-13T10:55:31.904654+0000","last_became_active":"2023-11-13T10:55:30.912019+0000","last_became_peered":"2023-11-13T10:55:30.912019+0000","last_unstale":"2023-11-13T10:55:31.904654+0000","last_undegraded":"2023-11-13T10:55:31.904654+0000","last_fullsized":"2023-11-13T10:55:31.904654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:16.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897711+0000","last_change":"2023-11-13T10:55:26.868126+0000","last_active":"2023-11-13T10:55:30.897711+0000","last_peered":"2023-11-13T10:55:30.897711+0000","last_clean":"2023-11-13T10:55:30.897711+0000","last_became_active":"2023-11-13T10:55:26.867810+0000","last_became_peered":"2023-11-13T10:55:26.867810+0000","last_unstale":"2023-11-13T10:55:30.897711+0000","last_undegraded":"2023-11-13T10:55:30.897711+0000","last_fullsized":"2023-11-13T10:55:30.897711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:16.740806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897637+0000","last_change":"2023-11-13T10:55:25.851131+0000","last_active":"2023-11-13T10:55:30.897637+0000","last_peered":"2023-11-13T10:55:30.897637+0000","last_clean":"2023-11-13T10:55:30.897637+0000","last_became_active":"2023-11-13T10:55:25.850827+0000","last_became_peered":"2023-11-13T10:55:25.850827+0000","last_unstale":"2023-11-13T10:55:30.897637+0000","last_undegraded":"2023-11-13T10:55:30.897637+0000","last_fullsized":"2023-11-13T10:55:30.897637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:05.431553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888952+0000","last_change":"2023-11-13T10:55:30.924096+0000","last_active":"2023-11-13T10:55:31.888952+0000","last_peered":"2023-11-13T10:55:31.888952+0000","last_clean":"2023-11-13T10:55:31.888952+0000","last_became_active":"2023-11-13T10:55:30.923375+0000","last_became_peered":"2023-11-13T10:55:30.923375+0000","last_unstale":"2023-11-13T10:55:31.888952+0000","last_undegraded":"2023-11-13T10:55:31.888952+0000","last_fullsized":"2023-11-13T10:55:31.888952+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:18:07.170524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892700+0000","last_change":"2023-11-13T10:55:26.852339+0000","last_active":"2023-11-13T10:55:30.892700+0000","last_peered":"2023-11-13T10:55:30.892700+0000","last_clean":"2023-11-13T10:55:30.892700+0000","last_became_active":"2023-11-13T10:55:26.852036+0000","last_became_peered":"2023-11-13T10:55:26.852036+0000","last_unstale":"2023-11-13T10:55:30.892700+0000","last_undegraded":"2023-11-13T10:55:30.892700+0000","last_fullsized":"2023-11-13T10:55:30.892700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:41:12.172895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898608+0000","last_change":"2023-11-13T10:55:25.854134+0000","last_active":"2023-11-13T10:55:30.898608+0000","last_peered":"2023-11-13T10:55:30.898608+0000","last_clean":"2023-11-13T10:55:30.898608+0000","last_became_active":"2023-11-13T10:55:25.853195+0000","last_became_peered":"2023-11-13T10:55:25.853195+0000","last_unstale":"2023-11-13T10:55:30.898608+0000","last_undegraded":"2023-11-13T10:55:30.898608+0000","last_fullsized":"2023-11-13T10:55:30.898608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:09.162069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.888654+0000","last_change":"2023-11-13T10:55:30.914433+0000","last_active":"2023-11-13T10:55:31.888654+0000","last_peered":"2023-11-13T10:55:31.888654+0000","last_clean":"2023-11-13T10:55:31.888654+0000","last_became_active":"2023-11-13T10:55:30.913947+0000","last_became_peered":"2023-11-13T10:55:30.913947+0000","last_unstale":"2023-11-13T10:55:31.888654+0000","last_undegraded":"2023-11-13T10:55:31.888654+0000","last_fullsized":"2023-11-13T10:55:31.888654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:22.321232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899143+0000","last_change":"2023-11-13T10:55:26.866642+0000","last_active":"2023-11-13T10:55:30.899143+0000","last_peered":"2023-11-13T10:55:30.899143+0000","last_clean":"2023-11-13T10:55:30.899143+0000","last_became_active":"2023-11-13T10:55:26.866294+0000","last_became_peered":"2023-11-13T10:55:26.866294+0000","last_unstale":"2023-11-13T10:55:30.899143+0000","last_undegraded":"2023-11-13T10:55:30.899143+0000","last_fullsized":"2023-11-13T10:55:30.899143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:01.842453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898826+0000","last_change":"2023-11-13T10:55:25.848632+0000","last_active":"2023-11-13T10:55:30.898826+0000","last_peered":"2023-11-13T10:55:30.898826+0000","last_clean":"2023-11-13T10:55:30.898826+0000","last_became_active":"2023-11-13T10:55:25.848324+0000","last_became_peered":"2023-11-13T10:55:25.848324+0000","last_unstale":"2023-11-13T10:55:30.898826+0000","last_undegraded":"2023-11-13T10:55:30.898826+0000","last_fullsized":"2023-11-13T10:55:30.898826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:30.550401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905683+0000","last_change":"2023-11-13T10:55:30.931468+0000","last_active":"2023-11-13T10:55:31.905683+0000","last_peered":"2023-11-13T10:55:31.905683+0000","last_clean":"2023-11-13T10:55:31.905683+0000","last_became_active":"2023-11-13T10:55:30.930565+0000","last_became_peered":"2023-11-13T10:55:30.930565+0000","last_unstale":"2023-11-13T10:55:31.905683+0000","last_undegraded":"2023-11-13T10:55:31.905683+0000","last_fullsized":"2023-11-13T10:55:31.905683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:47.860796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912956+0000","last_change":"2023-11-13T10:55:26.866047+0000","last_active":"2023-11-13T10:55:30.912956+0000","last_peered":"2023-11-13T10:55:30.912956+0000","last_clean":"2023-11-13T10:55:30.912956+0000","last_became_active":"2023-11-13T10:55:26.865641+0000","last_became_peered":"2023-11-13T10:55:26.865641+0000","last_unstale":"2023-11-13T10:55:30.912956+0000","last_undegraded":"2023-11-13T10:55:30.912956+0000","last_fullsized":"2023-11-13T10:55:30.912956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:12.028508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911415+0000","last_change":"2023-11-13T10:55:25.860552+0000","last_active":"2023-11-13T10:55:30.911415+0000","last_peered":"2023-11-13T10:55:30.911415+0000","last_clean":"2023-11-13T10:55:30.911415+0000","last_became_active":"2023-11-13T10:55:25.859833+0000","last_became_peered":"2023-11-13T10:55:25.859833+0000","last_unstale":"2023-11-13T10:55:30.911415+0000","last_undegraded":"2023-11-13T10:55:30.911415+0000","last_fullsized":"2023-11-13T10:55:30.911415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:03:57.732657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883273+0000","last_change":"2023-11-13T10:55:30.926972+0000","last_active":"2023-11-13T10:55:31.883273+0000","last_peered":"2023-11-13T10:55:31.883273+0000","last_clean":"2023-11-13T10:55:31.883273+0000","last_became_active":"2023-11-13T10:55:30.926026+0000","last_became_peered":"2023-11-13T10:55:30.926026+0000","last_unstale":"2023-11-13T10:55:31.883273+0000","last_undegraded":"2023-11-13T10:55:31.883273+0000","last_fullsized":"2023-11-13T10:55:31.883273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:41:10.341680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897975+0000","last_change":"2023-11-13T10:55:26.860060+0000","last_active":"2023-11-13T10:55:30.897975+0000","last_peered":"2023-11-13T10:55:30.897975+0000","last_clean":"2023-11-13T10:55:30.897975+0000","last_became_active":"2023-11-13T10:55:26.859793+0000","last_became_peered":"2023-11-13T10:55:26.859793+0000","last_unstale":"2023-11-13T10:55:30.897975+0000","last_undegraded":"2023-11-13T10:55:30.897975+0000","last_fullsized":"2023-11-13T10:55:30.897975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:13.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:45.351786+0000","last_change":"2023-11-13T10:55:25.857501+0000","last_active":"2023-11-13T10:55:45.351786+0000","last_peered":"2023-11-13T10:55:45.351786+0000","last_clean":"2023-11-13T10:55:45.351786+0000","last_became_active":"2023-11-13T10:55:25.853910+0000","last_became_peered":"2023-11-13T10:55:25.853910+0000","last_unstale":"2023-11-13T10:55:45.351786+0000","last_undegraded":"2023-11-13T10:55:45.351786+0000","last_fullsized":"2023-11-13T10:55:45.351786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:41:11.160621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883442+0000","last_change":"2023-11-13T10:55:30.921872+0000","last_active":"2023-11-13T10:55:31.883442+0000","last_peered":"2023-11-13T10:55:31.883442+0000","last_clean":"2023-11-13T10:55:31.883442+0000","last_became_active":"2023-11-13T10:55:30.921430+0000","last_became_peered":"2023-11-13T10:55:30.921430+0000","last_unstale":"2023-11-13T10:55:31.883442+0000","last_undegraded":"2023-11-13T10:55:31.883442+0000","last_fullsized":"2023-11-13T10:55:31.883442+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:25.376399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899011+0000","last_change":"2023-11-13T10:55:26.870526+0000","last_active":"2023-11-13T10:55:30.899011+0000","last_peered":"2023-11-13T10:55:30.899011+0000","last_clean":"2023-11-13T10:55:30.899011+0000","last_became_active":"2023-11-13T10:55:26.870044+0000","last_became_peered":"2023-11-13T10:55:26.870044+0000","last_unstale":"2023-11-13T10:55:30.899011+0000","last_undegraded":"2023-11-13T10:55:30.899011+0000","last_fullsized":"2023-11-13T10:55:30.899011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:05:27.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.915525+0000","last_change":"2023-11-13T10:55:25.863136+0000","last_active":"2023-11-13T10:55:30.915525+0000","last_peered":"2023-11-13T10:55:30.915525+0000","last_clean":"2023-11-13T10:55:30.915525+0000","last_became_active":"2023-11-13T10:55:25.862731+0000","last_became_peered":"2023-11-13T10:55:25.862731+0000","last_unstale":"2023-11-13T10:55:30.915525+0000","last_undegraded":"2023-11-13T10:55:30.915525+0000","last_fullsized":"2023-11-13T10:55:30.915525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:50.405195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.905039+0000","last_change":"2023-11-13T10:55:30.920930+0000","last_active":"2023-11-13T10:55:31.905039+0000","last_peered":"2023-11-13T10:55:31.905039+0000","last_clean":"2023-11-13T10:55:31.905039+0000","last_became_active":"2023-11-13T10:55:30.920498+0000","last_became_peered":"2023-11-13T10:55:30.920498+0000","last_unstale":"2023-11-13T10:55:31.905039+0000","last_undegraded":"2023-11-13T10:55:31.905039+0000","last_fullsized":"2023-11-13T10:55:31.905039+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:49.941362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912962+0000","last_change":"2023-11-13T10:55:26.860086+0000","last_active":"2023-11-13T10:55:30.912962+0000","last_peered":"2023-11-13T10:55:30.912962+0000","last_clean":"2023-11-13T10:55:30.912962+0000","last_became_active":"2023-11-13T10:55:26.859816+0000","last_became_peered":"2023-11-13T10:55:26.859816+0000","last_unstale":"2023-11-13T10:55:30.912962+0000","last_undegraded":"2023-11-13T10:55:30.912962+0000","last_fullsized":"2023-11-13T10:55:30.912962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902678+0000","last_change":"2023-11-13T10:55:25.846097+0000","last_active":"2023-11-13T10:55:30.902678+0000","last_peered":"2023-11-13T10:55:30.902678+0000","last_clean":"2023-11-13T10:55:30.902678+0000","last_became_active":"2023-11-13T10:55:25.845770+0000","last_became_peered":"2023-11-13T10:55:25.845770+0000","last_unstale":"2023-11-13T10:55:30.902678+0000","last_undegraded":"2023-11-13T10:55:30.902678+0000","last_fullsized":"2023-11-13T10:55:30.902678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:16.023464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.906043+0000","last_change":"2023-11-13T10:55:30.925552+0000","last_active":"2023-11-13T10:55:31.906043+0000","last_peered":"2023-11-13T10:55:31.906043+0000","last_clean":"2023-11-13T10:55:31.906043+0000","last_became_active":"2023-11-13T10:55:30.925120+0000","last_became_peered":"2023-11-13T10:55:30.925120+0000","last_unstale":"2023-11-13T10:55:31.906043+0000","last_undegraded":"2023-11-13T10:55:31.906043+0000","last_fullsized":"2023-11-13T10:55:31.906043+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:54:40.287372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886993+0000","last_change":"2023-11-13T10:55:26.876001+0000","last_active":"2023-11-13T10:55:30.886993+0000","last_peered":"2023-11-13T10:55:30.886993+0000","last_clean":"2023-11-13T10:55:30.886993+0000","last_became_active":"2023-11-13T10:55:26.875523+0000","last_became_peered":"2023-11-13T10:55:26.875523+0000","last_unstale":"2023-11-13T10:55:30.886993+0000","last_undegraded":"2023-11-13T10:55:30.886993+0000","last_fullsized":"2023-11-13T10:55:30.886993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:17.901362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896671+0000","last_change":"2023-11-13T10:55:25.848415+0000","last_active":"2023-11-13T10:55:30.896671+0000","last_peered":"2023-11-13T10:55:30.896671+0000","last_clean":"2023-11-13T10:55:30.896671+0000","last_became_active":"2023-11-13T10:55:25.848102+0000","last_became_peered":"2023-11-13T10:55:25.848102+0000","last_unstale":"2023-11-13T10:55:30.896671+0000","last_undegraded":"2023-11-13T10:55:30.896671+0000","last_fullsized":"2023-11-13T10:55:30.896671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T10:56:29.442219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.883284+0000","last_change":"2023-11-13T10:55:30.918365+0000","last_active":"2023-11-13T10:55:31.883284+0000","last_peered":"2023-11-13T10:55:31.883284+0000","last_clean":"2023-11-13T10:55:31.883284+0000","last_became_active":"2023-11-13T10:55:30.917240+0000","last_became_peered":"2023-11-13T10:55:30.917240+0000","last_unstale":"2023-11-13T10:55:31.883284+0000","last_undegraded":"2023-11-13T10:55:31.883284+0000","last_fullsized":"2023-11-13T10:55:31.883284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:08:10.095798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884505+0000","last_change":"2023-11-13T10:55:30.924912+0000","last_active":"2023-11-13T10:55:31.884505+0000","last_peered":"2023-11-13T10:55:31.884505+0000","last_clean":"2023-11-13T10:55:31.884505+0000","last_became_active":"2023-11-13T10:55:30.919446+0000","last_became_peered":"2023-11-13T10:55:30.919446+0000","last_unstale":"2023-11-13T10:55:31.884505+0000","last_undegraded":"2023-11-13T10:55:31.884505+0000","last_fullsized":"2023-11-13T10:55:31.884505+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:37:31.924677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.892009+0000","last_change":"2023-11-13T10:55:26.852233+0000","last_active":"2023-11-13T10:55:30.892009+0000","last_peered":"2023-11-13T10:55:30.892009+0000","last_clean":"2023-11-13T10:55:30.892009+0000","last_became_active":"2023-11-13T10:55:26.849611+0000","last_became_peered":"2023-11-13T10:55:26.849611+0000","last_unstale":"2023-11-13T10:55:30.892009+0000","last_undegraded":"2023-11-13T10:55:30.892009+0000","last_fullsized":"2023-11-13T10:55:30.892009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:53.364560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898832+0000","last_change":"2023-11-13T10:55:25.856192+0000","last_active":"2023-11-13T10:55:30.898832+0000","last_peered":"2023-11-13T10:55:30.898832+0000","last_clean":"2023-11-13T10:55:30.898832+0000","last_became_active":"2023-11-13T10:55:25.852741+0000","last_became_peered":"2023-11-13T10:55:25.852741+0000","last_unstale":"2023-11-13T10:55:30.898832+0000","last_undegraded":"2023-11-13T10:55:30.898832+0000","last_fullsized":"2023-11-13T10:55:30.898832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:15.762181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884097+0000","last_change":"2023-11-13T10:55:30.910075+0000","last_active":"2023-11-13T10:55:31.884097+0000","last_peered":"2023-11-13T10:55:31.884097+0000","last_clean":"2023-11-13T10:55:31.884097+0000","last_became_active":"2023-11-13T10:55:30.909468+0000","last_became_peered":"2023-11-13T10:55:30.909468+0000","last_unstale":"2023-11-13T10:55:31.884097+0000","last_undegraded":"2023-11-13T10:55:31.884097+0000","last_fullsized":"2023-11-13T10:55:31.884097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:04:21.045625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903339+0000","last_change":"2023-11-13T10:55:26.856127+0000","last_active":"2023-11-13T10:55:30.903339+0000","last_peered":"2023-11-13T10:55:30.903339+0000","last_clean":"2023-11-13T10:55:30.903339+0000","last_became_active":"2023-11-13T10:55:26.855805+0000","last_became_peered":"2023-11-13T10:55:26.855805+0000","last_unstale":"2023-11-13T10:55:30.903339+0000","last_undegraded":"2023-11-13T10:55:30.903339+0000","last_fullsized":"2023-11-13T10:55:30.903339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:11:13.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911772+0000","last_change":"2023-11-13T10:55:25.859736+0000","last_active":"2023-11-13T10:55:30.911772+0000","last_peered":"2023-11-13T10:55:30.911772+0000","last_clean":"2023-11-13T10:55:30.911772+0000","last_became_active":"2023-11-13T10:55:25.859335+0000","last_became_peered":"2023-11-13T10:55:25.859335+0000","last_unstale":"2023-11-13T10:55:30.911772+0000","last_undegraded":"2023-11-13T10:55:30.911772+0000","last_fullsized":"2023-11-13T10:55:30.911772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:24.061923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898735+0000","last_change":"2023-11-13T10:55:25.847310+0000","last_active":"2023-11-13T10:55:30.898735+0000","last_peered":"2023-11-13T10:55:30.898735+0000","last_clean":"2023-11-13T10:55:30.898735+0000","last_became_active":"2023-11-13T10:55:25.846471+0000","last_became_peered":"2023-11-13T10:55:25.846471+0000","last_unstale":"2023-11-13T10:55:30.898735+0000","last_undegraded":"2023-11-13T10:55:30.898735+0000","last_fullsized":"2023-11-13T10:55:30.898735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:57.678774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894953+0000","last_change":"2023-11-13T10:55:26.865930+0000","last_active":"2023-11-13T10:55:30.894953+0000","last_peered":"2023-11-13T10:55:30.894953+0000","last_clean":"2023-11-13T10:55:30.894953+0000","last_became_active":"2023-11-13T10:55:26.865603+0000","last_became_peered":"2023-11-13T10:55:26.865603+0000","last_unstale":"2023-11-13T10:55:30.894953+0000","last_undegraded":"2023-11-13T10:55:30.894953+0000","last_fullsized":"2023-11-13T10:55:30.894953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:31.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.884103+0000","last_change":"2023-11-13T10:55:30.922058+0000","last_active":"2023-11-13T10:55:31.884103+0000","last_peered":"2023-11-13T10:55:31.884103+0000","last_clean":"2023-11-13T10:55:31.884103+0000","last_became_active":"2023-11-13T10:55:30.921389+0000","last_became_peered":"2023-11-13T10:55:30.921389+0000","last_unstale":"2023-11-13T10:55:31.884103+0000","last_undegraded":"2023-11-13T10:55:31.884103+0000","last_fullsized":"2023-11-13T10:55:31.884103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:54.585715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909071+0000","last_change":"2023-11-13T10:55:25.859601+0000","last_active":"2023-11-13T10:55:30.909071+0000","last_peered":"2023-11-13T10:55:30.909071+0000","last_clean":"2023-11-13T10:55:30.909071+0000","last_became_active":"2023-11-13T10:55:25.859320+0000","last_became_peered":"2023-11-13T10:55:25.859320+0000","last_unstale":"2023-11-13T10:55:30.909071+0000","last_undegraded":"2023-11-13T10:55:30.909071+0000","last_fullsized":"2023-11-13T10:55:30.909071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:24.791735+0000","last_clean_scrub_stamp":"2023-11-13T10:55:24.791735+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:21.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.901962+0000","last_change":"2023-11-13T10:55:26.844138+0000","last_active":"2023-11-13T10:55:30.901962+0000","last_peered":"2023-11-13T10:55:30.901962+0000","last_clean":"2023-11-13T10:55:30.901962+0000","last_became_active":"2023-11-13T10:55:26.843850+0000","last_became_peered":"2023-11-13T10:55:26.843850+0000","last_unstale":"2023-11-13T10:55:30.901962+0000","last_undegraded":"2023-11-13T10:55:30.901962+0000","last_fullsized":"2023-11-13T10:55:30.901962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:16:43.105401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-13T10:55:31.907248+0000","last_change":"2023-11-13T10:55:30.929718+0000","last_active":"2023-11-13T10:55:31.907248+0000","last_peered":"2023-11-13T10:55:31.907248+0000","last_clean":"2023-11-13T10:55:31.907248+0000","last_became_active":"2023-11-13T10:55:30.929223+0000","last_became_peered":"2023-11-13T10:55:30.929223+0000","last_unstale":"2023-11-13T10:55:31.907248+0000","last_undegraded":"2023-11-13T10:55:31.907248+0000","last_fullsized":"2023-11-13T10:55:31.907248+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:29.864861+0000","last_clean_scrub_stamp":"2023-11-13T10:55:29.864861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:28:22.239712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886249+0000","last_change":"2023-11-13T10:55:26.858768+0000","last_active":"2023-11-13T10:55:30.886249+0000","last_peered":"2023-11-13T10:55:30.886249+0000","last_clean":"2023-11-13T10:55:30.886249+0000","last_became_active":"2023-11-13T10:55:26.858493+0000","last_became_peered":"2023-11-13T10:55:26.858493+0000","last_unstale":"2023-11-13T10:55:30.886249+0000","last_undegraded":"2023-11-13T10:55:30.886249+0000","last_fullsized":"2023-11-13T10:55:30.886249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:30:25.074197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911186+0000","last_change":"2023-11-13T10:55:26.866587+0000","last_active":"2023-11-13T10:55:30.911186+0000","last_peered":"2023-11-13T10:55:30.911186+0000","last_clean":"2023-11-13T10:55:30.911186+0000","last_became_active":"2023-11-13T10:55:26.866277+0000","last_became_peered":"2023-11-13T10:55:26.866277+0000","last_unstale":"2023-11-13T10:55:30.911186+0000","last_undegraded":"2023-11-13T10:55:30.911186+0000","last_fullsized":"2023-11-13T10:55:30.911186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:49:42.713290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898545+0000","last_change":"2023-11-13T10:55:26.869354+0000","last_active":"2023-11-13T10:55:30.898545+0000","last_peered":"2023-11-13T10:55:30.898545+0000","last_clean":"2023-11-13T10:55:30.898545+0000","last_became_active":"2023-11-13T10:55:26.869040+0000","last_became_peered":"2023-11-13T10:55:26.869040+0000","last_unstale":"2023-11-13T10:55:30.898545+0000","last_undegraded":"2023-11-13T10:55:30.898545+0000","last_fullsized":"2023-11-13T10:55:30.898545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:50.524863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.908909+0000","last_change":"2023-11-13T10:55:26.861905+0000","last_active":"2023-11-13T10:55:30.908909+0000","last_peered":"2023-11-13T10:55:30.908909+0000","last_clean":"2023-11-13T10:55:30.908909+0000","last_became_active":"2023-11-13T10:55:26.861613+0000","last_became_peered":"2023-11-13T10:55:26.861613+0000","last_unstale":"2023-11-13T10:55:30.908909+0000","last_undegraded":"2023-11-13T10:55:30.908909+0000","last_fullsized":"2023-11-13T10:55:30.908909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:46:31.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.913352+0000","last_change":"2023-11-13T10:55:26.873181+0000","last_active":"2023-11-13T10:55:30.913352+0000","last_peered":"2023-11-13T10:55:30.913352+0000","last_clean":"2023-11-13T10:55:30.913352+0000","last_became_active":"2023-11-13T10:55:26.872868+0000","last_became_peered":"2023-11-13T10:55:26.872868+0000","last_unstale":"2023-11-13T10:55:30.913352+0000","last_undegraded":"2023-11-13T10:55:30.913352+0000","last_fullsized":"2023-11-13T10:55:30.913352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:37:10.174343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.915906+0000","last_change":"2023-11-13T10:55:26.855441+0000","last_active":"2023-11-13T10:55:30.915906+0000","last_peered":"2023-11-13T10:55:30.915906+0000","last_clean":"2023-11-13T10:55:30.915906+0000","last_became_active":"2023-11-13T10:55:26.855101+0000","last_became_peered":"2023-11-13T10:55:26.855101+0000","last_unstale":"2023-11-13T10:55:30.915906+0000","last_undegraded":"2023-11-13T10:55:30.915906+0000","last_fullsized":"2023-11-13T10:55:30.915906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:53.364560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.886903+0000","last_change":"2023-11-13T10:55:26.864741+0000","last_active":"2023-11-13T10:55:30.886903+0000","last_peered":"2023-11-13T10:55:30.886903+0000","last_clean":"2023-11-13T10:55:30.886903+0000","last_became_active":"2023-11-13T10:55:26.863408+0000","last_became_peered":"2023-11-13T10:55:26.863408+0000","last_unstale":"2023-11-13T10:55:30.886903+0000","last_undegraded":"2023-11-13T10:55:30.886903+0000","last_fullsized":"2023-11-13T10:55:30.886903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:12.028508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912752+0000","last_change":"2023-11-13T10:55:26.856400+0000","last_active":"2023-11-13T10:55:30.912752+0000","last_peered":"2023-11-13T10:55:30.912752+0000","last_clean":"2023-11-13T10:55:30.912752+0000","last_became_active":"2023-11-13T10:55:26.853838+0000","last_became_peered":"2023-11-13T10:55:26.853838+0000","last_unstale":"2023-11-13T10:55:30.912752+0000","last_undegraded":"2023-11-13T10:55:30.912752+0000","last_fullsized":"2023-11-13T10:55:30.912752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:45.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898736+0000","last_change":"2023-11-13T10:55:26.854049+0000","last_active":"2023-11-13T10:55:30.898736+0000","last_peered":"2023-11-13T10:55:30.898736+0000","last_clean":"2023-11-13T10:55:30.898736+0000","last_became_active":"2023-11-13T10:55:26.853767+0000","last_became_peered":"2023-11-13T10:55:26.853767+0000","last_unstale":"2023-11-13T10:55:30.898736+0000","last_undegraded":"2023-11-13T10:55:30.898736+0000","last_fullsized":"2023-11-13T10:55:30.898736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:01:21.315547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903782+0000","last_change":"2023-11-13T10:55:26.855717+0000","last_active":"2023-11-13T10:55:30.903782+0000","last_peered":"2023-11-13T10:55:30.903782+0000","last_clean":"2023-11-13T10:55:30.903782+0000","last_became_active":"2023-11-13T10:55:26.853617+0000","last_became_peered":"2023-11-13T10:55:26.853617+0000","last_unstale":"2023-11-13T10:55:30.903782+0000","last_undegraded":"2023-11-13T10:55:30.903782+0000","last_fullsized":"2023-11-13T10:55:30.903782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.895441+0000","last_change":"2023-11-13T10:55:26.857625+0000","last_active":"2023-11-13T10:55:30.895441+0000","last_peered":"2023-11-13T10:55:30.895441+0000","last_clean":"2023-11-13T10:55:30.895441+0000","last_became_active":"2023-11-13T10:55:26.857214+0000","last_became_peered":"2023-11-13T10:55:26.857214+0000","last_unstale":"2023-11-13T10:55:30.895441+0000","last_undegraded":"2023-11-13T10:55:30.895441+0000","last_fullsized":"2023-11-13T10:55:30.895441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:06.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.894989+0000","last_change":"2023-11-13T10:55:26.870935+0000","last_active":"2023-11-13T10:55:30.894989+0000","last_peered":"2023-11-13T10:55:30.894989+0000","last_clean":"2023-11-13T10:55:30.894989+0000","last_became_active":"2023-11-13T10:55:26.870322+0000","last_became_peered":"2023-11-13T10:55:26.870322+0000","last_unstale":"2023-11-13T10:55:30.894989+0000","last_undegraded":"2023-11-13T10:55:30.894989+0000","last_fullsized":"2023-11-13T10:55:30.894989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:17.035738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900412+0000","last_change":"2023-11-13T10:55:26.867707+0000","last_active":"2023-11-13T10:55:30.900412+0000","last_peered":"2023-11-13T10:55:30.900412+0000","last_clean":"2023-11-13T10:55:30.900412+0000","last_became_active":"2023-11-13T10:55:26.864751+0000","last_became_peered":"2023-11-13T10:55:26.864751+0000","last_unstale":"2023-11-13T10:55:30.900412+0000","last_undegraded":"2023-11-13T10:55:30.900412+0000","last_fullsized":"2023-11-13T10:55:30.900412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:43.799944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898242+0000","last_change":"2023-11-13T10:55:26.855923+0000","last_active":"2023-11-13T10:55:30.898242+0000","last_peered":"2023-11-13T10:55:30.898242+0000","last_clean":"2023-11-13T10:55:30.898242+0000","last_became_active":"2023-11-13T10:55:26.855638+0000","last_became_peered":"2023-11-13T10:55:26.855638+0000","last_unstale":"2023-11-13T10:55:30.898242+0000","last_undegraded":"2023-11-13T10:55:30.898242+0000","last_fullsized":"2023-11-13T10:55:30.898242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:40:26.611005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.916348+0000","last_change":"2023-11-13T10:55:26.870707+0000","last_active":"2023-11-13T10:55:30.916348+0000","last_peered":"2023-11-13T10:55:30.916348+0000","last_clean":"2023-11-13T10:55:30.916348+0000","last_became_active":"2023-11-13T10:55:26.870417+0000","last_became_peered":"2023-11-13T10:55:26.870417+0000","last_unstale":"2023-11-13T10:55:30.916348+0000","last_undegraded":"2023-11-13T10:55:30.916348+0000","last_fullsized":"2023-11-13T10:55:30.916348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:24:16.048814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.899902+0000","last_change":"2023-11-13T10:55:26.873122+0000","last_active":"2023-11-13T10:55:30.899902+0000","last_peered":"2023-11-13T10:55:30.899902+0000","last_clean":"2023-11-13T10:55:30.899902+0000","last_became_active":"2023-11-13T10:55:26.872823+0000","last_became_peered":"2023-11-13T10:55:26.872823+0000","last_unstale":"2023-11-13T10:55:30.899902+0000","last_undegraded":"2023-11-13T10:55:30.899902+0000","last_fullsized":"2023-11-13T10:55:30.899902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:09:58.900197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898638+0000","last_change":"2023-11-13T10:55:26.869223+0000","last_active":"2023-11-13T10:55:30.898638+0000","last_peered":"2023-11-13T10:55:30.898638+0000","last_clean":"2023-11-13T10:55:30.898638+0000","last_became_active":"2023-11-13T10:55:26.868919+0000","last_became_peered":"2023-11-13T10:55:26.868919+0000","last_unstale":"2023-11-13T10:55:30.898638+0000","last_undegraded":"2023-11-13T10:55:30.898638+0000","last_fullsized":"2023-11-13T10:55:30.898638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:57:13.609895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903525+0000","last_change":"2023-11-13T10:55:26.865259+0000","last_active":"2023-11-13T10:55:30.903525+0000","last_peered":"2023-11-13T10:55:30.903525+0000","last_clean":"2023-11-13T10:55:30.903525+0000","last_became_active":"2023-11-13T10:55:26.864675+0000","last_became_peered":"2023-11-13T10:55:26.864675+0000","last_unstale":"2023-11-13T10:55:30.903525+0000","last_undegraded":"2023-11-13T10:55:30.903525+0000","last_fullsized":"2023-11-13T10:55:30.903525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:17.035738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.912854+0000","last_change":"2023-11-13T10:55:26.863002+0000","last_active":"2023-11-13T10:55:30.912854+0000","last_peered":"2023-11-13T10:55:30.912854+0000","last_clean":"2023-11-13T10:55:30.912854+0000","last_became_active":"2023-11-13T10:55:26.862257+0000","last_became_peered":"2023-11-13T10:55:26.862257+0000","last_unstale":"2023-11-13T10:55:30.912854+0000","last_undegraded":"2023-11-13T10:55:30.912854+0000","last_fullsized":"2023-11-13T10:55:30.912854+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:17.901362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.914921+0000","last_change":"2023-11-13T10:55:26.872307+0000","last_active":"2023-11-13T10:55:30.914921+0000","last_peered":"2023-11-13T10:55:30.914921+0000","last_clean":"2023-11-13T10:55:30.914921+0000","last_became_active":"2023-11-13T10:55:26.872087+0000","last_became_peered":"2023-11-13T10:55:26.872087+0000","last_unstale":"2023-11-13T10:55:30.914921+0000","last_undegraded":"2023-11-13T10:55:30.914921+0000","last_fullsized":"2023-11-13T10:55:30.914921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:14:22.528830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.900326+0000","last_change":"2023-11-13T10:55:26.872483+0000","last_active":"2023-11-13T10:55:30.900326+0000","last_peered":"2023-11-13T10:55:30.900326+0000","last_clean":"2023-11-13T10:55:30.900326+0000","last_became_active":"2023-11-13T10:55:26.872186+0000","last_became_peered":"2023-11-13T10:55:26.872186+0000","last_unstale":"2023-11-13T10:55:30.900326+0000","last_undegraded":"2023-11-13T10:55:30.900326+0000","last_fullsized":"2023-11-13T10:55:30.900326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:24:47.501708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.897861+0000","last_change":"2023-11-13T10:55:26.862513+0000","last_active":"2023-11-13T10:55:30.897861+0000","last_peered":"2023-11-13T10:55:30.897861+0000","last_clean":"2023-11-13T10:55:30.897861+0000","last_became_active":"2023-11-13T10:55:26.862167+0000","last_became_peered":"2023-11-13T10:55:26.862167+0000","last_unstale":"2023-11-13T10:55:30.897861+0000","last_undegraded":"2023-11-13T10:55:30.897861+0000","last_fullsized":"2023-11-13T10:55:30.897861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:45:29.179442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902597+0000","last_change":"2023-11-13T10:55:26.861425+0000","last_active":"2023-11-13T10:55:30.902597+0000","last_peered":"2023-11-13T10:55:30.902597+0000","last_clean":"2023-11-13T10:55:30.902597+0000","last_became_active":"2023-11-13T10:55:26.861132+0000","last_became_peered":"2023-11-13T10:55:26.861132+0000","last_unstale":"2023-11-13T10:55:30.902597+0000","last_undegraded":"2023-11-13T10:55:30.902597+0000","last_fullsized":"2023-11-13T10:55:30.902597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:29.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.896561+0000","last_change":"2023-11-13T10:55:26.858332+0000","last_active":"2023-11-13T10:55:30.896561+0000","last_peered":"2023-11-13T10:55:30.896561+0000","last_clean":"2023-11-13T10:55:30.896561+0000","last_became_active":"2023-11-13T10:55:26.858052+0000","last_became_peered":"2023-11-13T10:55:26.858052+0000","last_unstale":"2023-11-13T10:55:30.896561+0000","last_undegraded":"2023-11-13T10:55:30.896561+0000","last_fullsized":"2023-11-13T10:55:30.896561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.903167+0000","last_change":"2023-11-13T10:55:26.863579+0000","last_active":"2023-11-13T10:55:30.903167+0000","last_peered":"2023-11-13T10:55:30.903167+0000","last_clean":"2023-11-13T10:55:30.903167+0000","last_became_active":"2023-11-13T10:55:26.863247+0000","last_became_peered":"2023-11-13T10:55:26.863247+0000","last_unstale":"2023-11-13T10:55:30.903167+0000","last_undegraded":"2023-11-13T10:55:30.903167+0000","last_fullsized":"2023-11-13T10:55:30.903167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.774455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.898503+0000","last_change":"2023-11-13T10:55:26.856219+0000","last_active":"2023-11-13T10:55:30.898503+0000","last_peered":"2023-11-13T10:55:30.898503+0000","last_clean":"2023-11-13T10:55:30.898503+0000","last_became_active":"2023-11-13T10:55:26.855933+0000","last_became_peered":"2023-11-13T10:55:26.855933+0000","last_unstale":"2023-11-13T10:55:30.898503+0000","last_undegraded":"2023-11-13T10:55:30.898503+0000","last_fullsized":"2023-11-13T10:55:30.898503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:49:58.691048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887319+0000","last_change":"2023-11-13T10:55:26.853802+0000","last_active":"2023-11-13T10:55:30.887319+0000","last_peered":"2023-11-13T10:55:30.887319+0000","last_clean":"2023-11-13T10:55:30.887319+0000","last_became_active":"2023-11-13T10:55:26.853463+0000","last_became_peered":"2023-11-13T10:55:26.853463+0000","last_unstale":"2023-11-13T10:55:30.887319+0000","last_undegraded":"2023-11-13T10:55:30.887319+0000","last_fullsized":"2023-11-13T10:55:30.887319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:07:06.443827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902537+0000","last_change":"2023-11-13T10:55:26.865339+0000","last_active":"2023-11-13T10:55:30.902537+0000","last_peered":"2023-11-13T10:55:30.902537+0000","last_clean":"2023-11-13T10:55:30.902537+0000","last_became_active":"2023-11-13T10:55:26.865069+0000","last_became_peered":"2023-11-13T10:55:26.865069+0000","last_unstale":"2023-11-13T10:55:30.902537+0000","last_undegraded":"2023-11-13T10:55:30.902537+0000","last_fullsized":"2023-11-13T10:55:30.902537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:50.524863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.887342+0000","last_change":"2023-11-13T10:55:26.857434+0000","last_active":"2023-11-13T10:55:30.887342+0000","last_peered":"2023-11-13T10:55:30.887342+0000","last_clean":"2023-11-13T10:55:30.887342+0000","last_became_active":"2023-11-13T10:55:26.857164+0000","last_became_peered":"2023-11-13T10:55:26.857164+0000","last_unstale":"2023-11-13T10:55:30.887342+0000","last_undegraded":"2023-11-13T10:55:30.887342+0000","last_fullsized":"2023-11-13T10:55:30.887342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:52:41.188500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.909271+0000","last_change":"2023-11-13T10:55:26.845954+0000","last_active":"2023-11-13T10:55:30.909271+0000","last_peered":"2023-11-13T10:55:30.909271+0000","last_clean":"2023-11-13T10:55:30.909271+0000","last_became_active":"2023-11-13T10:55:26.845505+0000","last_became_peered":"2023-11-13T10:55:26.845505+0000","last_unstale":"2023-11-13T10:55:30.909271+0000","last_undegraded":"2023-11-13T10:55:30.909271+0000","last_fullsized":"2023-11-13T10:55:30.909271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:43.799944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.911061+0000","last_change":"2023-11-13T10:55:26.857565+0000","last_active":"2023-11-13T10:55:30.911061+0000","last_peered":"2023-11-13T10:55:30.911061+0000","last_clean":"2023-11-13T10:55:30.911061+0000","last_became_active":"2023-11-13T10:55:26.857291+0000","last_became_peered":"2023-11-13T10:55:26.857291+0000","last_unstale":"2023-11-13T10:55:30.911061+0000","last_undegraded":"2023-11-13T10:55:30.911061+0000","last_fullsized":"2023-11-13T10:55:30.911061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:38:29.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-13T10:55:30.902086+0000","last_change":"2023-11-13T10:55:26.858620+0000","last_active":"2023-11-13T10:55:30.902086+0000","last_peered":"2023-11-13T10:55:30.902086+0000","last_clean":"2023-11-13T10:55:30.902086+0000","last_became_active":"2023-11-13T10:55:26.858348+0000","last_became_peered":"2023-11-13T10:55:26.858348+0000","last_unstale":"2023-11-13T10:55:30.902086+0000","last_undegraded":"2023-11-13T10:55:30.902086+0000","last_fullsized":"2023-11-13T10:55:30.902086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:25.804009+0000","last_clean_scrub_stamp":"2023-11-13T10:55:25.804009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:37:45.880510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]}]},{"osd":5,"up_from":15,"seq":64424509453,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":695602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]}]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":109655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.627}]}]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":105724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":108983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":695580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2570000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3039999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7849999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T10:56:02.625 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-13T10:56:02.625 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-13T10:56:02.625 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-13T10:56:02.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-13T10:56:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.880+0000 7fd5fb060640 1 Processor -- start 2023-11-13T10:56:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.880+0000 7fd5fb060640 1 -- start start 2023-11-13T10:56:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5fb060640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5f41584d0 0x7fd5f40ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5fb060640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5f41588b0 0x7fd5f40adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5fb060640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f40712d0 0x7fd5f4071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5fb060640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f415b080 con 0x7fd5f41588b0 2023-11-13T10:56:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5fb060640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f4071cc0 con 0x7fd5f41584d0 2023-11-13T10:56:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5fb060640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f415b2c0 con 0x7fd5f40712d0 2023-11-13T10:56:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5f985d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5f41588b0 0x7fd5f40adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5f985d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5f41588b0 0x7fd5f40adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56318/0 (socket says 172.21.15.16:56318) 2023-11-13T10:56:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5f985d640 1 -- 172.21.15.16:0/2181901017 learned_addr learned my addr 172.21.15.16:0/2181901017 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5f985d640 1 -- 172.21.15.16:0/2181901017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f40712d0 msgr2=0x7fd5f4071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5f985d640 1 --2- 172.21.15.16:0/2181901017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f40712d0 0x7fd5f4071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5fa05e640 1 --2- 172.21.15.16:0/2181901017 >> v2:172.21.15.184:3300/0 conn(0x7fd5f41584d0 0x7fd5f40ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5f985d640 1 -- 172.21.15.16:0/2181901017 >> v2:172.21.15.184:3300/0 conn(0x7fd5f41584d0 msgr2=0x7fd5f40ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5f985d640 1 --2- 172.21.15.16:0/2181901017 >> v2:172.21.15.184:3300/0 conn(0x7fd5f41584d0 0x7fd5f40ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5f985d640 1 -- 172.21.15.16:0/2181901017 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f415b5a0 con 0x7fd5f41588b0 2023-11-13T10:56:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.882+0000 7fd5fa05e640 1 --2- 172.21.15.16:0/2181901017 >> v2:172.21.15.184:3300/0 conn(0x7fd5f41584d0 0x7fd5f40ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.884+0000 7fd5f985d640 1 --2- 172.21.15.16:0/2181901017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f41588b0 0x7fd5f40adc70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd5f406dec0 tx=0x7fd5e402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabcf1602a25faba server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.884+0000 7fd5f905c640 1 -- 172.21.15.16:0/2181901017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4031860 con 0x7fd5f41588b0 2023-11-13T10:56:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.884+0000 7fd5f905c640 1 -- 172.21.15.16:0/2181901017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e40319c0 con 0x7fd5f41588b0 2023-11-13T10:56:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.884+0000 7fd5f905c640 1 -- 172.21.15.16:0/2181901017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4031d50 con 0x7fd5f41588b0 2023-11-13T10:56:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.884+0000 7fd5fb060640 1 -- 172.21.15.16:0/2181901017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f41588b0 msgr2=0x7fd5f40adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.884+0000 7fd5fb060640 1 --2- 172.21.15.16:0/2181901017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f41588b0 0x7fd5f40adc70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd5f406dec0 tx=0x7fd5e402b900 comp rx=0 tx=0).stop 2023-11-13T10:56:02.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.885+0000 7fd5fb060640 1 -- 172.21.15.16:0/2181901017 shutdown_connections 2023-11-13T10:56:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.885+0000 7fd5fb060640 1 --2- 172.21.15.16:0/2181901017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f40712d0 0x7fd5f4071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.885+0000 7fd5fb060640 1 --2- 172.21.15.16:0/2181901017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f41588b0 0x7fd5f40adc70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.885+0000 7fd5fb060640 1 --2- 172.21.15.16:0/2181901017 >> v2:172.21.15.184:3300/0 conn(0x7fd5f41584d0 0x7fd5f40ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.885+0000 7fd5fb060640 1 -- 172.21.15.16:0/2181901017 >> 172.21.15.16:0/2181901017 conn(0x7fd5f406a190 msgr2=0x7fd5f406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.885+0000 7fd5fb060640 1 -- 172.21.15.16:0/2181901017 shutdown_connections 2023-11-13T10:56:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.885+0000 7fd5fb060640 1 -- 172.21.15.16:0/2181901017 wait complete. 2023-11-13T10:56:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.886+0000 7fd5fb060640 1 Processor -- start 2023-11-13T10:56:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.887+0000 7fd5fb060640 1 -- start start 2023-11-13T10:56:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fb060640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f40712d0 0x7fd5f4155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fb060640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f41584d0 0x7fd5f41515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fb060640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f41588b0 0x7fd5f415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fb060640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f4153fd0 con 0x7fd5f40712d0 2023-11-13T10:56:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fb060640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f41569f0 con 0x7fd5f41584d0 2023-11-13T10:56:02.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fb060640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f4154270 con 0x7fd5f41588b0 2023-11-13T10:56:02.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa05e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f40712d0 0x7fd5f4155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa05e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f40712d0 0x7fd5f4155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56326/0 (socket says 172.21.15.16:56326) 2023-11-13T10:56:02.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa05e640 1 -- 172.21.15.16:0/1414587039 learned_addr learned my addr 172.21.15.16:0/1414587039 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:02.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa05e640 1 -- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f41588b0 msgr2=0x7fd5f415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa85f640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f41588b0 0x7fd5f415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5f985d640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f41584d0 0x7fd5f41515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa05e640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f41588b0 0x7fd5f415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa05e640 1 -- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f41584d0 msgr2=0x7fd5f41515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa05e640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f41584d0 0x7fd5f41515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa05e640 1 -- 172.21.15.16:0/1414587039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e40094d0 con 0x7fd5f40712d0 2023-11-13T10:56:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5f985d640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f41584d0 0x7fd5f41515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.888+0000 7fd5fa85f640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f41588b0 0x7fd5f415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.889+0000 7fd5fa05e640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f40712d0 0x7fd5f4155ac0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000c7f0 tx=0x7fd5f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:02.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.889+0000 7fd5eaffd640 1 -- 172.21.15.16:0/1414587039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0007a30 con 0x7fd5f40712d0 2023-11-13T10:56:02.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.889+0000 7fd5eaffd640 1 -- 172.21.15.16:0/1414587039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f0007b90 con 0x7fd5f40712d0 2023-11-13T10:56:02.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.889+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f4152560 con 0x7fd5f40712d0 2023-11-13T10:56:02.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.889+0000 7fd5eaffd640 1 -- 172.21.15.16:0/1414587039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000ce40 con 0x7fd5f40712d0 2023-11-13T10:56:02.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.889+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f4156150 con 0x7fd5f40712d0 2023-11-13T10:56:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.891+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f404e4d0 con 0x7fd5f40712d0 2023-11-13T10:56:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.891+0000 7fd5eaffd640 1 -- 172.21.15.16:0/1414587039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd5f00113f0 con 0x7fd5f40712d0 2023-11-13T10:56:02.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.892+0000 7fd5eaffd640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd5d806aec0 0x7fd5d806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.892+0000 7fd5eaffd640 1 -- 172.21.15.16:0/1414587039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd5f0089930 con 0x7fd5f40712d0 2023-11-13T10:56:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.893+0000 7fd5f985d640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd5d806aec0 0x7fd5d806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.894+0000 7fd5f985d640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd5d806aec0 0x7fd5d806d380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4036000 tx=0x7fd5e403c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:02.896+0000 7fd5eaffd640 1 -- 172.21.15.16:0/1414587039 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5f0054860 con 0x7fd5f40712d0 2023-11-13T10:56:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.133+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd5f40b30a0 con 0x7fd5f40712d0 2023-11-13T10:56:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.134+0000 7fd5eaffd640 1 -- 172.21.15.16:0/1414587039 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd5f0057e80 con 0x7fd5f40712d0 2023-11-13T10:56:03.138 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:03.138 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-13T10:56:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.136+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd5d806aec0 msgr2=0x7fd5d806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.136+0000 7fd5fb060640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd5d806aec0 0x7fd5d806d380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4036000 tx=0x7fd5e403c040 comp rx=0 tx=0).stop 2023-11-13T10:56:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.136+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f40712d0 msgr2=0x7fd5f4155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.136+0000 7fd5fb060640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f40712d0 0x7fd5f4155ac0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000c7f0 tx=0x7fd5f000cbf0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.137+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 shutdown_connections 2023-11-13T10:56:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.137+0000 7fd5fb060640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd5d806aec0 0x7fd5d806d380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.137+0000 7fd5fb060640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f41588b0 0x7fd5f415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.137+0000 7fd5fb060640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f41584d0 0x7fd5f41515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.137+0000 7fd5fb060640 1 --2- 172.21.15.16:0/1414587039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f40712d0 0x7fd5f4155ac0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.137+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 >> 172.21.15.16:0/1414587039 conn(0x7fd5f406a190 msgr2=0x7fd5f4158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.137+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 shutdown_connections 2023-11-13T10:56:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.137+0000 7fd5fb060640 1 -- 172.21.15.16:0/1414587039 wait complete. 2023-11-13T10:56:03.159 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-13T10:56:03.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:03.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.375+0000 7fc4498df640 1 Processor -- start 2023-11-13T10:56:03.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.375+0000 7fc4498df640 1 -- start start 2023-11-13T10:56:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.377+0000 7fc4498df640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4440cb190 0x7fc44415b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.377+0000 7fc4498df640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 0x7fc44406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.377+0000 7fc4498df640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc44406a190 0x7fc44406a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.377+0000 7fc4498df640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc444158c20 con 0x7fc4440cb190 2023-11-13T10:56:03.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.377+0000 7fc4498df640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4440b1830 con 0x7fc44406a190 2023-11-13T10:56:03.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.377+0000 7fc4498df640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44406ab10 con 0x7fc4440716b0 2023-11-13T10:56:03.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc4488dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4440cb190 0x7fc44415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc4488dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4440cb190 0x7fc44415b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46680/0 (socket says 172.21.15.16:46680) 2023-11-13T10:56:03.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc4488dd640 1 -- 172.21.15.16:0/996741671 learned_addr learned my addr 172.21.15.16:0/996741671 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:03.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc4490de640 1 --2- 172.21.15.16:0/996741671 >> v2:172.21.15.184:3300/0 conn(0x7fc44406a190 0x7fc44406a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc43bfff640 1 --2- 172.21.15.16:0/996741671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 0x7fc44406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc4488dd640 1 -- 172.21.15.16:0/996741671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 msgr2=0x7fc44406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:03.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc4488dd640 1 --2- 172.21.15.16:0/996741671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 0x7fc44406f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc4488dd640 1 -- 172.21.15.16:0/996741671 >> v2:172.21.15.184:3300/0 conn(0x7fc44406a190 msgr2=0x7fc44406a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:03.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc4488dd640 1 --2- 172.21.15.16:0/996741671 >> v2:172.21.15.184:3300/0 conn(0x7fc44406a190 0x7fc44406a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.378+0000 7fc4488dd640 1 -- 172.21.15.16:0/996741671 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44415b780 con 0x7fc4440cb190 2023-11-13T10:56:03.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.380+0000 7fc4488dd640 1 --2- 172.21.15.16:0/996741671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4440cb190 0x7fc44415b190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc44406dc70 tx=0x7fc44002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cbad1c1e792840 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.380+0000 7fc43b7fe640 1 -- 172.21.15.16:0/996741671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440031870 con 0x7fc4440cb190 2023-11-13T10:56:03.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.380+0000 7fc43b7fe640 1 -- 172.21.15.16:0/996741671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4400319d0 con 0x7fc4440cb190 2023-11-13T10:56:03.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.380+0000 7fc43b7fe640 1 -- 172.21.15.16:0/996741671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440031d60 con 0x7fc4440cb190 2023-11-13T10:56:03.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.381+0000 7fc4498df640 1 -- 172.21.15.16:0/996741671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4440cb190 msgr2=0x7fc44415b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.381+0000 7fc4498df640 1 --2- 172.21.15.16:0/996741671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4440cb190 0x7fc44415b190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc44406dc70 tx=0x7fc44002b910 comp rx=0 tx=0).stop 2023-11-13T10:56:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.381+0000 7fc4498df640 1 -- 172.21.15.16:0/996741671 shutdown_connections 2023-11-13T10:56:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.381+0000 7fc4498df640 1 --2- 172.21.15.16:0/996741671 >> v2:172.21.15.184:3300/0 conn(0x7fc44406a190 0x7fc44406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.381+0000 7fc4498df640 1 --2- 172.21.15.16:0/996741671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 0x7fc44406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.381+0000 7fc4498df640 1 --2- 172.21.15.16:0/996741671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4440cb190 0x7fc44415b190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.381+0000 7fc4498df640 1 -- 172.21.15.16:0/996741671 >> 172.21.15.16:0/996741671 conn(0x7fc4440b04a0 msgr2=0x7fc4440b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.381+0000 7fc4498df640 1 -- 172.21.15.16:0/996741671 shutdown_connections 2023-11-13T10:56:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.381+0000 7fc4498df640 1 -- 172.21.15.16:0/996741671 wait complete. 2023-11-13T10:56:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.383+0000 7fc4498df640 1 Processor -- start 2023-11-13T10:56:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.384+0000 7fc4498df640 1 -- start start 2023-11-13T10:56:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4498df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc44406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4498df640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 0x7fc444145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4498df640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4440cb190 0x7fc4440c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4498df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4441f6bf0 con 0x7fc44406a190 2023-11-13T10:56:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4498df640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44406c420 con 0x7fc4440cb190 2023-11-13T10:56:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4498df640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44406c700 con 0x7fc4440716b0 2023-11-13T10:56:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4488dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc44406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4488dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc44406d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46686/0 (socket says 172.21.15.16:46686) 2023-11-13T10:56:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4488dd640 1 -- 172.21.15.16:0/2040023924 learned_addr learned my addr 172.21.15.16:0/2040023924 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc4490de640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4440cb190 0x7fc4440c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.385+0000 7fc43bfff640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 0x7fc444145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4488dd640 1 -- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 msgr2=0x7fc444145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4488dd640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 0x7fc444145170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4488dd640 1 -- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4440cb190 msgr2=0x7fc4440c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4488dd640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4440cb190 0x7fc4440c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4488dd640 1 -- 172.21.15.16:0/2040023924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4400094d0 con 0x7fc44406a190 2023-11-13T10:56:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4488dd640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc44406d1e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc44002b950 tx=0x7fc4400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4397fa640 1 -- 172.21.15.16:0/2040023924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44003ad30 con 0x7fc44406a190 2023-11-13T10:56:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4397fa640 1 -- 172.21.15.16:0/2040023924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4400443f0 con 0x7fc44406a190 2023-11-13T10:56:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4397fa640 1 -- 172.21.15.16:0/2040023924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400447a0 con 0x7fc44406a190 2023-11-13T10:56:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.386+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44406c9e0 con 0x7fc44406a190 2023-11-13T10:56:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.387+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44406d720 con 0x7fc44406a190 2023-11-13T10:56:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.388+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44406cf00 con 0x7fc44406a190 2023-11-13T10:56:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.389+0000 7fc4397fa640 1 -- 172.21.15.16:0/2040023924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc44004d450 con 0x7fc44406a190 2023-11-13T10:56:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.390+0000 7fc4397fa640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc43006ae40 0x7fc43006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.390+0000 7fc4397fa640 1 -- 172.21.15.16:0/2040023924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc44002d040 con 0x7fc44406a190 2023-11-13T10:56:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.390+0000 7fc43bfff640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc43006ae40 0x7fc43006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.391+0000 7fc43bfff640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc43006ae40 0x7fc43006d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4440c9c10 tx=0x7fc434008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.394+0000 7fc4397fa640 1 -- 172.21.15.16:0/2040023924 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc44406cf00 con 0x7fc44406a190 2023-11-13T10:56:03.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.589+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc44406cf00 con 0x7fc44406a190 2023-11-13T10:56:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.591+0000 7fc4397fa640 1 -- 172.21.15.16:0/2040023924 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7fc44406cf00 con 0x7fc44406a190 2023-11-13T10:56:03.595 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:03.596 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:03.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc43006ae40 msgr2=0x7fc43006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc43006ae40 0x7fc43006d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4440c9c10 tx=0x7fc434008040 comp rx=0 tx=0).stop 2023-11-13T10:56:03.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 msgr2=0x7fc44406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc44406d1e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc44002b950 tx=0x7fc4400046c0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 shutdown_connections 2023-11-13T10:56:03.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc43006ae40 0x7fc43006d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4440cb190 0x7fc4440c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4440716b0 0x7fc444145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 --2- 172.21.15.16:0/2040023924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc44406d1e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 >> 172.21.15.16:0/2040023924 conn(0x7fc4440b04a0 msgr2=0x7fc4440cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 shutdown_connections 2023-11-13T10:56:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.594+0000 7fc4498df640 1 -- 172.21.15.16:0/2040023924 wait complete. 2023-11-13T10:56:03.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:03.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.839+0000 7f66fee0d640 1 Processor -- start 2023-11-13T10:56:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.840+0000 7f66fee0d640 1 -- start start 2023-11-13T10:56:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.841+0000 7f66fee0d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f66f815ce70 0x7f66f80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.841+0000 7f66fee0d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f80ce0e0 con 0x7f66f80cdbb0 2023-11-13T10:56:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.841+0000 7f66fee0d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f80ce3c0 con 0x7f66f80d1850 2023-11-13T10:56:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.841+0000 7f66fee0d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f80ce6a0 con 0x7f66f815ce70 2023-11-13T10:56:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fd60a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f66f80cdbb0 0x7f66f80d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49342/0 (socket says 172.21.15.16:49342) 2023-11-13T10:56:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fd60a640 1 -- 172.21.15.16:0/1535091517 learned_addr learned my addr 172.21.15.16:0/1535091517 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fde0b640 1 --2- 172.21.15.16:0/1535091517 >> v2:172.21.15.184:3300/0 conn(0x7f66f815ce70 0x7f66f80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799378838 0 0) 0x7f66f80ce0e0 con 0x7f66f80cdbb0 2023-11-13T10:56:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66e4003520 con 0x7f66f80cdbb0 2023-11-13T10:56:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3026646507 0 0) 0x7f66e4003520 con 0x7f66f80cdbb0 2023-11-13T10:56:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 >> v2:172.21.15.184:3300/0 conn(0x7f66f815ce70 msgr2=0x7f66f80cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fce09640 1 --2- 172.21.15.16:0/1535091517 >> v2:172.21.15.184:3300/0 conn(0x7f66f815ce70 0x7f66f80cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 >> v1:172.21.15.184:6789/0 conn(0x7f66f80d1850 legacy=0x7f66f80c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.842+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f80ce980 con 0x7f66f80cdbb0 2023-11-13T10:56:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f66e8001670 con 0x7f66f80cdbb0 2023-11-13T10:56:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 >> v1:172.21.15.16:6789/0 conn(0x7f66f80cdbb0 legacy=0x7f66f80d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fce09640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fce09640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66e40047c0 0x7f66e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fce09640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66e4005180 0x7f66e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f80ce6a0 con 0x7f66e40039c0 2023-11-13T10:56:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e400c080 con 0x7f66e40047c0 2023-11-13T10:56:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e400c240 con 0x7f66e4005180 2023-11-13T10:56:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fde0b640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fd60a640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66e40047c0 0x7f66e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fde0b640 1 -- 172.21.15.16:0/1535091517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66e4005180 msgr2=0x7f66e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fde0b640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66e4005180 0x7f66e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fde0b640 1 -- 172.21.15.16:0/1535091517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66e40047c0 msgr2=0x7f66e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.843+0000 7f66fde0b640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66e40047c0 0x7f66e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.844+0000 7f66fde0b640 1 -- 172.21.15.16:0/1535091517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e4003520 con 0x7f66e40039c0 2023-11-13T10:56:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.844+0000 7f66fe60c640 1 -- 172.21.15.16:0/1535091517 reap_dead start 2023-11-13T10:56:03.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.845+0000 7f66fde0b640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66e4003de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009ac0 tx=0x7f66e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca5b27b123c2fd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:03.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.845+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f402c710 con 0x7f66e40039c0 2023-11-13T10:56:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.845+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f402c870 con 0x7f66e40039c0 2023-11-13T10:56:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.845+0000 7f66fce09640 1 -- 172.21.15.16:0/1535091517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f402cc00 con 0x7f66e40039c0 2023-11-13T10:56:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.846+0000 7f66fee0d640 1 -- 172.21.15.16:0/1535091517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 msgr2=0x7f66e4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.846+0000 7f66fee0d640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66e4003de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009ac0 tx=0x7f66e40057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.846+0000 7f66fee0d640 1 -- 172.21.15.16:0/1535091517 shutdown_connections 2023-11-13T10:56:03.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.846+0000 7f66fee0d640 1 --2- 172.21.15.16:0/1535091517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66e4003de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.846+0000 7f66fee0d640 1 -- 172.21.15.16:0/1535091517 >> 172.21.15.16:0/1535091517 conn(0x7f66f808c580 msgr2=0x7f66f809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:03.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.846+0000 7f66fee0d640 1 -- 172.21.15.16:0/1535091517 shutdown_connections 2023-11-13T10:56:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.846+0000 7f66fee0d640 1 -- 172.21.15.16:0/1535091517 wait complete. 2023-11-13T10:56:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.848+0000 7f66fee0d640 1 Processor -- start 2023-11-13T10:56:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.848+0000 7f66fee0d640 1 -- start start 2023-11-13T10:56:03.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fee0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66f80b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fee0d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f80b6f70 0x7f66f80b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fee0d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66f80b73f0 0x7f66f80b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fee0d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f80b5ca0 con 0x7f66e40039c0 2023-11-13T10:56:03.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fee0d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f80b6ae0 con 0x7f66f80b6f70 2023-11-13T10:56:03.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fee0d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f80b5ed0 con 0x7f66f80b73f0 2023-11-13T10:56:03.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fde0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66f80b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fde0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66f80b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46704/0 (socket says 172.21.15.16:46704) 2023-11-13T10:56:03.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fde0b640 1 -- 172.21.15.16:0/2125984735 learned_addr learned my addr 172.21.15.16:0/2125984735 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:03.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fd60a640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f80b6f70 0x7f66f80b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.849+0000 7f66fe60c640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66f80b73f0 0x7f66f80b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66fde0b640 1 -- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66f80b73f0 msgr2=0x7f66f80b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66fde0b640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66f80b73f0 0x7f66f80b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66fde0b640 1 -- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f80b6f70 msgr2=0x7f66f80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66fde0b640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f80b6f70 0x7f66f80b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66fde0b640 1 -- 172.21.15.16:0/2125984735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f80b61b0 con 0x7f66e40039c0 2023-11-13T10:56:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66fd60a640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f80b6f70 0x7f66f80b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66fe60c640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66f80b73f0 0x7f66f80b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66fde0b640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66f80b5760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009ac0 tx=0x7f66f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66ee7fc640 1 -- 172.21.15.16:0/2125984735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4003810 con 0x7f66e40039c0 2023-11-13T10:56:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66ee7fc640 1 -- 172.21.15.16:0/2125984735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f402de90 con 0x7f66e40039c0 2023-11-13T10:56:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f80b6490 con 0x7f66e40039c0 2023-11-13T10:56:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.850+0000 7f66ee7fc640 1 -- 172.21.15.16:0/2125984735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4003b90 con 0x7f66e40039c0 2023-11-13T10:56:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.851+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f814d7c0 con 0x7f66e40039c0 2023-11-13T10:56:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.852+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f80675b0 con 0x7f66e40039c0 2023-11-13T10:56:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.852+0000 7f66ee7fc640 1 -- 172.21.15.16:0/2125984735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66f4003cf0 con 0x7f66e40039c0 2023-11-13T10:56:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.854+0000 7f66ee7fc640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66dc06ae70 0x7f66dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.854+0000 7f66ee7fc640 1 -- 172.21.15.16:0/2125984735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f66f40ab8c0 con 0x7f66e40039c0 2023-11-13T10:56:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.854+0000 7f66fd60a640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66dc06ae70 0x7f66dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.855+0000 7f66fd60a640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66dc06ae70 0x7f66dc06d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f66f815bf50 tx=0x7f66e8000ef0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:03.858+0000 7f66ee7fc640 1 -- 172.21.15.16:0/2125984735 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66f40765e0 con 0x7f66e40039c0 2023-11-13T10:56:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.043+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66f80cef10 con 0x7f66e40039c0 2023-11-13T10:56:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.044+0000 7f66ee7fc640 1 -- 172.21.15.16:0/2125984735 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66f4079c00 con 0x7f66e40039c0 2023-11-13T10:56:04.048 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.046+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66dc06ae70 msgr2=0x7f66dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.046+0000 7f66fee0d640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66dc06ae70 0x7f66dc06d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f66f815bf50 tx=0x7f66e8000ef0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.046+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 msgr2=0x7f66f80b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.046+0000 7f66fee0d640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66f80b5760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009ac0 tx=0x7f66f4004730 comp rx=0 tx=0).stop 2023-11-13T10:56:04.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.047+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 shutdown_connections 2023-11-13T10:56:04.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.047+0000 7f66fee0d640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66dc06ae70 0x7f66dc06d330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.047+0000 7f66fee0d640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66f80b73f0 0x7f66f80b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.047+0000 7f66fee0d640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66f80b6f70 0x7f66f80b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.047+0000 7f66fee0d640 1 --2- 172.21.15.16:0/2125984735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66e40039c0 0x7f66f80b5760 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.047+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 >> 172.21.15.16:0/2125984735 conn(0x7f66f808c580 msgr2=0x7f66f80ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:04.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.047+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 shutdown_connections 2023-11-13T10:56:04.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.047+0000 7f66fee0d640 1 -- 172.21.15.16:0/2125984735 wait complete. 2023-11-13T10:56:04.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.287+0000 7f6822aa2640 1 Processor -- start 2023-11-13T10:56:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.288+0000 7f6822aa2640 1 -- start start 2023-11-13T10:56:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.289+0000 7f6822aa2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f681c06b510 0x7f681c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.289+0000 7f6822aa2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f681c06b8f0 0x7f681c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.289+0000 7f6822aa2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 0x7f681c15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f6822aa2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c0716a0 con 0x7f681c06b510 2023-11-13T10:56:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f6822aa2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c15b070 con 0x7f681c06b8f0 2023-11-13T10:56:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f6822aa2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c15b350 con 0x7f681c0ce9d0 2023-11-13T10:56:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f682083e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f681c06b510 0x7f681c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f682083e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f681c06b510 0x7f681c0ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46720/0 (socket says 172.21.15.16:46720) 2023-11-13T10:56:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f682083e640 1 -- 172.21.15.16:0/91206795 learned_addr learned my addr 172.21.15.16:0/91206795 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f682103f640 1 --2- 172.21.15.16:0/91206795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 0x7f681c15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f681bfff640 1 --2- 172.21.15.16:0/91206795 >> v2:172.21.15.184:3300/0 conn(0x7f681c06b8f0 0x7f681c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f682083e640 1 -- 172.21.15.16:0/91206795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 msgr2=0x7f681c15bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f682083e640 1 --2- 172.21.15.16:0/91206795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 0x7f681c15bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f682083e640 1 -- 172.21.15.16:0/91206795 >> v2:172.21.15.184:3300/0 conn(0x7f681c06b8f0 msgr2=0x7f681c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f682083e640 1 --2- 172.21.15.16:0/91206795 >> v2:172.21.15.184:3300/0 conn(0x7f681c06b8f0 0x7f681c15b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.290+0000 7f682083e640 1 -- 172.21.15.16:0/91206795 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0cd9b0 con 0x7f681c06b510 2023-11-13T10:56:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.292+0000 7f682083e640 1 --2- 172.21.15.16:0/91206795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 0x7f681c0ce490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f681c0712e0 tx=0x7f680c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772163d105bc8fdb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.292+0000 7f681b7fe640 1 -- 172.21.15.16:0/91206795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c02ca70 con 0x7f681c06b510 2023-11-13T10:56:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.292+0000 7f681b7fe640 1 -- 172.21.15.16:0/91206795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c02cbd0 con 0x7f681c06b510 2023-11-13T10:56:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.292+0000 7f681b7fe640 1 -- 172.21.15.16:0/91206795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0343f0 con 0x7f681c06b510 2023-11-13T10:56:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.293+0000 7f6822aa2640 1 -- 172.21.15.16:0/91206795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 msgr2=0x7f681c0ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.293+0000 7f6822aa2640 1 --2- 172.21.15.16:0/91206795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 0x7f681c0ce490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f681c0712e0 tx=0x7f680c02bed0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.293+0000 7f6822aa2640 1 -- 172.21.15.16:0/91206795 shutdown_connections 2023-11-13T10:56:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.293+0000 7f6822aa2640 1 --2- 172.21.15.16:0/91206795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 0x7f681c15bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.293+0000 7f6822aa2640 1 --2- 172.21.15.16:0/91206795 >> v2:172.21.15.184:3300/0 conn(0x7f681c06b8f0 0x7f681c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.293+0000 7f6822aa2640 1 --2- 172.21.15.16:0/91206795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 0x7f681c0ce490 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.293+0000 7f6822aa2640 1 -- 172.21.15.16:0/91206795 >> 172.21.15.16:0/91206795 conn(0x7f681c0b19a0 msgr2=0x7f681c0b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.293+0000 7f6822aa2640 1 -- 172.21.15.16:0/91206795 shutdown_connections 2023-11-13T10:56:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.293+0000 7f6822aa2640 1 -- 172.21.15.16:0/91206795 wait complete. 2023-11-13T10:56:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.295+0000 7f6822aa2640 1 Processor -- start 2023-11-13T10:56:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.295+0000 7f6822aa2640 1 -- start start 2023-11-13T10:56:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f6822aa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 0x7f681c152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f6822aa2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c06b8f0 0x7f681c138250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f6822aa2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 0x7f681c1357e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f6822aa2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c136c20 con 0x7f681c06b510 2023-11-13T10:56:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f6822aa2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c135d20 con 0x7f681c06b8f0 2023-11-13T10:56:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f6822aa2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c135fa0 con 0x7f681c0ce9d0 2023-11-13T10:56:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f682083e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 0x7f681c152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f682083e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 0x7f681c152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46730/0 (socket says 172.21.15.16:46730) 2023-11-13T10:56:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f682083e640 1 -- 172.21.15.16:0/2456456804 learned_addr learned my addr 172.21.15.16:0/2456456804 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f681bfff640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c06b8f0 0x7f681c138250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f682083e640 1 -- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 msgr2=0x7f681c1357e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f682103f640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 0x7f681c1357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f682083e640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 0x7f681c1357e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f682083e640 1 -- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c06b8f0 msgr2=0x7f681c138250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f682083e640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c06b8f0 0x7f681c138250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f682083e640 1 -- 172.21.15.16:0/2456456804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c009b30 con 0x7f681c06b510 2023-11-13T10:56:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.296+0000 7f681bfff640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c06b8f0 0x7f681c138250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.297+0000 7f682083e640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 0x7f681c152a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f680c000f80 tx=0x7f680c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.297+0000 7f68197fa640 1 -- 172.21.15.16:0/2456456804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c009510 con 0x7f681c06b510 2023-11-13T10:56:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.297+0000 7f68197fa640 1 -- 172.21.15.16:0/2456456804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c03e450 con 0x7f681c06b510 2023-11-13T10:56:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.297+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c136280 con 0x7f681c06b510 2023-11-13T10:56:04.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.297+0000 7f68197fa640 1 -- 172.21.15.16:0/2456456804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c03e800 con 0x7f681c06b510 2023-11-13T10:56:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.297+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c136840 con 0x7f681c06b510 2023-11-13T10:56:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.299+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681c072670 con 0x7f681c06b510 2023-11-13T10:56:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.300+0000 7f68197fa640 1 -- 172.21.15.16:0/2456456804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f680c03e960 con 0x7f681c06b510 2023-11-13T10:56:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.301+0000 7f68197fa640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f680806ae40 0x7f680806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.301+0000 7f68197fa640 1 -- 172.21.15.16:0/2456456804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f680c0ac7f0 con 0x7f681c06b510 2023-11-13T10:56:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.302+0000 7f681bfff640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f680806ae40 0x7f680806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.303+0000 7f681bfff640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f680806ae40 0x7f680806d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f681c134ca0 tx=0x7f6810008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.306+0000 7f68197fa640 1 -- 172.21.15.16:0/2456456804 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f680c077510 con 0x7f681c06b510 2023-11-13T10:56:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.498+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f681c159e60 con 0x7f681c06b510 2023-11-13T10:56:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.499+0000 7f68197fa640 1 -- 172.21.15.16:0/2456456804 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f680c07ab30 con 0x7f681c06b510 2023-11-13T10:56:04.503 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.501+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f680806ae40 msgr2=0x7f680806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.501+0000 7f6822aa2640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f680806ae40 0x7f680806d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f681c134ca0 tx=0x7f6810008040 comp rx=0 tx=0).stop 2023-11-13T10:56:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.501+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 msgr2=0x7f681c152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.501+0000 7f6822aa2640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 0x7f681c152a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f680c000f80 tx=0x7f680c004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.502+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 shutdown_connections 2023-11-13T10:56:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.502+0000 7f6822aa2640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f680806ae40 0x7f680806d300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.502+0000 7f6822aa2640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f681c0ce9d0 0x7f681c1357e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.502+0000 7f6822aa2640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f681c06b8f0 0x7f681c138250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.502+0000 7f6822aa2640 1 --2- 172.21.15.16:0/2456456804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f681c06b510 0x7f681c152a50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.502+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 >> 172.21.15.16:0/2456456804 conn(0x7f681c0b19a0 msgr2=0x7f681c0b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.502+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 shutdown_connections 2023-11-13T10:56:04.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.502+0000 7f6822aa2640 1 -- 172.21.15.16:0/2456456804 wait complete. 2023-11-13T10:56:04.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.761+0000 7f80f359e640 1 Processor -- start 2023-11-13T10:56:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.761+0000 7f80f359e640 1 -- start start 2023-11-13T10:56:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.763+0000 7f80f359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 0x7f80f40ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.763+0000 7f80f359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f40ad240 con 0x7f80f40715b0 2023-11-13T10:56:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.763+0000 7f80f359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f406bca0 con 0x7f80f40ace20 2023-11-13T10:56:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.763+0000 7f80f359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f406c920 con 0x7f80f406b6c0 2023-11-13T10:56:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.763+0000 7f80f259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80f40715b0 0x7f80f40b3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49348/0 (socket says 172.21.15.16:49348) 2023-11-13T10:56:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.763+0000 7f80f259c640 1 -- 172.21.15.16:0/2949936826 learned_addr learned my addr 172.21.15.16:0/2949936826 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f1d9b640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 0x7f80f40ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432621058 0 0) 0x7f80f40ad240 con 0x7f80f40715b0 2023-11-13T10:56:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80e0003520 con 0x7f80f40715b0 2023-11-13T10:56:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257741167 0 0) 0x7f80f406bca0 con 0x7f80f40ace20 2023-11-13T10:56:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f40ad240 con 0x7f80f40ace20 2023-11-13T10:56:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2303826633 0 0) 0x7f80e0003520 con 0x7f80f40715b0 2023-11-13T10:56:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 msgr2=0x7f80f40ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 0x7f80f40ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 >> v1:172.21.15.184:6789/0 conn(0x7f80f40ace20 legacy=0x7f80f406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f406bf80 con 0x7f80f40715b0 2023-11-13T10:56:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f80e40019f0 con 0x7f80f40715b0 2023-11-13T10:56:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f1d9b640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 0x7f80f40ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:56:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.764+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 >> v1:172.21.15.16:6789/0 conn(0x7f80f40715b0 legacy=0x7f80f40b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f159a640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80e0003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f159a640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e00047c0 0x7f80e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f159a640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 0x7f80e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f406c920 con 0x7f80e0003b30 2023-11-13T10:56:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e000ba90 con 0x7f80e00047c0 2023-11-13T10:56:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e000bc50 con 0x7f80f406b6c0 2023-11-13T10:56:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f259c640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80e0003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f1d9b640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e00047c0 0x7f80e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f259c640 1 -- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 msgr2=0x7f80e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f2d9d640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 0x7f80e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f259c640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 0x7f80e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f259c640 1 -- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e00047c0 msgr2=0x7f80e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f259c640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80e00047c0 0x7f80e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f259c640 1 -- 172.21.15.16:0/2949936826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc009400 con 0x7f80e0003b30 2023-11-13T10:56:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.765+0000 7f80f2d9d640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f406b6c0 0x7f80e0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:04.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.766+0000 7f80f259c640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80e0003f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80e4002230 tx=0x7f80e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8bc24f84d9b55 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.767+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4034430 con 0x7f80e0003b30 2023-11-13T10:56:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.767+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e402f070 con 0x7f80e0003b30 2023-11-13T10:56:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.767+0000 7f80f159a640 1 -- 172.21.15.16:0/2949936826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40347f0 con 0x7f80e0003b30 2023-11-13T10:56:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.767+0000 7f80f359e640 1 -- 172.21.15.16:0/2949936826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 msgr2=0x7f80e0003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.767+0000 7f80f359e640 1 --2- 172.21.15.16:0/2949936826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80e0003f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80e4002230 tx=0x7f80e00057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.767+0000 7f80f2d9d640 1 -- 172.21.15.16:0/2949936826 reap_dead start 2023-11-13T10:56:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.768+0000 7f80f359e640 1 -- 172.21.15.16:0/2949936826 shutdown_connections 2023-11-13T10:56:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.768+0000 7f80f359e640 1 -- 172.21.15.16:0/2949936826 >> 172.21.15.16:0/2949936826 conn(0x7f80f40b0970 msgr2=0x7f80f40b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.768+0000 7f80f359e640 1 -- 172.21.15.16:0/2949936826 shutdown_connections 2023-11-13T10:56:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.768+0000 7f80f359e640 1 -- 172.21.15.16:0/2949936826 wait complete. 2023-11-13T10:56:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.770+0000 7f80f359e640 1 Processor -- start 2023-11-13T10:56:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.770+0000 7f80f359e640 1 -- start start 2023-11-13T10:56:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80f40cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40c8520 0x7f80f40c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f415e2d0 0x7f80f415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4134b90 con 0x7f80e0003b30 2023-11-13T10:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f415ec90 con 0x7f80f40c8520 2023-11-13T10:56:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4133f10 con 0x7f80f415e2d0 2023-11-13T10:56:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80f40cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80f40cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46748/0 (socket says 172.21.15.16:46748) 2023-11-13T10:56:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f259c640 1 -- 172.21.15.16:0/614715398 learned_addr learned my addr 172.21.15.16:0/614715398 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f2d9d640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f415e2d0 0x7f80f415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f1d9b640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40c8520 0x7f80f40c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.771+0000 7f80f259c640 1 -- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f415e2d0 msgr2=0x7f80f415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80f259c640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f415e2d0 0x7f80f415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80f259c640 1 -- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40c8520 msgr2=0x7f80f40c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80f259c640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40c8520 0x7f80f40c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80f259c640 1 -- 172.21.15.16:0/614715398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f41341f0 con 0x7f80e0003b30 2023-11-13T10:56:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80f2d9d640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f415e2d0 0x7f80f415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80f1d9b640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40c8520 0x7f80f40c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80f259c640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80f40cd9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001c30 tx=0x7f80e4001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80eaffd640 1 -- 172.21.15.16:0/614715398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4034630 con 0x7f80e0003b30 2023-11-13T10:56:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80eaffd640 1 -- 172.21.15.16:0/614715398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e4034790 con 0x7f80e0003b30 2023-11-13T10:56:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80eaffd640 1 -- 172.21.15.16:0/614715398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4034b00 con 0x7f80e0003b30 2023-11-13T10:56:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f41344d0 con 0x7f80e0003b30 2023-11-13T10:56:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.772+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f40cdf30 con 0x7f80e0003b30 2023-11-13T10:56:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.774+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f404e4d0 con 0x7f80e0003b30 2023-11-13T10:56:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.774+0000 7f80eaffd640 1 -- 172.21.15.16:0/614715398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80e4031030 con 0x7f80e0003b30 2023-11-13T10:56:04.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.775+0000 7f80eaffd640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d406ae70 0x7f80d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:04.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.775+0000 7f80eaffd640 1 -- 172.21.15.16:0/614715398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f80e40abc10 con 0x7f80e0003b30 2023-11-13T10:56:04.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.776+0000 7f80f1d9b640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d406ae70 0x7f80d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:04.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.777+0000 7f80f1d9b640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d406ae70 0x7f80d406d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80f40acd60 tx=0x7f80dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:04.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.780+0000 7f80eaffd640 1 -- 172.21.15.16:0/614715398 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80e4076ae0 con 0x7f80e0003b30 2023-11-13T10:56:04.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.973+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80f404e4d0 con 0x7f80e0003b30 2023-11-13T10:56:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.974+0000 7f80eaffd640 1 -- 172.21.15.16:0/614715398 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80e407a100 con 0x7f80e0003b30 2023-11-13T10:56:04.978 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.976+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d406ae70 msgr2=0x7f80d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.976+0000 7f80f359e640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d406ae70 0x7f80d406d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80f40acd60 tx=0x7f80dc008040 comp rx=0 tx=0).stop 2023-11-13T10:56:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 msgr2=0x7f80f40cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80f40cd9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001c30 tx=0x7f80e4001d00 comp rx=0 tx=0).stop 2023-11-13T10:56:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 shutdown_connections 2023-11-13T10:56:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d406ae70 0x7f80d406d330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80f415e2d0 0x7f80f415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f40c8520 0x7f80f40c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 --2- 172.21.15.16:0/614715398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0003b30 0x7f80f40cd9f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 >> 172.21.15.16:0/614715398 conn(0x7f80f40b0970 msgr2=0x7f80f406ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 shutdown_connections 2023-11-13T10:56:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:04.977+0000 7f80f359e640 1 -- 172.21.15.16:0/614715398 wait complete. 2023-11-13T10:56:05.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:05.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.233+0000 7f47e359e640 1 Processor -- start 2023-11-13T10:56:05.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.233+0000 7f47e359e640 1 -- start start 2023-11-13T10:56:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.235+0000 7f47e359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40ce520 0x7f47e40cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.235+0000 7f47e359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47e40d1470 0x7f47e40cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.235+0000 7f47e359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e40ced70 con 0x7f47e415ad60 2023-11-13T10:56:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.235+0000 7f47e359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e40cb460 con 0x7f47e40d1470 2023-11-13T10:56:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.235+0000 7f47e359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e40cea90 con 0x7f47e40ce520 2023-11-13T10:56:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.235+0000 7f47e2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47e40d1470 0x7f47e40cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.235+0000 7f47e2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47e40d1470 0x7f47e40cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46756/0 (socket says 172.21.15.16:46756) 2023-11-13T10:56:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.235+0000 7f47e2d9d640 1 -- 172.21.15.16:0/1502018750 learned_addr learned my addr 172.21.15.16:0/1502018750 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.235+0000 7f47e1d9b640 1 --2- 172.21.15.16:0/1502018750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40ce520 0x7f47e40cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.236+0000 7f47e2d9d640 1 -- 172.21.15.16:0/1502018750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40ce520 msgr2=0x7f47e40cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:05.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.236+0000 7f47e2d9d640 1 --2- 172.21.15.16:0/1502018750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40ce520 0x7f47e40cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.236+0000 7f47e2d9d640 1 -- 172.21.15.16:0/1502018750 >> v1:172.21.15.184:6789/0 conn(0x7f47e415ad60 legacy=0x7f47e40cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.236+0000 7f47e2d9d640 1 -- 172.21.15.16:0/1502018750 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e40cb700 con 0x7f47e40d1470 2023-11-13T10:56:05.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.237+0000 7f47e2d9d640 1 --2- 172.21.15.16:0/1502018750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e40d1470 0x7f47e40cf470 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009e50 tx=0x7f47dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaba9662d50e5bd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:05.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e159a640 1 -- 172.21.15.16:0/1502018750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc02ca70 con 0x7f47e40d1470 2023-11-13T10:56:05.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e159a640 1 -- 172.21.15.16:0/1502018750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc02cbd0 con 0x7f47e40d1470 2023-11-13T10:56:05.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e159a640 1 -- 172.21.15.16:0/1502018750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0343f0 con 0x7f47e40d1470 2023-11-13T10:56:05.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e359e640 1 -- 172.21.15.16:0/1502018750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e40d1470 msgr2=0x7f47e40cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e359e640 1 --2- 172.21.15.16:0/1502018750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e40d1470 0x7f47e40cf470 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009e50 tx=0x7f47dc02bed0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e359e640 1 -- 172.21.15.16:0/1502018750 shutdown_connections 2023-11-13T10:56:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e359e640 1 --2- 172.21.15.16:0/1502018750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e40d1470 0x7f47e40cf470 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e359e640 1 --2- 172.21.15.16:0/1502018750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40ce520 0x7f47e40cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e359e640 1 -- 172.21.15.16:0/1502018750 >> 172.21.15.16:0/1502018750 conn(0x7f47e40d0950 msgr2=0x7f47e40d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.238+0000 7f47e359e640 1 -- 172.21.15.16:0/1502018750 shutdown_connections 2023-11-13T10:56:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.239+0000 7f47e359e640 1 -- 172.21.15.16:0/1502018750 wait complete. 2023-11-13T10:56:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.241+0000 7f47e359e640 1 Processor -- start 2023-11-13T10:56:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.241+0000 7f47e359e640 1 -- start start 2023-11-13T10:56:05.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e40ce520 0x7f47e40b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40d1470 0x7f47e41f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e415ad60 0x7f47e40b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e414d990 con 0x7f47e415ad60 2023-11-13T10:56:05.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e41f2680 con 0x7f47e40ce520 2023-11-13T10:56:05.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e415b7c0 con 0x7f47e40d1470 2023-11-13T10:56:05.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e415ad60 0x7f47e40b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e415ad60 0x7f47e40b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46772/0 (socket says 172.21.15.16:46772) 2023-11-13T10:56:05.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e2d9d640 1 -- 172.21.15.16:0/4205323936 learned_addr learned my addr 172.21.15.16:0/4205323936 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:05.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.242+0000 7f47e259c640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e40ce520 0x7f47e40b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47e1d9b640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40d1470 0x7f47e41f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47e2d9d640 1 -- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40d1470 msgr2=0x7f47e41f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47e2d9d640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40d1470 0x7f47e41f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47e2d9d640 1 -- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e40ce520 msgr2=0x7f47e40b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47e2d9d640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e40ce520 0x7f47e40b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47e2d9d640 1 -- 172.21.15.16:0/4205323936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc009b30 con 0x7f47e415ad60 2023-11-13T10:56:05.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47e2d9d640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e415ad60 0x7f47e40b41b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f47dc036040 tx=0x7f47dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47d2ffd640 1 -- 172.21.15.16:0/4205323936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc009510 con 0x7f47e415ad60 2023-11-13T10:56:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47d2ffd640 1 -- 172.21.15.16:0/4205323936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc03e450 con 0x7f47e415ad60 2023-11-13T10:56:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.243+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e40b4ab0 con 0x7f47e415ad60 2023-11-13T10:56:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.244+0000 7f47d2ffd640 1 -- 172.21.15.16:0/4205323936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc03e800 con 0x7f47e415ad60 2023-11-13T10:56:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.244+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e40b5070 con 0x7f47e415ad60 2023-11-13T10:56:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.245+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e415b970 con 0x7f47e415ad60 2023-11-13T10:56:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.246+0000 7f47d2ffd640 1 -- 172.21.15.16:0/4205323936 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47dc03e960 con 0x7f47e415ad60 2023-11-13T10:56:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.247+0000 7f47d2ffd640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47c406ae40 0x7f47c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.247+0000 7f47d2ffd640 1 -- 172.21.15.16:0/4205323936 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f47dc0ac630 con 0x7f47e415ad60 2023-11-13T10:56:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.247+0000 7f47e259c640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47c406ae40 0x7f47c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.249+0000 7f47e259c640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47c406ae40 0x7f47c406d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f47d4000f60 tx=0x7f47d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.251+0000 7f47d2ffd640 1 -- 172.21.15.16:0/4205323936 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47e415b970 con 0x7f47e415ad60 2023-11-13T10:56:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.443+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f47e415b930 con 0x7f47e415ad60 2023-11-13T10:56:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.444+0000 7f47d2ffd640 1 -- 172.21.15.16:0/4205323936 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f47dc0773f0 con 0x7f47e415ad60 2023-11-13T10:56:05.448 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47c406ae40 msgr2=0x7f47c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47c406ae40 0x7f47c406d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f47d4000f60 tx=0x7f47d4009040 comp rx=0 tx=0).stop 2023-11-13T10:56:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e415ad60 msgr2=0x7f47e40b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e415ad60 0x7f47e40b41b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f47dc036040 tx=0x7f47dc004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 shutdown_connections 2023-11-13T10:56:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f47c406ae40 0x7f47c406d300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e415ad60 0x7f47e40b41b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47e40d1470 0x7f47e41f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 --2- 172.21.15.16:0/4205323936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47e40ce520 0x7f47e40b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 >> 172.21.15.16:0/4205323936 conn(0x7f47e40d0950 msgr2=0x7f47e415e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 shutdown_connections 2023-11-13T10:56:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.447+0000 7f47e359e640 1 -- 172.21.15.16:0/4205323936 wait complete. 2023-11-13T10:56:05.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:56:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.693+0000 7fcd405c5640 1 Processor -- start 2023-11-13T10:56:05.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.694+0000 7fcd405c5640 1 -- start start 2023-11-13T10:56:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.695+0000 7fcd405c5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd380cd950 0x7fcd380cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.695+0000 7fcd405c5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd380cebd0 con 0x7fcd3815d150 2023-11-13T10:56:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.695+0000 7fcd405c5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd380cdfc0 con 0x7fcd380cfce0 2023-11-13T10:56:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.695+0000 7fcd405c5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380d3ac0 con 0x7fcd380cd950 2023-11-13T10:56:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.695+0000 7fcd3e361640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd3815d150 0x7fcd38161520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49350/0 (socket says 172.21.15.16:49350) 2023-11-13T10:56:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.695+0000 7fcd3e361640 1 -- 172.21.15.16:0/996654319 learned_addr learned my addr 172.21.15.16:0/996654319 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3db60640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd380cd950 0x7fcd380cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999849914 0 0) 0x7fcd380cebd0 con 0x7fcd3815d150 2023-11-13T10:56:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd24003520 con 0x7fcd3815d150 2023-11-13T10:56:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3624028061 0 0) 0x7fcd24003520 con 0x7fcd3815d150 2023-11-13T10:56:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd380cd950 msgr2=0x7fcd380cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd380cd950 0x7fcd380cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 >> v1:172.21.15.184:6789/0 conn(0x7fcd380cfce0 legacy=0x7fcd380d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd380ce2a0 con 0x7fcd3815d150 2023-11-13T10:56:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fcd34001670 con 0x7fcd3815d150 2023-11-13T10:56:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 >> v1:172.21.15.16:6789/0 conn(0x7fcd3815d150 legacy=0x7fcd38161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd240039a0 0x7fcd24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240047c0 0x7fcd24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd380cd950 0x7fcd24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2400ba30 con 0x7fcd240047c0 2023-11-13T10:56:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2400bba0 con 0x7fcd240039a0 2023-11-13T10:56:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2400bd60 con 0x7fcd380cd950 2023-11-13T10:56:05.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.696+0000 7fcd3db60640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240047c0 0x7fcd24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.697+0000 7fcd3e361640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd240039a0 0x7fcd24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.697+0000 7fcd3eb62640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd380cd950 0x7fcd24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.697+0000 7fcd3db60640 1 -- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd380cd950 msgr2=0x7fcd24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.697+0000 7fcd3db60640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd380cd950 0x7fcd24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.697+0000 7fcd3db60640 1 -- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd240039a0 msgr2=0x7fcd24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.697+0000 7fcd3db60640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd240039a0 0x7fcd24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.697+0000 7fcd3db60640 1 -- 172.21.15.16:0/996654319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24003520 con 0x7fcd240047c0 2023-11-13T10:56:05.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.698+0000 7fcd3db60640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240047c0 0x7fcd24004c40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009f90 tx=0x7fcd240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b2253e2e8e79f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:05.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.698+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c02cbd0 con 0x7fcd240047c0 2023-11-13T10:56:05.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.698+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c00f040 con 0x7fcd240047c0 2023-11-13T10:56:05.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.698+0000 7fcd3d35f640 1 -- 172.21.15.16:0/996654319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c034440 con 0x7fcd240047c0 2023-11-13T10:56:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.699+0000 7fcd405c5640 1 -- 172.21.15.16:0/996654319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240047c0 msgr2=0x7fcd24004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.699+0000 7fcd405c5640 1 --2- 172.21.15.16:0/996654319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240047c0 0x7fcd24004c40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009f90 tx=0x7fcd240057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.699+0000 7fcd3eb62640 1 -- 172.21.15.16:0/996654319 reap_dead start 2023-11-13T10:56:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.699+0000 7fcd405c5640 1 -- 172.21.15.16:0/996654319 shutdown_connections 2023-11-13T10:56:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.699+0000 7fcd405c5640 1 -- 172.21.15.16:0/996654319 >> 172.21.15.16:0/996654319 conn(0x7fcd3809c030 msgr2=0x7fcd380d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.699+0000 7fcd405c5640 1 -- 172.21.15.16:0/996654319 shutdown_connections 2023-11-13T10:56:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.699+0000 7fcd405c5640 1 -- 172.21.15.16:0/996654319 wait complete. 2023-11-13T10:56:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.701+0000 7fcd405c5640 1 Processor -- start 2023-11-13T10:56:05.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.701+0000 7fcd405c5640 1 -- start start 2023-11-13T10:56:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.702+0000 7fcd405c5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd240047c0 0x7fcd381e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.702+0000 7fcd405c5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd381e7390 0x7fcd381f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.702+0000 7fcd405c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd381e8940 0x7fcd381f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.702+0000 7fcd405c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd381e84b0 con 0x7fcd381e8940 2023-11-13T10:56:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.702+0000 7fcd405c5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd381e78a0 con 0x7fcd240047c0 2023-11-13T10:56:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.702+0000 7fcd405c5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd381e7b80 con 0x7fcd381e7390 2023-11-13T10:56:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.702+0000 7fcd3eb62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd381e8940 0x7fcd381f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.702+0000 7fcd3eb62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd381e8940 0x7fcd381f31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46784/0 (socket says 172.21.15.16:46784) 2023-11-13T10:56:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.702+0000 7fcd3eb62640 1 -- 172.21.15.16:0/2725185115 learned_addr learned my addr 172.21.15.16:0/2725185115 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd3e361640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd240047c0 0x7fcd381e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd3db60640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd381e7390 0x7fcd381f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd3eb62640 1 -- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd381e7390 msgr2=0x7fcd381f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd3eb62640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd381e7390 0x7fcd381f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd3eb62640 1 -- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd240047c0 msgr2=0x7fcd381e6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd3eb62640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd240047c0 0x7fcd381e6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd3eb62640 1 -- 172.21.15.16:0/2725185115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd381e7e60 con 0x7fcd381e8940 2023-11-13T10:56:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd3db60640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd381e7390 0x7fcd381f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd3eb62640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd381e8940 0x7fcd381f31f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd30000c00 tx=0x7fcd3000e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd2affd640 1 -- 172.21.15.16:0/2725185115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd300052e0 con 0x7fcd381e8940 2023-11-13T10:56:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd2affd640 1 -- 172.21.15.16:0/2725185115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd30005440 con 0x7fcd381e8940 2023-11-13T10:56:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd2affd640 1 -- 172.21.15.16:0/2725185115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30003830 con 0x7fcd381e8940 2023-11-13T10:56:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.703+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd381e8140 con 0x7fcd381e8940 2023-11-13T10:56:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.704+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3815a720 con 0x7fcd381e8940 2023-11-13T10:56:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.705+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd381e7cc0 con 0x7fcd381e8940 2023-11-13T10:56:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.706+0000 7fcd2affd640 1 -- 172.21.15.16:0/2725185115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd30003990 con 0x7fcd381e8940 2023-11-13T10:56:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.707+0000 7fcd2affd640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd1c06ae40 0x7fcd1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.707+0000 7fcd2affd640 1 -- 172.21.15.16:0/2725185115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcd30088ff0 con 0x7fcd381e8940 2023-11-13T10:56:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.708+0000 7fcd3e361640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd1c06ae40 0x7fcd1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:05.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.709+0000 7fcd3e361640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd1c06ae40 0x7fcd1c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd34001050 tx=0x7fcd34009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:05.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.712+0000 7fcd2affd640 1 -- 172.21.15.16:0/2725185115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd381e7cc0 con 0x7fcd381e8940 2023-11-13T10:56:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.951+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd380d0150 con 0x7fcd381e8940 2023-11-13T10:56:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.952+0000 7fcd2affd640 1 -- 172.21.15.16:0/2725185115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7fcd380d0150 con 0x7fcd381e8940 2023-11-13T10:56:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:05.958 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:53.560913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-13T10:56:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-11-13T10:56:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd1c06ae40 msgr2=0x7fcd1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd1c06ae40 0x7fcd1c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd34001050 tx=0x7fcd34009040 comp rx=0 tx=0).stop 2023-11-13T10:56:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd381e8940 msgr2=0x7fcd381f31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd381e8940 0x7fcd381f31f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd30000c00 tx=0x7fcd3000e8a0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 shutdown_connections 2023-11-13T10:56:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd1c06ae40 0x7fcd1c06d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd381e8940 0x7fcd381f31f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd381e7390 0x7fcd381f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 --2- 172.21.15.16:0/2725185115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd240047c0 0x7fcd381e6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.954+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 >> 172.21.15.16:0/2725185115 conn(0x7fcd3809c030 msgr2=0x7fcd3815feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.955+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 shutdown_connections 2023-11-13T10:56:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:05.955+0000 7fcd405c5640 1 -- 172.21.15.16:0/2725185115 wait complete. 2023-11-13T10:56:05.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.206+0000 7f74d20ff640 1 Processor -- start 2023-11-13T10:56:06.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.206+0000 7f74d20ff640 1 -- start start 2023-11-13T10:56:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.208+0000 7f74d20ff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc06a8a0 0x7f74cc070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.208+0000 7f74d20ff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f74cc0708a0 0x7f74cc06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.208+0000 7f74d20ff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74cc070d50 con 0x7f74cc15aa90 2023-11-13T10:56:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.208+0000 7f74d20ff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc15d630 con 0x7f74cc0708a0 2023-11-13T10:56:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.208+0000 7f74d20ff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc15d910 con 0x7f74cc06a8a0 2023-11-13T10:56:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.208+0000 7f74d10fd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f74cc15aa90 0x7f74cc06a190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49352/0 (socket says 172.21.15.16:49352) 2023-11-13T10:56:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.208+0000 7f74d10fd640 1 -- 172.21.15.16:0/942161357 learned_addr learned my addr 172.21.15.16:0/942161357 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74d08fc640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc06a8a0 0x7f74cc070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74d18fe640 1 --2- 172.21.15.16:0/942161357 >> v2:172.21.15.184:3300/0 conn(0x7f74cc0708a0 0x7f74cc06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722250319 0 0) 0x7f74cc070d50 con 0x7f74cc15aa90 2023-11-13T10:56:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74b8003520 con 0x7f74cc15aa90 2023-11-13T10:56:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3189781867 0 0) 0x7f74b8003520 con 0x7f74cc15aa90 2023-11-13T10:56:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc06a8a0 msgr2=0x7f74cc070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc06a8a0 0x7f74cc070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 >> v2:172.21.15.184:3300/0 conn(0x7f74cc0708a0 msgr2=0x7f74cc06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 --2- 172.21.15.16:0/942161357 >> v2:172.21.15.184:3300/0 conn(0x7f74cc0708a0 0x7f74cc06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc15dbf0 con 0x7f74cc15aa90 2023-11-13T10:56:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f74c8001670 con 0x7f74cc15aa90 2023-11-13T10:56:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 >> v1:172.21.15.16:6789/0 conn(0x7f74cc15aa90 legacy=0x7f74cc06a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc06a8a0 0x7f74b8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74b8004d00 0x7f74b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc15d630 con 0x7f74b8003f40 2023-11-13T10:56:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc15d910 con 0x7f74b8004d00 2023-11-13T10:56:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.209+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b800ba30 con 0x7f74cc06a8a0 2023-11-13T10:56:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.210+0000 7f74d08fc640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.210+0000 7f74d10fd640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc06a8a0 0x7f74b8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.210+0000 7f74d18fe640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74b8004d00 0x7f74b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.210+0000 7f74d08fc640 1 -- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc06a8a0 msgr2=0x7f74b8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.210+0000 7f74d08fc640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc06a8a0 0x7f74b8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.210+0000 7f74d08fc640 1 -- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74b8004d00 msgr2=0x7f74b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.210+0000 7f74d08fc640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74b8004d00 0x7f74b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.210+0000 7f74d08fc640 1 -- 172.21.15.16:0/942161357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b8003520 con 0x7f74b8003f40 2023-11-13T10:56:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.211+0000 7f74d08fc640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74b80047c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009ac0 tx=0x7f74b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722f14ec19528d96 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc02c6b0 con 0x7f74b8003f40 2023-11-13T10:56:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc02c810 con 0x7f74b8003f40 2023-11-13T10:56:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74c3fff640 1 -- 172.21.15.16:0/942161357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc02cba0 con 0x7f74b8003f40 2023-11-13T10:56:06.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74d20ff640 1 -- 172.21.15.16:0/942161357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 msgr2=0x7f74b80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74d20ff640 1 --2- 172.21.15.16:0/942161357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74b80047c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009ac0 tx=0x7f74b80057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74d18fe640 1 -- 172.21.15.16:0/942161357 reap_dead start 2023-11-13T10:56:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74d20ff640 1 -- 172.21.15.16:0/942161357 shutdown_connections 2023-11-13T10:56:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74d20ff640 1 -- 172.21.15.16:0/942161357 >> 172.21.15.16:0/942161357 conn(0x7f74cc0b04a0 msgr2=0x7f74cc0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:06.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74d20ff640 1 -- 172.21.15.16:0/942161357 shutdown_connections 2023-11-13T10:56:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.212+0000 7f74d20ff640 1 -- 172.21.15.16:0/942161357 wait complete. 2023-11-13T10:56:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.214+0000 7f74d20ff640 1 Processor -- start 2023-11-13T10:56:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.214+0000 7f74d20ff640 1 -- start start 2023-11-13T10:56:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.215+0000 7f74d20ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74cc0acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.215+0000 7f74d20ff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc0a84d0 0x7f74cc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.215+0000 7f74d20ff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc144390 0x7f74cc0abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.215+0000 7f74d20ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc0a8c90 con 0x7f74b8003f40 2023-11-13T10:56:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.215+0000 7f74d20ff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc0aadf0 con 0x7f74cc144390 2023-11-13T10:56:06.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.215+0000 7f74d20ff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc0ab0d0 con 0x7f74cc0a84d0 2023-11-13T10:56:06.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d10fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74cc0acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d10fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74cc0acfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46800/0 (socket says 172.21.15.16:46800) 2023-11-13T10:56:06.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d10fd640 1 -- 172.21.15.16:0/294678173 learned_addr learned my addr 172.21.15.16:0/294678173 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:06.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d08fc640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc0a84d0 0x7f74cc15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d18fe640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc144390 0x7f74cc0abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d10fd640 1 -- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc0a84d0 msgr2=0x7f74cc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d10fd640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc0a84d0 0x7f74cc15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d10fd640 1 -- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc144390 msgr2=0x7f74cc0abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d10fd640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc144390 0x7f74cc0abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d10fd640 1 -- 172.21.15.16:0/294678173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc0ab3b0 con 0x7f74b8003f40 2023-11-13T10:56:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d18fe640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc144390 0x7f74cc0abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.216+0000 7f74d10fd640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74cc0acfb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f74c8001860 tx=0x7f74c800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.217+0000 7f74c1ffb640 1 -- 172.21.15.16:0/294678173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c800c6c0 con 0x7f74b8003f40 2023-11-13T10:56:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.217+0000 7f74c1ffb640 1 -- 172.21.15.16:0/294678173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c800c840 con 0x7f74b8003f40 2023-11-13T10:56:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.217+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc0ab690 con 0x7f74b8003f40 2023-11-13T10:56:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.217+0000 7f74c1ffb640 1 -- 172.21.15.16:0/294678173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c800cbf0 con 0x7f74b8003f40 2023-11-13T10:56:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.217+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc15c920 con 0x7f74b8003f40 2023-11-13T10:56:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.218+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc0675b0 con 0x7f74b8003f40 2023-11-13T10:56:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.220+0000 7f74c1ffb640 1 -- 172.21.15.16:0/294678173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74c801d460 con 0x7f74b8003f40 2023-11-13T10:56:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.221+0000 7f74c1ffb640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74b006ae70 0x7f74b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.221+0000 7f74c1ffb640 1 -- 172.21.15.16:0/294678173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f74c8089af0 con 0x7f74b8003f40 2023-11-13T10:56:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.221+0000 7f74d08fc640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74b006ae70 0x7f74b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.222+0000 7f74d08fc640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74b006ae70 0x7f74b006d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f74bc036000 tx=0x7f74bc030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.225+0000 7f74c1ffb640 1 -- 172.21.15.16:0/294678173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74c80548a0 con 0x7f74b8003f40 2023-11-13T10:56:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.421+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74cc15cb40 con 0x7f74b8003f40 2023-11-13T10:56:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.423+0000 7f74c1ffb640 1 -- 172.21.15.16:0/294678173 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18362 (secure 0 0 0) 0x7f74c8057ec0 con 0x7f74b8003f40 2023-11-13T10:56:06.427 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:06.427 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:55:31.370190+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74b006ae70 msgr2=0x7f74b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74b006ae70 0x7f74b006d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f74bc036000 tx=0x7f74bc030000 comp rx=0 tx=0).stop 2023-11-13T10:56:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 msgr2=0x7f74cc0acfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74cc0acfb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f74c8001860 tx=0x7f74c800eb90 comp rx=0 tx=0).stop 2023-11-13T10:56:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 shutdown_connections 2023-11-13T10:56:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74b006ae70 0x7f74b006d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc144390 0x7f74cc0abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74cc0a84d0 0x7f74cc15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 --2- 172.21.15.16:0/294678173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8003f40 0x7f74cc0acfb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.425+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 >> 172.21.15.16:0/294678173 conn(0x7f74cc0b04a0 msgr2=0x7f74cc147180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.426+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 shutdown_connections 2023-11-13T10:56:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.426+0000 7f74d20ff640 1 -- 172.21.15.16:0/294678173 wait complete. 2023-11-13T10:56:06.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:06.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:06.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.686+0000 7f5b4b59e640 1 Processor -- start 2023-11-13T10:56:06.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.686+0000 7f5b4b59e640 1 -- start start 2023-11-13T10:56:06.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.688+0000 7f5b4b59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b4c15b8e0 0x7f5b4c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.688+0000 7f5b4b59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5b4c0ce260 0x7f5b4c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.688+0000 7f5b4b59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c06b860 con 0x7f5b4c15bcc0 2023-11-13T10:56:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.688+0000 7f5b4b59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c06bb40 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.688+0000 7f5b4b59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c06be20 con 0x7f5b4c0ce260 2023-11-13T10:56:06.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.688+0000 7f5b4a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b4c15b8e0 0x7f5b4c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.688+0000 7f5b4a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b4c15b8e0 0x7f5b4c06c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46802/0 (socket says 172.21.15.16:46802) 2023-11-13T10:56:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.688+0000 7f5b4a59c640 1 -- 172.21.15.16:0/2328587161 learned_addr learned my addr 172.21.15.16:0/2328587161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.689+0000 7f5b4ad9d640 1 --2- 172.21.15.16:0/2328587161 >> v2:172.21.15.184:3300/0 conn(0x7f5b4c0ce260 0x7f5b4c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.689+0000 7f5b4a59c640 1 -- 172.21.15.16:0/2328587161 >> v2:172.21.15.184:3300/0 conn(0x7f5b4c0ce260 msgr2=0x7f5b4c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.689+0000 7f5b4a59c640 1 --2- 172.21.15.16:0/2328587161 >> v2:172.21.15.184:3300/0 conn(0x7f5b4c0ce260 0x7f5b4c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.689+0000 7f5b4a59c640 1 -- 172.21.15.16:0/2328587161 >> v1:172.21.15.184:6789/0 conn(0x7f5b4c15bcc0 legacy=0x7f5b4c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.689+0000 7f5b4a59c640 1 -- 172.21.15.16:0/2328587161 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c06c100 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.690+0000 7f5b4a59c640 1 --2- 172.21.15.16:0/2328587161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 0x7f5b4c06c900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0cc830 tx=0x7f5b3c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf1bf003ea24b30 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:06.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.690+0000 7f5b4959a640 1 -- 172.21.15.16:0/2328587161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c02cb20 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4959a640 1 -- 172.21.15.16:0/2328587161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c02cc80 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4959a640 1 -- 172.21.15.16:0/2328587161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0343f0 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4b59e640 1 -- 172.21.15.16:0/2328587161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 msgr2=0x7f5b4c06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4b59e640 1 --2- 172.21.15.16:0/2328587161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 0x7f5b4c06c900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0cc830 tx=0x7f5b3c02bf80 comp rx=0 tx=0).stop 2023-11-13T10:56:06.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4b59e640 1 -- 172.21.15.16:0/2328587161 shutdown_connections 2023-11-13T10:56:06.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4b59e640 1 --2- 172.21.15.16:0/2328587161 >> v2:172.21.15.184:3300/0 conn(0x7f5b4c0ce260 0x7f5b4c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4b59e640 1 --2- 172.21.15.16:0/2328587161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 0x7f5b4c06c900 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4b59e640 1 -- 172.21.15.16:0/2328587161 >> 172.21.15.16:0/2328587161 conn(0x7f5b4c0b0970 msgr2=0x7f5b4c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:06.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4b59e640 1 -- 172.21.15.16:0/2328587161 shutdown_connections 2023-11-13T10:56:06.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.691+0000 7f5b4b59e640 1 -- 172.21.15.16:0/2328587161 wait complete. 2023-11-13T10:56:06.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.693+0000 7f5b4b59e640 1 Processor -- start 2023-11-13T10:56:06.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.693+0000 7f5b4b59e640 1 -- start start 2023-11-13T10:56:06.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.694+0000 7f5b4b59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b4c0ce260 0x7f5b4c14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.694+0000 7f5b4b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 0x7f5b4c1551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b4b59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b4c1556f0 0x7f5b4c153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b4b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c158b10 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b4b59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c159370 con 0x7f5b4c1556f0 2023-11-13T10:56:06.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b4b59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c155d00 con 0x7f5b4c0ce260 2023-11-13T10:56:06.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 0x7f5b4c1551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 0x7f5b4c1551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46806/0 (socket says 172.21.15.16:46806) 2023-11-13T10:56:06.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b49d9b640 1 -- 172.21.15.16:0/3773956539 learned_addr learned my addr 172.21.15.16:0/3773956539 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:06.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b4a59c640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b4c0ce260 0x7f5b4c14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b4ad9d640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b4c1556f0 0x7f5b4c153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b49d9b640 1 -- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b4c0ce260 msgr2=0x7f5b4c14e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b49d9b640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b4c0ce260 0x7f5b4c14e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b49d9b640 1 -- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b4c1556f0 msgr2=0x7f5b4c153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b49d9b640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b4c1556f0 0x7f5b4c153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b49d9b640 1 -- 172.21.15.16:0/3773956539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c009b30 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.695+0000 7f5b4a59c640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b4c0ce260 0x7f5b4c14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:06.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.696+0000 7f5b49d9b640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 0x7f5b4c1551b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400c400 tx=0x7f5b3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:06.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.696+0000 7f5b42ffd640 1 -- 172.21.15.16:0/3773956539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b340052d0 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.696+0000 7f5b42ffd640 1 -- 172.21.15.16:0/3773956539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b34005430 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.696+0000 7f5b42ffd640 1 -- 172.21.15.16:0/3773956539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34003830 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.696+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c158d20 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.696+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c14ea10 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.697+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c159520 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.698+0000 7f5b42ffd640 1 -- 172.21.15.16:0/3773956539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5b34003990 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.700+0000 7f5b42ffd640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b2c06ae90 0x7f5b2c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:06.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.700+0000 7f5b42ffd640 1 -- 172.21.15.16:0/3773956539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5b34088f10 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.700+0000 7f5b4a59c640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b2c06ae90 0x7f5b2c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:06.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.701+0000 7f5b4a59c640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b2c06ae90 0x7f5b2c06d350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c035000 tx=0x7f5b3c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:06.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.704+0000 7f5b42ffd640 1 -- 172.21.15.16:0/3773956539 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b4c159520 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.907+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b4c1594e0 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.908+0000 7f5b42ffd640 1 -- 172.21.15.16:0/3773956539 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b340170a0 con 0x7f5b4c15b8e0 2023-11-13T10:56:06.915 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:06.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.916+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b2c06ae90 msgr2=0x7f5b2c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.916+0000 7f5b4b59e640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b2c06ae90 0x7f5b2c06d350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c035000 tx=0x7f5b3c030000 comp rx=0 tx=0).stop 2023-11-13T10:56:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.916+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 msgr2=0x7f5b4c1551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.917+0000 7f5b4b59e640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 0x7f5b4c1551b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400c400 tx=0x7f5b3400c8c0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.917+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 shutdown_connections 2023-11-13T10:56:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.917+0000 7f5b4b59e640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b2c06ae90 0x7f5b2c06d350 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.917+0000 7f5b4b59e640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b4c1556f0 0x7f5b4c153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.917+0000 7f5b4b59e640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b4c15b8e0 0x7f5b4c1551b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.917+0000 7f5b4b59e640 1 --2- 172.21.15.16:0/3773956539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b4c0ce260 0x7f5b4c14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.917+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 >> 172.21.15.16:0/3773956539 conn(0x7f5b4c0b0970 msgr2=0x7f5b4c0cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.917+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 shutdown_connections 2023-11-13T10:56:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:06.917+0000 7f5b4b59e640 1 -- 172.21.15.16:0/3773956539 wait complete. 2023-11-13T10:56:06.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.153+0000 7f6392ee6640 1 Processor -- start 2023-11-13T10:56:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.154+0000 7f6392ee6640 1 -- start start 2023-11-13T10:56:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.155+0000 7f6392ee6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f638c15ce70 0x7f638c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.155+0000 7f6392ee6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f638c0cdbb0 0x7f638c0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.155+0000 7f6392ee6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638c0cac40 con 0x7f638c0d1850 2023-11-13T10:56:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.155+0000 7f6392ee6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c0ce0e0 con 0x7f638c0cdbb0 2023-11-13T10:56:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.155+0000 7f6392ee6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0ce3c0 con 0x7f638c15ce70 2023-11-13T10:56:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.156+0000 7f63916e3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f638c0cdbb0 0x7f638c0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.156+0000 7f63916e3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f638c0cdbb0 0x7f638c0d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46816/0 (socket says 172.21.15.16:46816) 2023-11-13T10:56:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.156+0000 7f63916e3640 1 -- 172.21.15.16:0/1448027901 learned_addr learned my addr 172.21.15.16:0/1448027901 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.156+0000 7f6391ee4640 1 --2- 172.21.15.16:0/1448027901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f638c15ce70 0x7f638c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.156+0000 7f63916e3640 1 -- 172.21.15.16:0/1448027901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f638c15ce70 msgr2=0x7f638c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.156+0000 7f63916e3640 1 --2- 172.21.15.16:0/1448027901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f638c15ce70 0x7f638c0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.156+0000 7f63916e3640 1 -- 172.21.15.16:0/1448027901 >> v1:172.21.15.184:6789/0 conn(0x7f638c0d1850 legacy=0x7f638c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.156+0000 7f63916e3640 1 -- 172.21.15.16:0/1448027901 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c0ce6a0 con 0x7f638c0cdbb0 2023-11-13T10:56:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.157+0000 7f63916e3640 1 --2- 172.21.15.16:0/1448027901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 0x7f638c0d27c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f637c0098a0 tx=0x7f637c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b917942e91a76bfd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.158+0000 7f6390ee2640 1 -- 172.21.15.16:0/1448027901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c030840 con 0x7f638c0cdbb0 2023-11-13T10:56:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.158+0000 7f6390ee2640 1 -- 172.21.15.16:0/1448027901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c0309a0 con 0x7f638c0cdbb0 2023-11-13T10:56:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.158+0000 7f6390ee2640 1 -- 172.21.15.16:0/1448027901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c030d30 con 0x7f638c0cdbb0 2023-11-13T10:56:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.158+0000 7f6392ee6640 1 -- 172.21.15.16:0/1448027901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 msgr2=0x7f638c0d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.158+0000 7f6392ee6640 1 --2- 172.21.15.16:0/1448027901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 0x7f638c0d27c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f637c0098a0 tx=0x7f637c02b980 comp rx=0 tx=0).stop 2023-11-13T10:56:07.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.159+0000 7f6392ee6640 1 -- 172.21.15.16:0/1448027901 shutdown_connections 2023-11-13T10:56:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.159+0000 7f6392ee6640 1 --2- 172.21.15.16:0/1448027901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 0x7f638c0d27c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.159+0000 7f6392ee6640 1 --2- 172.21.15.16:0/1448027901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f638c15ce70 0x7f638c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.159+0000 7f6392ee6640 1 -- 172.21.15.16:0/1448027901 >> 172.21.15.16:0/1448027901 conn(0x7f638c08c580 msgr2=0x7f638c09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:07.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.159+0000 7f6392ee6640 1 -- 172.21.15.16:0/1448027901 shutdown_connections 2023-11-13T10:56:07.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.159+0000 7f6392ee6640 1 -- 172.21.15.16:0/1448027901 wait complete. 2023-11-13T10:56:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.161+0000 7f6392ee6640 1 Processor -- start 2023-11-13T10:56:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.161+0000 7f6392ee6640 1 -- start start 2023-11-13T10:56:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6392ee6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 0x7f638c0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6392ee6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638c0d1850 0x7f638c0c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6392ee6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f638c15ce70 0x7f638c1f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6392ee6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0b6770 con 0x7f638c0cdbb0 2023-11-13T10:56:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6392ee6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0b6a50 con 0x7f638c0d1850 2023-11-13T10:56:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6392ee6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0b6d30 con 0x7f638c15ce70 2023-11-13T10:56:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6391ee4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 0x7f638c0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6391ee4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 0x7f638c0b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46828/0 (socket says 172.21.15.16:46828) 2023-11-13T10:56:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6391ee4640 1 -- 172.21.15.16:0/367846847 learned_addr learned my addr 172.21.15.16:0/367846847 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f63916e3640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638c0d1850 0x7f638c0c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6391ee4640 1 -- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f638c15ce70 msgr2=0x7f638c1f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6391ee4640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f638c15ce70 0x7f638c1f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6391ee4640 1 -- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638c0d1850 msgr2=0x7f638c0c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6391ee4640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638c0d1850 0x7f638c0c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f6391ee4640 1 -- 172.21.15.16:0/367846847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0094d0 con 0x7f638c0cdbb0 2023-11-13T10:56:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.162+0000 7f63916e3640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638c0d1850 0x7f638c0c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.163+0000 7f6391ee4640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 0x7f638c0b6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f638800d5d0 tx=0x7f638800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:07.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.163+0000 7f63827fc640 1 -- 172.21.15.16:0/367846847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880125f0 con 0x7f638c0cdbb0 2023-11-13T10:56:07.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.163+0000 7f63827fc640 1 -- 172.21.15.16:0/367846847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6388012770 con 0x7f638c0cdbb0 2023-11-13T10:56:07.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.163+0000 7f63827fc640 1 -- 172.21.15.16:0/367846847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388012b20 con 0x7f638c0cdbb0 2023-11-13T10:56:07.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.163+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c0b7010 con 0x7f638c0cdbb0 2023-11-13T10:56:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.163+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c0c8740 con 0x7f638c0cdbb0 2023-11-13T10:56:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.164+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c04e4d0 con 0x7f638c0cdbb0 2023-11-13T10:56:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.165+0000 7f63827fc640 1 -- 172.21.15.16:0/367846847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6388012c80 con 0x7f638c0cdbb0 2023-11-13T10:56:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.166+0000 7f63827fc640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f637006ae40 0x7f637006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.166+0000 7f63827fc640 1 -- 172.21.15.16:0/367846847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f6388089ee0 con 0x7f638c0cdbb0 2023-11-13T10:56:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.167+0000 7f63916e3640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f637006ae40 0x7f637006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.168+0000 7f63916e3640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f637006ae40 0x7f637006d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f637c036000 tx=0x7f637c03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:07.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.171+0000 7f63827fc640 1 -- 172.21.15.16:0/367846847 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6388054c40 con 0x7f638c0cdbb0 2023-11-13T10:56:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.367+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f638c072a50 con 0x7f638c0cdbb0 2023-11-13T10:56:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.367+0000 7f63827fc640 1 -- 172.21.15.16:0/367846847 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6388058260 con 0x7f638c0cdbb0 2023-11-13T10:56:07.371 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:07.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f637006ae40 msgr2=0x7f637006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f637006ae40 0x7f637006d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f637c036000 tx=0x7f637c03b040 comp rx=0 tx=0).stop 2023-11-13T10:56:07.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 msgr2=0x7f638c0b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 0x7f638c0b6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f638800d5d0 tx=0x7f638800da90 comp rx=0 tx=0).stop 2023-11-13T10:56:07.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 shutdown_connections 2023-11-13T10:56:07.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f637006ae40 0x7f637006d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f638c15ce70 0x7f638c1f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f638c0d1850 0x7f638c0c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 --2- 172.21.15.16:0/367846847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f638c0cdbb0 0x7f638c0b6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 >> 172.21.15.16:0/367846847 conn(0x7f638c08c580 msgr2=0x7f638c09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:07.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.370+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 shutdown_connections 2023-11-13T10:56:07.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.371+0000 7f6392ee6640 1 -- 172.21.15.16:0/367846847 wait complete. 2023-11-13T10:56:07.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.622+0000 7effeb59e640 1 Processor -- start 2023-11-13T10:56:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.622+0000 7effeb59e640 1 -- start start 2023-11-13T10:56:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.624+0000 7effeb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec15bcc0 0x7effec0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.624+0000 7effeb59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effec0ce090 0x7effec0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.624+0000 7effeb59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec06d010 con 0x7effec15b8e0 2023-11-13T10:56:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.624+0000 7effeb59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effec0ce540 con 0x7effec0ce090 2023-11-13T10:56:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.624+0000 7effeb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec06b860 con 0x7effec15bcc0 2023-11-13T10:56:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.625+0000 7effea59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7effec15b8e0 0x7effec06c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49368/0 (socket says 172.21.15.16:49368) 2023-11-13T10:56:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.625+0000 7effea59c640 1 -- 172.21.15.16:0/4148762887 learned_addr learned my addr 172.21.15.16:0/4148762887 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.625+0000 7effead9d640 1 --2- 172.21.15.16:0/4148762887 >> v2:172.21.15.16:3300/0 conn(0x7effec0ce090 0x7effec0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.625+0000 7effe1d9b640 1 --2- 172.21.15.16:0/4148762887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec15bcc0 0x7effec0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.625+0000 7effe9d9b640 1 -- 172.21.15.16:0/4148762887 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683875618 0 0) 0x7effec06d010 con 0x7effec15b8e0 2023-11-13T10:56:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.625+0000 7effe9d9b640 1 -- 172.21.15.16:0/4148762887 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effd8003520 con 0x7effec15b8e0 2023-11-13T10:56:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.626+0000 7effead9d640 1 -- 172.21.15.16:0/4148762887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec15bcc0 msgr2=0x7effec0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.626+0000 7effead9d640 1 --2- 172.21.15.16:0/4148762887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec15bcc0 0x7effec0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.626+0000 7effead9d640 1 -- 172.21.15.16:0/4148762887 >> v1:172.21.15.16:6789/0 conn(0x7effec15b8e0 legacy=0x7effec06c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.626+0000 7effead9d640 1 -- 172.21.15.16:0/4148762887 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec06bb40 con 0x7effec0ce090 2023-11-13T10:56:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.627+0000 7effead9d640 1 --2- 172.21.15.16:0/4148762887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec0ce090 0x7effec0cbae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effe40097d0 tx=0x7effe402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83c379431dc2aa9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.627+0000 7effe9d9b640 1 -- 172.21.15.16:0/4148762887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe402d040 con 0x7effec0ce090 2023-11-13T10:56:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.627+0000 7effe9d9b640 1 -- 172.21.15.16:0/4148762887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe4031810 con 0x7effec0ce090 2023-11-13T10:56:07.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.627+0000 7effe9d9b640 1 -- 172.21.15.16:0/4148762887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4031ba0 con 0x7effec0ce090 2023-11-13T10:56:07.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.627+0000 7effeb59e640 1 -- 172.21.15.16:0/4148762887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec0ce090 msgr2=0x7effec0cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.627+0000 7effeb59e640 1 --2- 172.21.15.16:0/4148762887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec0ce090 0x7effec0cbae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effe40097d0 tx=0x7effe402b8b0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.628+0000 7effeb59e640 1 -- 172.21.15.16:0/4148762887 shutdown_connections 2023-11-13T10:56:07.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.628+0000 7effeb59e640 1 --2- 172.21.15.16:0/4148762887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec0ce090 0x7effec0cbae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.628+0000 7effeb59e640 1 --2- 172.21.15.16:0/4148762887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec15bcc0 0x7effec0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.628+0000 7effeb59e640 1 -- 172.21.15.16:0/4148762887 >> 172.21.15.16:0/4148762887 conn(0x7effec0b0970 msgr2=0x7effec0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:07.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.628+0000 7effeb59e640 1 -- 172.21.15.16:0/4148762887 shutdown_connections 2023-11-13T10:56:07.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.628+0000 7effeb59e640 1 -- 172.21.15.16:0/4148762887 wait complete. 2023-11-13T10:56:07.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.630+0000 7effeb59e640 1 Processor -- start 2023-11-13T10:56:07.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.630+0000 7effeb59e640 1 -- start start 2023-11-13T10:56:07.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effeb59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effec0ce090 0x7effec150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effeb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec15b8e0 0x7effec153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effeb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec151840 0x7effec153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effeb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec154170 con 0x7effec15b8e0 2023-11-13T10:56:07.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effeb59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec1556d0 con 0x7effec0ce090 2023-11-13T10:56:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effeb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec1559b0 con 0x7effec151840 2023-11-13T10:56:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effe1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec15b8e0 0x7effec153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effe1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec15b8e0 0x7effec153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46848/0 (socket says 172.21.15.16:46848) 2023-11-13T10:56:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effe1d9b640 1 -- 172.21.15.16:0/3598356390 learned_addr learned my addr 172.21.15.16:0/3598356390 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effea59c640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effec0ce090 0x7effec150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effead9d640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec151840 0x7effec153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effe1d9b640 1 -- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec151840 msgr2=0x7effec153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effe1d9b640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec151840 0x7effec153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effe1d9b640 1 -- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effec0ce090 msgr2=0x7effec150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effe1d9b640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effec0ce090 0x7effec150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.631+0000 7effe1d9b640 1 -- 172.21.15.16:0/3598356390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe4009400 con 0x7effec15b8e0 2023-11-13T10:56:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.632+0000 7effe1d9b640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec15b8e0 0x7effec153500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7effd400efc0 tx=0x7effd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.632+0000 7effe37fe640 1 -- 172.21.15.16:0/3598356390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4009b80 con 0x7effec15b8e0 2023-11-13T10:56:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.632+0000 7effe37fe640 1 -- 172.21.15.16:0/3598356390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd400f040 con 0x7effec15b8e0 2023-11-13T10:56:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.632+0000 7effe37fe640 1 -- 172.21.15.16:0/3598356390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40146a0 con 0x7effec15b8e0 2023-11-13T10:56:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.632+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec155c90 con 0x7effec15b8e0 2023-11-13T10:56:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.632+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec0ae7c0 con 0x7effec15b8e0 2023-11-13T10:56:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.633+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec0cc000 con 0x7effec15b8e0 2023-11-13T10:56:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.634+0000 7effe37fe640 1 -- 172.21.15.16:0/3598356390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effd4014800 con 0x7effec15b8e0 2023-11-13T10:56:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.635+0000 7effe37fe640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effcc06ae70 0x7effcc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.635+0000 7effe37fe640 1 -- 172.21.15.16:0/3598356390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6576+0+0 (secure 0 0 0) 0x7effd408a560 con 0x7effec15b8e0 2023-11-13T10:56:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.636+0000 7effea59c640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effcc06ae70 0x7effcc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.637+0000 7effea59c640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effcc06ae70 0x7effcc06d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7effdc0011e0 tx=0x7effdc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.640+0000 7effe37fe640 1 -- 172.21.15.16:0/3598356390 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7effec0cc000 con 0x7effec15b8e0 2023-11-13T10:56:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.832+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7effec0cc000 con 0x7effec15b8e0 2023-11-13T10:56:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.833+0000 7effe37fe640 1 -- 172.21.15.16:0/3598356390 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7effec0cc000 con 0x7effec15b8e0 2023-11-13T10:56:07.837 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:07.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.835+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effcc06ae70 msgr2=0x7effcc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.835+0000 7effeb59e640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effcc06ae70 0x7effcc06d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7effdc0011e0 tx=0x7effdc009040 comp rx=0 tx=0).stop 2023-11-13T10:56:07.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.835+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec15b8e0 msgr2=0x7effec153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:07.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.835+0000 7effeb59e640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec15b8e0 0x7effec153500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7effd400efc0 tx=0x7effd400c490 comp rx=0 tx=0).stop 2023-11-13T10:56:07.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.836+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 shutdown_connections 2023-11-13T10:56:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.836+0000 7effeb59e640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effcc06ae70 0x7effcc06d330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.836+0000 7effeb59e640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effec151840 0x7effec153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.836+0000 7effeb59e640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effec15b8e0 0x7effec153500 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.836+0000 7effeb59e640 1 --2- 172.21.15.16:0/3598356390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effec0ce090 0x7effec150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.836+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 >> 172.21.15.16:0/3598356390 conn(0x7effec0b0970 msgr2=0x7effec0ad0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:07.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.836+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 shutdown_connections 2023-11-13T10:56:07.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:07.836+0000 7effeb59e640 1 -- 172.21.15.16:0/3598356390 wait complete. 2023-11-13T10:56:07.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.087+0000 7fb1b5aec640 1 Processor -- start 2023-11-13T10:56:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.087+0000 7fb1b5aec640 1 -- start start 2023-11-13T10:56:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.089+0000 7fb1b5aec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb1b00b2290 0x7fb1b00ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.089+0000 7fb1b5aec640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb1b00b2720 0x7fb1b00adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.089+0000 7fb1b5aec640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b00ccab0 con 0x7fb1b0159bd0 2023-11-13T10:56:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.089+0000 7fb1b5aec640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b00ccd30 con 0x7fb1b00b2720 2023-11-13T10:56:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.089+0000 7fb1b5aec640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b00cd010 con 0x7fb1b00b2290 2023-11-13T10:56:08.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.089+0000 7fb1b52eb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb1b00b2720 0x7fb1b00adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.089+0000 7fb1b52eb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb1b00b2720 0x7fb1b00adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46864/0 (socket says 172.21.15.16:46864) 2023-11-13T10:56:08.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.089+0000 7fb1b52eb640 1 -- 172.21.15.16:0/3775290560 learned_addr learned my addr 172.21.15.16:0/3775290560 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.090+0000 7fb1affff640 1 --2- 172.21.15.16:0/3775290560 >> v2:172.21.15.184:3300/0 conn(0x7fb1b00b2290 0x7fb1b00ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.090+0000 7fb1b52eb640 1 -- 172.21.15.16:0/3775290560 >> v2:172.21.15.184:3300/0 conn(0x7fb1b00b2290 msgr2=0x7fb1b00ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.090+0000 7fb1b52eb640 1 --2- 172.21.15.16:0/3775290560 >> v2:172.21.15.184:3300/0 conn(0x7fb1b00b2290 0x7fb1b00ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.090+0000 7fb1b52eb640 1 -- 172.21.15.16:0/3775290560 >> v1:172.21.15.184:6789/0 conn(0x7fb1b0159bd0 legacy=0x7fb1b006f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.090+0000 7fb1b52eb640 1 -- 172.21.15.16:0/3775290560 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b00cd2f0 con 0x7fb1b00b2720 2023-11-13T10:56:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.091+0000 7fb1b52eb640 1 --2- 172.21.15.16:0/3775290560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 0x7fb1b00adc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00cbca0 tx=0x7fb1a802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37446547f99d71f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.091+0000 7fb1af7fe640 1 -- 172.21.15.16:0/3775290560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a802c9f0 con 0x7fb1b00b2720 2023-11-13T10:56:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.091+0000 7fb1af7fe640 1 -- 172.21.15.16:0/3775290560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a802cb50 con 0x7fb1b00b2720 2023-11-13T10:56:08.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.091+0000 7fb1af7fe640 1 -- 172.21.15.16:0/3775290560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80343f0 con 0x7fb1b00b2720 2023-11-13T10:56:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.092+0000 7fb1b5aec640 1 -- 172.21.15.16:0/3775290560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 msgr2=0x7fb1b00adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.092+0000 7fb1b5aec640 1 --2- 172.21.15.16:0/3775290560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 0x7fb1b00adc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00cbca0 tx=0x7fb1a802be50 comp rx=0 tx=0).stop 2023-11-13T10:56:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.092+0000 7fb1b5aec640 1 -- 172.21.15.16:0/3775290560 shutdown_connections 2023-11-13T10:56:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.092+0000 7fb1b5aec640 1 --2- 172.21.15.16:0/3775290560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 0x7fb1b00adc70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.092+0000 7fb1b5aec640 1 --2- 172.21.15.16:0/3775290560 >> v2:172.21.15.184:3300/0 conn(0x7fb1b00b2290 0x7fb1b00ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.092+0000 7fb1b5aec640 1 -- 172.21.15.16:0/3775290560 >> 172.21.15.16:0/3775290560 conn(0x7fb1b00b0970 msgr2=0x7fb1b00b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:08.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.092+0000 7fb1b5aec640 1 -- 172.21.15.16:0/3775290560 shutdown_connections 2023-11-13T10:56:08.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.092+0000 7fb1b5aec640 1 -- 172.21.15.16:0/3775290560 wait complete. 2023-11-13T10:56:08.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.094+0000 7fb1b5aec640 1 Processor -- start 2023-11-13T10:56:08.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.094+0000 7fb1b5aec640 1 -- start start 2023-11-13T10:56:08.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.095+0000 7fb1b5aec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1b00b2290 0x7fb1b01537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.095+0000 7fb1b5aec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 0x7fb1b014d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.095+0000 7fb1b5aec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1b015a9b0 0x7fb1b015c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.095+0000 7fb1b5aec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b015b190 con 0x7fb1b00b2720 2023-11-13T10:56:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.095+0000 7fb1b5aec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b015b470 con 0x7fb1b015a9b0 2023-11-13T10:56:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.095+0000 7fb1b5aec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b015b750 con 0x7fb1b00b2290 2023-11-13T10:56:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.095+0000 7fb1affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 0x7fb1b014d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.095+0000 7fb1affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 0x7fb1b014d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46868/0 (socket says 172.21.15.16:46868) 2023-11-13T10:56:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.095+0000 7fb1affff640 1 -- 172.21.15.16:0/4131382399 learned_addr learned my addr 172.21.15.16:0/4131382399 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1b4aea640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1b00b2290 0x7fb1b01537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1b52eb640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1b015a9b0 0x7fb1b015c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1affff640 1 -- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1b00b2290 msgr2=0x7fb1b01537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1affff640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1b00b2290 0x7fb1b01537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1affff640 1 -- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1b015a9b0 msgr2=0x7fb1b015c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1affff640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1b015a9b0 0x7fb1b015c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1affff640 1 -- 172.21.15.16:0/4131382399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8009b30 con 0x7fb1b00b2720 2023-11-13T10:56:08.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1affff640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 0x7fb1b014d210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00d800 tx=0x7fb19c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1ad7fa640 1 -- 172.21.15.16:0/4131382399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00b6c0 con 0x7fb1b00b2720 2023-11-13T10:56:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1ad7fa640 1 -- 172.21.15.16:0/4131382399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00b820 con 0x7fb1b00b2720 2023-11-13T10:56:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.096+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b015ba30 con 0x7fb1b00b2720 2023-11-13T10:56:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.097+0000 7fb1ad7fa640 1 -- 172.21.15.16:0/4131382399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00bbb0 con 0x7fb1b00b2720 2023-11-13T10:56:08.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.097+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b015ae30 con 0x7fb1b00b2720 2023-11-13T10:56:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.098+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b004e4d0 con 0x7fb1b00b2720 2023-11-13T10:56:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.099+0000 7fb1ad7fa640 1 -- 172.21.15.16:0/4131382399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb19c00bd10 con 0x7fb1b00b2720 2023-11-13T10:56:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.100+0000 7fb1ad7fa640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb19406ae70 0x7fb19406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.100+0000 7fb1ad7fa640 1 -- 172.21.15.16:0/4131382399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fb19c089ec0 con 0x7fb1b00b2720 2023-11-13T10:56:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.100+0000 7fb1b4aea640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb19406ae70 0x7fb19406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.101+0000 7fb1b4aea640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb19406ae70 0x7fb19406d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4006fd0 tx=0x7fb1a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:08.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.104+0000 7fb1ad7fa640 1 -- 172.21.15.16:0/4131382399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb19c054c20 con 0x7fb1b00b2720 2023-11-13T10:56:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.303+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb1b0154880 con 0x7fb1b00b2720 2023-11-13T10:56:08.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.304+0000 7fb1ad7fa640 1 -- 172.21.15.16:0/4131382399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb19c058240 con 0x7fb1b00b2720 2023-11-13T10:56:08.307 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.306+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb19406ae70 msgr2=0x7fb19406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.306+0000 7fb1b5aec640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb19406ae70 0x7fb19406d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4006fd0 tx=0x7fb1a4008040 comp rx=0 tx=0).stop 2023-11-13T10:56:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.306+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 msgr2=0x7fb1b014d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.306+0000 7fb1b5aec640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 0x7fb1b014d210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00d800 tx=0x7fb19c00dcc0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.307+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 shutdown_connections 2023-11-13T10:56:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.307+0000 7fb1b5aec640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb19406ae70 0x7fb19406d330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.307+0000 7fb1b5aec640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1b015a9b0 0x7fb1b015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.307+0000 7fb1b5aec640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b00b2720 0x7fb1b014d210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.307+0000 7fb1b5aec640 1 --2- 172.21.15.16:0/4131382399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1b00b2290 0x7fb1b01537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.307+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 >> 172.21.15.16:0/4131382399 conn(0x7fb1b00b0970 msgr2=0x7fb1b0158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.307+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 shutdown_connections 2023-11-13T10:56:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.307+0000 7fb1b5aec640 1 -- 172.21.15.16:0/4131382399 wait complete. 2023-11-13T10:56:08.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:56:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.559+0000 7ff2c0165640 1 Processor -- start 2023-11-13T10:56:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.559+0000 7ff2c0165640 1 -- start start 2023-11-13T10:56:08.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.561+0000 7ff2c0165640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff2b80cfe20 0x7ff2b80b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.561+0000 7ff2c0165640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b80d3c70 con 0x7ff2b815d0f0 2023-11-13T10:56:08.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.561+0000 7ff2c0165640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b80b45a0 con 0x7ff2b80d19d0 2023-11-13T10:56:08.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.561+0000 7ff2c0165640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b80c69d0 con 0x7ff2b80cfe20 2023-11-13T10:56:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.561+0000 7ff2bdf01640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff2b815d0f0 0x7ff2b80d12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49376/0 (socket says 172.21.15.16:49376) 2023-11-13T10:56:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.561+0000 7ff2bdf01640 1 -- 172.21.15.16:0/3073155873 learned_addr learned my addr 172.21.15.16:0/3073155873 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2be702640 1 --2- 172.21.15.16:0/3073155873 >> v2:172.21.15.184:3300/0 conn(0x7ff2b80cfe20 0x7ff2b80b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014073328 0 0) 0x7ff2a8001490 con 0x7ff2b815d0f0 2023-11-13T10:56:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2a0003520 con 0x7ff2b815d0f0 2023-11-13T10:56:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2876415941 0 0) 0x7ff2a0003520 con 0x7ff2b815d0f0 2023-11-13T10:56:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 >> v2:172.21.15.184:3300/0 conn(0x7ff2b80cfe20 msgr2=0x7ff2b80b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 --2- 172.21.15.16:0/3073155873 >> v2:172.21.15.184:3300/0 conn(0x7ff2b80cfe20 0x7ff2b80b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 >> v1:172.21.15.184:6789/0 conn(0x7ff2b80d19d0 legacy=0x7ff2b80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b80c6cb0 con 0x7ff2b815d0f0 2023-11-13T10:56:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7ff2a8001b80 con 0x7ff2b815d0f0 2023-11-13T10:56:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 >> v1:172.21.15.16:6789/0 conn(0x7ff2b815d0f0 legacy=0x7ff2b80d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2a0003970 0x7ff2a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a00047c0 0x7ff2a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80c69d0 con 0x7ff2a0005180 2023-11-13T10:56:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a000c030 con 0x7ff2a00047c0 2023-11-13T10:56:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.562+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a000c1f0 con 0x7ff2a0003970 2023-11-13T10:56:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.563+0000 7ff2be702640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.563+0000 7ff2bdf01640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2a0003970 0x7ff2a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.563+0000 7ff2bd700640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a00047c0 0x7ff2a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.563+0000 7ff2be702640 1 -- 172.21.15.16:0/3073155873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2a0003970 msgr2=0x7ff2a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.563+0000 7ff2be702640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2a0003970 0x7ff2a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.563+0000 7ff2be702640 1 -- 172.21.15.16:0/3073155873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a00047c0 msgr2=0x7ff2a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.563+0000 7ff2be702640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2a00047c0 0x7ff2a0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.563+0000 7ff2be702640 1 -- 172.21.15.16:0/3073155873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a0003520 con 0x7ff2a0005180 2023-11-13T10:56:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.564+0000 7ff2be702640 1 -- 172.21.15.16:0/3073155873 reap_dead start 2023-11-13T10:56:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.564+0000 7ff2be702640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2a000b980 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009ac0 tx=0x7ff2a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c78fe7a25666ad server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.564+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0004020 con 0x7ff2a0005180 2023-11-13T10:56:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.564+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b0004180 con 0x7ff2a0005180 2023-11-13T10:56:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.565+0000 7ff2bceff640 1 -- 172.21.15.16:0/3073155873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b002c710 con 0x7ff2a0005180 2023-11-13T10:56:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.565+0000 7ff2c0165640 1 -- 172.21.15.16:0/3073155873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 msgr2=0x7ff2a000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.565+0000 7ff2c0165640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2a000b980 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009ac0 tx=0x7ff2a00057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.565+0000 7ff2c0165640 1 -- 172.21.15.16:0/3073155873 shutdown_connections 2023-11-13T10:56:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.565+0000 7ff2c0165640 1 --2- 172.21.15.16:0/3073155873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2a000b980 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.565+0000 7ff2c0165640 1 -- 172.21.15.16:0/3073155873 >> 172.21.15.16:0/3073155873 conn(0x7ff2b80d2ce0 msgr2=0x7ff2b80d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.565+0000 7ff2c0165640 1 -- 172.21.15.16:0/3073155873 shutdown_connections 2023-11-13T10:56:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.565+0000 7ff2c0165640 1 -- 172.21.15.16:0/3073155873 wait complete. 2023-11-13T10:56:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.567+0000 7ff2c0165640 1 Processor -- start 2023-11-13T10:56:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.567+0000 7ff2c0165640 1 -- start start 2023-11-13T10:56:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.568+0000 7ff2c0165640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2b80b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.568+0000 7ff2c0165640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b80b8610 0x7ff2b815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.568+0000 7ff2c0165640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b815c000 0x7ff2b80cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.568+0000 7ff2c0165640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b81e6f60 con 0x7ff2a0005180 2023-11-13T10:56:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.568+0000 7ff2c0165640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80b7340 con 0x7ff2b815c000 2023-11-13T10:56:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.568+0000 7ff2c0165640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80b8180 con 0x7ff2b80b8610 2023-11-13T10:56:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.569+0000 7ff2bdf01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2b80b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.569+0000 7ff2bdf01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2b80b6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46876/0 (socket says 172.21.15.16:46876) 2023-11-13T10:56:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.569+0000 7ff2bdf01640 1 -- 172.21.15.16:0/855964956 learned_addr learned my addr 172.21.15.16:0/855964956 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.569+0000 7ff2bdf01640 1 -- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b80b8610 msgr2=0x7ff2b815b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.569+0000 7ff2bdf01640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b80b8610 0x7ff2b815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.569+0000 7ff2bdf01640 1 -- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b815c000 msgr2=0x7ff2b80cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.569+0000 7ff2bdf01640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b815c000 0x7ff2b80cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.569+0000 7ff2bdf01640 1 -- 172.21.15.16:0/855964956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b80b7570 con 0x7ff2a0005180 2023-11-13T10:56:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.569+0000 7ff2bdf01640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2b80b6e00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80015f0 tx=0x7ff2a800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.570+0000 7ff2b67fc640 1 -- 172.21.15.16:0/855964956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800ee50 con 0x7ff2a0005180 2023-11-13T10:56:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.570+0000 7ff2b67fc640 1 -- 172.21.15.16:0/855964956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a8013420 con 0x7ff2a0005180 2023-11-13T10:56:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.570+0000 7ff2b67fc640 1 -- 172.21.15.16:0/855964956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80137d0 con 0x7ff2a0005180 2023-11-13T10:56:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.570+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b80b7850 con 0x7ff2a0005180 2023-11-13T10:56:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.570+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b80b7e10 con 0x7ff2a0005180 2023-11-13T10:56:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.571+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b80b7fa0 con 0x7ff2a0005180 2023-11-13T10:56:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.572+0000 7ff2b67fc640 1 -- 172.21.15.16:0/855964956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2a8013930 con 0x7ff2a0005180 2023-11-13T10:56:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.573+0000 7ff2b67fc640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2a406ae70 0x7ff2a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.573+0000 7ff2b67fc640 1 -- 172.21.15.16:0/855964956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7ff2a808a170 con 0x7ff2a0005180 2023-11-13T10:56:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.574+0000 7ff2bd700640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2a406ae70 0x7ff2a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.575+0000 7ff2bd700640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2a406ae70 0x7ff2a406d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80b8a50 tx=0x7ff2ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.577+0000 7ff2b67fc640 1 -- 172.21.15.16:0/855964956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2a8054db0 con 0x7ff2a0005180 2023-11-13T10:56:08.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.810+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b80d3c70 con 0x7ff2a0005180 2023-11-13T10:56:08.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.811+0000 7ff2b67fc640 1 -- 172.21.15.16:0/855964956 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7ff2b80d3c70 con 0x7ff2a0005180 2023-11-13T10:56:08.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:08.816 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:53.560913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-13T10:56:08.816 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-11-13T10:56:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2a406ae70 msgr2=0x7ff2a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2a406ae70 0x7ff2a406d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80b8a50 tx=0x7ff2ac009040 comp rx=0 tx=0).stop 2023-11-13T10:56:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 msgr2=0x7ff2b80b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2b80b6e00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80015f0 tx=0x7ff2a800e530 comp rx=0 tx=0).stop 2023-11-13T10:56:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 shutdown_connections 2023-11-13T10:56:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2a406ae70 0x7ff2a406d330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2b815c000 0x7ff2b80cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2b80b8610 0x7ff2b815b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 --2- 172.21.15.16:0/855964956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a0005180 0x7ff2b80b6e00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 >> 172.21.15.16:0/855964956 conn(0x7ff2b80d2ce0 msgr2=0x7ff2b806ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 shutdown_connections 2023-11-13T10:56:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:08.814+0000 7ff2c0165640 1 -- 172.21.15.16:0/855964956 wait complete. 2023-11-13T10:56:08.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.059+0000 7f194a3fb640 1 Processor -- start 2023-11-13T10:56:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.060+0000 7f194a3fb640 1 -- start start 2023-11-13T10:56:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.061+0000 7f194a3fb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f19440ccc60 0x7f1944071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.061+0000 7f194a3fb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1944071980 0x7f194406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.061+0000 7f194a3fb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19440cd110 con 0x7f1944155410 2023-11-13T10:56:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.061+0000 7f194a3fb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1944071e10 con 0x7f1944071980 2023-11-13T10:56:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.061+0000 7f194a3fb640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194415b500 con 0x7f19440ccc60 2023-11-13T10:56:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.061+0000 7f1943fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1944155410 0x7f19440ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49390/0 (socket says 172.21.15.16:49390) 2023-11-13T10:56:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.061+0000 7f1943fff640 1 -- 172.21.15.16:0/2634630513 learned_addr learned my addr 172.21.15.16:0/2634630513 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.061+0000 7f1948998640 1 --2- 172.21.15.16:0/2634630513 >> v2:172.21.15.16:3300/0 conn(0x7f1944071980 0x7f194406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.062+0000 7f19437fe640 1 --2- 172.21.15.16:0/2634630513 >> v2:172.21.15.184:3300/0 conn(0x7f19440ccc60 0x7f1944071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.062+0000 7f1942ffd640 1 -- 172.21.15.16:0/2634630513 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197442937 0 0) 0x7f19440cd110 con 0x7f1944155410 2023-11-13T10:56:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.062+0000 7f1942ffd640 1 -- 172.21.15.16:0/2634630513 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f192c003520 con 0x7f1944155410 2023-11-13T10:56:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.062+0000 7f1948998640 1 -- 172.21.15.16:0/2634630513 >> v2:172.21.15.184:3300/0 conn(0x7f19440ccc60 msgr2=0x7f1944071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.062+0000 7f1948998640 1 --2- 172.21.15.16:0/2634630513 >> v2:172.21.15.184:3300/0 conn(0x7f19440ccc60 0x7f1944071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.062+0000 7f1948998640 1 -- 172.21.15.16:0/2634630513 >> v1:172.21.15.16:6789/0 conn(0x7f1944155410 legacy=0x7f19440ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.062+0000 7f1948998640 1 -- 172.21.15.16:0/2634630513 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194415b7e0 con 0x7f1944071980 2023-11-13T10:56:09.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.063+0000 7f1948998640 1 --2- 172.21.15.16:0/2634630513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1944071980 0x7f194406fe40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f194406e000 tx=0x7f193c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b2fd5115610397 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:09.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.063+0000 7f1942ffd640 1 -- 172.21.15.16:0/2634630513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c02cc70 con 0x7f1944071980 2023-11-13T10:56:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.063+0000 7f1942ffd640 1 -- 172.21.15.16:0/2634630513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c02cdd0 con 0x7f1944071980 2023-11-13T10:56:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.064+0000 7f1942ffd640 1 -- 172.21.15.16:0/2634630513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c034640 con 0x7f1944071980 2023-11-13T10:56:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.064+0000 7f194a3fb640 1 -- 172.21.15.16:0/2634630513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1944071980 msgr2=0x7f194406fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.064+0000 7f194a3fb640 1 --2- 172.21.15.16:0/2634630513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1944071980 0x7f194406fe40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f194406e000 tx=0x7f193c02bec0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.064+0000 7f194a3fb640 1 -- 172.21.15.16:0/2634630513 shutdown_connections 2023-11-13T10:56:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.064+0000 7f194a3fb640 1 --2- 172.21.15.16:0/2634630513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1944071980 0x7f194406fe40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.064+0000 7f194a3fb640 1 --2- 172.21.15.16:0/2634630513 >> v2:172.21.15.184:3300/0 conn(0x7f19440ccc60 0x7f1944071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.064+0000 7f194a3fb640 1 -- 172.21.15.16:0/2634630513 >> 172.21.15.16:0/2634630513 conn(0x7f194406a190 msgr2=0x7f194406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.064+0000 7f194a3fb640 1 -- 172.21.15.16:0/2634630513 shutdown_connections 2023-11-13T10:56:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.064+0000 7f194a3fb640 1 -- 172.21.15.16:0/2634630513 wait complete. 2023-11-13T10:56:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.066+0000 7f194a3fb640 1 Processor -- start 2023-11-13T10:56:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.066+0000 7f194a3fb640 1 -- start start 2023-11-13T10:56:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f194a3fb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1944071980 0x7f1944144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f194a3fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440ccc60 0x7f194406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f194a3fb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1944155410 0x7f19440ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f194a3fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194406bf30 con 0x7f19440ccc60 2023-11-13T10:56:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f194a3fb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194406c210 con 0x7f1944071980 2023-11-13T10:56:09.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f194a3fb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194406c4f0 con 0x7f1944155410 2023-11-13T10:56:09.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f19437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440ccc60 0x7f194406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f19437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440ccc60 0x7f194406d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46896/0 (socket says 172.21.15.16:46896) 2023-11-13T10:56:09.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f19437fe640 1 -- 172.21.15.16:0/180769719 learned_addr learned my addr 172.21.15.16:0/180769719 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:09.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f1948998640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1944155410 0x7f19440ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f19437fe640 1 -- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1944155410 msgr2=0x7f19440ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f19437fe640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1944155410 0x7f19440ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f19437fe640 1 -- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1944071980 msgr2=0x7f1944144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f19437fe640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1944071980 0x7f1944144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.068+0000 7f19437fe640 1 -- 172.21.15.16:0/180769719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c009a70 con 0x7f19440ccc60 2023-11-13T10:56:09.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.069+0000 7f19437fe640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440ccc60 0x7f194406d4c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f193800d800 tx=0x7f193800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:09.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.069+0000 7f1940ff9640 1 -- 172.21.15.16:0/180769719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193800b6c0 con 0x7f19440ccc60 2023-11-13T10:56:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.069+0000 7f1940ff9640 1 -- 172.21.15.16:0/180769719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193800b820 con 0x7f19440ccc60 2023-11-13T10:56:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.069+0000 7f1940ff9640 1 -- 172.21.15.16:0/180769719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193800bbb0 con 0x7f19440ccc60 2023-11-13T10:56:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.069+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194406c7d0 con 0x7f19440ccc60 2023-11-13T10:56:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.069+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1944144840 con 0x7f19440ccc60 2023-11-13T10:56:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.070+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1944067810 con 0x7f19440ccc60 2023-11-13T10:56:09.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.071+0000 7f1940ff9640 1 -- 172.21.15.16:0/180769719 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f193800bd10 con 0x7f19440ccc60 2023-11-13T10:56:09.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.072+0000 7f1940ff9640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f193006ae40 0x7f193006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.072+0000 7f1940ff9640 1 -- 172.21.15.16:0/180769719 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f1938089f80 con 0x7f19440ccc60 2023-11-13T10:56:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.073+0000 7f1943fff640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f193006ae40 0x7f193006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.074+0000 7f1943fff640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f193006ae40 0x7f193006d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1934001610 tx=0x7f1934007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.077+0000 7f1940ff9640 1 -- 172.21.15.16:0/180769719 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1938054c50 con 0x7f19440ccc60 2023-11-13T10:56:09.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.278+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1944067090 con 0x7f19440ccc60 2023-11-13T10:56:09.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.280+0000 7f1940ff9640 1 -- 172.21.15.16:0/180769719 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f1938058270 con 0x7f19440ccc60 2023-11-13T10:56:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:09.286 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f193006ae40 msgr2=0x7f193006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f193006ae40 0x7f193006d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1934001610 tx=0x7f1934007930 comp rx=0 tx=0).stop 2023-11-13T10:56:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440ccc60 msgr2=0x7f194406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440ccc60 0x7f194406d4c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f193800d800 tx=0x7f193800dcc0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 shutdown_connections 2023-11-13T10:56:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f193006ae40 0x7f193006d300 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1934001610 tx=0x7f1934007930 comp rx=0 tx=0).stop 2023-11-13T10:56:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1944155410 0x7f19440ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19440ccc60 0x7f194406d4c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 --2- 172.21.15.16:0/180769719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1944071980 0x7f1944144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 >> 172.21.15.16:0/180769719 conn(0x7f194406a190 msgr2=0x7f1944154600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:09.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 shutdown_connections 2023-11-13T10:56:09.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.283+0000 7f194a3fb640 1 -- 172.21.15.16:0/180769719 wait complete. 2023-11-13T10:56:09.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:56:09.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.521+0000 7f7ea2a7e640 1 Processor -- start 2023-11-13T10:56:09.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.522+0000 7f7ea2a7e640 1 -- start start 2023-11-13T10:56:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.523+0000 7f7ea2a7e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e9c161060 0x7f7e9c1621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.523+0000 7f7ea2a7e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7e9c06a190 0x7f7e9c06a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.523+0000 7f7ea2a7e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c162700 con 0x7f7e9c06f970 2023-11-13T10:56:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.523+0000 7f7ea2a7e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c06ab60 con 0x7f7e9c161060 2023-11-13T10:56:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.523+0000 7f7ea2a7e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c0b1840 con 0x7f7e9c06a190 2023-11-13T10:56:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.524+0000 7f7ea1a7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e9c161060 0x7f7e9c1621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.524+0000 7f7ea1a7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e9c161060 0x7f7e9c1621c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46910/0 (socket says 172.21.15.16:46910) 2023-11-13T10:56:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.524+0000 7f7ea1a7c640 1 -- 172.21.15.16:0/916298446 learned_addr learned my addr 172.21.15.16:0/916298446 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.524+0000 7f7ea227d640 1 --2- 172.21.15.16:0/916298446 >> v2:172.21.15.184:3300/0 conn(0x7f7e9c06a190 0x7f7e9c06a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.524+0000 7f7ea1a7c640 1 -- 172.21.15.16:0/916298446 >> v2:172.21.15.184:3300/0 conn(0x7f7e9c06a190 msgr2=0x7f7e9c06a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.524+0000 7f7ea1a7c640 1 --2- 172.21.15.16:0/916298446 >> v2:172.21.15.184:3300/0 conn(0x7f7e9c06a190 0x7f7e9c06a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.524+0000 7f7ea1a7c640 1 -- 172.21.15.16:0/916298446 >> v1:172.21.15.184:6789/0 conn(0x7f7e9c06f970 legacy=0x7f7e9c162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:09.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.524+0000 7f7ea1a7c640 1 -- 172.21.15.16:0/916298446 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c071600 con 0x7f7e9c161060 2023-11-13T10:56:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.525+0000 7f7ea1a7c640 1 --2- 172.21.15.16:0/916298446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 0x7f7e9c1621c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0cde10 tx=0x7f7e9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5526d41efbc68be7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea0a7a640 1 -- 172.21.15.16:0/916298446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98031870 con 0x7f7e9c161060 2023-11-13T10:56:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea0a7a640 1 -- 172.21.15.16:0/916298446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e980319d0 con 0x7f7e9c161060 2023-11-13T10:56:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea0a7a640 1 -- 172.21.15.16:0/916298446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98031d60 con 0x7f7e9c161060 2023-11-13T10:56:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/916298446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 msgr2=0x7f7e9c1621c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea2a7e640 1 --2- 172.21.15.16:0/916298446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 0x7f7e9c1621c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0cde10 tx=0x7f7e9802b910 comp rx=0 tx=0).stop 2023-11-13T10:56:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/916298446 shutdown_connections 2023-11-13T10:56:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea2a7e640 1 --2- 172.21.15.16:0/916298446 >> v2:172.21.15.184:3300/0 conn(0x7f7e9c06a190 0x7f7e9c06a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea2a7e640 1 --2- 172.21.15.16:0/916298446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 0x7f7e9c1621c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/916298446 >> 172.21.15.16:0/916298446 conn(0x7f7e9c0b04a0 msgr2=0x7f7e9c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/916298446 shutdown_connections 2023-11-13T10:56:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.526+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/916298446 wait complete. 2023-11-13T10:56:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.528+0000 7f7ea2a7e640 1 Processor -- start 2023-11-13T10:56:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.528+0000 7f7ea2a7e640 1 -- start start 2023-11-13T10:56:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.529+0000 7f7ea2a7e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e9c06a190 0x7f7e9c15dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.529+0000 7f7ea2a7e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e9c06f970 0x7f7e9c1f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.529+0000 7f7ea2a7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 0x7f7e9c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.529+0000 7f7ea2a7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c159670 con 0x7f7e9c161060 2023-11-13T10:56:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.529+0000 7f7ea2a7e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c159340 con 0x7f7e9c06f970 2023-11-13T10:56:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.529+0000 7f7ea2a7e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1f7670 con 0x7f7e9c06a190 2023-11-13T10:56:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea227d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 0x7f7e9c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea227d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 0x7f7e9c155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46924/0 (socket says 172.21.15.16:46924) 2023-11-13T10:56:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea227d640 1 -- 172.21.15.16:0/4162174953 learned_addr learned my addr 172.21.15.16:0/4162174953 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea127b640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e9c06f970 0x7f7e9c1f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea1a7c640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e9c06a190 0x7f7e9c15dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea227d640 1 -- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e9c06a190 msgr2=0x7f7e9c15dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea227d640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e9c06a190 0x7f7e9c15dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea227d640 1 -- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e9c06f970 msgr2=0x7f7e9c1f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea227d640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e9c06f970 0x7f7e9c1f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea227d640 1 -- 172.21.15.16:0/4162174953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980094d0 con 0x7f7e9c161060 2023-11-13T10:56:09.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.530+0000 7f7ea227d640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 0x7f7e9c155b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e94007e40 tx=0x7f7e9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:09.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.531+0000 7f7e927fc640 1 -- 172.21.15.16:0/4162174953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94014620 con 0x7f7e9c161060 2023-11-13T10:56:09.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.531+0000 7f7e927fc640 1 -- 172.21.15.16:0/4162174953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9400ee90 con 0x7f7e9c161060 2023-11-13T10:56:09.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.531+0000 7f7e927fc640 1 -- 172.21.15.16:0/4162174953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940149e0 con 0x7f7e9c161060 2023-11-13T10:56:09.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.531+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c1f7950 con 0x7f7e9c161060 2023-11-13T10:56:09.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.531+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c1f72f0 con 0x7f7e9c161060 2023-11-13T10:56:09.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.532+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c159820 con 0x7f7e9c161060 2023-11-13T10:56:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.534+0000 7f7e927fc640 1 -- 172.21.15.16:0/4162174953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e94014b40 con 0x7f7e9c161060 2023-11-13T10:56:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.534+0000 7f7e927fc640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e8006ae40 0x7f7e8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.535+0000 7f7e927fc640 1 -- 172.21.15.16:0/4162174953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f7e9408a320 con 0x7f7e9c161060 2023-11-13T10:56:09.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.535+0000 7f7ea1a7c640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e8006ae40 0x7f7e8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:09.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.536+0000 7f7ea1a7c640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e8006ae40 0x7f7e8006d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e98036000 tx=0x7f7e9803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:09.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.539+0000 7f7e927fc640 1 -- 172.21.15.16:0/4162174953 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e9c159820 con 0x7f7e9c161060 2023-11-13T10:56:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.769+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e9c04e4d0 con 0x7f7e9c161060 2023-11-13T10:56:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.770+0000 7f7e927fc640 1 -- 172.21.15.16:0/4162174953 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f7e94055070 con 0x7f7e9c161060 2023-11-13T10:56:09.774 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:09.774 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:53.560913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-13T10:56:09.775 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-11-13T10:56:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.772+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e8006ae40 msgr2=0x7f7e8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e8006ae40 0x7f7e8006d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e98036000 tx=0x7f7e9803c040 comp rx=0 tx=0).stop 2023-11-13T10:56:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 msgr2=0x7f7e9c155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 0x7f7e9c155b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e94007e40 tx=0x7f7e9400e4b0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 shutdown_connections 2023-11-13T10:56:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e8006ae40 0x7f7e8006d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c161060 0x7f7e9c155b40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e9c06f970 0x7f7e9c1f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 --2- 172.21.15.16:0/4162174953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e9c06a190 0x7f7e9c15dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 >> 172.21.15.16:0/4162174953 conn(0x7f7e9c0b04a0 msgr2=0x7f7e9c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 shutdown_connections 2023-11-13T10:56:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:09.773+0000 7f7ea2a7e640 1 -- 172.21.15.16:0/4162174953 wait complete. 2023-11-13T10:56:10.199 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-13T10:55:26.312366+0000', 'modified': '2023-11-13T10:55:53.560913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4251}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4251': {'gid': 4251, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/3826676517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 3826676517}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 3826676517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-13T10:56:10.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-13T10:56:10.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-13T10:56:10.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.436+0000 7fce31f61640 1 Processor -- start 2023-11-13T10:56:10.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.436+0000 7fce31f61640 1 -- start start 2023-11-13T10:56:10.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce31f61640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fce2c15cf20 0x7fce2c06cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce31f61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 0x7fce2c0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce31f61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce2c0b2d90 0x7fce2c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce31f61640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c15d300 con 0x7fce2c0b2d90 2023-11-13T10:56:10.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce31f61640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c0b2900 con 0x7fce2c15cf20 2023-11-13T10:56:10.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce31f61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0b1fd0 con 0x7fce2c06bb90 2023-11-13T10:56:10.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce2bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce2c0b2d90 0x7fce2c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce2bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce2c0b2d90 0x7fce2c0ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46932/0 (socket says 172.21.15.16:46932) 2023-11-13T10:56:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce2bfff640 1 -- 172.21.15.16:0/1436994555 learned_addr learned my addr 172.21.15.16:0/1436994555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/1436994555 >> v2:172.21.15.184:3300/0 conn(0x7fce2c15cf20 0x7fce2c06cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.438+0000 7fce2affd640 1 --2- 172.21.15.16:0/1436994555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 0x7fce2c0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.439+0000 7fce2bfff640 1 -- 172.21.15.16:0/1436994555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 msgr2=0x7fce2c0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.439+0000 7fce2bfff640 1 --2- 172.21.15.16:0/1436994555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 0x7fce2c0ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.439+0000 7fce2bfff640 1 -- 172.21.15.16:0/1436994555 >> v2:172.21.15.184:3300/0 conn(0x7fce2c15cf20 msgr2=0x7fce2c06cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.439+0000 7fce2bfff640 1 --2- 172.21.15.16:0/1436994555 >> v2:172.21.15.184:3300/0 conn(0x7fce2c15cf20 0x7fce2c06cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.439+0000 7fce2bfff640 1 -- 172.21.15.16:0/1436994555 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0b22b0 con 0x7fce2c0b2d90 2023-11-13T10:56:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.440+0000 7fce2bfff640 1 --2- 172.21.15.16:0/1436994555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 0x7fce2c0ceaf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0cf9f0 tx=0x7fce2402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cf62f05b95b90b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.440+0000 7fce2a7fc640 1 -- 172.21.15.16:0/1436994555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24031860 con 0x7fce2c0b2d90 2023-11-13T10:56:10.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.440+0000 7fce2a7fc640 1 -- 172.21.15.16:0/1436994555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce240319c0 con 0x7fce2c0b2d90 2023-11-13T10:56:10.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.440+0000 7fce2a7fc640 1 -- 172.21.15.16:0/1436994555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24031d50 con 0x7fce2c0b2d90 2023-11-13T10:56:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.441+0000 7fce31f61640 1 -- 172.21.15.16:0/1436994555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 msgr2=0x7fce2c0ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:10.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.441+0000 7fce31f61640 1 --2- 172.21.15.16:0/1436994555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 0x7fce2c0ceaf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0cf9f0 tx=0x7fce2402b900 comp rx=0 tx=0).stop 2023-11-13T10:56:10.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.441+0000 7fce31f61640 1 -- 172.21.15.16:0/1436994555 shutdown_connections 2023-11-13T10:56:10.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.441+0000 7fce31f61640 1 --2- 172.21.15.16:0/1436994555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 0x7fce2c0ceaf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.441+0000 7fce31f61640 1 --2- 172.21.15.16:0/1436994555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 0x7fce2c0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.441+0000 7fce31f61640 1 --2- 172.21.15.16:0/1436994555 >> v2:172.21.15.184:3300/0 conn(0x7fce2c15cf20 0x7fce2c06cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.441+0000 7fce31f61640 1 -- 172.21.15.16:0/1436994555 >> 172.21.15.16:0/1436994555 conn(0x7fce2c06a190 msgr2=0x7fce2c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.441+0000 7fce31f61640 1 -- 172.21.15.16:0/1436994555 shutdown_connections 2023-11-13T10:56:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.441+0000 7fce31f61640 1 -- 172.21.15.16:0/1436994555 wait complete. 2023-11-13T10:56:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.444+0000 7fce31f61640 1 Processor -- start 2023-11-13T10:56:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.444+0000 7fce31f61640 1 -- start start 2023-11-13T10:56:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.445+0000 7fce31f61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 0x7fce2c1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.445+0000 7fce31f61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 0x7fce2c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.445+0000 7fce31f61640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2c15cf20 0x7fce2c0b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.445+0000 7fce31f61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c1e24a0 con 0x7fce2c0b2d90 2023-11-13T10:56:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.445+0000 7fce31f61640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c155b60 con 0x7fce2c15cf20 2023-11-13T10:56:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.445+0000 7fce31f61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c155e40 con 0x7fce2c06bb90 2023-11-13T10:56:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.445+0000 7fce2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 0x7fce2c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.445+0000 7fce2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 0x7fce2c0afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46936/0 (socket says 172.21.15.16:46936) 2023-11-13T10:56:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.445+0000 7fce2affd640 1 -- 172.21.15.16:0/3031441296 learned_addr learned my addr 172.21.15.16:0/3031441296 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2bfff640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2c15cf20 0x7fce2c0b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 0x7fce2c1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2affd640 1 -- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 msgr2=0x7fce2c1516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2affd640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 0x7fce2c1516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2affd640 1 -- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2c15cf20 msgr2=0x7fce2c0b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2affd640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2c15cf20 0x7fce2c0b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2affd640 1 -- 172.21.15.16:0/3031441296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240094d0 con 0x7fce2c0b2d90 2023-11-13T10:56:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2bfff640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2c15cf20 0x7fce2c0b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 0x7fce2c1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.446+0000 7fce2affd640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 0x7fce2c0afec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce1400d800 tx=0x7fce1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.447+0000 7fce23fff640 1 -- 172.21.15.16:0/3031441296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce14009640 con 0x7fce2c0b2d90 2023-11-13T10:56:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.447+0000 7fce23fff640 1 -- 172.21.15.16:0/3031441296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce140097a0 con 0x7fce2c0b2d90 2023-11-13T10:56:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.447+0000 7fce31f61640 1 -- 172.21.15.16:0/3031441296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c156120 con 0x7fce2c0b2d90 2023-11-13T10:56:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.447+0000 7fce23fff640 1 -- 172.21.15.16:0/3031441296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce14009b30 con 0x7fce2c0b2d90 2023-11-13T10:56:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.447+0000 7fce31f61640 1 -- 172.21.15.16:0/3031441296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c1566e0 con 0x7fce2c0b2d90 2023-11-13T10:56:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.449+0000 7fce23fff640 1 -- 172.21.15.16:0/3031441296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce14009c90 con 0x7fce2c0b2d90 2023-11-13T10:56:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.450+0000 7fce23fff640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce0c06ae40 0x7fce0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.450+0000 7fce23fff640 1 -- 172.21.15.16:0/3031441296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fce14089dc0 con 0x7fce2c0b2d90 2023-11-13T10:56:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.451+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce0c06ae40 0x7fce0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.451+0000 7fce34bd4740 1 Processor -- start 2023-11-13T10:56:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.451+0000 7fce34bd4740 1 -- start start 2023-11-13T10:56:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce0c06ae40 0x7fce0c06d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fce2c1e2cb0 tx=0x7fce1c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123de0 0x5630a81241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 0x5630a8122fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123520 0x5630a81674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce34bd4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a8082880 con 0x5630a8123520 2023-11-13T10:56:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce34bd4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a8091ae0 con 0x5630a8123de0 2023-11-13T10:56:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce34bd4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630a808f1d0 con 0x5630a811eae0 2023-11-13T10:56:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123520 0x5630a81674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123520 0x5630a81674a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46944/0 (socket says 172.21.15.16:46944) 2023-11-13T10:56:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.452+0000 7fce2bfff640 1 -- 172.21.15.16:0/2351832425 learned_addr learned my addr 172.21.15.16:0/2351832425 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/2351832425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123de0 0x5630a81241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce2affd640 1 --2- 172.21.15.16:0/2351832425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 0x5630a8122fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce2bfff640 1 -- 172.21.15.16:0/2351832425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 msgr2=0x5630a8122fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce2bfff640 1 --2- 172.21.15.16:0/2351832425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 0x5630a8122fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce2bfff640 1 -- 172.21.15.16:0/2351832425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123de0 msgr2=0x5630a81241c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce2bfff640 1 --2- 172.21.15.16:0/2351832425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123de0 0x5630a81241c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce2bfff640 1 -- 172.21.15.16:0/2351832425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630a8052860 con 0x5630a8123520 2023-11-13T10:56:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce2bfff640 1 --2- 172.21.15.16:0/2351832425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123520 0x5630a81674a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce24009930 tx=0x7fce24008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31ecc412970ad1 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce217fa640 1 -- 172.21.15.16:0/2351832425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce240317e0 con 0x5630a8123520 2023-11-13T10:56:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce217fa640 1 -- 172.21.15.16:0/2351832425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24031b30 con 0x5630a8123520 2023-11-13T10:56:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.453+0000 7fce217fa640 1 -- 172.21.15.16:0/2351832425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2403a420 con 0x5630a8123520 2023-11-13T10:56:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.454+0000 7fce34bd4740 1 -- 172.21.15.16:0/2351832425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123520 msgr2=0x5630a81674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.454+0000 7fce34bd4740 1 --2- 172.21.15.16:0/2351832425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123520 0x5630a81674a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce24009930 tx=0x7fce24008030 comp rx=0 tx=0).stop 2023-11-13T10:56:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.454+0000 7fce34bd4740 1 -- 172.21.15.16:0/2351832425 shutdown_connections 2023-11-13T10:56:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.454+0000 7fce34bd4740 1 --2- 172.21.15.16:0/2351832425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123520 0x5630a81674a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.454+0000 7fce34bd4740 1 --2- 172.21.15.16:0/2351832425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 0x5630a8122fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.454+0000 7fce34bd4740 1 --2- 172.21.15.16:0/2351832425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123de0 0x5630a81241c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.454+0000 7fce34bd4740 1 -- 172.21.15.16:0/2351832425 >> 172.21.15.16:0/2351832425 conn(0x5630a80e5bb0 msgr2=0x5630a81698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.454+0000 7fce34bd4740 1 -- 172.21.15.16:0/2351832425 shutdown_connections 2023-11-13T10:56:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.454+0000 7fce34bd4740 1 -- 172.21.15.16:0/2351832425 wait complete. 2023-11-13T10:56:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.455+0000 7fce34bd4740 1 -- start start 2023-11-13T10:56:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.455+0000 7fce34bd4740 1 Processor -- start 2023-11-13T10:56:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 0x5630a8148f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a81494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123de0 0x5630a81499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce34bd4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a808aac0 con 0x5630a8123de0 2023-11-13T10:56:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce34bd4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a80272f0 con 0x5630a8123520 2023-11-13T10:56:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce34bd4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630a8044c70 con 0x5630a811eae0 2023-11-13T10:56:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce34bd4740 10 client.0 _ll_register_callbacks cb 0x5630a80493b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T10:56:10.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123de0 0x5630a81499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123de0 0x5630a81499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46956/0 (socket says 172.21.15.16:46956) 2023-11-13T10:56:10.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.456+0000 7fce2bfff640 1 -- 172.21.15.16:0/4191059295 learned_addr learned my addr 172.21.15.16:0/4191059295 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:10.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 0x5630a8148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce2affd640 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a81494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce2bfff640 1 -- 172.21.15.16:0/4191059295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 msgr2=0x5630a8148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce2bfff640 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 0x5630a8148f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce2bfff640 1 -- 172.21.15.16:0/4191059295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 msgr2=0x5630a81494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce2bfff640 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a81494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce2bfff640 1 -- 172.21.15.16:0/4191059295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c0b22b0 con 0x5630a8123de0 2023-11-13T10:56:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce2bfff640 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123de0 0x5630a81499e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce24043000 tx=0x7fce24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e6e1778e20c0fe server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T10:56:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce0a7fc640 1 -- 172.21.15.16:0/4191059295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24003810 con 0x5630a8123de0 2023-11-13T10:56:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce0a7fc640 1 -- 172.21.15.16:0/4191059295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fce24003970 con 0x5630a8123de0 2023-11-13T10:56:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.457+0000 7fce34bd4740 1 -- 172.21.15.16:0/4191059295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630a8083aa0 con 0x5630a8123de0 2023-11-13T10:56:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce0a7fc640 1 -- 172.21.15.16:0/4191059295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce24003b30 con 0x5630a8123de0 2023-11-13T10:56:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce34bd4740 10 client.4806 fetch_fsmap learned FSMap version 9 2023-11-13T10:56:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce34bd4740 1 -- 172.21.15.16:0/4191059295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5630a8089140 con 0x5630a8123de0 2023-11-13T10:56:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce0a7fc640 1 -- 172.21.15.16:0/4191059295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5630a8083aa0 con 0x5630a8123de0 2023-11-13T10:56:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce0a7fc640 1 -- 172.21.15.16:0/4191059295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fce2404b030 con 0x5630a8123de0 2023-11-13T10:56:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce34bd4740 10 client.4806 fetch_fsmap finished waiting for FSMap version 9 2023-11-13T10:56:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce34bd4740 10 client.4806 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T10:56:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce34bd4740 20 client.4806 populate_metadata read hostname 'smithi016' 2023-11-13T10:56:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce34bd4740 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x5630a814d200 0x5630a8186d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce34bd4740 4 client.4806 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T10:56:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce34bd4740 1 -- 172.21.15.16:0/4191059295 --> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630a80837c0 con 0x5630a814d200 2023-11-13T10:56:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.458+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x5630a814d200 0x5630a8186d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.459+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x5630a814d200 0x5630a8186d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a87e2e91465062ef server_cookie=c56dc8c4bf7bf14d in_seq=0 out_seq=0 2023-11-13T10:56:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.459+0000 7fce0a7fc640 10 client.4806 ms_handle_connect on v2:172.21.15.16:6834/3826676517 2023-11-13T10:56:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce0a7fc640 1 -- 172.21.15.16:0/4191059295 <== mds.0 v2:172.21.15.16:6834/3826676517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5630a80837c0 con 0x5630a814d200 2023-11-13T10:56:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce0a7fc640 10 client.4806 handle_command_reply: tid=0 2023-11-13T10:56:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 client.4806 shutdown 2023-11-13T10:56:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 -- 172.21.15.16:0/4191059295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123de0 msgr2=0x5630a81499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123de0 0x5630a81499e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce24043000 tx=0x7fce24004730 comp rx=0 tx=0).stop 2023-11-13T10:56:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 -- 172.21.15.16:0/4191059295 shutdown_connections 2023-11-13T10:56:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x5630a814d200 0x5630a8186d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a8123de0 0x5630a81499e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a81494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 --2- 172.21.15.16:0/4191059295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a811eae0 0x5630a8148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce0a7fc640 0 client.4806 ms_handle_reset on v2:172.21.15.16:6834/3826676517 2023-11-13T10:56:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 -- 172.21.15.16:0/4191059295 >> 172.21.15.16:0/4191059295 conn(0x5630a80e5bb0 msgr2=0x5630a811dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 -- 172.21.15.16:0/4191059295 shutdown_connections 2023-11-13T10:56:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 1 -- 172.21.15.16:0/4191059295 wait complete. 2023-11-13T10:56:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.460+0000 7fce34bd4740 20 client.4806 trim_cache size 0 max 16384 2023-11-13T10:56:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.478+0000 7fce34bd4740 1 Processor -- start 2023-11-13T10:56:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.478+0000 7fce34bd4740 1 -- start start 2023-11-13T10:56:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a811e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a8179080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a8123de0 0x5630a8168d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce34bd4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a7dedf20 con 0x5630a811eae0 2023-11-13T10:56:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce34bd4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a8124b00 con 0x5630a8123520 2023-11-13T10:56:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce34bd4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630a8178b30 con 0x5630a8123de0 2023-11-13T10:56:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a811e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a811e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46960/0 (socket says 172.21.15.16:46960) 2023-11-13T10:56:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce2b7fe640 1 -- 172.21.15.16:0/1342832841 learned_addr learned my addr 172.21.15.16:0/1342832841 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce2b7fe640 1 -- 172.21.15.16:0/1342832841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a8123de0 msgr2=0x5630a8168d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/1342832841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a8123de0 0x5630a8168d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce2b7fe640 1 -- 172.21.15.16:0/1342832841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 msgr2=0x5630a8179080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/1342832841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a8179080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.479+0000 7fce2b7fe640 1 -- 172.21.15.16:0/1342832841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630a8026b80 con 0x5630a811eae0 2023-11-13T10:56:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.480+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/1342832841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a811e230 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009530 tx=0x7fce1c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea1e67c33c0c3c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.480+0000 7fce21ffb640 1 -- 172.21.15.16:0/1342832841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c003c50 con 0x5630a811eae0 2023-11-13T10:56:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.480+0000 7fce21ffb640 1 -- 172.21.15.16:0/1342832841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c003db0 con 0x5630a811eae0 2023-11-13T10:56:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.480+0000 7fce21ffb640 1 -- 172.21.15.16:0/1342832841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c004250 con 0x5630a811eae0 2023-11-13T10:56:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 -- 172.21.15.16:0/1342832841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 msgr2=0x5630a811e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 --2- 172.21.15.16:0/1342832841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a811e230 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009530 tx=0x7fce1c012000 comp rx=0 tx=0).stop 2023-11-13T10:56:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 -- 172.21.15.16:0/1342832841 shutdown_connections 2023-11-13T10:56:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 --2- 172.21.15.16:0/1342832841 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a8123de0 0x5630a8168d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 --2- 172.21.15.16:0/1342832841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a8179080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 --2- 172.21.15.16:0/1342832841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a811e230 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 -- 172.21.15.16:0/1342832841 >> 172.21.15.16:0/1342832841 conn(0x5630a80e5bb0 msgr2=0x5630a811a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 -- 172.21.15.16:0/1342832841 shutdown_connections 2023-11-13T10:56:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 -- 172.21.15.16:0/1342832841 wait complete. 2023-11-13T10:56:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.481+0000 7fce34bd4740 1 -- start start 2023-11-13T10:56:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.482+0000 7fce34bd4740 1 Processor -- start 2023-11-13T10:56:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a8167bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a814b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce34bd4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a8123de0 0x5630a814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce34bd4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a81793b0 con 0x5630a811eae0 2023-11-13T10:56:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce34bd4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a814bc30 con 0x5630a8123520 2023-11-13T10:56:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce34bd4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630a8187540 con 0x5630a8123de0 2023-11-13T10:56:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce34bd4740 10 client.0 _ll_register_callbacks cb 0x7fce1c007f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T10:56:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a8167bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a8167bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46974/0 (socket says 172.21.15.16:46974) 2023-11-13T10:56:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2b7fe640 1 -- 172.21.15.16:0/3602251196 learned_addr learned my addr 172.21.15.16:0/3602251196 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2affd640 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a814b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2bfff640 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a8123de0 0x5630a814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2b7fe640 1 -- 172.21.15.16:0/3602251196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a8123de0 msgr2=0x5630a814b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a8123de0 0x5630a814b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2b7fe640 1 -- 172.21.15.16:0/3602251196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 msgr2=0x5630a814b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a814b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2b7fe640 1 -- 172.21.15.16:0/3602251196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c01e040 con 0x5630a811eae0 2023-11-13T10:56:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.483+0000 7fce2affd640 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.484+0000 7fce2b7fe640 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a8167bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fce1c003d40 tx=0x7fce1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1edd3e0222ea90b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.484+0000 7fce0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T10:56:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.484+0000 7fce0a7fc640 1 -- 172.21.15.16:0/3602251196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c027240 con 0x5630a811eae0 2023-11-13T10:56:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.484+0000 7fce0a7fc640 1 -- 172.21.15.16:0/3602251196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fce1c0273a0 con 0x5630a811eae0 2023-11-13T10:56:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.484+0000 7fce34bd4740 1 -- 172.21.15.16:0/3602251196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce24031b30 con 0x5630a811eae0 2023-11-13T10:56:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.484+0000 7fce0a7fc640 1 -- 172.21.15.16:0/3602251196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce1c027560 con 0x5630a811eae0 2023-11-13T10:56:10.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.484+0000 7fce34bd4740 10 client.4812 fetch_fsmap learned FSMap version 9 2023-11-13T10:56:10.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.484+0000 7fce34bd4740 1 -- 172.21.15.16:0/3602251196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5630a804c170 con 0x5630a811eae0 2023-11-13T10:56:10.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.485+0000 7fce0a7fc640 1 -- 172.21.15.16:0/3602251196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24031b30 con 0x5630a811eae0 2023-11-13T10:56:10.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.485+0000 7fce0a7fc640 1 -- 172.21.15.16:0/3602251196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fce1c0276c0 con 0x5630a811eae0 2023-11-13T10:56:10.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.485+0000 7fce34bd4740 10 client.4812 fetch_fsmap finished waiting for FSMap version 9 2023-11-13T10:56:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.485+0000 7fce34bd4740 10 client.4812 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T10:56:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.485+0000 7fce34bd4740 20 client.4812 populate_metadata read hostname 'smithi016' 2023-11-13T10:56:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.485+0000 7fce34bd4740 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x5630a81518a0 0x5630a8151c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:10.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.485+0000 7fce34bd4740 4 client.4812 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-13T10:56:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.485+0000 7fce34bd4740 1 -- 172.21.15.16:0/3602251196 --> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5630a7d8b6c0 con 0x5630a81518a0 2023-11-13T10:56:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.485+0000 7fce2affd640 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x5630a81518a0 0x5630a8151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce2affd640 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x5630a81518a0 0x5630a8151c80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86c88aa39513917 server_cookie=a2eb8f45a26e046 in_seq=0 out_seq=0 2023-11-13T10:56:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce0a7fc640 10 client.4812 ms_handle_connect on v2:172.21.15.16:6834/3826676517 2023-11-13T10:56:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce0a7fc640 1 -- 172.21.15.16:0/3602251196 <== mds.0 v2:172.21.15.16:6834/3826676517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5630a7d8b6c0 con 0x5630a81518a0 2023-11-13T10:56:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce0a7fc640 10 client.4812 handle_command_reply: tid=0 2023-11-13T10:56:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 client.4812 shutdown 2023-11-13T10:56:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 -- 172.21.15.16:0/3602251196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 msgr2=0x5630a8167bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a8167bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fce1c003d40 tx=0x7fce1c004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 -- 172.21.15.16:0/3602251196 shutdown_connections 2023-11-13T10:56:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.16:6834/3826676517,v1:172.21.15.16:6835/3826676517] conn(0x5630a81518a0 0x5630a8151c80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5630a8123de0 0x5630a814b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5630a8123520 0x5630a814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce0a7fc640 0 client.4812 ms_handle_reset on v2:172.21.15.16:6834/3826676517 2023-11-13T10:56:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 --2- 172.21.15.16:0/3602251196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5630a811eae0 0x5630a8167bb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 -- 172.21.15.16:0/3602251196 >> 172.21.15.16:0/3602251196 conn(0x5630a80e5bb0 msgr2=0x5630a81226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 -- 172.21.15.16:0/3602251196 shutdown_connections 2023-11-13T10:56:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.486+0000 7fce34bd4740 1 -- 172.21.15.16:0/3602251196 wait complete. 2023-11-13T10:56:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce34bd4740 20 client.4812 trim_cache size 0 max 16384 2023-11-13T10:56:10.582 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T10:56:10.582 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9", 2023-11-13T10:56:10.583 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-13T10:56:10.583 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4251, 2023-11-13T10:56:10.583 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-13T10:56:10.584 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-13T10:56:10.584 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-13T10:56:10.584 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 43.640420218000003, 2023-11-13T10:56:10.584 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 9, 2023-11-13T10:56:10.584 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 28, 2023-11-13T10:56:10.585 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 26, 2023-11-13T10:56:10.585 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 49.386657552000003 2023-11-13T10:56:10.585 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T10:56:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 -- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce0c06ae40 msgr2=0x7fce0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:10.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce0c06ae40 0x7fce0c06d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fce2c1e2cb0 tx=0x7fce1c00b040 comp rx=0 tx=0).stop 2023-11-13T10:56:10.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 -- 172.21.15.16:0/3031441296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 msgr2=0x7fce2c0afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:10.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 0x7fce2c0afec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce1400d800 tx=0x7fce1400dcc0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 -- 172.21.15.16:0/3031441296 shutdown_connections 2023-11-13T10:56:10.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce0c06ae40 0x7fce0c06d300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2c15cf20 0x7fce2c0b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2c0b2d90 0x7fce2c0afec0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 --2- 172.21.15.16:0/3031441296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce2c06bb90 0x7fce2c1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:10.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 -- 172.21.15.16:0/3031441296 >> 172.21.15.16:0/3031441296 conn(0x7fce2c06a190 msgr2=0x7fce2c15beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 -- 172.21.15.16:0/3031441296 shutdown_connections 2023-11-13T10:56:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:10.487+0000 7fce31f61640 1 -- 172.21.15.16:0/3031441296 wait complete. 2023-11-13T10:56:10.603 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-13T10:56:10.622 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-13T10:56:10.622 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-13T10:56:10.623 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-13T10:56:10.623 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-13T10:56:10.623 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:10.624 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-13T10:56:10.624 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-13T10:56:10.624 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-13T10:56:10.624 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-13T10:56:10.636 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-13T10:56:10.636 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:10.646 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-13T10:56:10.647 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-13T10:56:10.647 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-13T10:56:10.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:10.648 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-11-13T10:56:10.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:10.714 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-11-13T10:56:10.868 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2023-11-13T10:56:10.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T10:56:10.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:10.871 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-13T10:56:10.938 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-13T10:56:10.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:10.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-13T10:56:11.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:11.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2023-11-13T10:56:11.201 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-13T10:56:11.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-13T10:56:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.397+0000 7f80a194c640 1 Processor -- start 2023-11-13T10:56:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.397+0000 7f80a194c640 1 -- start start 2023-11-13T10:56:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.399+0000 7f80a194c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f809c071b50 0x7f809c06fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.399+0000 7f80a194c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809c0cd110 con 0x7f809c155410 2023-11-13T10:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.399+0000 7f80a194c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809c15b500 con 0x7f809c0ccc60 2023-11-13T10:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.399+0000 7f80a194c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809c15b7e0 con 0x7f809c071b50 2023-11-13T10:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.399+0000 7f809affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f809c155410 0x7f809c0ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49392/0 (socket says 172.21.15.16:49392) 2023-11-13T10:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.399+0000 7f809affd640 1 -- 172.21.15.16:0/2140770464 learned_addr learned my addr 172.21.15.16:0/2140770464 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.399+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638521692 0 0) 0x7f809c0cd110 con 0x7f809c155410 2023-11-13T10:56:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.399+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8084003520 con 0x7f809c155410 2023-11-13T10:56:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f809b7fe640 1 --2- 172.21.15.16:0/2140770464 >> v2:172.21.15.184:3300/0 conn(0x7f809c071b50 0x7f809c06fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 159654718 0 0) 0x7f8084003520 con 0x7f809c155410 2023-11-13T10:56:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 >> v2:172.21.15.184:3300/0 conn(0x7f809c071b50 msgr2=0x7f809c06fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 --2- 172.21.15.16:0/2140770464 >> v2:172.21.15.184:3300/0 conn(0x7f809c071b50 0x7f809c06fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 >> v1:172.21.15.184:6789/0 conn(0x7f809c0ccc60 legacy=0x7f809c071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c15bac0 con 0x7f809c155410 2023-11-13T10:56:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f808c001670 con 0x7f809c155410 2023-11-13T10:56:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 >> v1:172.21.15.16:6789/0 conn(0x7f809c155410 legacy=0x7f809c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8084003970 0x7f8084003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f8084004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8084005180 0x7f808400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c15b7e0 con 0x7f80840047c0 2023-11-13T10:56:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808400c0d0 con 0x7f8084003970 2023-11-13T10:56:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808400c290 con 0x7f8084005180 2023-11-13T10:56:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.400+0000 7f809a7fc640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f8084004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.401+0000 7f809affd640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8084003970 0x7f8084003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.401+0000 7f809b7fe640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8084005180 0x7f808400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.401+0000 7f809a7fc640 1 -- 172.21.15.16:0/2140770464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8084005180 msgr2=0x7f808400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.401+0000 7f809a7fc640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8084005180 0x7f808400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.401+0000 7f809a7fc640 1 -- 172.21.15.16:0/2140770464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8084003970 msgr2=0x7f8084003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.401+0000 7f809a7fc640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8084003970 0x7f8084003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.401+0000 7f809a7fc640 1 -- 172.21.15.16:0/2140770464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8084003520 con 0x7f80840047c0 2023-11-13T10:56:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.401+0000 7f809b7fe640 1 -- 172.21.15.16:0/2140770464 reap_dead start 2023-11-13T10:56:11.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.402+0000 7f809a7fc640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f8084004c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8090001750 tx=0x7f80840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d1dab0d9932ce5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:11.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.402+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80900052d0 con 0x7f80840047c0 2023-11-13T10:56:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.402+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809002e050 con 0x7f80840047c0 2023-11-13T10:56:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.402+0000 7f8099ffb640 1 -- 172.21.15.16:0/2140770464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090003830 con 0x7f80840047c0 2023-11-13T10:56:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.403+0000 7f80a194c640 1 -- 172.21.15.16:0/2140770464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 msgr2=0x7f8084004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.403+0000 7f80a194c640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f8084004c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8090001750 tx=0x7f80840057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.403+0000 7f80a194c640 1 -- 172.21.15.16:0/2140770464 shutdown_connections 2023-11-13T10:56:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.403+0000 7f80a194c640 1 --2- 172.21.15.16:0/2140770464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f8084004c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.403+0000 7f80a194c640 1 -- 172.21.15.16:0/2140770464 >> 172.21.15.16:0/2140770464 conn(0x7f809c06a190 msgr2=0x7f809c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:11.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.403+0000 7f80a194c640 1 -- 172.21.15.16:0/2140770464 shutdown_connections 2023-11-13T10:56:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.403+0000 7f80a194c640 1 -- 172.21.15.16:0/2140770464 wait complete. 2023-11-13T10:56:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.405+0000 7f80a194c640 1 Processor -- start 2023-11-13T10:56:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.405+0000 7f80a194c640 1 -- start start 2023-11-13T10:56:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.406+0000 7f80a194c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f809c146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.406+0000 7f80a194c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c156170 0x7f809c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.406+0000 7f80a194c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f809c157ca0 0x7f809c1e3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.406+0000 7f80a194c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c157340 con 0x7f80840047c0 2023-11-13T10:56:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.406+0000 7f80a194c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1566c0 con 0x7f809c157ca0 2023-11-13T10:56:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.406+0000 7f80a194c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1569a0 con 0x7f809c156170 2023-11-13T10:56:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.406+0000 7f809affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f809c146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f809c146720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46988/0 (socket says 172.21.15.16:46988) 2023-11-13T10:56:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809affd640 1 -- 172.21.15.16:0/2930663655 learned_addr learned my addr 172.21.15.16:0/2930663655 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809a7fc640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c156170 0x7f809c157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809b7fe640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f809c157ca0 0x7f809c1e3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809affd640 1 -- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c156170 msgr2=0x7f809c157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809affd640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c156170 0x7f809c157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809affd640 1 -- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f809c157ca0 msgr2=0x7f809c1e3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809affd640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f809c157ca0 0x7f809c1e3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809affd640 1 -- 172.21.15.16:0/2930663655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c156c80 con 0x7f80840047c0 2023-11-13T10:56:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.407+0000 7f809affd640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f809c146720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f808c0011e0 tx=0x7f808c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:11.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.408+0000 7f8083fff640 1 -- 172.21.15.16:0/2930663655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c001650 con 0x7f80840047c0 2023-11-13T10:56:11.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.408+0000 7f8083fff640 1 -- 172.21.15.16:0/2930663655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c00f940 con 0x7f80840047c0 2023-11-13T10:56:11.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.408+0000 7f8083fff640 1 -- 172.21.15.16:0/2930663655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c00fcd0 con 0x7f80840047c0 2023-11-13T10:56:11.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.408+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c156f60 con 0x7f80840047c0 2023-11-13T10:56:11.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.408+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c1476d0 con 0x7f80840047c0 2023-11-13T10:56:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.409+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c067090 con 0x7f80840047c0 2023-11-13T10:56:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.410+0000 7f8083fff640 1 -- 172.21.15.16:0/2930663655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f808c01d460 con 0x7f80840047c0 2023-11-13T10:56:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.411+0000 7f8083fff640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f808806ae40 0x7f808806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.411+0000 7f8083fff640 1 -- 172.21.15.16:0/2930663655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f808c089a30 con 0x7f80840047c0 2023-11-13T10:56:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.412+0000 7f809a7fc640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f808806ae40 0x7f808806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.413+0000 7f809a7fc640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f808806ae40 0x7f808806d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8090033000 tx=0x7f8090009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.416+0000 7f8083fff640 1 -- 172.21.15.16:0/2930663655 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f808c054780 con 0x7f80840047c0 2023-11-13T10:56:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.650+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f809c0b21f0 con 0x7f80840047c0 2023-11-13T10:56:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.651+0000 7f8083fff640 1 -- 172.21.15.16:0/2930663655 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f808c057da0 con 0x7f80840047c0 2023-11-13T10:56:11.655 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-13T10:56:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f808806ae40 msgr2=0x7f808806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f808806ae40 0x7f808806d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8090033000 tx=0x7f8090009030 comp rx=0 tx=0).stop 2023-11-13T10:56:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 msgr2=0x7f809c146720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f809c146720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f808c0011e0 tx=0x7f808c007720 comp rx=0 tx=0).stop 2023-11-13T10:56:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 shutdown_connections 2023-11-13T10:56:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f808806ae40 0x7f808806d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f809c157ca0 0x7f809c1e3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c156170 0x7f809c157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 --2- 172.21.15.16:0/2930663655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80840047c0 0x7f809c146720 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 >> 172.21.15.16:0/2930663655 conn(0x7f809c06a190 msgr2=0x7f809c0ad9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 shutdown_connections 2023-11-13T10:56:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:11.654+0000 7f80a194c640 1 -- 172.21.15.16:0/2930663655 wait complete. 2023-11-13T10:56:11.678 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-13T10:56:11.678 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-13T10:56:11.679 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-13T10:56:11.679 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-13T10:56:11.679 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:11.679 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-13T10:56:11.680 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-13T10:56:11.680 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-13T10:56:11.680 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-13T10:56:11.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:11.681 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-13T10:56:11.743 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:56:11.743 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:56:11.743 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:56:11.744 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:11.744 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-13T10:56:11.744 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:11.744 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:11.745 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:11.745 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-13T10:56:11.745 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:11.745 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:11.745 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-13T10:56:11.746 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-13T10:56:11.746 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:11.746 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-13T10:56:11.746 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-13T10:56:11.747 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42270sec preferred_lft 42270sec 2023-11-13T10:56:11.747 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-13T10:56:11.747 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:11.747 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:11.748 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:11.754 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-13T10:56:11.758 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-13T10:56:11.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:11.760 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:56:11.760 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:56:11.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2023-11-13T10:56:11.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2023-11-13T10:56:11.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2023-11-13T10:56:11.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-13T10:56:11.761 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:56:11.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:11.999 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-13T10:56:12.086 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-13T10:56:12.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:12.092 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-13T10:56:12.145 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-13T10:56:12.146 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-13T10:56:12.146 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-13T10:56:12.146 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-13T10:56:12.146 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-13T10:56:12.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:12.147 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:56:12.148 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:56:12.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-13T10:56:12.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-13T10:56:12.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-13T10:56:12.149 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:56:12.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:12.342 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-13T10:56:12.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:12.410 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-11-13T10:56:12.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:12.467 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:56:12.468 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:56:12.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:12.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-13T10:56:12.469 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:56:12.623 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-13T10:56:12.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:12.624 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:56:12.624 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:56:12.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-13T10:56:12.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-13T10:56:12.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-13T10:56:12.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-13T10:56:12.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-13T10:56:12.626 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:56:12.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:12.901 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:56:12.901 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:56:12.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-13T10:56:12.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2023-11-13T10:56:12.902 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:56:13.077 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-13T10:56:13.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:56:13.078 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:13.123 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-13T10:56:13.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:56:13.124 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:13.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T10:56:13.324 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:56:13.382 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T10:56:13.383 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T10:56:13.383 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T10:56:13.383 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T10:56:13.383 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T10:56:13.384 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T10:56:13.384 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T10:56:13.384 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T10:56:13.384 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T10:56:13.384 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T10:56:13.385 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T10:56:13.385 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:56:13.385 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T10:56:13.385 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T10:56:13.385 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T10:56:13.385 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T10:56:13.386 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:56:13.386 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T10:56:13.386 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T10:56:13.386 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:56:13.386 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:56:13.387 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:56:13.387 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T10:56:13.387 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T10:56:13.387 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T10:56:13.388 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T10:56:13.388 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T10:56:13.388 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T10:56:13.388 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T10:56:13.388 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:13.389 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:13.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:13.401 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T10:56:13.457 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-13T10:56:13.457 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-13T10:56:13.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T10:56:13.589 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:56:13.639 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T10:56:13.640 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T10:56:13.640 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T10:56:13.640 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T10:56:13.640 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T10:56:13.640 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T10:56:13.641 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T10:56:13.641 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T10:56:13.641 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T10:56:13.641 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T10:56:13.642 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T10:56:13.642 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:56:13.642 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T10:56:13.642 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T10:56:13.642 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T10:56:13.643 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T10:56:13.643 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:56:13.643 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T10:56:13.643 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T10:56:13.644 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:56:13.644 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:56:13.644 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:56:13.644 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T10:56:13.644 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T10:56:13.645 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T10:56:13.645 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T10:56:13.645 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T10:56:13.645 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T10:56:13.645 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T10:56:13.646 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:13.646 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:13.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:13.650 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T10:56:13.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-13T10:56:13.691+0000 7fcc9bc3a480 -1 init, newargv = 0x563ce4b1b170 newargc=15 2023-11-13T10:56:13.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[96211]: starting ceph client 2023-11-13T10:56:13.706 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[96211]: starting fuse 2023-11-13T10:56:13.735 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-13T10:56:13.735 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-13T10:56:13.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-13T10:56:13.736 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-13T10:56:13.736 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-13T10:56:13.737 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T10:56:13.737 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-13T10:56:13.737 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:56:13.738 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-13T10:56:13.740 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-13T10:56:13.740 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-13T10:56:13.740 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-13T10:56:13.741 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:56:13.744 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-13T10:56:13.744 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-13T10:56:13.744 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-13T10:56:13.745 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-13T10:56:13.745 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:56:13.745 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-13T10:56:13.746 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-13T10:56:13.746 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-13T10:56:13.746 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-13T10:56:13.747 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-13T10:56:13.751 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-13T10:56:13.751 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-13T10:56:13.751 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-13T10:56:13.752 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:56:13.752 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-13T10:56:13.752 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T10:56:13.946 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.96211.asok 2023-11-13T10:56:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T10:56:13.961 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.96211.asok 2023-11-13T10:56:13.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:13.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.96211.asok status 2023-11-13T10:56:14.185 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T10:56:14.185 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-13T10:56:14.186 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "35f0debb3cc28955b7132ef4c509cd138a09d56b", 2023-11-13T10:56:14.186 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7288-g35f0debb (35f0debb3cc28955b7132ef4c509cd138a09d56b) reef (dev)", 2023-11-13T10:56:14.186 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-13T10:56:14.186 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-13T10:56:14.187 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-13T10:56:14.187 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "96211", 2023-11-13T10:56:14.187 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-13T10:56:14.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:56:14.187 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-13T10:56:14.188 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-13T10:56:14.188 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4827, 2023-11-13T10:56:14.188 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-13T10:56:14.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-13T10:56:14.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-13T10:56:14.189 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4827 2023-11-13T10:56:14.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:56:14.189 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T10:56:14.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T10:56:14.189 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T10:56:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1355066039 2023-11-13T10:56:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:56:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:56:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T10:56:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T10:56:14.191 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T10:56:14.191 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1355066039 2023-11-13T10:56:14.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:56:14.191 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4827 192.168.0.1:0/1355066039", 2023-11-13T10:56:14.191 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1355066039", 2023-11-13T10:56:14.192 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-13T10:56:14.192 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 9, 2023-11-13T10:56:14.192 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 28, 2023-11-13T10:56:14.192 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-13T10:56:14.192 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-13T10:56:14.193 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-13T10:56:14.193 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T10:56:14.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-13T10:56:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.413+0000 7f94b359e640 1 Processor -- start 2023-11-13T10:56:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.413+0000 7f94b359e640 1 -- start start 2023-11-13T10:56:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.415+0000 7f94b359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b406a190 0x7f94b406a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.415+0000 7f94b359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b4158c20 con 0x7f94b415b190 2023-11-13T10:56:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.415+0000 7f94b359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b40b1830 con 0x7f94b40cb190 2023-11-13T10:56:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.415+0000 7f94b359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b406ab40 con 0x7f94b406a190 2023-11-13T10:56:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.415+0000 7f94b1d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f94b415b190 0x7f94b406f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46220/0 (socket says 172.21.15.16:46220) 2023-11-13T10:56:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.415+0000 7f94b1d9b640 1 -- 172.21.15.16:0/2756393339 learned_addr learned my addr 172.21.15.16:0/2756393339 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.415+0000 7f94b2d9d640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b406a190 0x7f94b406a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496757839 0 0) 0x7f94b4158c20 con 0x7f94b415b190 2023-11-13T10:56:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f949c003520 con 0x7f94b415b190 2023-11-13T10:56:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2962320560 0 0) 0x7f949c003520 con 0x7f94b415b190 2023-11-13T10:56:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b406a190 msgr2=0x7f94b406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b406a190 0x7f94b406a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 >> v1:172.21.15.184:6789/0 conn(0x7f94b40cb190 legacy=0x7f94b40716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b415b780 con 0x7f94b415b190 2023-11-13T10:56:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f94a8001690 con 0x7f94b415b190 2023-11-13T10:56:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 >> v1:172.21.15.16:6789/0 conn(0x7f94b415b190 legacy=0x7f94b406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f949c0039a0 0x7f949c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0047c0 0x7f949c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b406a190 0x7f949c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b406ab40 con 0x7f949c0047c0 2023-11-13T10:56:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c00bb40 con 0x7f949c0039a0 2023-11-13T10:56:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.416+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c00bd00 con 0x7f94b406a190 2023-11-13T10:56:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.417+0000 7f94b1d9b640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0047c0 0x7f949c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.417+0000 7f94b259c640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f949c0039a0 0x7f949c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.417+0000 7f94b2d9d640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b406a190 0x7f949c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.417+0000 7f94b1d9b640 1 -- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b406a190 msgr2=0x7f949c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.417+0000 7f94b1d9b640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b406a190 0x7f949c005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.417+0000 7f94b1d9b640 1 -- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f949c0039a0 msgr2=0x7f949c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.417+0000 7f94b1d9b640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f949c0039a0 0x7f949c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.417+0000 7f94b1d9b640 1 -- 172.21.15.16:0/2756393339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c003520 con 0x7f949c0047c0 2023-11-13T10:56:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.418+0000 7f94b1d9b640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0047c0 0x7f949c004c40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f94a8007fd0 tx=0x7f949c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fe8903f9b10272 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.418+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8005ad0 con 0x7f949c0047c0 2023-11-13T10:56:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.418+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a8005c30 con 0x7f949c0047c0 2023-11-13T10:56:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.418+0000 7f94b159a640 1 -- 172.21.15.16:0/2756393339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a802c7c0 con 0x7f949c0047c0 2023-11-13T10:56:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.419+0000 7f94b359e640 1 -- 172.21.15.16:0/2756393339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0047c0 msgr2=0x7f949c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.419+0000 7f94b359e640 1 --2- 172.21.15.16:0/2756393339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f949c0047c0 0x7f949c004c40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f94a8007fd0 tx=0x7f949c0057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.419+0000 7f94b2d9d640 1 -- 172.21.15.16:0/2756393339 reap_dead start 2023-11-13T10:56:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.419+0000 7f94b359e640 1 -- 172.21.15.16:0/2756393339 shutdown_connections 2023-11-13T10:56:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.419+0000 7f94b359e640 1 -- 172.21.15.16:0/2756393339 >> 172.21.15.16:0/2756393339 conn(0x7f94b40b04a0 msgr2=0x7f94b40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.419+0000 7f94b359e640 1 -- 172.21.15.16:0/2756393339 shutdown_connections 2023-11-13T10:56:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.419+0000 7f94b359e640 1 -- 172.21.15.16:0/2756393339 wait complete. 2023-11-13T10:56:14.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.421+0000 7f94b359e640 1 Processor -- start 2023-11-13T10:56:14.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.421+0000 7f94b359e640 1 -- start start 2023-11-13T10:56:14.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.422+0000 7f94b359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f949c0047c0 0x7f94b4154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:14.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.422+0000 7f94b359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b41f3dd0 0x7f94b41f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.422+0000 7f94b359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b41f4250 0x7f94b41f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.422+0000 7f94b359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41f10a0 con 0x7f94b41f3dd0 2023-11-13T10:56:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.422+0000 7f94b359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41f1380 con 0x7f949c0047c0 2023-11-13T10:56:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.422+0000 7f94b359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4153300 con 0x7f94b41f4250 2023-11-13T10:56:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.422+0000 7f94b1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b41f3dd0 0x7f94b41f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.422+0000 7f94b1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b41f3dd0 0x7f94b41f0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43950/0 (socket says 172.21.15.16:43950) 2023-11-13T10:56:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.422+0000 7f94b1d9b640 1 -- 172.21.15.16:0/3698184154 learned_addr learned my addr 172.21.15.16:0/3698184154 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b259c640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f949c0047c0 0x7f94b4154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b2d9d640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b41f4250 0x7f94b41f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:14.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b1d9b640 1 -- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b41f4250 msgr2=0x7f94b41f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:14.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b1d9b640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b41f4250 0x7f94b41f0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b1d9b640 1 -- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f949c0047c0 msgr2=0x7f94b4154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:14.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b1d9b640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f949c0047c0 0x7f94b4154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b1d9b640 1 -- 172.21.15.16:0/3698184154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b41535e0 con 0x7f94b41f3dd0 2023-11-13T10:56:14.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b1d9b640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b41f3dd0 0x7f94b41f0570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94a800c1d0 tx=0x7f94b40b17a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:14.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94a2ffd640 1 -- 172.21.15.16:0/3698184154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a803c070 con 0x7f94b41f3dd0 2023-11-13T10:56:14.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94a2ffd640 1 -- 172.21.15.16:0/3698184154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a802ca90 con 0x7f94b41f3dd0 2023-11-13T10:56:14.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94a2ffd640 1 -- 172.21.15.16:0/3698184154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a802ce20 con 0x7f94b41f3dd0 2023-11-13T10:56:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b4152ff0 con 0x7f94b41f3dd0 2023-11-13T10:56:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.423+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b4154cb0 con 0x7f94b41f3dd0 2023-11-13T10:56:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.425+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b4153770 con 0x7f94b41f3dd0 2023-11-13T10:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.425+0000 7f94a2ffd640 1 -- 172.21.15.16:0/3698184154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94a8032070 con 0x7f94b41f3dd0 2023-11-13T10:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.426+0000 7f94a2ffd640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f949406ae40 0x7f949406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.426+0000 7f94a2ffd640 1 -- 172.21.15.16:0/3698184154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f94a80aa880 con 0x7f94b41f3dd0 2023-11-13T10:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.427+0000 7f94b259c640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f949406ae40 0x7f949406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.428+0000 7f94b259c640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f949406ae40 0x7f949406d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f94b40cbd30 tx=0x7f94a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.434+0000 7f94a2ffd640 1 -- 172.21.15.16:0/3698184154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94a80755e0 con 0x7f94b41f3dd0 2023-11-13T10:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.688+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f94b41f1540 con 0x7f94b41f3dd0 2023-11-13T10:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.688+0000 7f94a2ffd640 1 -- 172.21.15.16:0/3698184154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f94a8078c00 con 0x7f94b41f3dd0 2023-11-13T10:56:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-13T10:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f949406ae40 msgr2=0x7f949406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f949406ae40 0x7f949406d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f94b40cbd30 tx=0x7f94a4009040 comp rx=0 tx=0).stop 2023-11-13T10:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b41f3dd0 msgr2=0x7f94b41f0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b41f3dd0 0x7f94b41f0570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94a800c1d0 tx=0x7f94b40b17a0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 shutdown_connections 2023-11-13T10:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f949406ae40 0x7f949406d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94b41f4250 0x7f94b41f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b41f3dd0 0x7f94b41f0570 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 --2- 172.21.15.16:0/3698184154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f949c0047c0 0x7f94b4154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 >> 172.21.15.16:0/3698184154 conn(0x7f94b40b04a0 msgr2=0x7f94b40cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 shutdown_connections 2023-11-13T10:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:14.691+0000 7f94b359e640 1 -- 172.21.15.16:0/3698184154 wait complete. 2023-11-13T10:56:14.714 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-13T10:56:14.714 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-13T10:56:14.715 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-13T10:56:14.715 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-13T10:56:14.715 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-13T10:56:14.716 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-13T10:56:14.716 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-13T10:56:14.716 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-13T10:56:14.717 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-13T10:56:14.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:14.717 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-13T10:56:14.781 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:56:14.781 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:56:14.782 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:56:14.782 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:14.782 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-13T10:56:14.782 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:14.782 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:14.783 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:14.783 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-13T10:56:14.783 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:14.783 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:14.783 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-13T10:56:14.784 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-13T10:56:14.784 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:14.784 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-13T10:56:14.791 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-13T10:56:14.792 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42267sec preferred_lft 42267sec 2023-11-13T10:56:14.792 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-13T10:56:14.800 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:14.800 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:14.800 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:14.802 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-13T10:56:14.802 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-13T10:56:14.802 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1a:84:8e:80:25:76 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:14.803 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-13T10:56:14.803 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:14.803 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5426:dff:feb9:4760/64 scope link 2023-11-13T10:56:14.803 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:14.803 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-13T10:56:14.804 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1a:84:8e:80:25:76 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:14.804 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::1884:8eff:fe80:2576/64 scope link 2023-11-13T10:56:14.804 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:14.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:14.805 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-13T10:56:14.919 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-13T10:56:14.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:14.923 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-13T10:56:14.975 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-13T10:56:14.975 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-13T10:56:14.975 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-13T10:56:14.975 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-13T10:56:14.976 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-13T10:56:14.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:14.977 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:56:14.977 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:56:14.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-13T10:56:14.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-13T10:56:14.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-13T10:56:14.978 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:56:15.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:15.181 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-13T10:56:15.240 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-13T10:56:15.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:15.241 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-11-13T10:56:15.300 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-13T10:56:15.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:15.302 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:56:15.302 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:56:15.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:56:15.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-13T10:56:15.303 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:56:15.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:15.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-13T10:56:15.554 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:56:15.555 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:56:15.555 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:56:15.555 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:15.556 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-13T10:56:15.556 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:15.556 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-13T10:56:15.561 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 26:8e:fb:11:eb:d7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-13T10:56:15.562 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-13T10:56:15.562 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:15.562 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::248e:fbff:fe11:ebd7/64 scope link 2023-11-13T10:56:15.562 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:15.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:15.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-13T10:56:15.672 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:56:15.673 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:56:15.673 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:56:15.673 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:15.674 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-13T10:56:15.674 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:15.674 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-13T10:56:15.674 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 26:8e:fb:11:eb:d7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-13T10:56:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-13T10:56:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::248e:fbff:fe11:ebd7/64 scope link 2023-11-13T10:56:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:15.677 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-13T10:56:15.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:15.677 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:56:15.678 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:56:15.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-13T10:56:15.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-13T10:56:15.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-13T10:56:15.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-13T10:56:15.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-13T10:56:15.681 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:56:15.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:15.979 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:56:15.980 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:56:15.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2023-11-13T10:56:15.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2023-11-13T10:56:15.980 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:56:16.164 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-13T10:56:16.164 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:56:16.165 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-13T10:56:16.217 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-13T10:56:16.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:56:16.218 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-13T10:56:16.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T10:56:16.379 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:56:16.439 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T10:56:16.439 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T10:56:16.439 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T10:56:16.440 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T10:56:16.440 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T10:56:16.440 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T10:56:16.440 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T10:56:16.441 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T10:56:16.441 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T10:56:16.441 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T10:56:16.441 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T10:56:16.441 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:56:16.442 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T10:56:16.442 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T10:56:16.442 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T10:56:16.442 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T10:56:16.442 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:56:16.443 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T10:56:16.443 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T10:56:16.443 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:56:16.443 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:56:16.444 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:56:16.444 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T10:56:16.444 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T10:56:16.444 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T10:56:16.444 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T10:56:16.445 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T10:56:16.445 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T10:56:16.445 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T10:56:16.445 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:16.445 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:16.446 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:16.449 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:56:16.449 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:56:16.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:16.453 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T10:56:16.518 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-13T10:56:16.519 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-11-13T10:56:16.519 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-13T10:56:16.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T10:56:16.606 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:56:16.651 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T10:56:16.651 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T10:56:16.651 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T10:56:16.651 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T10:56:16.652 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T10:56:16.652 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T10:56:16.652 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T10:56:16.652 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T10:56:16.653 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T10:56:16.653 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T10:56:16.653 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T10:56:16.653 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:56:16.653 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T10:56:16.654 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T10:56:16.654 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T10:56:16.654 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T10:56:16.654 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:56:16.654 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T10:56:16.655 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T10:56:16.655 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:56:16.655 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:56:16.655 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:56:16.656 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T10:56:16.656 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T10:56:16.656 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T10:56:16.657 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T10:56:16.657 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T10:56:16.657 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T10:56:16.657 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T10:56:16.658 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:16.658 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:56:16.659 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:16.659 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:56:16.659 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:56:16.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:16.662 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T10:56:16.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:2023-11-13T10:56:16.709+0000 7fec18377480 -1 init, newargv = 0x563d6478a340 newargc=15 2023-11-13T10:56:16.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[96713]: starting ceph client 2023-11-13T10:56:16.722 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[96713]: starting fuse 2023-11-13T10:56:16.749 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-13T10:56:16.749 INFO:teuthology.orchestra.run.smithi016.stdout:42 2023-11-13T10:56:16.749 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 42] 2023-11-13T10:56:16.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-13T10:56:16.750 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-13T10:56:16.750 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-13T10:56:16.751 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T10:56:16.751 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-13T10:56:16.751 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:56:16.752 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-13T10:56:16.752 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-13T10:56:16.752 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-13T10:56:16.753 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-13T10:56:16.753 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:56:16.753 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-13T10:56:16.754 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-13T10:56:16.757 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-13T10:56:16.757 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-13T10:56:16.757 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:56:16.758 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-13T10:56:16.758 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-13T10:56:16.758 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-13T10:56:16.759 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-13T10:56:16.759 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-13T10:56:16.759 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-13T10:56:16.760 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-13T10:56:16.760 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-13T10:56:16.760 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:56:16.761 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.2")) 2023-11-13T10:56:16.761 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T10:56:16.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:56:16.806 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:56:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.2.96713.asok 2023-11-13T10:56:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T10:56:16.974 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.96713.asok 2023-11-13T10:56:16.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:16.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.96713.asok status 2023-11-13T10:56:17.184 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T10:56:17.185 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-13T10:56:17.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "35f0debb3cc28955b7132ef4c509cd138a09d56b", 2023-11-13T10:56:17.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7288-g35f0debb (35f0debb3cc28955b7132ef4c509cd138a09d56b) reef (dev)", 2023-11-13T10:56:17.185 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "2", 2023-11-13T10:56:17.186 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-13T10:56:17.186 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-13T10:56:17.186 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "96713", 2023-11-13T10:56:17.186 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-13T10:56:17.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:56:17.187 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-13T10:56:17.187 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-13T10:56:17.187 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4839, 2023-11-13T10:56:17.187 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-13T10:56:17.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-13T10:56:17.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-13T10:56:17.188 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4839 2023-11-13T10:56:17.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:56:17.188 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T10:56:17.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T10:56:17.189 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2023-11-13T10:56:17.189 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1423569755 2023-11-13T10:56:17.190 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:56:17.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:56:17.191 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T10:56:17.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T10:56:17.192 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2023-11-13T10:56:17.192 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1423569755 2023-11-13T10:56:17.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:56:17.192 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4839 192.168.0.2:0/1423569755", 2023-11-13T10:56:17.193 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.2:0/1423569755", 2023-11-13T10:56:17.193 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-13T10:56:17.193 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 9, 2023-11-13T10:56:17.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 28, 2023-11-13T10:56:17.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-13T10:56:17.194 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-13T10:56:17.194 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-13T10:56:17.194 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T10:56:17.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-13T10:56:17.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.428+0000 7fcc62a60640 1 Processor -- start 2023-11-13T10:56:17.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.428+0000 7fcc62a60640 1 -- start start 2023-11-13T10:56:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.430+0000 7fcc62a60640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc5c06a190 0x7fcc5c06a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.430+0000 7fcc62a60640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c158c20 con 0x7fcc5c15b190 2023-11-13T10:56:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.430+0000 7fcc62a60640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c0b1830 con 0x7fcc5c0cb190 2023-11-13T10:56:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.430+0000 7fcc62a60640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c06ab40 con 0x7fcc5c06a190 2023-11-13T10:56:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.430+0000 7fcc6225f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc5c06a190 0x7fcc5c06a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:17.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.430+0000 7fcc6225f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc5c06a190 0x7fcc5c06a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43952/0 (socket says 172.21.15.16:43952) 2023-11-13T10:56:17.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.430+0000 7fcc6125d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcc5c15b190 0x7fcc5c06f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46224/0 (socket says 172.21.15.16:46224) 2023-11-13T10:56:17.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.430+0000 7fcc6225f640 1 -- 172.21.15.16:0/3453238821 learned_addr learned my addr 172.21.15.16:0/3453238821 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:17.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.431+0000 7fcc6225f640 1 -- 172.21.15.16:0/3453238821 >> v1:172.21.15.184:6789/0 conn(0x7fcc5c0cb190 legacy=0x7fcc5c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:17.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.431+0000 7fcc6225f640 1 -- 172.21.15.16:0/3453238821 >> v1:172.21.15.16:6789/0 conn(0x7fcc5c15b190 legacy=0x7fcc5c06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:17.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.431+0000 7fcc6225f640 1 -- 172.21.15.16:0/3453238821 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c15b780 con 0x7fcc5c06a190 2023-11-13T10:56:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.431+0000 7fcc60a5c640 1 -- 172.21.15.16:0/3453238821 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345101899 0 0) 0x7fcc5c158c20 con 0x7fcc5c15b190 2023-11-13T10:56:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.432+0000 7fcc6225f640 1 --2- 172.21.15.16:0/3453238821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c06a190 0x7fcc5c06a600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009e50 tx=0x7fcc5402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d2b6fc848edf1b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.432+0000 7fcc60a5c640 1 -- 172.21.15.16:0/3453238821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5402ca80 con 0x7fcc5c06a190 2023-11-13T10:56:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.432+0000 7fcc60a5c640 1 -- 172.21.15.16:0/3453238821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5402cbe0 con 0x7fcc5c06a190 2023-11-13T10:56:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.432+0000 7fcc60a5c640 1 -- 172.21.15.16:0/3453238821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc540343f0 con 0x7fcc5c06a190 2023-11-13T10:56:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.433+0000 7fcc62a60640 1 -- 172.21.15.16:0/3453238821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c06a190 msgr2=0x7fcc5c06a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.433+0000 7fcc62a60640 1 --2- 172.21.15.16:0/3453238821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c06a190 0x7fcc5c06a600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009e50 tx=0x7fcc5402bee0 comp rx=0 tx=0).stop 2023-11-13T10:56:17.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.433+0000 7fcc62a60640 1 -- 172.21.15.16:0/3453238821 shutdown_connections 2023-11-13T10:56:17.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.433+0000 7fcc62a60640 1 --2- 172.21.15.16:0/3453238821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c06a190 0x7fcc5c06a600 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:17.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.433+0000 7fcc62a60640 1 -- 172.21.15.16:0/3453238821 >> 172.21.15.16:0/3453238821 conn(0x7fcc5c0b04a0 msgr2=0x7fcc5c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:17.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.433+0000 7fcc62a60640 1 -- 172.21.15.16:0/3453238821 shutdown_connections 2023-11-13T10:56:17.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.433+0000 7fcc62a60640 1 -- 172.21.15.16:0/3453238821 wait complete. 2023-11-13T10:56:17.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.435+0000 7fcc62a60640 1 Processor -- start 2023-11-13T10:56:17.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.435+0000 7fcc62a60640 1 -- start start 2023-11-13T10:56:17.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc62a60640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc5c06a190 0x7fcc5c150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:17.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc62a60640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c0cb190 0x7fcc5c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:17.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc62a60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c15b190 0x7fcc5c1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:17.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc62a60640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c15cf50 con 0x7fcc5c15b190 2023-11-13T10:56:17.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc62a60640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c15d230 con 0x7fcc5c06a190 2023-11-13T10:56:17.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc62a60640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c15d510 con 0x7fcc5c0cb190 2023-11-13T10:56:17.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc6225f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c15b190 0x7fcc5c1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc6225f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c15b190 0x7fcc5c1594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43954/0 (socket says 172.21.15.16:43954) 2023-11-13T10:56:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc6225f640 1 -- 172.21.15.16:0/3487560352 learned_addr learned my addr 172.21.15.16:0/3487560352 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc6125d640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c0cb190 0x7fcc5c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc61a5e640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc5c06a190 0x7fcc5c150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc6225f640 1 -- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c0cb190 msgr2=0x7fcc5c15c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.437+0000 7fcc6225f640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c0cb190 0x7fcc5c15c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.438+0000 7fcc6225f640 1 -- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc5c06a190 msgr2=0x7fcc5c150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.438+0000 7fcc6225f640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc5c06a190 0x7fcc5c150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.438+0000 7fcc6225f640 1 -- 172.21.15.16:0/3487560352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54009b30 con 0x7fcc5c15b190 2023-11-13T10:56:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.438+0000 7fcc6225f640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c15b190 0x7fcc5c1594f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcc54036040 tx=0x7fcc54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.438+0000 7fcc527fc640 1 -- 172.21.15.16:0/3487560352 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5403e450 con 0x7fcc5c15b190 2023-11-13T10:56:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.438+0000 7fcc527fc640 1 -- 172.21.15.16:0/3487560352 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5403e5d0 con 0x7fcc5c15b190 2023-11-13T10:56:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.438+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c15d7f0 con 0x7fcc5c15b190 2023-11-13T10:56:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.438+0000 7fcc527fc640 1 -- 172.21.15.16:0/3487560352 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5403e980 con 0x7fcc5c15b190 2023-11-13T10:56:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.438+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c15cad0 con 0x7fcc5c15b190 2023-11-13T10:56:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.440+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c15da30 con 0x7fcc5c15b190 2023-11-13T10:56:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.441+0000 7fcc527fc640 1 -- 172.21.15.16:0/3487560352 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc5402cd50 con 0x7fcc5c15b190 2023-11-13T10:56:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.442+0000 7fcc527fc640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc4006ae70 0x7fcc4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.442+0000 7fcc527fc640 1 -- 172.21.15.16:0/3487560352 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fcc540ac2d0 con 0x7fcc5c15b190 2023-11-13T10:56:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.442+0000 7fcc61a5e640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc4006ae70 0x7fcc4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.443+0000 7fcc61a5e640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc4006ae70 0x7fcc4006d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc58003d90 tx=0x7fcc58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.446+0000 7fcc527fc640 1 -- 172.21.15.16:0/3487560352 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc5c15da30 con 0x7fcc5c15b190 2023-11-13T10:56:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.686+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcc5c15d090 con 0x7fcc5c15b190 2023-11-13T10:56:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.687+0000 7fcc527fc640 1 -- 172.21.15.16:0/3487560352 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fcc5c15d090 con 0x7fcc5c15b190 2023-11-13T10:56:17.692 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-13T10:56:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc4006ae70 msgr2=0x7fcc4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc4006ae70 0x7fcc4006d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc58003d90 tx=0x7fcc58009040 comp rx=0 tx=0).stop 2023-11-13T10:56:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c15b190 msgr2=0x7fcc5c1594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c15b190 0x7fcc5c1594f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcc54036040 tx=0x7fcc54004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 shutdown_connections 2023-11-13T10:56:17.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc4006ae70 0x7fcc4006d330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc5c15b190 0x7fcc5c1594f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c0cb190 0x7fcc5c15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 --2- 172.21.15.16:0/3487560352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc5c06a190 0x7fcc5c150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 >> 172.21.15.16:0/3487560352 conn(0x7fcc5c0b04a0 msgr2=0x7fcc5c157350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 shutdown_connections 2023-11-13T10:56:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:17.690+0000 7fcc62a60640 1 -- 172.21.15.16:0/3487560352 wait complete. 2023-11-13T10:56:17.712 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-13T10:56:17.713 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-13T10:56:17.714 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2023-11-13T10:56:17.714 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-13T10:56:17.714 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:17.715 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-13T10:56:17.715 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-13T10:56:17.715 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-13T10:56:17.715 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-13T10:56:17.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:17.716 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2023-11-13T10:56:17.777 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:56:17.777 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:56:17.778 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:56:17.778 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:17.778 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-11-13T10:56:17.778 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:17.779 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:17.779 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:17.779 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2023-11-13T10:56:17.779 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:17.779 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:17.780 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2023-11-13T10:56:17.780 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:17.780 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:17.780 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-13T10:56:17.780 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:17.780 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-13T10:56:17.781 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42226sec preferred_lft 42226sec 2023-11-13T10:56:17.781 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link 2023-11-13T10:56:17.781 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:17.782 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-13T10:56:17.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:17.783 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-11-13T10:56:17.783 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-11-13T10:56:17.784 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add name ceph-brx type bridge 2023-11-13T10:56:17.785 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr flush dev ceph-brx 2023-11-13T10:56:17.785 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set ceph-brx up 2023-11-13T10:56:17.785 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-13T10:56:17.786 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-11-13T10:56:18.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:18.267 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-13T10:56:18.348 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-11-13T10:56:18.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:18.352 DEBUG:teuthology.orchestra.run.smithi184:> route 2023-11-13T10:56:18.411 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2023-11-13T10:56:18.411 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-13T10:56:18.412 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-13T10:56:18.412 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-13T10:56:18.412 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-13T10:56:18.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:18.413 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-11-13T10:56:18.413 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-11-13T10:56:18.414 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-13T10:56:18.414 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-13T10:56:18.415 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-13T10:56:18.415 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-11-13T10:56:19.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:19.173 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-11-13T10:56:19.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:19.232 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2023-11-13T10:56:19.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:19.289 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-11-13T10:56:19.289 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-11-13T10:56:19.290 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:19.290 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-13T10:56:19.290 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-11-13T10:56:19.441 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-13T10:56:19.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:19.443 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-11-13T10:56:19.443 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-11-13T10:56:19.444 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-13T10:56:19.444 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-13T10:56:19.445 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-13T10:56:19.446 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-13T10:56:19.447 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-13T10:56:19.449 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-11-13T10:56:19.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:19.727 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-11-13T10:56:19.728 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-11-13T10:56:19.728 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2023-11-13T10:56:19.728 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.0 master ceph-brx 2023-11-13T10:56:19.728 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-11-13T10:56:19.895 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-13T10:56:19.898 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:56:19.901 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:19.949 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-13T10:56:19.950 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:56:19.950 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:20.047 DEBUG:teuthology.orchestra.run.smithi184:> sudo modprobe fuse 2023-11-13T10:56:20.137 DEBUG:teuthology.orchestra.run.smithi184:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:56:20.188 INFO:teuthology.orchestra.run.smithi184.stdout:/proc 2023-11-13T10:56:20.188 INFO:teuthology.orchestra.run.smithi184.stdout:/sys 2023-11-13T10:56:20.189 INFO:teuthology.orchestra.run.smithi184.stdout:/dev 2023-11-13T10:56:20.189 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/security 2023-11-13T10:56:20.189 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/shm 2023-11-13T10:56:20.189 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/pts 2023-11-13T10:56:20.190 INFO:teuthology.orchestra.run.smithi184.stdout:/run 2023-11-13T10:56:20.190 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/cgroup 2023-11-13T10:56:20.190 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/pstore 2023-11-13T10:56:20.190 INFO:teuthology.orchestra.run.smithi184.stdout:/ 2023-11-13T10:56:20.190 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/selinux 2023-11-13T10:56:20.191 INFO:teuthology.orchestra.run.smithi184.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:56:20.191 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/mqueue 2023-11-13T10:56:20.191 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/hugepages 2023-11-13T10:56:20.191 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/debug 2023-11-13T10:56:20.192 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/tracing 2023-11-13T10:56:20.195 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/fuse/connections 2023-11-13T10:56:20.206 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/config 2023-11-13T10:56:20.207 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:56:20.207 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:56:20.208 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:56:20.208 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:56:20.208 INFO:teuthology.orchestra.run.smithi184.stdout:/run/user/1000 2023-11-13T10:56:20.209 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph 2023-11-13T10:56:20.209 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-4 2023-11-13T10:56:20.218 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-5 2023-11-13T10:56:20.219 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-6 2023-11-13T10:56:20.219 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-7 2023-11-13T10:56:20.219 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns 2023-11-13T10:56:20.219 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:20.219 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:20.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:20.221 DEBUG:teuthology.orchestra.run.smithi184:> ls /sys/fs/fuse/connections 2023-11-13T10:56:20.279 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-13T10:56:20.280 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-13T10:56:20.289 DEBUG:teuthology.orchestra.run.smithi184:> sudo modprobe fuse 2023-11-13T10:56:20.369 DEBUG:teuthology.orchestra.run.smithi184:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:56:20.415 INFO:teuthology.orchestra.run.smithi184.stdout:/proc 2023-11-13T10:56:20.416 INFO:teuthology.orchestra.run.smithi184.stdout:/sys 2023-11-13T10:56:20.416 INFO:teuthology.orchestra.run.smithi184.stdout:/dev 2023-11-13T10:56:20.416 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/security 2023-11-13T10:56:20.417 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/shm 2023-11-13T10:56:20.417 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/pts 2023-11-13T10:56:20.417 INFO:teuthology.orchestra.run.smithi184.stdout:/run 2023-11-13T10:56:20.417 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/cgroup 2023-11-13T10:56:20.417 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/pstore 2023-11-13T10:56:20.418 INFO:teuthology.orchestra.run.smithi184.stdout:/ 2023-11-13T10:56:20.418 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/selinux 2023-11-13T10:56:20.418 INFO:teuthology.orchestra.run.smithi184.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:56:20.418 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/mqueue 2023-11-13T10:56:20.418 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/hugepages 2023-11-13T10:56:20.419 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/debug 2023-11-13T10:56:20.419 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/tracing 2023-11-13T10:56:20.419 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/fuse/connections 2023-11-13T10:56:20.419 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/config 2023-11-13T10:56:20.419 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:56:20.420 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:56:20.420 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:56:20.420 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:56:20.420 INFO:teuthology.orchestra.run.smithi184.stdout:/run/user/1000 2023-11-13T10:56:20.420 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph 2023-11-13T10:56:20.423 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-4 2023-11-13T10:56:20.423 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-5 2023-11-13T10:56:20.423 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-6 2023-11-13T10:56:20.424 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-7 2023-11-13T10:56:20.424 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns 2023-11-13T10:56:20.424 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:20.425 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:20.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:20.426 DEBUG:teuthology.orchestra.run.smithi184:> ls /sys/fs/fuse/connections 2023-11-13T10:56:20.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi184.stderr:2023-11-13T10:56:20.472+0000 7f0890e43480 -1 init, newargv = 0x55a152ace080 newargc=15 2023-11-13T10:56:20.477 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi184.stderr:ceph-fuse[92575]: starting ceph client 2023-11-13T10:56:20.489 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi184.stderr:ceph-fuse[92575]: starting fuse 2023-11-13T10:56:20.501 INFO:teuthology.orchestra.run.smithi184.stdout:40 2023-11-13T10:56:20.502 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-13T10:56:20.502 DEBUG:teuthology.orchestra.run.smithi184:> sudo stdin-killer -- python3 -c ' 2023-11-13T10:56:20.503 DEBUG:teuthology.orchestra.run.smithi184:> import glob 2023-11-13T10:56:20.503 DEBUG:teuthology.orchestra.run.smithi184:> import re 2023-11-13T10:56:20.503 DEBUG:teuthology.orchestra.run.smithi184:> import os 2023-11-13T10:56:20.503 DEBUG:teuthology.orchestra.run.smithi184:> import subprocess 2023-11-13T10:56:20.504 DEBUG:teuthology.orchestra.run.smithi184:> 2023-11-13T10:56:20.504 DEBUG:teuthology.orchestra.run.smithi184:> def _find_admin_socket(client_name): 2023-11-13T10:56:20.504 DEBUG:teuthology.orchestra.run.smithi184:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-13T10:56:20.505 DEBUG:teuthology.orchestra.run.smithi184:> files = glob.glob(asok_path) 2023-11-13T10:56:20.505 DEBUG:teuthology.orchestra.run.smithi184:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-13T10:56:20.505 DEBUG:teuthology.orchestra.run.smithi184:> 2023-11-13T10:56:20.506 DEBUG:teuthology.orchestra.run.smithi184:> # Given a non-glob path, it better be there 2023-11-13T10:56:20.506 DEBUG:teuthology.orchestra.run.smithi184:> if "*" not in asok_path: 2023-11-13T10:56:20.508 DEBUG:teuthology.orchestra.run.smithi184:> assert(len(files) == 1) 2023-11-13T10:56:20.508 DEBUG:teuthology.orchestra.run.smithi184:> return files[0] 2023-11-13T10:56:20.512 DEBUG:teuthology.orchestra.run.smithi184:> 2023-11-13T10:56:20.513 DEBUG:teuthology.orchestra.run.smithi184:> for f in files: 2023-11-13T10:56:20.514 DEBUG:teuthology.orchestra.run.smithi184:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-13T10:56:20.515 DEBUG:teuthology.orchestra.run.smithi184:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-13T10:56:20.524 DEBUG:teuthology.orchestra.run.smithi184:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-13T10:56:20.525 DEBUG:teuthology.orchestra.run.smithi184:> contents = proc_f.read() 2023-11-13T10:56:20.526 DEBUG:teuthology.orchestra.run.smithi184:> if mountpoint in contents: 2023-11-13T10:56:20.527 DEBUG:teuthology.orchestra.run.smithi184:> return f 2023-11-13T10:56:20.528 DEBUG:teuthology.orchestra.run.smithi184:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-13T10:56:20.529 DEBUG:teuthology.orchestra.run.smithi184:> 2023-11-13T10:56:20.530 DEBUG:teuthology.orchestra.run.smithi184:> print(_find_admin_socket("client.1")) 2023-11-13T10:56:20.532 DEBUG:teuthology.orchestra.run.smithi184:> ' 2023-11-13T10:56:20.711 INFO:teuthology.orchestra.run.smithi184.stdout:/var/run/ceph/ceph-client.1.92575.asok 2023-11-13T10:56:20.713 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:56:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T10:56:20.721 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.92575.asok 2023-11-13T10:56:20.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:20.723 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.92575.asok status 2023-11-13T10:56:20.965 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2023-11-13T10:56:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: "metadata": { 2023-11-13T10:56:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_sha1": "35f0debb3cc28955b7132ef4c509cd138a09d56b", 2023-11-13T10:56:20.965 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 18.0.0-7288-g35f0debb (35f0debb3cc28955b7132ef4c509cd138a09d56b) reef (dev)", 2023-11-13T10:56:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "entity_id": "1", 2023-11-13T10:56:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "hostname": "smithi184", 2023-11-13T10:56:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-13T10:56:20.966 INFO:teuthology.orchestra.run.smithi184.stdout: "pid": "92575", 2023-11-13T10:56:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "root": "/" 2023-11-13T10:56:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-11-13T10:56:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "dentry_count": 0, 2023-11-13T10:56:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "dentry_pinned_count": 0, 2023-11-13T10:56:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "id": 4678, 2023-11-13T10:56:20.967 INFO:teuthology.orchestra.run.smithi184.stdout: "inst": { 2023-11-13T10:56:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "name": { 2023-11-13T10:56:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "type": "client", 2023-11-13T10:56:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "num": 4678 2023-11-13T10:56:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-11-13T10:56:20.968 INFO:teuthology.orchestra.run.smithi184.stdout: "addr": { 2023-11-13T10:56:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "type": "v1", 2023-11-13T10:56:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "addr": "192.168.0.1:0", 2023-11-13T10:56:20.969 INFO:teuthology.orchestra.run.smithi184.stdout: "nonce": 400954550 2023-11-13T10:56:20.992 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-11-13T10:56:20.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-11-13T10:56:20.992 INFO:teuthology.orchestra.run.smithi184.stdout: "addr": { 2023-11-13T10:56:20.992 INFO:teuthology.orchestra.run.smithi184.stdout: "type": "v1", 2023-11-13T10:56:20.992 INFO:teuthology.orchestra.run.smithi184.stdout: "addr": "192.168.0.1:0", 2023-11-13T10:56:20.993 INFO:teuthology.orchestra.run.smithi184.stdout: "nonce": 400954550 2023-11-13T10:56:20.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-11-13T10:56:20.993 INFO:teuthology.orchestra.run.smithi184.stdout: "inst_str": "client.4678 192.168.0.1:0/400954550", 2023-11-13T10:56:20.993 INFO:teuthology.orchestra.run.smithi184.stdout: "addr_str": "192.168.0.1:0/400954550", 2023-11-13T10:56:20.993 INFO:teuthology.orchestra.run.smithi184.stdout: "inode_count": 1, 2023-11-13T10:56:20.994 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_epoch": 9, 2023-11-13T10:56:20.994 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_epoch": 28, 2023-11-13T10:56:20.994 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_epoch_barrier": 0, 2023-11-13T10:56:20.994 INFO:teuthology.orchestra.run.smithi184.stdout: "blocklisted": false, 2023-11-13T10:56:20.994 INFO:teuthology.orchestra.run.smithi184.stdout: "fs_name": "cephfs" 2023-11-13T10:56:20.995 INFO:teuthology.orchestra.run.smithi184.stdout:} 2023-11-13T10:56:20.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-13T10:56:21.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.228+0000 7f2fcb7fb640 1 Processor -- start 2023-11-13T10:56:21.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.228+0000 7f2fcb7fb640 1 -- start start 2023-11-13T10:56:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.230+0000 7f2fcb7fb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2fc4155410 0x7f2fc40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.230+0000 7f2fcb7fb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc415b500 con 0x7f2fc40ce990 2023-11-13T10:56:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.230+0000 7f2fcb7fb640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc415b7e0 con 0x7f2fc4071b50 2023-11-13T10:56:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.230+0000 7f2fcb7fb640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc415bac0 con 0x7f2fc4155410 2023-11-13T10:56:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.230+0000 7f2fc8d96640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2fc40ce990 0x7f2fc4071440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46240/0 (socket says 172.21.15.16:46240) 2023-11-13T10:56:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.230+0000 7f2fc8d96640 1 -- 172.21.15.16:0/1412043161 learned_addr learned my addr 172.21.15.16:0/1412043161 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676189582 0 0) 0x7f2fc415b500 con 0x7f2fc40ce990 2023-11-13T10:56:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fb0003520 con 0x7f2fc40ce990 2023-11-13T10:56:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fc9597640 1 --2- 172.21.15.16:0/1412043161 >> v2:172.21.15.184:3300/0 conn(0x7f2fc4155410 0x7f2fc40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4119309357 0 0) 0x7f2fb0003520 con 0x7f2fc40ce990 2023-11-13T10:56:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 >> v2:172.21.15.184:3300/0 conn(0x7f2fc4155410 msgr2=0x7f2fc40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 --2- 172.21.15.16:0/1412043161 >> v2:172.21.15.184:3300/0 conn(0x7f2fc4155410 0x7f2fc40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 >> v1:172.21.15.184:6789/0 conn(0x7f2fc4071b50 legacy=0x7f2fc406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc415bda0 con 0x7f2fc40ce990 2023-11-13T10:56:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f2fb4001670 con 0x7f2fc40ce990 2023-11-13T10:56:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 >> v1:172.21.15.16:6789/0 conn(0x7f2fc40ce990 legacy=0x7f2fc4071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb00039c0 0x7f2fb0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fb00047c0 0x7f2fb0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0005180 0x7f2fb000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc415bac0 con 0x7f2fb0005180 2023-11-13T10:56:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb000c080 con 0x7f2fb00039c0 2023-11-13T10:56:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.231+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb000c240 con 0x7f2fb00047c0 2023-11-13T10:56:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.232+0000 7f2fc9d98640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0005180 0x7f2fb000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.232+0000 7f2fc8d96640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fb00047c0 0x7f2fb0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.232+0000 7f2fc9597640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb00039c0 0x7f2fb0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.232+0000 7f2fc9d98640 1 -- 172.21.15.16:0/1412043161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fb00047c0 msgr2=0x7f2fb0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.232+0000 7f2fc9d98640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fb00047c0 0x7f2fb0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.232+0000 7f2fc9d98640 1 -- 172.21.15.16:0/1412043161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb00039c0 msgr2=0x7f2fb0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.232+0000 7f2fc9d98640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb00039c0 0x7f2fb0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.232+0000 7f2fc9d98640 1 -- 172.21.15.16:0/1412043161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb0003520 con 0x7f2fb0005180 2023-11-13T10:56:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.233+0000 7f2fc9d98640 1 -- 172.21.15.16:0/1412043161 reap_dead start 2023-11-13T10:56:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.233+0000 7f2fc9d98640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0005180 0x7f2fb000b980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc001e50 tx=0x7f2fb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20e5884af751dd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc003b70 con 0x7f2fb0005180 2023-11-13T10:56:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc003d10 con 0x7f2fb0005180 2023-11-13T10:56:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fbbfff640 1 -- 172.21.15.16:0/1412043161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc005310 con 0x7f2fb0005180 2023-11-13T10:56:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/1412043161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0005180 msgr2=0x7f2fb000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:21.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fcb7fb640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0005180 0x7f2fb000b980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc001e50 tx=0x7f2fb00057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/1412043161 shutdown_connections 2023-11-13T10:56:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fcb7fb640 1 --2- 172.21.15.16:0/1412043161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0005180 0x7f2fb000b980 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/1412043161 >> 172.21.15.16:0/1412043161 conn(0x7f2fc406a190 msgr2=0x7f2fc406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/1412043161 shutdown_connections 2023-11-13T10:56:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.234+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/1412043161 wait complete. 2023-11-13T10:56:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.236+0000 7f2fcb7fb640 1 Processor -- start 2023-11-13T10:56:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.236+0000 7f2fcb7fb640 1 -- start start 2023-11-13T10:56:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.237+0000 7f2fcb7fb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb0005180 0x7f2fc415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.237+0000 7f2fcb7fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fc41f6250 0x7f2fc4142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.237+0000 7f2fcb7fb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fc406ce40 0x7f2fc406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.237+0000 7f2fcb7fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc406d800 con 0x7f2fc41f6250 2023-11-13T10:56:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.237+0000 7f2fcb7fb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc4155870 con 0x7f2fb0005180 2023-11-13T10:56:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.237+0000 7f2fcb7fb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc41f69c0 con 0x7f2fc406ce40 2023-11-13T10:56:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc8d96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fc41f6250 0x7f2fc4142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc8d96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fc41f6250 0x7f2fc4142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43972/0 (socket says 172.21.15.16:43972) 2023-11-13T10:56:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc8d96640 1 -- 172.21.15.16:0/138823989 learned_addr learned my addr 172.21.15.16:0/138823989 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc9597640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb0005180 0x7f2fc415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:21.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc8d96640 1 -- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fc406ce40 msgr2=0x7f2fc406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:21.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc8d96640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fc406ce40 0x7f2fc406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc8d96640 1 -- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb0005180 msgr2=0x7f2fc415aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc8d96640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb0005180 0x7f2fc415aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc8d96640 1 -- 172.21.15.16:0/138823989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc41f6ca0 con 0x7f2fc41f6250 2023-11-13T10:56:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.238+0000 7f2fc8d96640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fc41f6250 0x7f2fc4142250 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400e730 tx=0x7f2fb400eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.239+0000 7f2fb9ffb640 1 -- 172.21.15.16:0/138823989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb400c6c0 con 0x7f2fc41f6250 2023-11-13T10:56:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.239+0000 7f2fb9ffb640 1 -- 172.21.15.16:0/138823989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb400c840 con 0x7f2fc41f6250 2023-11-13T10:56:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.239+0000 7f2fb9ffb640 1 -- 172.21.15.16:0/138823989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb400cbf0 con 0x7f2fc41f6250 2023-11-13T10:56:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.239+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc41f6f80 con 0x7f2fc41f6250 2023-11-13T10:56:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.239+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc41f66d0 con 0x7f2fc41f6250 2023-11-13T10:56:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.240+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc4155d00 con 0x7f2fc41f6250 2023-11-13T10:56:21.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.241+0000 7f2fb9ffb640 1 -- 172.21.15.16:0/138823989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fb400cd50 con 0x7f2fc41f6250 2023-11-13T10:56:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.242+0000 7f2fb9ffb640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2fa806ae70 0x7f2fa806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.242+0000 7f2fb9ffb640 1 -- 172.21.15.16:0/138823989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f2fb4089810 con 0x7f2fc41f6250 2023-11-13T10:56:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.243+0000 7f2fc9597640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2fa806ae70 0x7f2fa806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.244+0000 7f2fc9597640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2fa806ae70 0x7f2fa806d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009990 tx=0x7f2fc0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:21.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.247+0000 7f2fb9ffb640 1 -- 172.21.15.16:0/138823989 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2fc4155d00 con 0x7f2fc41f6250 2023-11-13T10:56:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.478+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2fc4155d00 con 0x7f2fc41f6250 2023-11-13T10:56:21.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.479+0000 7f2fb9ffb640 1 -- 172.21.15.16:0/138823989 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2fb4054570 con 0x7f2fc41f6250 2023-11-13T10:56:21.482 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-13T10:56:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2fa806ae70 msgr2=0x7f2fa806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:21.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2fa806ae70 0x7f2fa806d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009990 tx=0x7f2fc0008040 comp rx=0 tx=0).stop 2023-11-13T10:56:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fc41f6250 msgr2=0x7f2fc4142250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fc41f6250 0x7f2fc4142250 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400e730 tx=0x7f2fb400eb30 comp rx=0 tx=0).stop 2023-11-13T10:56:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 shutdown_connections 2023-11-13T10:56:21.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2fa806ae70 0x7f2fa806d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fc406ce40 0x7f2fc406d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fc41f6250 0x7f2fc4142250 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 --2- 172.21.15.16:0/138823989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb0005180 0x7f2fc415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:21.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.481+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 >> 172.21.15.16:0/138823989 conn(0x7f2fc406a190 msgr2=0x7f2fc40704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.482+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 shutdown_connections 2023-11-13T10:56:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:21.482+0000 7f2fcb7fb640 1 -- 172.21.15.16:0/138823989 wait complete. 2023-11-13T10:56:21.508 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-13T10:56:21.508 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-13T10:56:21.508 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2023-11-13T10:56:21.509 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-13T10:56:21.509 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-13T10:56:21.509 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-13T10:56:21.509 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-13T10:56:21.510 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-13T10:56:21.510 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-13T10:56:21.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:21.511 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2023-11-13T10:56:21.577 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:56:21.577 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:56:21.577 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:56:21.578 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:21.578 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-11-13T10:56:21.579 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:21.579 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:21.579 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:21.579 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2023-11-13T10:56:21.580 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:21.580 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:21.580 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2023-11-13T10:56:21.580 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:56:21.580 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:21.581 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-13T10:56:21.582 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:21.582 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-13T10:56:21.582 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42222sec preferred_lft 42222sec 2023-11-13T10:56:21.582 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link 2023-11-13T10:56:21.583 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:21.583 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-13T10:56:21.583 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether b6:d0:cd:f1:8a:a6 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:56:21.584 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-13T10:56:21.585 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:21.585 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::822:cbff:fe28:6caf/64 scope link 2023-11-13T10:56:21.585 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:21.586 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-13T10:56:21.586 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether b6:d0:cd:f1:8a:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:21.593 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::b4d0:cdff:fef1:8aa6/64 scope link 2023-11-13T10:56:21.593 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:21.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:21.594 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-13T10:56:21.693 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-11-13T10:56:21.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:21.694 DEBUG:teuthology.orchestra.run.smithi184:> route 2023-11-13T10:56:21.745 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2023-11-13T10:56:21.745 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-13T10:56:21.745 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-13T10:56:21.745 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-13T10:56:21.746 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-13T10:56:21.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:21.747 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-11-13T10:56:21.747 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-11-13T10:56:21.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-13T10:56:21.748 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-13T10:56:21.750 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-13T10:56:21.750 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-11-13T10:56:21.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:21.936 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-11-13T10:56:21.985 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-13T10:56:21.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:21.987 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2023-11-13T10:56:22.087 INFO:teuthology.orchestra.run.smithi184.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-13T10:56:22.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:22.088 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-11-13T10:56:22.088 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-11-13T10:56:22.089 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-13T10:56:22.089 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-13T10:56:22.089 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-11-13T10:56:22.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:22.230 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-13T10:56:22.314 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:56:22.314 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:56:22.315 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:56:22.315 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:22.315 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-11-13T10:56:22.315 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:22.316 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-13T10:56:22.316 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 9e:cd:7d:54:d1:7e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-13T10:56:22.316 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-13T10:56:22.316 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:22.316 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::9ccd:7dff:fe54:d17e/64 scope link 2023-11-13T10:56:22.317 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:22.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:22.324 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-13T10:56:22.417 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:56:22.417 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:56:22.417 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:56:22.417 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-11-13T10:56:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:22.418 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-13T10:56:22.418 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 9e:cd:7d:54:d1:7e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-13T10:56:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-13T10:56:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::9ccd:7dff:fe54:d17e/64 scope link 2023-11-13T10:56:22.419 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:56:22.421 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-13T10:56:22.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:22.422 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-11-13T10:56:22.423 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-11-13T10:56:22.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-13T10:56:22.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-13T10:56:22.424 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-13T10:56:22.424 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-13T10:56:22.424 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-13T10:56:22.425 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-11-13T10:56:22.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:22.700 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-11-13T10:56:22.700 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-11-13T10:56:22.701 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.1 up 2023-11-13T10:56:22.701 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.1 master ceph-brx 2023-11-13T10:56:22.702 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-11-13T10:56:22.876 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-13T10:56:22.876 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:56:22.877 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-13T10:56:22.925 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-13T10:56:22.926 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:56:22.928 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-13T10:56:23.025 DEBUG:teuthology.orchestra.run.smithi184:> sudo modprobe fuse 2023-11-13T10:56:23.126 DEBUG:teuthology.orchestra.run.smithi184:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:56:23.186 INFO:teuthology.orchestra.run.smithi184.stdout:/proc 2023-11-13T10:56:23.187 INFO:teuthology.orchestra.run.smithi184.stdout:/sys 2023-11-13T10:56:23.187 INFO:teuthology.orchestra.run.smithi184.stdout:/dev 2023-11-13T10:56:23.187 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/security 2023-11-13T10:56:23.187 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/shm 2023-11-13T10:56:23.187 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/pts 2023-11-13T10:56:23.188 INFO:teuthology.orchestra.run.smithi184.stdout:/run 2023-11-13T10:56:23.188 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/cgroup 2023-11-13T10:56:23.188 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/pstore 2023-11-13T10:56:23.188 INFO:teuthology.orchestra.run.smithi184.stdout:/ 2023-11-13T10:56:23.188 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/selinux 2023-11-13T10:56:23.189 INFO:teuthology.orchestra.run.smithi184.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:56:23.189 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/mqueue 2023-11-13T10:56:23.189 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/hugepages 2023-11-13T10:56:23.189 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/debug 2023-11-13T10:56:23.189 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/tracing 2023-11-13T10:56:23.190 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/fuse/connections 2023-11-13T10:56:23.190 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/config 2023-11-13T10:56:23.190 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:56:23.190 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:56:23.191 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:56:23.191 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:56:23.191 INFO:teuthology.orchestra.run.smithi184.stdout:/run/user/1000 2023-11-13T10:56:23.191 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph 2023-11-13T10:56:23.192 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-4 2023-11-13T10:56:23.192 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-5 2023-11-13T10:56:23.192 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-6 2023-11-13T10:56:23.193 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-7 2023-11-13T10:56:23.193 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns 2023-11-13T10:56:23.193 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:23.193 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:23.193 INFO:teuthology.orchestra.run.smithi184.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:23.194 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-13T10:56:23.194 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-13T10:56:23.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:23.211 DEBUG:teuthology.orchestra.run.smithi184:> ls /sys/fs/fuse/connections 2023-11-13T10:56:23.279 INFO:teuthology.orchestra.run.smithi184.stdout:40 2023-11-13T10:56:23.280 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-11-13T10:56:23.280 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-13T10:56:23.287 DEBUG:teuthology.orchestra.run.smithi184:> sudo modprobe fuse 2023-11-13T10:56:23.373 DEBUG:teuthology.orchestra.run.smithi184:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:56:23.420 INFO:teuthology.orchestra.run.smithi184.stdout:/proc 2023-11-13T10:56:23.420 INFO:teuthology.orchestra.run.smithi184.stdout:/sys 2023-11-13T10:56:23.420 INFO:teuthology.orchestra.run.smithi184.stdout:/dev 2023-11-13T10:56:23.421 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/security 2023-11-13T10:56:23.421 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/shm 2023-11-13T10:56:23.421 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/pts 2023-11-13T10:56:23.421 INFO:teuthology.orchestra.run.smithi184.stdout:/run 2023-11-13T10:56:23.422 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/cgroup 2023-11-13T10:56:23.422 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/pstore 2023-11-13T10:56:23.422 INFO:teuthology.orchestra.run.smithi184.stdout:/ 2023-11-13T10:56:23.422 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/selinux 2023-11-13T10:56:23.422 INFO:teuthology.orchestra.run.smithi184.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:56:23.423 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/mqueue 2023-11-13T10:56:23.423 INFO:teuthology.orchestra.run.smithi184.stdout:/dev/hugepages 2023-11-13T10:56:23.423 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/debug 2023-11-13T10:56:23.423 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/tracing 2023-11-13T10:56:23.423 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/fs/fuse/connections 2023-11-13T10:56:23.424 INFO:teuthology.orchestra.run.smithi184.stdout:/sys/kernel/config 2023-11-13T10:56:23.424 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:56:23.424 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:56:23.424 INFO:teuthology.orchestra.run.smithi184.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:56:23.424 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:56:23.425 INFO:teuthology.orchestra.run.smithi184.stdout:/run/user/1000 2023-11-13T10:56:23.425 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph 2023-11-13T10:56:23.425 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-4 2023-11-13T10:56:23.425 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-5 2023-11-13T10:56:23.425 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-6 2023-11-13T10:56:23.426 INFO:teuthology.orchestra.run.smithi184.stdout:/var/lib/ceph/osd/ceph-7 2023-11-13T10:56:23.427 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns 2023-11-13T10:56:23.427 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:23.427 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T10:56:23.427 INFO:teuthology.orchestra.run.smithi184.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:23.428 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-13T10:56:23.428 INFO:teuthology.orchestra.run.smithi184.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-13T10:56:23.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:23.435 DEBUG:teuthology.orchestra.run.smithi184:> ls /sys/fs/fuse/connections 2023-11-13T10:56:23.488 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi184.stderr:2023-11-13T10:56:23.479+0000 7fb535a35480 -1 init, newargv = 0x5565e65b8160 newargc=15 2023-11-13T10:56:23.489 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi184.stderr:ceph-fuse[93027]: starting ceph client 2023-11-13T10:56:23.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi184.stderr:ceph-fuse[93027]: starting fuse 2023-11-13T10:56:23.522 INFO:teuthology.orchestra.run.smithi184.stdout:40 2023-11-13T10:56:23.523 INFO:teuthology.orchestra.run.smithi184.stdout:42 2023-11-13T10:56:23.523 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 42] 2023-11-13T10:56:23.524 DEBUG:teuthology.orchestra.run.smithi184:> sudo stdin-killer -- python3 -c ' 2023-11-13T10:56:23.524 DEBUG:teuthology.orchestra.run.smithi184:> import glob 2023-11-13T10:56:23.525 DEBUG:teuthology.orchestra.run.smithi184:> import re 2023-11-13T10:56:23.525 DEBUG:teuthology.orchestra.run.smithi184:> import os 2023-11-13T10:56:23.525 DEBUG:teuthology.orchestra.run.smithi184:> import subprocess 2023-11-13T10:56:23.531 DEBUG:teuthology.orchestra.run.smithi184:> 2023-11-13T10:56:23.531 DEBUG:teuthology.orchestra.run.smithi184:> def _find_admin_socket(client_name): 2023-11-13T10:56:23.532 DEBUG:teuthology.orchestra.run.smithi184:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-13T10:56:23.532 DEBUG:teuthology.orchestra.run.smithi184:> files = glob.glob(asok_path) 2023-11-13T10:56:23.532 DEBUG:teuthology.orchestra.run.smithi184:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-13T10:56:23.533 DEBUG:teuthology.orchestra.run.smithi184:> 2023-11-13T10:56:23.533 DEBUG:teuthology.orchestra.run.smithi184:> # Given a non-glob path, it better be there 2023-11-13T10:56:23.533 DEBUG:teuthology.orchestra.run.smithi184:> if "*" not in asok_path: 2023-11-13T10:56:23.534 DEBUG:teuthology.orchestra.run.smithi184:> assert(len(files) == 1) 2023-11-13T10:56:23.534 DEBUG:teuthology.orchestra.run.smithi184:> return files[0] 2023-11-13T10:56:23.534 DEBUG:teuthology.orchestra.run.smithi184:> 2023-11-13T10:56:23.535 DEBUG:teuthology.orchestra.run.smithi184:> for f in files: 2023-11-13T10:56:23.535 DEBUG:teuthology.orchestra.run.smithi184:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-13T10:56:23.535 DEBUG:teuthology.orchestra.run.smithi184:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-13T10:56:23.535 DEBUG:teuthology.orchestra.run.smithi184:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-13T10:56:23.536 DEBUG:teuthology.orchestra.run.smithi184:> contents = proc_f.read() 2023-11-13T10:56:23.536 DEBUG:teuthology.orchestra.run.smithi184:> if mountpoint in contents: 2023-11-13T10:56:23.537 DEBUG:teuthology.orchestra.run.smithi184:> return f 2023-11-13T10:56:23.537 DEBUG:teuthology.orchestra.run.smithi184:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-13T10:56:23.537 DEBUG:teuthology.orchestra.run.smithi184:> 2023-11-13T10:56:23.538 DEBUG:teuthology.orchestra.run.smithi184:> print(_find_admin_socket("client.3")) 2023-11-13T10:56:23.544 DEBUG:teuthology.orchestra.run.smithi184:> ' 2023-11-13T10:56:23.731 INFO:teuthology.orchestra.run.smithi184.stdout:/var/run/ceph/ceph-client.3.93027.asok 2023-11-13T10:56:23.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-11-13T10:56:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T10:56:23.751 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.93027.asok 2023-11-13T10:56:23.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:23.751 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.93027.asok status 2023-11-13T10:56:23.993 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2023-11-13T10:56:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "metadata": { 2023-11-13T10:56:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_sha1": "35f0debb3cc28955b7132ef4c509cd138a09d56b", 2023-11-13T10:56:23.993 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 18.0.0-7288-g35f0debb (35f0debb3cc28955b7132ef4c509cd138a09d56b) reef (dev)", 2023-11-13T10:56:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "entity_id": "3", 2023-11-13T10:56:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "hostname": "smithi184", 2023-11-13T10:56:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-13T10:56:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "pid": "93027", 2023-11-13T10:56:23.994 INFO:teuthology.orchestra.run.smithi184.stdout: "root": "/" 2023-11-13T10:56:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-11-13T10:56:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "dentry_count": 0, 2023-11-13T10:56:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "dentry_pinned_count": 0, 2023-11-13T10:56:23.995 INFO:teuthology.orchestra.run.smithi184.stdout: "id": 4541, 2023-11-13T10:56:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "inst": { 2023-11-13T10:56:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "name": { 2023-11-13T10:56:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "type": "client", 2023-11-13T10:56:23.996 INFO:teuthology.orchestra.run.smithi184.stdout: "num": 4541 2023-11-13T10:56:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-11-13T10:56:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "addr": { 2023-11-13T10:56:24.003 INFO:teuthology.orchestra.run.smithi184.stdout: "type": "v1", 2023-11-13T10:56:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: "addr": "192.168.0.2:0", 2023-11-13T10:56:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: "nonce": 3350400459 2023-11-13T10:56:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-11-13T10:56:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-11-13T10:56:24.004 INFO:teuthology.orchestra.run.smithi184.stdout: "addr": { 2023-11-13T10:56:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "type": "v1", 2023-11-13T10:56:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "addr": "192.168.0.2:0", 2023-11-13T10:56:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "nonce": 3350400459 2023-11-13T10:56:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-11-13T10:56:24.005 INFO:teuthology.orchestra.run.smithi184.stdout: "inst_str": "client.4541 192.168.0.2:0/3350400459", 2023-11-13T10:56:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "addr_str": "192.168.0.2:0/3350400459", 2023-11-13T10:56:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "inode_count": 1, 2023-11-13T10:56:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_epoch": 9, 2023-11-13T10:56:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_epoch": 28, 2023-11-13T10:56:24.006 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_epoch_barrier": 0, 2023-11-13T10:56:24.013 INFO:teuthology.orchestra.run.smithi184.stdout: "blocklisted": false, 2023-11-13T10:56:24.014 INFO:teuthology.orchestra.run.smithi184.stdout: "fs_name": "cephfs" 2023-11-13T10:56:24.014 INFO:teuthology.orchestra.run.smithi184.stdout:} 2023-11-13T10:56:24.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:24.023 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-13T10:56:24.023 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:24.106 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-13T10:56:24.107 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:24.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:24.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:24.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:24.224 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-13T10:56:24.224 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-13T10:56:24.286 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-13T10:56:24.287 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-13T10:56:24.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:24.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-13T10:56:24.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:24.392 DEBUG:teuthology.orchestra.run.smithi184:> stat --file-system '--printf=%T 2023-11-13T10:56:24.393 DEBUG:teuthology.orchestra.run.smithi184:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:24.453 INFO:teuthology.orchestra.run.smithi184.stdout:fuseblk 2023-11-13T10:56:24.454 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:24.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:24.455 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:24.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:24.568 DEBUG:teuthology.orchestra.run.smithi184:> stat --file-system '--printf=%T 2023-11-13T10:56:24.568 DEBUG:teuthology.orchestra.run.smithi184:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-13T10:56:24.625 INFO:teuthology.orchestra.run.smithi184.stdout:fuseblk 2023-11-13T10:56:24.625 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-13T10:56:24.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:24.626 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-13T10:56:24.725 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-13T10:56:24.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:24.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.991+0000 7f8236d67640 1 Processor -- start 2023-11-13T10:56:24.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.992+0000 7f8236d67640 1 -- start start 2023-11-13T10:56:25.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.993+0000 7f8236d67640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82300d2ae0 0x7f82300d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.993+0000 7f8236d67640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82300b3cf0 con 0x7f82300b29d0 2023-11-13T10:56:25.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.993+0000 7f8236d67640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82300d3480 con 0x7f823015cfc0 2023-11-13T10:56:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.993+0000 7f8236d67640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82300cde80 con 0x7f82300d2ae0 2023-11-13T10:56:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.994+0000 7f8236566640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f82300b29d0 0x7f82300b2e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49324/0 (socket says 172.21.15.16:49324) 2023-11-13T10:56:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.994+0000 7f8236566640 1 -- 172.21.15.16:0/625459921 learned_addr learned my addr 172.21.15.16:0/625459921 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.994+0000 7f8235564640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82300d2ae0 0x7f82300d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.994+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045201904 0 0) 0x7f82300b3cf0 con 0x7f82300b29d0 2023-11-13T10:56:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.994+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8218003520 con 0x7f82300b29d0 2023-11-13T10:56:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.994+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2486143007 0 0) 0x7f8218003520 con 0x7f82300b29d0 2023-11-13T10:56:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.994+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82300d2ae0 msgr2=0x7f82300d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.994+0000 7f8234d63640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82300d2ae0 0x7f82300d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 >> v1:172.21.15.184:6789/0 conn(0x7f823015cfc0 legacy=0x7f82300cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300b3730 con 0x7f82300b29d0 2023-11-13T10:56:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f8228001670 con 0x7f82300b29d0 2023-11-13T10:56:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 >> v1:172.21.15.16:6789/0 conn(0x7f82300b29d0 legacy=0x7f82300b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82300d2ae0 0x7f82180039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8218003fd0 0x7f82180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8218004d00 0x7f8218005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300cde80 con 0x7f8218004d00 2023-11-13T10:56:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821800bbc0 con 0x7f8218003fd0 2023-11-13T10:56:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821800bd80 con 0x7f82300d2ae0 2023-11-13T10:56:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8236566640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8218004d00 0x7f8218005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8235564640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8218003fd0 0x7f82180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.995+0000 7f8235d65640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82300d2ae0 0x7f82180039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.996+0000 7f8236566640 1 -- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82300d2ae0 msgr2=0x7f82180039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.996+0000 7f8236566640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82300d2ae0 0x7f82180039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.996+0000 7f8236566640 1 -- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8218003fd0 msgr2=0x7f82180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.996+0000 7f8236566640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8218003fd0 0x7f82180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.996+0000 7f8236566640 1 -- 172.21.15.16:0/625459921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8218003520 con 0x7f8218004d00 2023-11-13T10:56:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.997+0000 7f8236566640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8218004d00 0x7f8218005180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f822800c240 tx=0x7f82180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9882ac8c74142e18 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.997+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228005ad0 con 0x7f8218004d00 2023-11-13T10:56:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.997+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228005c30 con 0x7f8218004d00 2023-11-13T10:56:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.997+0000 7f8234d63640 1 -- 172.21.15.16:0/625459921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822802caf0 con 0x7f8218004d00 2023-11-13T10:56:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.998+0000 7f8236d67640 1 -- 172.21.15.16:0/625459921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8218004d00 msgr2=0x7f8218005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.998+0000 7f8236d67640 1 --2- 172.21.15.16:0/625459921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8218004d00 0x7f8218005180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f822800c240 tx=0x7f82180057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.998+0000 7f8236566640 1 -- 172.21.15.16:0/625459921 reap_dead start 2023-11-13T10:56:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.998+0000 7f8236d67640 1 -- 172.21.15.16:0/625459921 shutdown_connections 2023-11-13T10:56:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.998+0000 7f8236d67640 1 -- 172.21.15.16:0/625459921 >> 172.21.15.16:0/625459921 conn(0x7f823008c570 msgr2=0x7f823009be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.998+0000 7f8236d67640 1 -- 172.21.15.16:0/625459921 shutdown_connections 2023-11-13T10:56:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:24.998+0000 7f8236d67640 1 -- 172.21.15.16:0/625459921 wait complete. 2023-11-13T10:56:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.000+0000 7f8236d67640 1 Processor -- start 2023-11-13T10:56:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.000+0000 7f8236d67640 1 -- start start 2023-11-13T10:56:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.001+0000 7f8236d67640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8218004d00 0x7f8230152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.001+0000 7f8236d67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82300cc360 0x7f82300c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.001+0000 7f8236d67640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f823015de60 0x7f82300c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.001+0000 7f8236d67640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300c8ce0 con 0x7f82300cc360 2023-11-13T10:56:25.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.001+0000 7f8236d67640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300c8ab0 con 0x7f823015de60 2023-11-13T10:56:25.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.001+0000 7f8236d67640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82300c9c50 con 0x7f8218004d00 2023-11-13T10:56:25.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.001+0000 7f8235564640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82300cc360 0x7f82300c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.001+0000 7f8235564640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82300cc360 0x7f82300c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58084/0 (socket says 172.21.15.16:58084) 2023-11-13T10:56:25.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.001+0000 7f8235564640 1 -- 172.21.15.16:0/3326045275 learned_addr learned my addr 172.21.15.16:0/3326045275 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:25.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f8235d65640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8218004d00 0x7f8230152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f8236566640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f823015de60 0x7f82300c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f8235564640 1 -- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8218004d00 msgr2=0x7f8230152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f8235564640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8218004d00 0x7f8230152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f8235564640 1 -- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f823015de60 msgr2=0x7f82300c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f8235564640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f823015de60 0x7f82300c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f8235564640 1 -- 172.21.15.16:0/3326045275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300cc7e0 con 0x7f82300cc360 2023-11-13T10:56:25.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f8236566640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f823015de60 0x7f82300c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:25.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f8235564640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82300cc360 0x7f82300c8570 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f822400f580 tx=0x7f822400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.002+0000 7f822e7fc640 1 -- 172.21.15.16:0/3326045275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224014870 con 0x7f82300cc360 2023-11-13T10:56:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.003+0000 7f822e7fc640 1 -- 172.21.15.16:0/3326045275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82240149d0 con 0x7f82300cc360 2023-11-13T10:56:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.003+0000 7f822e7fc640 1 -- 172.21.15.16:0/3326045275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224014d80 con 0x7f82300cc360 2023-11-13T10:56:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.003+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82300c9ea0 con 0x7f82300cc360 2023-11-13T10:56:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.003+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82300ca460 con 0x7f82300cc360 2023-11-13T10:56:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.004+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823004e4d0 con 0x7f82300cc360 2023-11-13T10:56:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.005+0000 7f822e7fc640 1 -- 172.21.15.16:0/3326045275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8224015410 con 0x7f82300cc360 2023-11-13T10:56:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.006+0000 7f822e7fc640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f821c06ae70 0x7f821c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.006+0000 7f822e7fc640 1 -- 172.21.15.16:0/3326045275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f822408a3a0 con 0x7f82300cc360 2023-11-13T10:56:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.006+0000 7f8235d65640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f821c06ae70 0x7f821c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.008+0000 7f8235d65640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f821c06ae70 0x7f821c06d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8220007fd0 tx=0x7f8220009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.010+0000 7f822e7fc640 1 -- 172.21.15.16:0/3326045275 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8224055080 con 0x7f82300cc360 2023-11-13T10:56:25.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.207+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82300b8b00 con 0x7f82300cc360 2023-11-13T10:56:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.209+0000 7f822e7fc640 1 -- 172.21.15.16:0/3326045275 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f82240586a0 con 0x7f82300cc360 2023-11-13T10:56:25.214 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:25.215 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:25.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f821c06ae70 msgr2=0x7f821c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f821c06ae70 0x7f821c06d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8220007fd0 tx=0x7f8220009040 comp rx=0 tx=0).stop 2023-11-13T10:56:25.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82300cc360 msgr2=0x7f82300c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82300cc360 0x7f82300c8570 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f822400f580 tx=0x7f822400fa40 comp rx=0 tx=0).stop 2023-11-13T10:56:25.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 shutdown_connections 2023-11-13T10:56:25.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f821c06ae70 0x7f821c06d330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f823015de60 0x7f82300c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82300cc360 0x7f82300c8570 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 --2- 172.21.15.16:0/3326045275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8218004d00 0x7f8230152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 >> 172.21.15.16:0/3326045275 conn(0x7f823008c570 msgr2=0x7f823006ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:25.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.212+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 shutdown_connections 2023-11-13T10:56:25.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.213+0000 7f8236d67640 1 -- 172.21.15.16:0/3326045275 wait complete. 2023-11-13T10:56:25.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:25.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.451+0000 7f3791caa640 1 Processor -- start 2023-11-13T10:56:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.451+0000 7f3791caa640 1 -- start start 2023-11-13T10:56:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.453+0000 7f3791caa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f378c15b8e0 0x7f378c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.453+0000 7f3791caa640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c06b860 con 0x7f378c15bcc0 2023-11-13T10:56:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.453+0000 7f3791caa640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c06bb40 con 0x7f378c0ce260 2023-11-13T10:56:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.453+0000 7f3791caa640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378c06be20 con 0x7f378c15b8e0 2023-11-13T10:56:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.454+0000 7f3790ca8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f378c15b8e0 0x7f378c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.454+0000 7f378bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f378c15bcc0 0x7f378c0cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49326/0 (socket says 172.21.15.16:49326) 2023-11-13T10:56:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.454+0000 7f378bfff640 1 -- 172.21.15.16:0/1615928156 learned_addr learned my addr 172.21.15.16:0/1615928156 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.454+0000 7f3790ca8640 1 -- 172.21.15.16:0/1615928156 >> v1:172.21.15.184:6789/0 conn(0x7f378c0ce260 legacy=0x7f378c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.454+0000 7f3790ca8640 1 -- 172.21.15.16:0/1615928156 >> v1:172.21.15.16:6789/0 conn(0x7f378c15bcc0 legacy=0x7f378c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.454+0000 7f3790ca8640 1 -- 172.21.15.16:0/1615928156 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c06c100 con 0x7f378c15b8e0 2023-11-13T10:56:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.454+0000 7f378b7fe640 1 -- 172.21.15.16:0/1615928156 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315133107 0 0) 0x7f378c06b860 con 0x7f378c15bcc0 2023-11-13T10:56:25.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.455+0000 7f3790ca8640 1 --2- 172.21.15.16:0/1615928156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c15b8e0 0x7f378c06c900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f37800097a0 tx=0x7f378002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddf7c2ddf27cc5e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.456+0000 7f378b7fe640 1 -- 172.21.15.16:0/1615928156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780031830 con 0x7f378c15b8e0 2023-11-13T10:56:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.456+0000 7f378b7fe640 1 -- 172.21.15.16:0/1615928156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3780031990 con 0x7f378c15b8e0 2023-11-13T10:56:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.456+0000 7f378b7fe640 1 -- 172.21.15.16:0/1615928156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780031d20 con 0x7f378c15b8e0 2023-11-13T10:56:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.456+0000 7f3791caa640 1 -- 172.21.15.16:0/1615928156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c15b8e0 msgr2=0x7f378c06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.456+0000 7f3791caa640 1 --2- 172.21.15.16:0/1615928156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c15b8e0 0x7f378c06c900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f37800097a0 tx=0x7f378002b8d0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.456+0000 7f3791caa640 1 -- 172.21.15.16:0/1615928156 shutdown_connections 2023-11-13T10:56:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.456+0000 7f3791caa640 1 --2- 172.21.15.16:0/1615928156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c15b8e0 0x7f378c06c900 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.456+0000 7f3791caa640 1 -- 172.21.15.16:0/1615928156 >> 172.21.15.16:0/1615928156 conn(0x7f378c0b0970 msgr2=0x7f378c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.457+0000 7f3791caa640 1 -- 172.21.15.16:0/1615928156 shutdown_connections 2023-11-13T10:56:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.457+0000 7f3791caa640 1 -- 172.21.15.16:0/1615928156 wait complete. 2023-11-13T10:56:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.458+0000 7f3791caa640 1 Processor -- start 2023-11-13T10:56:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.459+0000 7f3791caa640 1 -- start start 2023-11-13T10:56:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.459+0000 7f3791caa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c0ce260 0x7f378c146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.459+0000 7f3791caa640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15b8e0 0x7f378c0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3791caa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f378c071700 0x7f378c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3791caa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c153f70 con 0x7f378c0ce260 2023-11-13T10:56:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3791caa640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c154250 con 0x7f378c071700 2023-11-13T10:56:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3791caa640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c154530 con 0x7f378c15b8e0 2023-11-13T10:56:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3790ca8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c0ce260 0x7f378c146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3790ca8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c0ce260 0x7f378c146190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58100/0 (socket says 172.21.15.16:58100) 2023-11-13T10:56:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3790ca8640 1 -- 172.21.15.16:0/3011529491 learned_addr learned my addr 172.21.15.16:0/3011529491 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f378bfff640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15b8e0 0x7f378c0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3790ca8640 1 -- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15b8e0 msgr2=0x7f378c0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3790ca8640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15b8e0 0x7f378c0711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3790ca8640 1 -- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f378c071700 msgr2=0x7f378c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3790ca8640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f378c071700 0x7f378c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.460+0000 7f3790ca8640 1 -- 172.21.15.16:0/3011529491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3780009400 con 0x7f378c0ce260 2023-11-13T10:56:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.461+0000 7f3790ca8640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c0ce260 0x7f378c146190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f37800098c0 tx=0x7f37800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:25.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.461+0000 7f37897fa640 1 -- 172.21.15.16:0/3011529491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378003ba70 con 0x7f378c0ce260 2023-11-13T10:56:25.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.461+0000 7f37897fa640 1 -- 172.21.15.16:0/3011529491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378003bbf0 con 0x7f378c0ce260 2023-11-13T10:56:25.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.461+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c154810 con 0x7f378c0ce260 2023-11-13T10:56:25.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.461+0000 7f37897fa640 1 -- 172.21.15.16:0/3011529491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37800453f0 con 0x7f378c0ce260 2023-11-13T10:56:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.461+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c153bd0 con 0x7f378c0ce260 2023-11-13T10:56:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.462+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378c154d30 con 0x7f378c0ce260 2023-11-13T10:56:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.463+0000 7f37897fa640 1 -- 172.21.15.16:0/3011529491 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f378004e450 con 0x7f378c0ce260 2023-11-13T10:56:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.464+0000 7f37897fa640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f377006ae40 0x7f377006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.464+0000 7f37897fa640 1 -- 172.21.15.16:0/3011529491 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f37800abc50 con 0x7f378c0ce260 2023-11-13T10:56:25.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.465+0000 7f378bfff640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f377006ae40 0x7f377006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.466+0000 7f378bfff640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f377006ae40 0x7f377006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f378c15f000 tx=0x7f3778009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:25.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.468+0000 7f37897fa640 1 -- 172.21.15.16:0/3011529491 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f378c154d30 con 0x7f378c0ce260 2023-11-13T10:56:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.664+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f378c154d30 con 0x7f378c0ce260 2023-11-13T10:56:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.665+0000 7f37897fa640 1 -- 172.21.15.16:0/3011529491 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f378c154d30 con 0x7f378c0ce260 2023-11-13T10:56:25.670 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:25.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f377006ae40 msgr2=0x7f377006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f377006ae40 0x7f377006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f378c15f000 tx=0x7f3778009040 comp rx=0 tx=0).stop 2023-11-13T10:56:25.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c0ce260 msgr2=0x7f378c146190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c0ce260 0x7f378c146190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f37800098c0 tx=0x7f37800046c0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 shutdown_connections 2023-11-13T10:56:25.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f377006ae40 0x7f377006d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f378c071700 0x7f378c15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15b8e0 0x7f378c0711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 --2- 172.21.15.16:0/3011529491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378c0ce260 0x7f378c146190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 >> 172.21.15.16:0/3011529491 conn(0x7f378c0b0970 msgr2=0x7f378c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 shutdown_connections 2023-11-13T10:56:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.668+0000 7f3791caa640 1 -- 172.21.15.16:0/3011529491 wait complete. 2023-11-13T10:56:25.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:25.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.929+0000 7fdb6ffff640 1 Processor -- start 2023-11-13T10:56:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.930+0000 7fdb6ffff640 1 -- start start 2023-11-13T10:56:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.932+0000 7fdb6ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdb700ce260 0x7fdb700cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.932+0000 7fdb6ffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7006d010 con 0x7fdb7015bcc0 2023-11-13T10:56:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.932+0000 7fdb6ffff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7006b860 con 0x7fdb7015b8e0 2023-11-13T10:56:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.932+0000 7fdb6ffff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7006bb40 con 0x7fdb700ce260 2023-11-13T10:56:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.932+0000 7fdb67fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdb7015bcc0 0x7fdb700cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49328/0 (socket says 172.21.15.16:49328) 2023-11-13T10:56:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.932+0000 7fdb67fff640 1 -- 172.21.15.16:0/2956671970 learned_addr learned my addr 172.21.15.16:0/2956671970 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:25.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6f7fe640 1 --2- 172.21.15.16:0/2956671970 >> v2:172.21.15.184:3300/0 conn(0x7fdb700ce260 0x7fdb700cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452006819 0 0) 0x7fdb7006d010 con 0x7fdb7015bcc0 2023-11-13T10:56:25.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb5c003520 con 0x7fdb7015bcc0 2023-11-13T10:56:25.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 860844240 0 0) 0x7fdb5c003520 con 0x7fdb7015bcc0 2023-11-13T10:56:25.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 >> v2:172.21.15.184:3300/0 conn(0x7fdb700ce260 msgr2=0x7fdb700cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:25.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6e7fc640 1 --2- 172.21.15.16:0/2956671970 >> v2:172.21.15.184:3300/0 conn(0x7fdb700ce260 0x7fdb700cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 >> v1:172.21.15.184:6789/0 conn(0x7fdb7015b8e0 legacy=0x7fdb7006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7006be20 con 0x7fdb7015bcc0 2023-11-13T10:56:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fdb58001670 con 0x7fdb7015bcc0 2023-11-13T10:56:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.933+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 >> v1:172.21.15.16:6789/0 conn(0x7fdb7015bcc0 legacy=0x7fdb700cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6e7fc640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb5c003970 0x7fdb5c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6e7fc640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c0047c0 0x7fdb5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6e7fc640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb5c005180 0x7fdb5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7006bb40 con 0x7fdb5c003970 2023-11-13T10:56:25.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c00c020 con 0x7fdb5c005180 2023-11-13T10:56:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c00c1e0 con 0x7fdb5c0047c0 2023-11-13T10:56:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6effd640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb5c003970 0x7fdb5c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb67fff640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c0047c0 0x7fdb5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6f7fe640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb5c005180 0x7fdb5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6effd640 1 -- 172.21.15.16:0/2956671970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c0047c0 msgr2=0x7fdb5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6effd640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c0047c0 0x7fdb5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6effd640 1 -- 172.21.15.16:0/2956671970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb5c005180 msgr2=0x7fdb5c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:25.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6effd640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb5c005180 0x7fdb5c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6effd640 1 -- 172.21.15.16:0/2956671970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c003520 con 0x7fdb5c003970 2023-11-13T10:56:25.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb6f7fe640 1 -- 172.21.15.16:0/2956671970 reap_dead start 2023-11-13T10:56:25.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.934+0000 7fdb67fff640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c0047c0 0x7fdb5c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:25.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.935+0000 7fdb6effd640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb5c003970 0x7fdb5c003d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdb60001750 tx=0x7fdb5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96beb9232c0892d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:25.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.936+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb600052d0 con 0x7fdb5c003970 2023-11-13T10:56:25.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.936+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6002e050 con 0x7fdb5c003970 2023-11-13T10:56:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.936+0000 7fdb6e7fc640 1 -- 172.21.15.16:0/2956671970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60003830 con 0x7fdb5c003970 2023-11-13T10:56:25.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.936+0000 7fdb6ffff640 1 -- 172.21.15.16:0/2956671970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb5c003970 msgr2=0x7fdb5c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.936+0000 7fdb6ffff640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb5c003970 0x7fdb5c003d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdb60001750 tx=0x7fdb5c0057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.936+0000 7fdb6ffff640 1 -- 172.21.15.16:0/2956671970 shutdown_connections 2023-11-13T10:56:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.936+0000 7fdb6ffff640 1 --2- 172.21.15.16:0/2956671970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb5c003970 0x7fdb5c003d90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.936+0000 7fdb6ffff640 1 -- 172.21.15.16:0/2956671970 >> 172.21.15.16:0/2956671970 conn(0x7fdb700b0970 msgr2=0x7fdb700b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.937+0000 7fdb6ffff640 1 -- 172.21.15.16:0/2956671970 shutdown_connections 2023-11-13T10:56:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.937+0000 7fdb6ffff640 1 -- 172.21.15.16:0/2956671970 wait complete. 2023-11-13T10:56:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.938+0000 7fdb6ffff640 1 Processor -- start 2023-11-13T10:56:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.938+0000 7fdb6ffff640 1 -- start start 2023-11-13T10:56:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.939+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c003970 0x7fdb701550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.939+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb7014fe80 0x7fdb7014f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.939+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb70071350 0x7fdb700af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.939+0000 7fdb6ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70154c40 con 0x7fdb7014fe80 2023-11-13T10:56:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.939+0000 7fdb6ffff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70154030 con 0x7fdb70071350 2023-11-13T10:56:25.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.939+0000 7fdb6ffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70154310 con 0x7fdb5c003970 2023-11-13T10:56:25.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb7014fe80 0x7fdb7014f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb7014fe80 0x7fdb7014f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58108/0 (socket says 172.21.15.16:58108) 2023-11-13T10:56:25.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb67fff640 1 -- 172.21.15.16:0/4258979086 learned_addr learned my addr 172.21.15.16:0/4258979086 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:25.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb6effd640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c003970 0x7fdb701550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb6f7fe640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb70071350 0x7fdb700af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb67fff640 1 -- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c003970 msgr2=0x7fdb701550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb67fff640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c003970 0x7fdb701550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb67fff640 1 -- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb70071350 msgr2=0x7fdb700af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb67fff640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb70071350 0x7fdb700af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb67fff640 1 -- 172.21.15.16:0/4258979086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb701545f0 con 0x7fdb7014fe80 2023-11-13T10:56:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb6effd640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c003970 0x7fdb701550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.940+0000 7fdb67fff640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb7014fe80 0x7fdb7014f810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800d770 tx=0x7fdb5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:25.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.941+0000 7fdb677fe640 1 -- 172.21.15.16:0/4258979086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb580074e0 con 0x7fdb7014fe80 2023-11-13T10:56:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.941+0000 7fdb677fe640 1 -- 172.21.15.16:0/4258979086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb58007640 con 0x7fdb7014fe80 2023-11-13T10:56:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.941+0000 7fdb677fe640 1 -- 172.21.15.16:0/4258979086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58004a00 con 0x7fdb7014fe80 2023-11-13T10:56:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.941+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb701548d0 con 0x7fdb7014fe80 2023-11-13T10:56:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.941+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb70071990 con 0x7fdb7014fe80 2023-11-13T10:56:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.942+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb70154db0 con 0x7fdb7014fe80 2023-11-13T10:56:25.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.943+0000 7fdb677fe640 1 -- 172.21.15.16:0/4258979086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb58004b60 con 0x7fdb7014fe80 2023-11-13T10:56:25.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.945+0000 7fdb677fe640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb5006ae70 0x7fdb5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:25.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.945+0000 7fdb677fe640 1 -- 172.21.15.16:0/4258979086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fdb58089100 con 0x7fdb7014fe80 2023-11-13T10:56:25.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.945+0000 7fdb6effd640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb5006ae70 0x7fdb5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:25.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.946+0000 7fdb6effd640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb5006ae70 0x7fdb5006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdb60033000 tx=0x7fdb60009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:25.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:25.949+0000 7fdb677fe640 1 -- 172.21.15.16:0/4258979086 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb58053e60 con 0x7fdb7014fe80 2023-11-13T10:56:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.145+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb7004e4d0 con 0x7fdb7014fe80 2023-11-13T10:56:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.146+0000 7fdb677fe640 1 -- 172.21.15.16:0/4258979086 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb58057480 con 0x7fdb7014fe80 2023-11-13T10:56:26.155 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb5006ae70 msgr2=0x7fdb5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb5006ae70 0x7fdb5006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdb60033000 tx=0x7fdb60009030 comp rx=0 tx=0).stop 2023-11-13T10:56:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb7014fe80 msgr2=0x7fdb7014f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb7014fe80 0x7fdb7014f810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800d770 tx=0x7fdb5800dc30 comp rx=0 tx=0).stop 2023-11-13T10:56:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 shutdown_connections 2023-11-13T10:56:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb5006ae70 0x7fdb5006d330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb70071350 0x7fdb700af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb7014fe80 0x7fdb7014f810 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 --2- 172.21.15.16:0/4258979086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb5c003970 0x7fdb701550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 >> 172.21.15.16:0/4258979086 conn(0x7fdb700b0970 msgr2=0x7fdb7015e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 shutdown_connections 2023-11-13T10:56:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.149+0000 7fdb6ffff640 1 -- 172.21.15.16:0/4258979086 wait complete. 2023-11-13T10:56:26.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:26.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.400+0000 7fe3f8dc5640 1 Processor -- start 2023-11-13T10:56:26.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.401+0000 7fe3f8dc5640 1 -- start start 2023-11-13T10:56:26.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.402+0000 7fe3f8dc5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f415cf90 0x7fe3f40b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.402+0000 7fe3f8dc5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3f406c3b0 0x7fe3f40b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.402+0000 7fe3f8dc5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe3f40b39c0 0x7fe3f40ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.402+0000 7fe3f8dc5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f40b3e50 con 0x7fe3f406c3b0 2023-11-13T10:56:26.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.402+0000 7fe3f8dc5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f40b2e00 con 0x7fe3f40b39c0 2023-11-13T10:56:26.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.402+0000 7fe3f8dc5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40b21f0 con 0x7fe3f415cf90 2023-11-13T10:56:26.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3e9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3f406c3b0 0x7fe3f40b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3e9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3f406c3b0 0x7fe3f40b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58116/0 (socket says 172.21.15.16:58116) 2023-11-13T10:56:26.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3e9d9b640 1 -- 172.21.15.16:0/3126810386 learned_addr learned my addr 172.21.15.16:0/3126810386 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:26.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3f259c640 1 --2- 172.21.15.16:0/3126810386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f415cf90 0x7fe3f40b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3e9d9b640 1 -- 172.21.15.16:0/3126810386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f415cf90 msgr2=0x7fe3f40b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:26.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3e9d9b640 1 --2- 172.21.15.16:0/3126810386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f415cf90 0x7fe3f40b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3f2d9d640 1 --2- 172.21.15.16:0/3126810386 >> v2:172.21.15.184:3300/0 conn(0x7fe3f40b39c0 0x7fe3f40ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3e9d9b640 1 -- 172.21.15.16:0/3126810386 >> v2:172.21.15.184:3300/0 conn(0x7fe3f40b39c0 msgr2=0x7fe3f40ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:26.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3e9d9b640 1 --2- 172.21.15.16:0/3126810386 >> v2:172.21.15.184:3300/0 conn(0x7fe3f40b39c0 0x7fe3f40ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.403+0000 7fe3e9d9b640 1 -- 172.21.15.16:0/3126810386 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f40b24d0 con 0x7fe3f406c3b0 2023-11-13T10:56:26.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.404+0000 7fe3e9d9b640 1 --2- 172.21.15.16:0/3126810386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f406c3b0 0x7fe3f40b3480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe3f406dbb0 tx=0x7fe3dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44bb01138fd5439 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:26.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.405+0000 7fe3f1d9b640 1 -- 172.21.15.16:0/3126810386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc031870 con 0x7fe3f406c3b0 2023-11-13T10:56:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.405+0000 7fe3f1d9b640 1 -- 172.21.15.16:0/3126810386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc0319d0 con 0x7fe3f406c3b0 2023-11-13T10:56:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.405+0000 7fe3f1d9b640 1 -- 172.21.15.16:0/3126810386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc031d60 con 0x7fe3f406c3b0 2023-11-13T10:56:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.405+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3126810386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f406c3b0 msgr2=0x7fe3f40b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.405+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3126810386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f406c3b0 0x7fe3f40b3480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe3f406dbb0 tx=0x7fe3dc02b910 comp rx=0 tx=0).stop 2023-11-13T10:56:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.406+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3126810386 shutdown_connections 2023-11-13T10:56:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.406+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3126810386 >> v2:172.21.15.184:3300/0 conn(0x7fe3f40b39c0 0x7fe3f40ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.406+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3126810386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f406c3b0 0x7fe3f40b3480 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.406+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3126810386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f415cf90 0x7fe3f40b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.406+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3126810386 >> 172.21.15.16:0/3126810386 conn(0x7fe3f406a190 msgr2=0x7fe3f406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.406+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3126810386 shutdown_connections 2023-11-13T10:56:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.406+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3126810386 wait complete. 2023-11-13T10:56:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.408+0000 7fe3f8dc5640 1 Processor -- start 2023-11-13T10:56:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.408+0000 7fe3f8dc5640 1 -- start start 2023-11-13T10:56:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f8dc5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3f406c3b0 0x7fe3f414bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f8dc5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f40b39c0 0x7fe3f414b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f8dc5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f415cf90 0x7fe3f40cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f8dc5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4152110 con 0x7fe3f415cf90 2023-11-13T10:56:26.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f8dc5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4150820 con 0x7fe3f406c3b0 2023-11-13T10:56:26.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f8dc5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f415e900 con 0x7fe3f40b39c0 2023-11-13T10:56:26.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f415cf90 0x7fe3f40cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f415cf90 0x7fe3f40cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58120/0 (socket says 172.21.15.16:58120) 2023-11-13T10:56:26.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f2d9d640 1 -- 172.21.15.16:0/3436517142 learned_addr learned my addr 172.21.15.16:0/3436517142 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:26.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3f259c640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3f406c3b0 0x7fe3f414bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.409+0000 7fe3e9d9b640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f40b39c0 0x7fe3f414b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3f2d9d640 1 -- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f40b39c0 msgr2=0x7fe3f414b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3f2d9d640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f40b39c0 0x7fe3f414b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3f2d9d640 1 -- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3f406c3b0 msgr2=0x7fe3f414bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3f2d9d640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3f406c3b0 0x7fe3f414bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3f2d9d640 1 -- 172.21.15.16:0/3436517142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0094d0 con 0x7fe3f415cf90 2023-11-13T10:56:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3e9d9b640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f40b39c0 0x7fe3f414b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3f2d9d640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f415cf90 0x7fe3f40cc860 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00c7f0 tx=0x7fe3ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3eb7fe640 1 -- 172.21.15.16:0/3436517142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec004220 con 0x7fe3f415cf90 2023-11-13T10:56:26.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3eb7fe640 1 -- 172.21.15.16:0/3436517142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec004380 con 0x7fe3f415cf90 2023-11-13T10:56:26.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3eb7fe640 1 -- 172.21.15.16:0/3436517142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec007860 con 0x7fe3f415cf90 2023-11-13T10:56:26.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f415ebe0 con 0x7fe3f415cf90 2023-11-13T10:56:26.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.410+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f415f1a0 con 0x7fe3f415cf90 2023-11-13T10:56:26.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.412+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f4072520 con 0x7fe3f415cf90 2023-11-13T10:56:26.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.413+0000 7fe3eb7fe640 1 -- 172.21.15.16:0/3436517142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3ec007bd0 con 0x7fe3f415cf90 2023-11-13T10:56:26.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.413+0000 7fe3eb7fe640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe3d406ae40 0x7fe3d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.414+0000 7fe3eb7fe640 1 -- 172.21.15.16:0/3436517142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fe3ec089940 con 0x7fe3f415cf90 2023-11-13T10:56:26.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.414+0000 7fe3f259c640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe3d406ae40 0x7fe3d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.415+0000 7fe3f259c640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe3d406ae40 0x7fe3d406d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4007130 tx=0x7fe3e4007060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:26.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.418+0000 7fe3eb7fe640 1 -- 172.21.15.16:0/3436517142 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3ec0546a0 con 0x7fe3f415cf90 2023-11-13T10:56:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.610+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3f40adea0 con 0x7fe3f415cf90 2023-11-13T10:56:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.611+0000 7fe3eb7fe640 1 -- 172.21.15.16:0/3436517142 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3ec057cc0 con 0x7fe3f415cf90 2023-11-13T10:56:26.616 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.613+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe3d406ae40 msgr2=0x7fe3d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.613+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe3d406ae40 0x7fe3d406d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4007130 tx=0x7fe3e4007060 comp rx=0 tx=0).stop 2023-11-13T10:56:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.613+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f415cf90 msgr2=0x7fe3f40cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.613+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f415cf90 0x7fe3f40cc860 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00c7f0 tx=0x7fe3ec00ccb0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.614+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 shutdown_connections 2023-11-13T10:56:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.614+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe3d406ae40 0x7fe3d406d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.614+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f415cf90 0x7fe3f40cc860 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.614+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3f40b39c0 0x7fe3f414b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.614+0000 7fe3f8dc5640 1 --2- 172.21.15.16:0/3436517142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3f406c3b0 0x7fe3f414bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.614+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 >> 172.21.15.16:0/3436517142 conn(0x7fe3f406a190 msgr2=0x7fe3f406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.614+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 shutdown_connections 2023-11-13T10:56:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.614+0000 7fe3f8dc5640 1 -- 172.21.15.16:0/3436517142 wait complete. 2023-11-13T10:56:26.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.859+0000 7fea0c7d7640 1 Processor -- start 2023-11-13T10:56:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.859+0000 7fea0c7d7640 1 -- start start 2023-11-13T10:56:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.861+0000 7fea0c7d7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea040ce1d0 0x7fea040ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.861+0000 7fea0c7d7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0415d8d0 con 0x7fea040ce5b0 2023-11-13T10:56:26.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.861+0000 7fea0c7d7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0415af00 con 0x7fea0415e390 2023-11-13T10:56:26.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.861+0000 7fea0c7d7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0415db90 con 0x7fea040ce1d0 2023-11-13T10:56:26.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.861+0000 7fea09d72640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea040ce5b0 0x7fea0415a130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49332/0 (socket says 172.21.15.16:49332) 2023-11-13T10:56:26.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.861+0000 7fea09d72640 1 -- 172.21.15.16:0/3350377610 learned_addr learned my addr 172.21.15.16:0/3350377610 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:26.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea0a573640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea040ce1d0 0x7fea040ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125526780 0 0) 0x7fea0415d8d0 con 0x7fea040ce5b0 2023-11-13T10:56:26.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f0003520 con 0x7fea040ce5b0 2023-11-13T10:56:26.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633437228 0 0) 0x7fe9f0003520 con 0x7fea040ce5b0 2023-11-13T10:56:26.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea040ce1d0 msgr2=0x7fea040ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:26.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea040ce1d0 0x7fea040ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 >> v1:172.21.15.184:6789/0 conn(0x7fea0415e390 legacy=0x7fea040b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0415de70 con 0x7fea040ce5b0 2023-11-13T10:56:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fe9f4001670 con 0x7fea040ce5b0 2023-11-13T10:56:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 >> v1:172.21.15.16:6789/0 conn(0x7fea040ce5b0 legacy=0x7fea0415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fe9f0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea040ce1d0 0x7fe9f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0004d00 0x7fe9f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0415db90 con 0x7fe9f00039a0 2023-11-13T10:56:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f000ba60 con 0x7fe9f0004d00 2023-11-13T10:56:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.862+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f000bc20 con 0x7fea040ce1d0 2023-11-13T10:56:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.863+0000 7fea0a573640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fe9f0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.863+0000 7fea09d72640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea040ce1d0 0x7fe9f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.863+0000 7fea0ad74640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0004d00 0x7fe9f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.863+0000 7fea0a573640 1 -- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea040ce1d0 msgr2=0x7fe9f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.863+0000 7fea0a573640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea040ce1d0 0x7fe9f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.863+0000 7fea0a573640 1 -- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0004d00 msgr2=0x7fe9f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.863+0000 7fea0a573640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f0004d00 0x7fe9f0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.863+0000 7fea0a573640 1 -- 172.21.15.16:0/3350377610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0003520 con 0x7fe9f00039a0 2023-11-13T10:56:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.864+0000 7fea0a573640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fe9f0003dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fea00009a90 tx=0x7fe9f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870d07a2b4aeb28b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.864+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0002c680 con 0x7fe9f00039a0 2023-11-13T10:56:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.865+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0002c7e0 con 0x7fe9f00039a0 2023-11-13T10:56:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.865+0000 7fea09571640 1 -- 172.21.15.16:0/3350377610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0002cb70 con 0x7fe9f00039a0 2023-11-13T10:56:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.865+0000 7fea0c7d7640 1 -- 172.21.15.16:0/3350377610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 msgr2=0x7fe9f0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.865+0000 7fea0c7d7640 1 --2- 172.21.15.16:0/3350377610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fe9f0003dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fea00009a90 tx=0x7fe9f00057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.865+0000 7fea0ad74640 1 -- 172.21.15.16:0/3350377610 reap_dead start 2023-11-13T10:56:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.865+0000 7fea0c7d7640 1 -- 172.21.15.16:0/3350377610 shutdown_connections 2023-11-13T10:56:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.865+0000 7fea0c7d7640 1 -- 172.21.15.16:0/3350377610 >> 172.21.15.16:0/3350377610 conn(0x7fea0406a190 msgr2=0x7fea0406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.865+0000 7fea0c7d7640 1 -- 172.21.15.16:0/3350377610 shutdown_connections 2023-11-13T10:56:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.866+0000 7fea0c7d7640 1 -- 172.21.15.16:0/3350377610 wait complete. 2023-11-13T10:56:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.867+0000 7fea0c7d7640 1 Processor -- start 2023-11-13T10:56:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.867+0000 7fea0c7d7640 1 -- start start 2023-11-13T10:56:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.868+0000 7fea0c7d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fea041534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.868+0000 7fea0c7d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea0414f550 0x7fea04138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.868+0000 7fea0c7d7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea041f6250 0x7fea041f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.868+0000 7fea0c7d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0414c8b0 con 0x7fe9f00039a0 2023-11-13T10:56:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.868+0000 7fea0c7d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041f2be0 con 0x7fea0414f550 2023-11-13T10:56:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.868+0000 7fea0c7d7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea04153b10 con 0x7fea041f6250 2023-11-13T10:56:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.869+0000 7fea0a573640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fea041534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.869+0000 7fea0a573640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fea041534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58142/0 (socket says 172.21.15.16:58142) 2023-11-13T10:56:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.869+0000 7fea0a573640 1 -- 172.21.15.16:0/884358488 learned_addr learned my addr 172.21.15.16:0/884358488 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.869+0000 7fea0a573640 1 -- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea041f6250 msgr2=0x7fea041f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.869+0000 7fea0a573640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea041f6250 0x7fea041f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.869+0000 7fea0a573640 1 -- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea0414f550 msgr2=0x7fea04138a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.869+0000 7fea0a573640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea0414f550 0x7fea04138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.869+0000 7fea0a573640 1 -- 172.21.15.16:0/884358488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0414e790 con 0x7fe9f00039a0 2023-11-13T10:56:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.869+0000 7fea0a573640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fea041534f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea00009a90 tx=0x7fea00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.870+0000 7fe9faffd640 1 -- 172.21.15.16:0/884358488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0002dea0 con 0x7fe9f00039a0 2023-11-13T10:56:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.870+0000 7fe9faffd640 1 -- 172.21.15.16:0/884358488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea00003810 con 0x7fe9f00039a0 2023-11-13T10:56:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.870+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0414ea70 con 0x7fe9f00039a0 2023-11-13T10:56:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.870+0000 7fe9faffd640 1 -- 172.21.15.16:0/884358488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00003ba0 con 0x7fe9f00039a0 2023-11-13T10:56:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.870+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0414f030 con 0x7fe9f00039a0 2023-11-13T10:56:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.871+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea04072520 con 0x7fe9f00039a0 2023-11-13T10:56:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.872+0000 7fe9faffd640 1 -- 172.21.15.16:0/884358488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea00003d00 con 0x7fe9f00039a0 2023-11-13T10:56:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.873+0000 7fe9faffd640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe9e806ae70 0x7fe9e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.873+0000 7fe9faffd640 1 -- 172.21.15.16:0/884358488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fea000ab850 con 0x7fe9f00039a0 2023-11-13T10:56:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.874+0000 7fea09d72640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe9e806ae70 0x7fe9e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.875+0000 7fea09d72640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe9e806ae70 0x7fe9e806d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4001050 tx=0x7fe9f4000f60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:26.877+0000 7fe9faffd640 1 -- 172.21.15.16:0/884358488 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea00076520 con 0x7fe9f00039a0 2023-11-13T10:56:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.078+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fea0404e5b0 con 0x7fe9f00039a0 2023-11-13T10:56:27.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.079+0000 7fe9faffd640 1 -- 172.21.15.16:0/884358488 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fea000044e0 con 0x7fe9f00039a0 2023-11-13T10:56:27.084 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe9e806ae70 msgr2=0x7fe9e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe9e806ae70 0x7fe9e806d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4001050 tx=0x7fe9f4000f60 comp rx=0 tx=0).stop 2023-11-13T10:56:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 msgr2=0x7fea041534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fea041534f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea00009a90 tx=0x7fea00004730 comp rx=0 tx=0).stop 2023-11-13T10:56:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 shutdown_connections 2023-11-13T10:56:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe9e806ae70 0x7fe9e806d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea041f6250 0x7fea041f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea0414f550 0x7fea04138a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 --2- 172.21.15.16:0/884358488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9f00039a0 0x7fea041534f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 >> 172.21.15.16:0/884358488 conn(0x7fea0406a190 msgr2=0x7fea0406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.082+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 shutdown_connections 2023-11-13T10:56:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.083+0000 7fea0c7d7640 1 -- 172.21.15.16:0/884358488 wait complete. 2023-11-13T10:56:27.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.329+0000 7f033cd3c640 1 Processor -- start 2023-11-13T10:56:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.329+0000 7f033cd3c640 1 -- start start 2023-11-13T10:56:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.331+0000 7f033cd3c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f033815cfc0 0x7f03380cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.331+0000 7f033cd3c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03380d2ae0 0x7f03380d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.331+0000 7f033cd3c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03380b3cf0 con 0x7f03380b29d0 2023-11-13T10:56:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.331+0000 7f033cd3c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03380d3480 con 0x7f033815cfc0 2023-11-13T10:56:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.331+0000 7f033cd3c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380b3730 con 0x7f03380d2ae0 2023-11-13T10:56:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.331+0000 7f03377fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f033815cfc0 0x7f03380cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.331+0000 7f03377fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f033815cfc0 0x7f03380cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58150/0 (socket says 172.21.15.16:58150) 2023-11-13T10:56:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.331+0000 7f03377fe640 1 -- 172.21.15.16:0/2905172344 learned_addr learned my addr 172.21.15.16:0/2905172344 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.332+0000 7f0336ffd640 1 --2- 172.21.15.16:0/2905172344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03380d2ae0 0x7f03380d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.332+0000 7f03377fe640 1 -- 172.21.15.16:0/2905172344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03380d2ae0 msgr2=0x7f03380d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.332+0000 7f03377fe640 1 --2- 172.21.15.16:0/2905172344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03380d2ae0 0x7f03380d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.332+0000 7f03377fe640 1 -- 172.21.15.16:0/2905172344 >> v1:172.21.15.184:6789/0 conn(0x7f03380b29d0 legacy=0x7f03380b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.332+0000 7f03377fe640 1 -- 172.21.15.16:0/2905172344 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380b3a10 con 0x7f033815cfc0 2023-11-13T10:56:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.333+0000 7f03377fe640 1 --2- 172.21.15.16:0/2905172344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f033815cfc0 0x7f03380cd770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0328009e50 tx=0x7f032802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c5b9a9632ce4b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.333+0000 7f03367fc640 1 -- 172.21.15.16:0/2905172344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032802c9e0 con 0x7f033815cfc0 2023-11-13T10:56:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f03367fc640 1 -- 172.21.15.16:0/2905172344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032802cb40 con 0x7f033815cfc0 2023-11-13T10:56:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f03367fc640 1 -- 172.21.15.16:0/2905172344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03280343f0 con 0x7f033815cfc0 2023-11-13T10:56:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f033cd3c640 1 -- 172.21.15.16:0/2905172344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f033815cfc0 msgr2=0x7f03380cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f033cd3c640 1 --2- 172.21.15.16:0/2905172344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f033815cfc0 0x7f03380cd770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0328009e50 tx=0x7f032802be40 comp rx=0 tx=0).stop 2023-11-13T10:56:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f033cd3c640 1 -- 172.21.15.16:0/2905172344 shutdown_connections 2023-11-13T10:56:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f033cd3c640 1 --2- 172.21.15.16:0/2905172344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03380d2ae0 0x7f03380d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f033cd3c640 1 --2- 172.21.15.16:0/2905172344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f033815cfc0 0x7f03380cd770 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f033cd3c640 1 -- 172.21.15.16:0/2905172344 >> 172.21.15.16:0/2905172344 conn(0x7f033808c570 msgr2=0x7f033809be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f033cd3c640 1 -- 172.21.15.16:0/2905172344 shutdown_connections 2023-11-13T10:56:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.334+0000 7f033cd3c640 1 -- 172.21.15.16:0/2905172344 wait complete. 2023-11-13T10:56:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.336+0000 7f033cd3c640 1 Processor -- start 2023-11-13T10:56:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.336+0000 7f033cd3c640 1 -- start start 2023-11-13T10:56:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.337+0000 7f033cd3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380b29d0 0x7f03380cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.337+0000 7f033cd3c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03380d2ae0 0x7f0338158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.337+0000 7f033cd3c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f033815cfc0 0x7f03380c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.337+0000 7f033cd3c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338158960 con 0x7f03380b29d0 2023-11-13T10:56:27.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.337+0000 7f033cd3c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380cad50 con 0x7f033815cfc0 2023-11-13T10:56:27.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.337+0000 7f033cd3c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380cc710 con 0x7f03380d2ae0 2023-11-13T10:56:27.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.337+0000 7f03377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380b29d0 0x7f03380cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:27.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.338+0000 7f03377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380b29d0 0x7f03380cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58162/0 (socket says 172.21.15.16:58162) 2023-11-13T10:56:27.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.338+0000 7f03377fe640 1 -- 172.21.15.16:0/3765406293 learned_addr learned my addr 172.21.15.16:0/3765406293 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:27.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.338+0000 7f03377fe640 1 -- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03380d2ae0 msgr2=0x7f0338158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.338+0000 7f03377fe640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03380d2ae0 0x7f0338158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.338+0000 7f03377fe640 1 -- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f033815cfc0 msgr2=0x7f03380c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.338+0000 7f03377fe640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f033815cfc0 0x7f03380c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.338+0000 7f03377fe640 1 -- 172.21.15.16:0/3765406293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328009b30 con 0x7f03380b29d0 2023-11-13T10:56:27.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.338+0000 7f03377fe640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380b29d0 0x7f03380cb1e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0328000f80 tx=0x7f0328004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.338+0000 7f031ffff640 1 -- 172.21.15.16:0/3765406293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328009510 con 0x7f03380b29d0 2023-11-13T10:56:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.339+0000 7f031ffff640 1 -- 172.21.15.16:0/3765406293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032803e450 con 0x7f03380b29d0 2023-11-13T10:56:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.339+0000 7f031ffff640 1 -- 172.21.15.16:0/3765406293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032803e800 con 0x7f03380b29d0 2023-11-13T10:56:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.339+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03380cc9f0 con 0x7f03380b29d0 2023-11-13T10:56:27.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.339+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03380ca9e0 con 0x7f03380b29d0 2023-11-13T10:56:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.340+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033804e4d0 con 0x7f03380b29d0 2023-11-13T10:56:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.341+0000 7f031ffff640 1 -- 172.21.15.16:0/3765406293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f032803e960 con 0x7f03380b29d0 2023-11-13T10:56:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.342+0000 7f031ffff640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f032406ae40 0x7f032406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.342+0000 7f031ffff640 1 -- 172.21.15.16:0/3765406293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f03280ac7a0 con 0x7f03380b29d0 2023-11-13T10:56:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.342+0000 7f0336ffd640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f032406ae40 0x7f032406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.343+0000 7f0336ffd640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f032406ae40 0x7f032406d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f032c006fd0 tx=0x7f032c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.346+0000 7f031ffff640 1 -- 172.21.15.16:0/3765406293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0328077610 con 0x7f03380b29d0 2023-11-13T10:56:27.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.540+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03380b1a50 con 0x7f03380b29d0 2023-11-13T10:56:27.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.543+0000 7f031ffff640 1 -- 172.21.15.16:0/3765406293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f032807ac30 con 0x7f03380b29d0 2023-11-13T10:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:27.549 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:27.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.545+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f032406ae40 msgr2=0x7f032406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.545+0000 7f033cd3c640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f032406ae40 0x7f032406d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f032c006fd0 tx=0x7f032c008040 comp rx=0 tx=0).stop 2023-11-13T10:56:27.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.545+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380b29d0 msgr2=0x7f03380cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.545+0000 7f033cd3c640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380b29d0 0x7f03380cb1e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0328000f80 tx=0x7f0328004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:27.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.546+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 shutdown_connections 2023-11-13T10:56:27.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.546+0000 7f033cd3c640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f032406ae40 0x7f032406d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.546+0000 7f033cd3c640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f033815cfc0 0x7f03380c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.546+0000 7f033cd3c640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03380d2ae0 0x7f0338158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.546+0000 7f033cd3c640 1 --2- 172.21.15.16:0/3765406293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03380b29d0 0x7f03380cb1e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.546+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 >> 172.21.15.16:0/3765406293 conn(0x7f033808c570 msgr2=0x7f033806bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:27.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.546+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 shutdown_connections 2023-11-13T10:56:27.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.546+0000 7f033cd3c640 1 -- 172.21.15.16:0/3765406293 wait complete. 2023-11-13T10:56:27.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:27.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:27.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.795+0000 7f61060df640 1 Processor -- start 2023-11-13T10:56:27.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.796+0000 7f61060df640 1 -- start start 2023-11-13T10:56:27.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.797+0000 7f61060df640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f610015ce70 0x7f61000cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.797+0000 7f61060df640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000d1850 0x7f61000c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.797+0000 7f61060df640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61000ce0e0 con 0x7f61000cdbb0 2023-11-13T10:56:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.797+0000 7f61060df640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61000ce3c0 con 0x7f610015ce70 2023-11-13T10:56:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.797+0000 7f61060df640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61000ce6a0 con 0x7f61000d1850 2023-11-13T10:56:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.798+0000 7f61050dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f610015ce70 0x7f61000cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.798+0000 7f61048dc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f61000cdbb0 0x7f61000d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49340/0 (socket says 172.21.15.16:49340) 2023-11-13T10:56:27.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.798+0000 7f61048dc640 1 -- 172.21.15.16:0/4119858259 learned_addr learned my addr 172.21.15.16:0/4119858259 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:27.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.798+0000 7f61058de640 1 --2- 172.21.15.16:0/4119858259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000d1850 0x7f61000c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:27.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.798+0000 7f61050dd640 1 -- 172.21.15.16:0/4119858259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000d1850 msgr2=0x7f61000c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:27.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.798+0000 7f61050dd640 1 --2- 172.21.15.16:0/4119858259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000d1850 0x7f61000c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.798+0000 7f61050dd640 1 -- 172.21.15.16:0/4119858259 >> v1:172.21.15.16:6789/0 conn(0x7f61000cdbb0 legacy=0x7f61000d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.798+0000 7f61050dd640 1 -- 172.21.15.16:0/4119858259 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61000ce980 con 0x7f610015ce70 2023-11-13T10:56:27.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.798+0000 7f60fffff640 1 -- 172.21.15.16:0/4119858259 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382890761 0 0) 0x7f61000ce0e0 con 0x7f61000cdbb0 2023-11-13T10:56:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.799+0000 7f61050dd640 1 --2- 172.21.15.16:0/4119858259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 0x7f61000cd670 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009d90 tx=0x7f60f002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2654463e34e1e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.800+0000 7f60fffff640 1 -- 172.21.15.16:0/4119858259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f002cc60 con 0x7f610015ce70 2023-11-13T10:56:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.800+0000 7f60fffff640 1 -- 172.21.15.16:0/4119858259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f002cdc0 con 0x7f610015ce70 2023-11-13T10:56:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.800+0000 7f60fffff640 1 -- 172.21.15.16:0/4119858259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0034660 con 0x7f610015ce70 2023-11-13T10:56:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.800+0000 7f61060df640 1 -- 172.21.15.16:0/4119858259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 msgr2=0x7f61000cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.800+0000 7f61060df640 1 --2- 172.21.15.16:0/4119858259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 0x7f61000cd670 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009d90 tx=0x7f60f002beb0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.800+0000 7f61060df640 1 -- 172.21.15.16:0/4119858259 shutdown_connections 2023-11-13T10:56:27.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.800+0000 7f61060df640 1 --2- 172.21.15.16:0/4119858259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000d1850 0x7f61000c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.800+0000 7f61060df640 1 --2- 172.21.15.16:0/4119858259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 0x7f61000cd670 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.800+0000 7f61060df640 1 -- 172.21.15.16:0/4119858259 >> 172.21.15.16:0/4119858259 conn(0x7f610008c580 msgr2=0x7f610009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:27.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.801+0000 7f61060df640 1 -- 172.21.15.16:0/4119858259 shutdown_connections 2023-11-13T10:56:27.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.801+0000 7f61060df640 1 -- 172.21.15.16:0/4119858259 wait complete. 2023-11-13T10:56:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.802+0000 7f61060df640 1 Processor -- start 2023-11-13T10:56:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.802+0000 7f61060df640 1 -- start start 2023-11-13T10:56:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.803+0000 7f61060df640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000cdbb0 0x7f6100152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.803+0000 7f61060df640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61000d1850 0x7f61001f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.803+0000 7f61060df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 0x7f61000b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.803+0000 7f61060df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61000b5c90 con 0x7f610015ce70 2023-11-13T10:56:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.803+0000 7f61060df640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61001f6df0 con 0x7f61000d1850 2023-11-13T10:56:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.803+0000 7f61060df640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61001f6830 con 0x7f61000cdbb0 2023-11-13T10:56:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.804+0000 7f61058de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 0x7f61000b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.804+0000 7f61058de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 0x7f61000b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58174/0 (socket says 172.21.15.16:58174) 2023-11-13T10:56:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.804+0000 7f61058de640 1 -- 172.21.15.16:0/262636794 learned_addr learned my addr 172.21.15.16:0/262636794 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.804+0000 7f61050dd640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000cdbb0 0x7f6100152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.804+0000 7f61058de640 1 -- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000cdbb0 msgr2=0x7f6100152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.804+0000 7f61058de640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000cdbb0 0x7f6100152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.804+0000 7f61058de640 1 -- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61000d1850 msgr2=0x7f61001f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.804+0000 7f61058de640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61000d1850 0x7f61001f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.804+0000 7f61058de640 1 -- 172.21.15.16:0/262636794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f0009a70 con 0x7f610015ce70 2023-11-13T10:56:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.805+0000 7f61058de640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 0x7f61000b60b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f60f800c7c0 tx=0x7f60f800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.805+0000 7f60fdffb640 1 -- 172.21.15.16:0/262636794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f8004220 con 0x7f610015ce70 2023-11-13T10:56:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.805+0000 7f60fdffb640 1 -- 172.21.15.16:0/262636794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f8004380 con 0x7f610015ce70 2023-11-13T10:56:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.805+0000 7f60fdffb640 1 -- 172.21.15.16:0/262636794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f8007840 con 0x7f610015ce70 2023-11-13T10:56:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.805+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61001f6b10 con 0x7f610015ce70 2023-11-13T10:56:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.805+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61001525a0 con 0x7f610015ce70 2023-11-13T10:56:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.806+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61000b5dd0 con 0x7f610015ce70 2023-11-13T10:56:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.807+0000 7f60fdffb640 1 -- 172.21.15.16:0/262636794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f60f80079a0 con 0x7f610015ce70 2023-11-13T10:56:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.808+0000 7f60fdffb640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f60ec06ae40 0x7f60ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.808+0000 7f60fdffb640 1 -- 172.21.15.16:0/262636794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f60f8089640 con 0x7f610015ce70 2023-11-13T10:56:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.809+0000 7f61050dd640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f60ec06ae40 0x7f60ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.810+0000 7f61050dd640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f60ec06ae40 0x7f60ec06d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60f0036000 tx=0x7f60f0030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:27.812+0000 7f60fdffb640 1 -- 172.21.15.16:0/262636794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61000b5dd0 con 0x7f610015ce70 2023-11-13T10:56:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.011+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61000b5dd0 con 0x7f610015ce70 2023-11-13T10:56:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.011+0000 7f60fdffb640 1 -- 172.21.15.16:0/262636794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61000b5dd0 con 0x7f610015ce70 2023-11-13T10:56:28.021 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f60ec06ae40 msgr2=0x7f60ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f60ec06ae40 0x7f60ec06d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60f0036000 tx=0x7f60f0030000 comp rx=0 tx=0).stop 2023-11-13T10:56:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 msgr2=0x7f61000b60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 0x7f61000b60b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f60f800c7c0 tx=0x7f60f800cc80 comp rx=0 tx=0).stop 2023-11-13T10:56:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 shutdown_connections 2023-11-13T10:56:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f60ec06ae40 0x7f60ec06d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610015ce70 0x7f61000b60b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61000d1850 0x7f61001f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 --2- 172.21.15.16:0/262636794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61000cdbb0 0x7f6100152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 >> 172.21.15.16:0/262636794 conn(0x7f610008c580 msgr2=0x7f61000cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.014+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 shutdown_connections 2023-11-13T10:56:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.015+0000 7f61060df640 1 -- 172.21.15.16:0/262636794 wait complete. 2023-11-13T10:56:28.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.264+0000 7f20c7389640 1 Processor -- start 2023-11-13T10:56:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.264+0000 7f20c7389640 1 -- start start 2023-11-13T10:56:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.266+0000 7f20c7389640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f20c015cf90 0x7f20c00b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.266+0000 7f20c7389640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006c3b0 0x7f20c00b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.266+0000 7f20c7389640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c00b2e00 con 0x7f20c00b39c0 2023-11-13T10:56:28.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.267+0000 7f20c7389640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c00b3e50 con 0x7f20c015cf90 2023-11-13T10:56:28.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.267+0000 7f20c7389640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b21f0 con 0x7f20c006c3b0 2023-11-13T10:56:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.267+0000 7f20c4924640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006c3b0 0x7f20c00b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.267+0000 7f20c4924640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006c3b0 0x7f20c00b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50216/0 (socket says 172.21.15.16:50216) 2023-11-13T10:56:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.267+0000 7f20c4924640 1 -- 172.21.15.16:0/2256661028 learned_addr learned my addr 172.21.15.16:0/2256661028 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.267+0000 7f20c5926640 1 --1- 172.21.15.16:0/2256661028 >> v1:172.21.15.184:6789/0 conn(0x7f20c00b39c0 0x7f20c00ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:52796/0 (socket says 172.21.15.16:52796) 2023-11-13T10:56:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.267+0000 7f20c5125640 1 --2- 172.21.15.16:0/2256661028 >> v2:172.21.15.184:3300/0 conn(0x7f20c015cf90 0x7f20c00b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.268+0000 7f20b7fff640 1 -- 172.21.15.16:0/2256661028 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433559203 0 0) 0x7f20c00b2e00 con 0x7f20c00b39c0 2023-11-13T10:56:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.268+0000 7f20b7fff640 1 -- 172.21.15.16:0/2256661028 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac003520 con 0x7f20c00b39c0 2023-11-13T10:56:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.268+0000 7f20c4924640 1 -- 172.21.15.16:0/2256661028 >> v2:172.21.15.184:3300/0 conn(0x7f20c015cf90 msgr2=0x7f20c00b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.268+0000 7f20c4924640 1 --2- 172.21.15.16:0/2256661028 >> v2:172.21.15.184:3300/0 conn(0x7f20c015cf90 0x7f20c00b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.268+0000 7f20c4924640 1 -- 172.21.15.16:0/2256661028 >> v1:172.21.15.184:6789/0 conn(0x7f20c00b39c0 legacy=0x7f20c00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.268+0000 7f20c4924640 1 -- 172.21.15.16:0/2256661028 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00b24d0 con 0x7f20c006c3b0 2023-11-13T10:56:28.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.269+0000 7f20c4924640 1 --2- 172.21.15.16:0/2256661028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006c3b0 0x7f20c00b3480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009870 tx=0x7f20b002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=139463b917ecd436 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:28.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.270+0000 7f20b7fff640 1 -- 172.21.15.16:0/2256661028 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0030840 con 0x7f20c006c3b0 2023-11-13T10:56:28.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.270+0000 7f20b7fff640 1 -- 172.21.15.16:0/2256661028 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b00309a0 con 0x7f20c006c3b0 2023-11-13T10:56:28.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.270+0000 7f20b7fff640 1 -- 172.21.15.16:0/2256661028 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0030d30 con 0x7f20c006c3b0 2023-11-13T10:56:28.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.270+0000 7f20c7389640 1 -- 172.21.15.16:0/2256661028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006c3b0 msgr2=0x7f20c00b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.270+0000 7f20c7389640 1 --2- 172.21.15.16:0/2256661028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006c3b0 0x7f20c00b3480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009870 tx=0x7f20b002b980 comp rx=0 tx=0).stop 2023-11-13T10:56:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.271+0000 7f20c7389640 1 -- 172.21.15.16:0/2256661028 shutdown_connections 2023-11-13T10:56:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.271+0000 7f20c7389640 1 --2- 172.21.15.16:0/2256661028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c006c3b0 0x7f20c00b3480 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.271+0000 7f20c7389640 1 --2- 172.21.15.16:0/2256661028 >> v2:172.21.15.184:3300/0 conn(0x7f20c015cf90 0x7f20c00b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.271+0000 7f20c7389640 1 -- 172.21.15.16:0/2256661028 >> 172.21.15.16:0/2256661028 conn(0x7f20c006a190 msgr2=0x7f20c006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.271+0000 7f20c7389640 1 -- 172.21.15.16:0/2256661028 shutdown_connections 2023-11-13T10:56:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.271+0000 7f20c7389640 1 -- 172.21.15.16:0/2256661028 wait complete. 2023-11-13T10:56:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.273+0000 7f20c7389640 1 Processor -- start 2023-11-13T10:56:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.273+0000 7f20c7389640 1 -- start start 2023-11-13T10:56:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c7389640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c006c3b0 0x7f20c015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c7389640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c00b39c0 0x7f20c015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c7389640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c015cf90 0x7f20c015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c7389640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00cbae0 con 0x7f20c00b39c0 2023-11-13T10:56:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c7389640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00cbdc0 con 0x7f20c006c3b0 2023-11-13T10:56:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c7389640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00cc0a0 con 0x7f20c015cf90 2023-11-13T10:56:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c4924640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c00b39c0 0x7f20c015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c4924640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c00b39c0 0x7f20c015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58180/0 (socket says 172.21.15.16:58180) 2023-11-13T10:56:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c4924640 1 -- 172.21.15.16:0/1220716440 learned_addr learned my addr 172.21.15.16:0/1220716440 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.274+0000 7f20c5125640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c006c3b0 0x7f20c015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20c5926640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c015cf90 0x7f20c015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20c4924640 1 -- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c015cf90 msgr2=0x7f20c015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20c4924640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c015cf90 0x7f20c015f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20c4924640 1 -- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c006c3b0 msgr2=0x7f20c015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20c4924640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c006c3b0 0x7f20c015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20c4924640 1 -- 172.21.15.16:0/1220716440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00094d0 con 0x7f20c00b39c0 2023-11-13T10:56:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20c4924640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c00b39c0 0x7f20c015d8b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009870 tx=0x7f20b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20b5ffb640 1 -- 172.21.15.16:0/1220716440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0039c90 con 0x7f20c00b39c0 2023-11-13T10:56:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20b5ffb640 1 -- 172.21.15.16:0/1220716440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b0039df0 con 0x7f20c00b39c0 2023-11-13T10:56:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.275+0000 7f20b5ffb640 1 -- 172.21.15.16:0/1220716440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0043640 con 0x7f20c00b39c0 2023-11-13T10:56:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.276+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c00cc380 con 0x7f20c00b39c0 2023-11-13T10:56:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.276+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c015f7c0 con 0x7f20c00b39c0 2023-11-13T10:56:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.277+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c00cc8a0 con 0x7f20c00b39c0 2023-11-13T10:56:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.278+0000 7f20b5ffb640 1 -- 172.21.15.16:0/1220716440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20b00437a0 con 0x7f20c00b39c0 2023-11-13T10:56:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.279+0000 7f20b5ffb640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20a406ae70 0x7f20a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.279+0000 7f20b5ffb640 1 -- 172.21.15.16:0/1220716440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f20b003e070 con 0x7f20c00b39c0 2023-11-13T10:56:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.280+0000 7f20c5125640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20a406ae70 0x7f20a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.281+0000 7f20c5125640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20a406ae70 0x7f20a406d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009a30 tx=0x7f20bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.284+0000 7f20b5ffb640 1 -- 172.21.15.16:0/1220716440 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20c00cc8a0 con 0x7f20c00b39c0 2023-11-13T10:56:28.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.477+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20c00cc8a0 con 0x7f20c00b39c0 2023-11-13T10:56:28.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.478+0000 7f20b5ffb640 1 -- 172.21.15.16:0/1220716440 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20c00cc8a0 con 0x7f20c00b39c0 2023-11-13T10:56:28.484 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20a406ae70 msgr2=0x7f20a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20a406ae70 0x7f20a406d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009a30 tx=0x7f20bc008040 comp rx=0 tx=0).stop 2023-11-13T10:56:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c00b39c0 msgr2=0x7f20c015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c00b39c0 0x7f20c015d8b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009870 tx=0x7f20b0004730 comp rx=0 tx=0).stop 2023-11-13T10:56:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 shutdown_connections 2023-11-13T10:56:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20a406ae70 0x7f20a406d330 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20c015cf90 0x7f20c015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c00b39c0 0x7f20c015d8b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 --2- 172.21.15.16:0/1220716440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c006c3b0 0x7f20c015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.481+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 >> 172.21.15.16:0/1220716440 conn(0x7f20c006a190 msgr2=0x7f20c015c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.482+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 shutdown_connections 2023-11-13T10:56:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.482+0000 7f20c7389640 1 -- 172.21.15.16:0/1220716440 wait complete. 2023-11-13T10:56:28.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.732+0000 7faa2d5a0640 1 Processor -- start 2023-11-13T10:56:28.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.733+0000 7faa2d5a0640 1 -- start start 2023-11-13T10:56:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2d5a0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faa2815bcc0 0x7faa280cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2d5a0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa280ce090 0x7faa280cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2d5a0640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2806d010 con 0x7faa2815b8e0 2023-11-13T10:56:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2d5a0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa280ce540 con 0x7faa280ce090 2023-11-13T10:56:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2d5a0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2806b860 con 0x7faa2815bcc0 2023-11-13T10:56:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2cd9f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa280ce090 0x7faa280cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2cd9f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa280ce090 0x7faa280cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58182/0 (socket says 172.21.15.16:58182) 2023-11-13T10:56:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2cd9f640 1 -- 172.21.15.16:0/262489041 learned_addr learned my addr 172.21.15.16:0/262489041 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa1ffff640 1 --2- 172.21.15.16:0/262489041 >> v2:172.21.15.184:3300/0 conn(0x7faa2815bcc0 0x7faa280cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2cd9f640 1 -- 172.21.15.16:0/262489041 >> v2:172.21.15.184:3300/0 conn(0x7faa2815bcc0 msgr2=0x7faa280cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.735+0000 7faa2cd9f640 1 --2- 172.21.15.16:0/262489041 >> v2:172.21.15.184:3300/0 conn(0x7faa2815bcc0 0x7faa280cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.736+0000 7faa2cd9f640 1 -- 172.21.15.16:0/262489041 >> v1:172.21.15.184:6789/0 conn(0x7faa2815b8e0 legacy=0x7faa2806c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.736+0000 7faa2cd9f640 1 -- 172.21.15.16:0/262489041 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2806bb40 con 0x7faa280ce090 2023-11-13T10:56:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.736+0000 7faa1ffff640 1 --2- 172.21.15.16:0/262489041 >> v2:172.21.15.184:3300/0 conn(0x7faa2815bcc0 0x7faa280cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:28.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.737+0000 7faa2cd9f640 1 --2- 172.21.15.16:0/262489041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa280ce090 0x7faa280cbae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa28071d20 tx=0x7faa2002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860b17986c4a309f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.737+0000 7faa277fe640 1 -- 172.21.15.16:0/262489041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2002ca80 con 0x7faa280ce090 2023-11-13T10:56:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.737+0000 7faa277fe640 1 -- 172.21.15.16:0/262489041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2002cbe0 con 0x7faa280ce090 2023-11-13T10:56:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.737+0000 7faa277fe640 1 -- 172.21.15.16:0/262489041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa200343f0 con 0x7faa280ce090 2023-11-13T10:56:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.738+0000 7faa2d5a0640 1 -- 172.21.15.16:0/262489041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa280ce090 msgr2=0x7faa280cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.738+0000 7faa2d5a0640 1 --2- 172.21.15.16:0/262489041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa280ce090 0x7faa280cbae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa28071d20 tx=0x7faa2002bee0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.738+0000 7faa2d5a0640 1 -- 172.21.15.16:0/262489041 shutdown_connections 2023-11-13T10:56:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.738+0000 7faa2d5a0640 1 --2- 172.21.15.16:0/262489041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa280ce090 0x7faa280cbae0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.738+0000 7faa2d5a0640 1 --2- 172.21.15.16:0/262489041 >> v2:172.21.15.184:3300/0 conn(0x7faa2815bcc0 0x7faa280cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.738+0000 7faa2d5a0640 1 -- 172.21.15.16:0/262489041 >> 172.21.15.16:0/262489041 conn(0x7faa280b0970 msgr2=0x7faa280b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.738+0000 7faa2d5a0640 1 -- 172.21.15.16:0/262489041 shutdown_connections 2023-11-13T10:56:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.738+0000 7faa2d5a0640 1 -- 172.21.15.16:0/262489041 wait complete. 2023-11-13T10:56:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.740+0000 7faa2d5a0640 1 Processor -- start 2023-11-13T10:56:28.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.740+0000 7faa2d5a0640 1 -- start start 2023-11-13T10:56:28.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.741+0000 7faa2d5a0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa280ce090 0x7faa28148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.741+0000 7faa2d5a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2815b8e0 0x7faa280aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.741+0000 7faa2d5a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa280af330 0x7faa280b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.741+0000 7faa2d5a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa281e4390 con 0x7faa2815b8e0 2023-11-13T10:56:28.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.741+0000 7faa2d5a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28148c70 con 0x7faa280af330 2023-11-13T10:56:28.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.741+0000 7faa2d5a0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa280af870 con 0x7faa280ce090 2023-11-13T10:56:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.741+0000 7faa1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2815b8e0 0x7faa280aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.741+0000 7faa1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2815b8e0 0x7faa280aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58186/0 (socket says 172.21.15.16:58186) 2023-11-13T10:56:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.741+0000 7faa1ffff640 1 -- 172.21.15.16:0/1330245178 learned_addr learned my addr 172.21.15.16:0/1330245178 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa1ffff640 1 -- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa280ce090 msgr2=0x7faa28148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa1ffff640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa280ce090 0x7faa28148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa1ffff640 1 -- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa280af330 msgr2=0x7faa280b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa1ffff640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa280af330 0x7faa280b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa1ffff640 1 -- 172.21.15.16:0/1330245178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa20009b30 con 0x7faa2815b8e0 2023-11-13T10:56:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa1ffff640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2815b8e0 0x7faa280aedf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c7f0 tx=0x7faa1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa257fa640 1 -- 172.21.15.16:0/1330245178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10007820 con 0x7faa2815b8e0 2023-11-13T10:56:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa257fa640 1 -- 172.21.15.16:0/1330245178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa10007980 con 0x7faa2815b8e0 2023-11-13T10:56:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa257fa640 1 -- 172.21.15.16:0/1330245178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10007cf0 con 0x7faa2815b8e0 2023-11-13T10:56:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa280afb50 con 0x7faa2815b8e0 2023-11-13T10:56:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.742+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa281e48c0 con 0x7faa2815b8e0 2023-11-13T10:56:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.744+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa280afd90 con 0x7faa2815b8e0 2023-11-13T10:56:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.745+0000 7faa257fa640 1 -- 172.21.15.16:0/1330245178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa100113f0 con 0x7faa2815b8e0 2023-11-13T10:56:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.745+0000 7faa257fa640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faa0806ae40 0x7faa0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.746+0000 7faa257fa640 1 -- 172.21.15.16:0/1330245178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7faa10089d00 con 0x7faa2815b8e0 2023-11-13T10:56:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.746+0000 7faa27fff640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faa0806ae40 0x7faa0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.747+0000 7faa27fff640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faa0806ae40 0x7faa0806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faa18003cc0 tx=0x7faa180079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.750+0000 7faa257fa640 1 -- 172.21.15.16:0/1330245178 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa100549d0 con 0x7faa2815b8e0 2023-11-13T10:56:28.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.941+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa2804e4d0 con 0x7faa2815b8e0 2023-11-13T10:56:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.942+0000 7faa257fa640 1 -- 172.21.15.16:0/1330245178 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa1001f0a0 con 0x7faa2815b8e0 2023-11-13T10:56:28.951 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:28.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faa0806ae40 msgr2=0x7faa0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faa0806ae40 0x7faa0806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faa18003cc0 tx=0x7faa180079b0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2815b8e0 msgr2=0x7faa280aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2815b8e0 0x7faa280aedf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c7f0 tx=0x7faa1000ccb0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 shutdown_connections 2023-11-13T10:56:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faa0806ae40 0x7faa0806d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faa280af330 0x7faa280b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2815b8e0 0x7faa280aedf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 --2- 172.21.15.16:0/1330245178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faa280ce090 0x7faa28148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 >> 172.21.15.16:0/1330245178 conn(0x7faa280b0970 msgr2=0x7faa280caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 shutdown_connections 2023-11-13T10:56:28.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:28.944+0000 7faa2d5a0640 1 -- 172.21.15.16:0/1330245178 wait complete. 2023-11-13T10:56:28.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.202+0000 7f9e299b6640 1 Processor -- start 2023-11-13T10:56:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.202+0000 7f9e299b6640 1 -- start start 2023-11-13T10:56:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.204+0000 7f9e299b6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9e24144570 0x7f9e2415f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.204+0000 7f9e299b6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e240ce550 0x7f9e240ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.204+0000 7f9e299b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2406bb30 0x7f9e2406bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.204+0000 7f9e299b6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240cef20 con 0x7f9e240ce550 2023-11-13T10:56:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.204+0000 7f9e299b6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e24144950 con 0x7f9e24144570 2023-11-13T10:56:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.204+0000 7f9e299b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2415f740 con 0x7f9e2406bb30 2023-11-13T10:56:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.204+0000 7f9e227fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e240ce550 0x7f9e240ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.205+0000 7f9e227fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e240ce550 0x7f9e240ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58192/0 (socket says 172.21.15.16:58192) 2023-11-13T10:56:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.205+0000 7f9e227fc640 1 -- 172.21.15.16:0/667455916 learned_addr learned my addr 172.21.15.16:0/667455916 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.205+0000 7f9e22ffd640 1 --2- 172.21.15.16:0/667455916 >> v2:172.21.15.184:3300/0 conn(0x7f9e24144570 0x7f9e2415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.205+0000 7f9e227fc640 1 -- 172.21.15.16:0/667455916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2406bb30 msgr2=0x7f9e2406bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.205+0000 7f9e227fc640 1 --2- 172.21.15.16:0/667455916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2406bb30 0x7f9e2406bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.205+0000 7f9e227fc640 1 -- 172.21.15.16:0/667455916 >> v2:172.21.15.184:3300/0 conn(0x7f9e24144570 msgr2=0x7f9e2415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.205+0000 7f9e227fc640 1 --2- 172.21.15.16:0/667455916 >> v2:172.21.15.184:3300/0 conn(0x7f9e24144570 0x7f9e2415f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.205+0000 7f9e227fc640 1 -- 172.21.15.16:0/667455916 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240ce300 con 0x7f9e240ce550 2023-11-13T10:56:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.205+0000 7f9e22ffd640 1 --2- 172.21.15.16:0/667455916 >> v2:172.21.15.184:3300/0 conn(0x7f9e24144570 0x7f9e2415f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.206+0000 7f9e227fc640 1 --2- 172.21.15.16:0/667455916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e240ce550 0x7f9e240ce9e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e240cf5e0 tx=0x7f9e1802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e32794e6ed24bf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:29.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.206+0000 7f9e21ffb640 1 -- 172.21.15.16:0/667455916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18031880 con 0x7f9e240ce550 2023-11-13T10:56:29.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.206+0000 7f9e21ffb640 1 -- 172.21.15.16:0/667455916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e180319e0 con 0x7f9e240ce550 2023-11-13T10:56:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e21ffb640 1 -- 172.21.15.16:0/667455916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18031d70 con 0x7f9e240ce550 2023-11-13T10:56:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e299b6640 1 -- 172.21.15.16:0/667455916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e240ce550 msgr2=0x7f9e240ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e299b6640 1 --2- 172.21.15.16:0/667455916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e240ce550 0x7f9e240ce9e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e240cf5e0 tx=0x7f9e1802b920 comp rx=0 tx=0).stop 2023-11-13T10:56:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e299b6640 1 -- 172.21.15.16:0/667455916 shutdown_connections 2023-11-13T10:56:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e299b6640 1 --2- 172.21.15.16:0/667455916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e2406bb30 0x7f9e2406bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e299b6640 1 --2- 172.21.15.16:0/667455916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e240ce550 0x7f9e240ce9e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e299b6640 1 --2- 172.21.15.16:0/667455916 >> v2:172.21.15.184:3300/0 conn(0x7f9e24144570 0x7f9e2415f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e299b6640 1 -- 172.21.15.16:0/667455916 >> 172.21.15.16:0/667455916 conn(0x7f9e2406a190 msgr2=0x7f9e2406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:29.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e299b6640 1 -- 172.21.15.16:0/667455916 shutdown_connections 2023-11-13T10:56:29.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.207+0000 7f9e299b6640 1 -- 172.21.15.16:0/667455916 wait complete. 2023-11-13T10:56:29.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.209+0000 7f9e299b6640 1 Processor -- start 2023-11-13T10:56:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.209+0000 7f9e299b6640 1 -- start start 2023-11-13T10:56:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.210+0000 7f9e299b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e2406bb30 0x7f9e2415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.210+0000 7f9e299b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e240ce550 0x7f9e2415cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.210+0000 7f9e299b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e24144570 0x7f9e241f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.210+0000 7f9e299b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241e4e00 con 0x7f9e2406bb30 2023-11-13T10:56:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.210+0000 7f9e299b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241e50e0 con 0x7f9e240ce550 2023-11-13T10:56:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.210+0000 7f9e299b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241e53c0 con 0x7f9e24144570 2023-11-13T10:56:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.210+0000 7f9e22ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e2406bb30 0x7f9e2415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.210+0000 7f9e22ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e2406bb30 0x7f9e2415c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58200/0 (socket says 172.21.15.16:58200) 2023-11-13T10:56:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.210+0000 7f9e22ffd640 1 -- 172.21.15.16:0/2037855453 learned_addr learned my addr 172.21.15.16:0/2037855453 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.211+0000 7f9e227fc640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e240ce550 0x7f9e2415cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.211+0000 7f9e237fe640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e24144570 0x7f9e241f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.211+0000 7f9e22ffd640 1 -- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e24144570 msgr2=0x7f9e241f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.211+0000 7f9e22ffd640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e24144570 0x7f9e241f2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.211+0000 7f9e22ffd640 1 -- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e240ce550 msgr2=0x7f9e2415cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.211+0000 7f9e22ffd640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e240ce550 0x7f9e2415cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.211+0000 7f9e22ffd640 1 -- 172.21.15.16:0/2037855453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180094d0 con 0x7f9e2406bb30 2023-11-13T10:56:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.211+0000 7f9e22ffd640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e2406bb30 0x7f9e2415c7f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400c7f0 tx=0x7f9e1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.211+0000 7f9e0bfff640 1 -- 172.21.15.16:0/2037855453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14007a30 con 0x7f9e2406bb30 2023-11-13T10:56:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.212+0000 7f9e0bfff640 1 -- 172.21.15.16:0/2037855453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e14007b90 con 0x7f9e2406bb30 2023-11-13T10:56:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.212+0000 7f9e0bfff640 1 -- 172.21.15.16:0/2037855453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400ce40 con 0x7f9e2406bb30 2023-11-13T10:56:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.212+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e241e56a0 con 0x7f9e2406bb30 2023-11-13T10:56:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.212+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2415cd30 con 0x7f9e2406bb30 2023-11-13T10:56:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.213+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e241e5bc0 con 0x7f9e2406bb30 2023-11-13T10:56:29.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.214+0000 7f9e0bfff640 1 -- 172.21.15.16:0/2037855453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e140113f0 con 0x7f9e2406bb30 2023-11-13T10:56:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.215+0000 7f9e0bfff640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e1006ae40 0x7f9e1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.215+0000 7f9e0bfff640 1 -- 172.21.15.16:0/2037855453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f9e14089b50 con 0x7f9e2406bb30 2023-11-13T10:56:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.216+0000 7f9e227fc640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e1006ae40 0x7f9e1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.217+0000 7f9e227fc640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e1006ae40 0x7f9e1006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9e18036000 tx=0x7f9e1803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.219+0000 7f9e0bfff640 1 -- 172.21.15.16:0/2037855453 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e241e5bc0 con 0x7f9e2406bb30 2023-11-13T10:56:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.414+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9e241e5bc0 con 0x7f9e2406bb30 2023-11-13T10:56:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.415+0000 7f9e0bfff640 1 -- 172.21.15.16:0/2037855453 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9e241e5bc0 con 0x7f9e2406bb30 2023-11-13T10:56:29.420 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e1006ae40 msgr2=0x7f9e1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e1006ae40 0x7f9e1006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9e18036000 tx=0x7f9e1803c040 comp rx=0 tx=0).stop 2023-11-13T10:56:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e2406bb30 msgr2=0x7f9e2415c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e2406bb30 0x7f9e2415c7f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400c7f0 tx=0x7f9e1400cbf0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 shutdown_connections 2023-11-13T10:56:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e1006ae40 0x7f9e1006d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e24144570 0x7f9e241f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e240ce550 0x7f9e2415cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 --2- 172.21.15.16:0/2037855453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e2406bb30 0x7f9e2415c7f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 >> 172.21.15.16:0/2037855453 conn(0x7f9e2406a190 msgr2=0x7f9e240cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 shutdown_connections 2023-11-13T10:56:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.418+0000 7f9e299b6640 1 -- 172.21.15.16:0/2037855453 wait complete. 2023-11-13T10:56:29.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:29.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.685+0000 7ff6b6154640 1 Processor -- start 2023-11-13T10:56:29.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.685+0000 7ff6b6154640 1 -- start start 2023-11-13T10:56:29.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.687+0000 7ff6b6154640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff6b00b3ea0 0x7ff6b00b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.687+0000 7ff6b6154640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b006d0d0 con 0x7ff6b00b3040 2023-11-13T10:56:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.687+0000 7ff6b6154640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b00cc630 con 0x7ff6b00b2720 2023-11-13T10:56:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.687+0000 7ff6b6154640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b00ace30 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.687+0000 7ff6b4951640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff6b00b3ea0 0x7ff6b00b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.687+0000 7ff6b5953640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff6b00b3040 0x7ff6b01646d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49344/0 (socket says 172.21.15.16:49344) 2023-11-13T10:56:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.687+0000 7ff6b5953640 1 -- 172.21.15.16:0/3901085873 learned_addr learned my addr 172.21.15.16:0/3901085873 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.688+0000 7ff6b4951640 1 -- 172.21.15.16:0/3901085873 >> v1:172.21.15.184:6789/0 conn(0x7ff6b00b2720 legacy=0x7ff6b00ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.688+0000 7ff6b4951640 1 -- 172.21.15.16:0/3901085873 >> v1:172.21.15.16:6789/0 conn(0x7ff6b00b3040 legacy=0x7ff6b01646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.688+0000 7ff6b4951640 1 -- 172.21.15.16:0/3901085873 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b006c520 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.688+0000 7ff6a7fff640 1 -- 172.21.15.16:0/3901085873 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381925212 0 0) 0x7ff6b006d0d0 con 0x7ff6b00b3040 2023-11-13T10:56:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.689+0000 7ff6b4951640 1 --2- 172.21.15.16:0/3901085873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 0x7ff6b00b2b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00097a0 tx=0x7ff6a002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4ba30d604ec4cf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.689+0000 7ff6a7fff640 1 -- 172.21.15.16:0/3901085873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0030610 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.689+0000 7ff6a7fff640 1 -- 172.21.15.16:0/3901085873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a0030770 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.689+0000 7ff6a7fff640 1 -- 172.21.15.16:0/3901085873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0030b00 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.690+0000 7ff6b6154640 1 -- 172.21.15.16:0/3901085873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 msgr2=0x7ff6b00b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.690+0000 7ff6b6154640 1 --2- 172.21.15.16:0/3901085873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 0x7ff6b00b2b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00097a0 tx=0x7ff6a002b960 comp rx=0 tx=0).stop 2023-11-13T10:56:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.690+0000 7ff6b6154640 1 -- 172.21.15.16:0/3901085873 shutdown_connections 2023-11-13T10:56:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.690+0000 7ff6b6154640 1 --2- 172.21.15.16:0/3901085873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 0x7ff6b00b2b00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.690+0000 7ff6b6154640 1 -- 172.21.15.16:0/3901085873 >> 172.21.15.16:0/3901085873 conn(0x7ff6b006a190 msgr2=0x7ff6b006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.690+0000 7ff6b6154640 1 -- 172.21.15.16:0/3901085873 shutdown_connections 2023-11-13T10:56:29.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.690+0000 7ff6b6154640 1 -- 172.21.15.16:0/3901085873 wait complete. 2023-11-13T10:56:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.692+0000 7ff6b6154640 1 Processor -- start 2023-11-13T10:56:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.692+0000 7ff6b6154640 1 -- start start 2023-11-13T10:56:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.693+0000 7ff6b6154640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6b00b2720 0x7ff6b0154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.693+0000 7ff6b6154640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b00b3040 0x7ff6b014d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.693+0000 7ff6b6154640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 0x7ff6b0159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.693+0000 7ff6b6154640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b014daa0 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.693+0000 7ff6b6154640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0159d60 con 0x7ff6b00b3040 2023-11-13T10:56:29.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.693+0000 7ff6b6154640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0070010 con 0x7ff6b00b2720 2023-11-13T10:56:29.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5953640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 0x7ff6b0159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5953640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 0x7ff6b0159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58216/0 (socket says 172.21.15.16:58216) 2023-11-13T10:56:29.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5953640 1 -- 172.21.15.16:0/3676151649 learned_addr learned my addr 172.21.15.16:0/3676151649 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:29.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b4951640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b00b3040 0x7ff6b014d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5152640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6b00b2720 0x7ff6b0154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5953640 1 -- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6b00b2720 msgr2=0x7ff6b0154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5953640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6b00b2720 0x7ff6b0154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5953640 1 -- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b00b3040 msgr2=0x7ff6b014d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5953640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b00b3040 0x7ff6b014d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5953640 1 -- 172.21.15.16:0/3676151649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0009400 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.694+0000 7ff6b5953640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 0x7ff6b0159820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800c7a0 tx=0x7ff6a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.695+0000 7ff6a5ffb640 1 -- 172.21.15.16:0/3676151649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800a640 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.695+0000 7ff6a5ffb640 1 -- 172.21.15.16:0/3676151649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a800a7a0 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.695+0000 7ff6a5ffb640 1 -- 172.21.15.16:0/3676151649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800abd0 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.695+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b0159f90 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.695+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b015a550 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.696+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b00675b0 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.697+0000 7ff6a5ffb640 1 -- 172.21.15.16:0/3676151649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff6a800ad30 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.699+0000 7ff6a5ffb640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff69406ae40 0x7ff69406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.699+0000 7ff6a5ffb640 1 -- 172.21.15.16:0/3676151649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7ff6a8088ff0 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.700+0000 7ff6b5152640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff69406ae40 0x7ff69406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:29.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.701+0000 7ff6b5152640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff69406ae40 0x7ff69406d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac000f60 tx=0x7ff6ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:29.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.703+0000 7ff6a5ffb640 1 -- 172.21.15.16:0/3676151649 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6a80bbe90 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.901+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6b015a8f0 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.904+0000 7ff6a5ffb640 1 -- 172.21.15.16:0/3676151649 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7ff6a8018030 con 0x7ff6b00b3ea0 2023-11-13T10:56:29.910 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:29.910 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff69406ae40 msgr2=0x7ff69406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff69406ae40 0x7ff69406d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac000f60 tx=0x7ff6ac009040 comp rx=0 tx=0).stop 2023-11-13T10:56:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 msgr2=0x7ff6b0159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 0x7ff6b0159820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800c7a0 tx=0x7ff6a800cc60 comp rx=0 tx=0).stop 2023-11-13T10:56:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 shutdown_connections 2023-11-13T10:56:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff69406ae40 0x7ff69406d300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b00b3ea0 0x7ff6b0159820 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b00b3040 0x7ff6b014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 --2- 172.21.15.16:0/3676151649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6b00b2720 0x7ff6b0154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.907+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 >> 172.21.15.16:0/3676151649 conn(0x7ff6b006a190 msgr2=0x7ff6b006a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.908+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 shutdown_connections 2023-11-13T10:56:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:29.908+0000 7ff6b6154640 1 -- 172.21.15.16:0/3676151649 wait complete. 2023-11-13T10:56:29.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:29.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:30.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.185+0000 7f63c6f75640 1 Processor -- start 2023-11-13T10:56:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.185+0000 7f63c6f75640 1 -- start start 2023-11-13T10:56:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.186+0000 7f63c6f75640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f63c00cb710 0x7f63c015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.186+0000 7f63c6f75640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63c00cc200 0x7f63c006bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.186+0000 7f63c6f75640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c00ce140 con 0x7f63c00cd090 2023-11-13T10:56:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.186+0000 7f63c6f75640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c0072630 con 0x7f63c00cb710 2023-11-13T10:56:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.186+0000 7f63c6f75640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00cd530 con 0x7f63c00cc200 2023-11-13T10:56:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.186+0000 7f63c5772640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f63c00cd090 0x7f63c00cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49354/0 (socket says 172.21.15.16:49354) 2023-11-13T10:56:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.186+0000 7f63c5772640 1 -- 172.21.15.16:0/1379192590 learned_addr learned my addr 172.21.15.16:0/1379192590 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c5f73640 1 --2- 172.21.15.16:0/1379192590 >> v2:172.21.15.184:3300/0 conn(0x7f63c00cb710 0x7f63c015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c6774640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63c00cc200 0x7f63c006bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045694431 0 0) 0x7f63b0001110 con 0x7f63c00cd090 2023-11-13T10:56:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63ac003520 con 0x7f63c00cd090 2023-11-13T10:56:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 983784404 0 0) 0x7f63ac003520 con 0x7f63c00cd090 2023-11-13T10:56:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63c00cc200 msgr2=0x7f63c006bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c4f71640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63c00cc200 0x7f63c006bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 >> v2:172.21.15.184:3300/0 conn(0x7f63c00cb710 msgr2=0x7f63c015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c4f71640 1 --2- 172.21.15.16:0/1379192590 >> v2:172.21.15.184:3300/0 conn(0x7f63c00cb710 0x7f63c015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c00cd810 con 0x7f63c00cd090 2023-11-13T10:56:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.187+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f63b0001800 con 0x7f63c00cd090 2023-11-13T10:56:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 >> v1:172.21.15.16:6789/0 conn(0x7f63c00cd090 legacy=0x7f63c00cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c4f71640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63ac003970 0x7f63ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c4f71640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac0047c0 0x7f63ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c4f71640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac005180 0x7f63ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c0072630 con 0x7f63ac003970 2023-11-13T10:56:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00cd530 con 0x7f63ac0047c0 2023-11-13T10:56:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac004590 con 0x7f63ac005180 2023-11-13T10:56:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c5f73640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63ac003970 0x7f63ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c5772640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac0047c0 0x7f63ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c5f73640 1 -- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac005180 msgr2=0x7f63ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c6774640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac005180 0x7f63ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c5f73640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac005180 0x7f63ac00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c5f73640 1 -- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac0047c0 msgr2=0x7f63ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c5f73640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac0047c0 0x7f63ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c5f73640 1 -- 172.21.15.16:0/1379192590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac003520 con 0x7f63ac003970 2023-11-13T10:56:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.188+0000 7f63c5772640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63ac0047c0 0x7f63ac004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.189+0000 7f63c5f73640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63ac003970 0x7f63ac003d90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009fc0 tx=0x7f63ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6f33ed15ba589b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.190+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc02cc60 con 0x7f63ac003970 2023-11-13T10:56:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.190+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc00f040 con 0x7f63ac003970 2023-11-13T10:56:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.190+0000 7f63c4f71640 1 -- 172.21.15.16:0/1379192590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc034440 con 0x7f63ac003970 2023-11-13T10:56:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.190+0000 7f63c6f75640 1 -- 172.21.15.16:0/1379192590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63ac003970 msgr2=0x7f63ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.190+0000 7f63c6f75640 1 --2- 172.21.15.16:0/1379192590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63ac003970 0x7f63ac003d90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009fc0 tx=0x7f63ac0057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.190+0000 7f63c6774640 1 -- 172.21.15.16:0/1379192590 reap_dead start 2023-11-13T10:56:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.190+0000 7f63c6f75640 1 -- 172.21.15.16:0/1379192590 shutdown_connections 2023-11-13T10:56:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.190+0000 7f63c6f75640 1 -- 172.21.15.16:0/1379192590 >> 172.21.15.16:0/1379192590 conn(0x7f63c00d08e0 msgr2=0x7f63c00d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.191+0000 7f63c6f75640 1 -- 172.21.15.16:0/1379192590 shutdown_connections 2023-11-13T10:56:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.191+0000 7f63c6f75640 1 -- 172.21.15.16:0/1379192590 wait complete. 2023-11-13T10:56:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.192+0000 7f63c6f75640 1 Processor -- start 2023-11-13T10:56:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.193+0000 7f63c6f75640 1 -- start start 2023-11-13T10:56:30.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.193+0000 7f63c6f75640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac003970 0x7f63c00b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.193+0000 7f63c6f75640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c00b3430 0x7f63c00b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.193+0000 7f63c6f75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c00ca130 0x7f63c00ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.193+0000 7f63c6f75640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00b3df0 con 0x7f63c00ca130 2023-11-13T10:56:30.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.193+0000 7f63c6f75640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00caaf0 con 0x7f63c00b3430 2023-11-13T10:56:30.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.193+0000 7f63c6f75640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00b62d0 con 0x7f63ac003970 2023-11-13T10:56:30.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.194+0000 7f63c6774640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c00ca130 0x7f63c00ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.194+0000 7f63c6774640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c00ca130 0x7f63c00ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58236/0 (socket says 172.21.15.16:58236) 2023-11-13T10:56:30.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.194+0000 7f63c6774640 1 -- 172.21.15.16:0/3158590051 learned_addr learned my addr 172.21.15.16:0/3158590051 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:30.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.194+0000 7f63c5f73640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac003970 0x7f63c00b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.194+0000 7f63c6774640 1 -- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac003970 msgr2=0x7f63c00b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.194+0000 7f63c6774640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac003970 0x7f63c00b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.194+0000 7f63c6774640 1 -- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c00b3430 msgr2=0x7f63c00b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.194+0000 7f63c6774640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c00b3430 0x7f63c00b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.194+0000 7f63c6774640 1 -- 172.21.15.16:0/3158590051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c00c5430 con 0x7f63c00ca130 2023-11-13T10:56:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.195+0000 7f63c6774640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c00ca130 0x7f63c00ca5b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f63c0073a50 tx=0x7f63b8010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.195+0000 7f63b67fc640 1 -- 172.21.15.16:0/3158590051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b800e8a0 con 0x7f63c00ca130 2023-11-13T10:56:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.195+0000 7f63b67fc640 1 -- 172.21.15.16:0/3158590051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b800ea00 con 0x7f63c00ca130 2023-11-13T10:56:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.195+0000 7f63b67fc640 1 -- 172.21.15.16:0/3158590051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b800edb0 con 0x7f63c00ca130 2023-11-13T10:56:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.195+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c00c5710 con 0x7f63c00ca130 2023-11-13T10:56:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.195+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c00c5cd0 con 0x7f63c00ca130 2023-11-13T10:56:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.196+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c004e4d0 con 0x7f63c00ca130 2023-11-13T10:56:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.197+0000 7f63b67fc640 1 -- 172.21.15.16:0/3158590051 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63b8020410 con 0x7f63c00ca130 2023-11-13T10:56:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.198+0000 7f63b67fc640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f63a406ae70 0x7f63a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.198+0000 7f63b67fc640 1 -- 172.21.15.16:0/3158590051 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f63b8089250 con 0x7f63c00ca130 2023-11-13T10:56:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.199+0000 7f63c5f73640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f63a406ae70 0x7f63a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.200+0000 7f63c5f73640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f63a406ae70 0x7f63a406d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63bc036000 tx=0x7f63bc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.202+0000 7f63b67fc640 1 -- 172.21.15.16:0/3158590051 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63b8053f20 con 0x7f63c00ca130 2023-11-13T10:56:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.395+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63c0071c50 con 0x7f63c00ca130 2023-11-13T10:56:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.396+0000 7f63b67fc640 1 -- 172.21.15.16:0/3158590051 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63b8057540 con 0x7f63c00ca130 2023-11-13T10:56:30.401 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:30.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f63a406ae70 msgr2=0x7f63a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f63a406ae70 0x7f63a406d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63bc036000 tx=0x7f63bc02e000 comp rx=0 tx=0).stop 2023-11-13T10:56:30.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c00ca130 msgr2=0x7f63c00ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c00ca130 0x7f63c00ca5b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f63c0073a50 tx=0x7f63b8010c20 comp rx=0 tx=0).stop 2023-11-13T10:56:30.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 shutdown_connections 2023-11-13T10:56:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f63a406ae70 0x7f63a406d330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c00ca130 0x7f63c00ca5b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c00b3430 0x7f63c00b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 --2- 172.21.15.16:0/3158590051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f63ac003970 0x7f63c00b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 >> 172.21.15.16:0/3158590051 conn(0x7f63c00d08e0 msgr2=0x7f63c006bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:30.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 shutdown_connections 2023-11-13T10:56:30.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.399+0000 7f63c6f75640 1 -- 172.21.15.16:0/3158590051 wait complete. 2023-11-13T10:56:30.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.644+0000 7f51ffef7640 1 Processor -- start 2023-11-13T10:56:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.644+0000 7f51ffef7640 1 -- start start 2023-11-13T10:56:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.645+0000 7f51ffef7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f8144570 0x7f51f815f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.645+0000 7f51ffef7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f8144950 con 0x7f51f80ce550 2023-11-13T10:56:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.645+0000 7f51ffef7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f806c950 con 0x7f51f80ce9b0 2023-11-13T10:56:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.645+0000 7f51ffef7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f815f740 con 0x7f51f8144570 2023-11-13T10:56:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.646+0000 7f51fd492640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f51f80ce550 0x7f51f806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49364/0 (socket says 172.21.15.16:49364) 2023-11-13T10:56:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.646+0000 7f51fd492640 1 -- 172.21.15.16:0/4018228907 learned_addr learned my addr 172.21.15.16:0/4018228907 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.646+0000 7f51fdc93640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f8144570 0x7f51f815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.646+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331636439 0 0) 0x7f51f8144950 con 0x7f51f80ce550 2023-11-13T10:56:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.646+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e4003520 con 0x7f51f80ce550 2023-11-13T10:56:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.646+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3108399983 0 0) 0x7f51e4003520 con 0x7f51f80ce550 2023-11-13T10:56:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.646+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f8144570 msgr2=0x7f51f815f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f8144570 0x7f51f815f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 >> v1:172.21.15.184:6789/0 conn(0x7f51f80ce9b0 legacy=0x7f51f806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f80ce300 con 0x7f51f80ce550 2023-11-13T10:56:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f51e8001670 con 0x7f51f80ce550 2023-11-13T10:56:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 >> v1:172.21.15.16:6789/0 conn(0x7f51f80ce550 legacy=0x7f51f806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e40039a0 0x7f51e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f8144570 0x7f51e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e4004d00 0x7f51e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f815f740 con 0x7f51e40039a0 2023-11-13T10:56:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e400ba30 con 0x7f51e4004d00 2023-11-13T10:56:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e400bbf0 con 0x7f51f8144570 2023-11-13T10:56:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fdc93640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e40039a0 0x7f51e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fe494640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e4004d00 0x7f51e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fd492640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f8144570 0x7f51e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fdc93640 1 -- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f8144570 msgr2=0x7f51e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.647+0000 7f51fdc93640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51f8144570 0x7f51e40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.648+0000 7f51fdc93640 1 -- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e4004d00 msgr2=0x7f51e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.648+0000 7f51fdc93640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e4004d00 0x7f51e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.648+0000 7f51fdc93640 1 -- 172.21.15.16:0/4018228907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4003520 con 0x7f51e40039a0 2023-11-13T10:56:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.648+0000 7f51fe494640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e4004d00 0x7f51e4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.649+0000 7f51fdc93640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e40039a0 0x7f51e4003dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009f90 tx=0x7f51e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac3d7a093f4250f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.649+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f402cbd0 con 0x7f51e40039a0 2023-11-13T10:56:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.649+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f400f040 con 0x7f51e40039a0 2023-11-13T10:56:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.649+0000 7f51fcc91640 1 -- 172.21.15.16:0/4018228907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4034440 con 0x7f51e40039a0 2023-11-13T10:56:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.650+0000 7f51ffef7640 1 -- 172.21.15.16:0/4018228907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e40039a0 msgr2=0x7f51e4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.650+0000 7f51ffef7640 1 --2- 172.21.15.16:0/4018228907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e40039a0 0x7f51e4003dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009f90 tx=0x7f51e40057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.650+0000 7f51fe494640 1 -- 172.21.15.16:0/4018228907 reap_dead start 2023-11-13T10:56:30.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.650+0000 7f51ffef7640 1 -- 172.21.15.16:0/4018228907 shutdown_connections 2023-11-13T10:56:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.650+0000 7f51ffef7640 1 -- 172.21.15.16:0/4018228907 >> 172.21.15.16:0/4018228907 conn(0x7f51f806a190 msgr2=0x7f51f806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.650+0000 7f51ffef7640 1 -- 172.21.15.16:0/4018228907 shutdown_connections 2023-11-13T10:56:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.650+0000 7f51ffef7640 1 -- 172.21.15.16:0/4018228907 wait complete. 2023-11-13T10:56:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.652+0000 7f51ffef7640 1 Processor -- start 2023-11-13T10:56:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.652+0000 7f51ffef7640 1 -- start start 2023-11-13T10:56:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.653+0000 7f51ffef7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40039a0 0x7f51f815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.653+0000 7f51ffef7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51f8139ca0 0x7f51f8070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.653+0000 7f51ffef7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813a120 0x7f51f8071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.653+0000 7f51ffef7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81e6fb0 con 0x7f51f813a120 2023-11-13T10:56:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.653+0000 7f51ffef7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81e5f10 con 0x7f51f8139ca0 2023-11-13T10:56:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.653+0000 7f51ffef7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81e61f0 con 0x7f51e40039a0 2023-11-13T10:56:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.653+0000 7f51fe494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813a120 0x7f51f8071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.653+0000 7f51fe494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813a120 0x7f51f8071450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58242/0 (socket says 172.21.15.16:58242) 2023-11-13T10:56:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.653+0000 7f51fe494640 1 -- 172.21.15.16:0/3421051782 learned_addr learned my addr 172.21.15.16:0/3421051782 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51fd492640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51f8139ca0 0x7f51f8070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51fdc93640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40039a0 0x7f51f815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51fe494640 1 -- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40039a0 msgr2=0x7f51f815e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51fe494640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40039a0 0x7f51f815e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51fe494640 1 -- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51f8139ca0 msgr2=0x7f51f8070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51fe494640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51f8139ca0 0x7f51f8070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51fe494640 1 -- 172.21.15.16:0/3421051782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f81e64d0 con 0x7f51f813a120 2023-11-13T10:56:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51fe494640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813a120 0x7f51f8071450 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f51f8071e60 tx=0x7f51f000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3421051782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f000b070 con 0x7f51f813a120 2023-11-13T10:56:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3421051782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f0007030 con 0x7f51f813a120 2023-11-13T10:56:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3421051782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00073e0 con 0x7f51f813a120 2023-11-13T10:56:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f81e67b0 con 0x7f51f813a120 2023-11-13T10:56:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.654+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f81e5c00 con 0x7f51f813a120 2023-11-13T10:56:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.656+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f8072520 con 0x7f51f813a120 2023-11-13T10:56:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.657+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3421051782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51f0007540 con 0x7f51f813a120 2023-11-13T10:56:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.657+0000 7f51ee7fc640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f51dc06ae70 0x7f51dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.658+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3421051782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f51f0089440 con 0x7f51f813a120 2023-11-13T10:56:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.658+0000 7f51fdc93640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f51dc06ae70 0x7f51dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.659+0000 7f51fdc93640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f51dc06ae70 0x7f51dc06d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51f4036000 tx=0x7f51f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.662+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3421051782 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51f00541a0 con 0x7f51f813a120 2023-11-13T10:56:30.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.859+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51f804e5b0 con 0x7f51f813a120 2023-11-13T10:56:30.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.860+0000 7f51ee7fc640 1 -- 172.21.15.16:0/3421051782 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51f00577c0 con 0x7f51f813a120 2023-11-13T10:56:30.864 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:30.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f51dc06ae70 msgr2=0x7f51dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f51dc06ae70 0x7f51dc06d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51f4036000 tx=0x7f51f402e000 comp rx=0 tx=0).stop 2023-11-13T10:56:30.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813a120 msgr2=0x7f51f8071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813a120 0x7f51f8071450 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f51f8071e60 tx=0x7f51f000d890 comp rx=0 tx=0).stop 2023-11-13T10:56:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 shutdown_connections 2023-11-13T10:56:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f51dc06ae70 0x7f51dc06d330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813a120 0x7f51f8071450 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51f8139ca0 0x7f51f8070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 --2- 172.21.15.16:0/3421051782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e40039a0 0x7f51f815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 >> 172.21.15.16:0/3421051782 conn(0x7f51f806a190 msgr2=0x7f51f8142000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.862+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 shutdown_connections 2023-11-13T10:56:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:30.863+0000 7f51ffef7640 1 -- 172.21.15.16:0/3421051782 wait complete. 2023-11-13T10:56:30.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.112+0000 7f45ab154640 1 Processor -- start 2023-11-13T10:56:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.112+0000 7f45ab154640 1 -- start start 2023-11-13T10:56:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.113+0000 7f45ab154640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45a40cf640 0x7f45a40c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.113+0000 7f45ab154640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a40ced70 con 0x7f45a415ad60 2023-11-13T10:56:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.114+0000 7f45ab154640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a40cb460 con 0x7f45a40ce520 2023-11-13T10:56:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.114+0000 7f45ab154640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40cea90 con 0x7f45a40cf640 2023-11-13T10:56:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.114+0000 7f45aa152640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f45a415ad60 0x7f45a40cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49378/0 (socket says 172.21.15.16:49378) 2023-11-13T10:56:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.114+0000 7f45aa152640 1 -- 172.21.15.16:0/2799532952 learned_addr learned my addr 172.21.15.16:0/2799532952 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.114+0000 7f45aa953640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45a40cf640 0x7f45a40c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.114+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462504525 0 0) 0x7f45a0001490 con 0x7f45a415ad60 2023-11-13T10:56:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4590003520 con 0x7f45a415ad60 2023-11-13T10:56:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2216069838 0 0) 0x7f4590003520 con 0x7f45a415ad60 2023-11-13T10:56:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45a40cf640 msgr2=0x7f45a40c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45a40cf640 0x7f45a40c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 >> v1:172.21.15.184:6789/0 conn(0x7f45a40ce520 legacy=0x7f45a40cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40cb700 con 0x7f45a415ad60 2023-11-13T10:56:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f45a0001b80 con 0x7f45a415ad60 2023-11-13T10:56:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 >> v1:172.21.15.16:6789/0 conn(0x7f45a415ad60 legacy=0x7f45a40cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45900039a0 0x7f4590003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45a40cf640 0x7f45900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4590004d00 0x7f4590005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40cea90 con 0x7f4590004d00 2023-11-13T10:56:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459000ba90 con 0x7f45900039a0 2023-11-13T10:56:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459000bc50 con 0x7f45a40cf640 2023-11-13T10:56:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.115+0000 7f45aa953640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4590004d00 0x7f4590005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.116+0000 7f45aa152640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45900039a0 0x7f4590003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.116+0000 7f45a9951640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45a40cf640 0x7f45900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.116+0000 7f45aa953640 1 -- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45a40cf640 msgr2=0x7f45900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.116+0000 7f45aa953640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45a40cf640 0x7f45900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.116+0000 7f45aa953640 1 -- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45900039a0 msgr2=0x7f4590003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.116+0000 7f45aa953640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45900039a0 0x7f4590003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.116+0000 7f45aa953640 1 -- 172.21.15.16:0/2799532952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4590003520 con 0x7f4590004d00 2023-11-13T10:56:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.117+0000 7f45aa953640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4590004d00 0x7f4590005180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f459c009a90 tx=0x7f45900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d4c4eab246366f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.117+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c02c680 con 0x7f4590004d00 2023-11-13T10:56:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.117+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c02c7e0 con 0x7f4590004d00 2023-11-13T10:56:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.117+0000 7f45a9150640 1 -- 172.21.15.16:0/2799532952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c02cb70 con 0x7f4590004d00 2023-11-13T10:56:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.118+0000 7f45ab154640 1 -- 172.21.15.16:0/2799532952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4590004d00 msgr2=0x7f4590005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.118+0000 7f45ab154640 1 --2- 172.21.15.16:0/2799532952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4590004d00 0x7f4590005180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f459c009a90 tx=0x7f45900057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.118+0000 7f45aa953640 1 -- 172.21.15.16:0/2799532952 reap_dead start 2023-11-13T10:56:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.118+0000 7f45ab154640 1 -- 172.21.15.16:0/2799532952 shutdown_connections 2023-11-13T10:56:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.118+0000 7f45ab154640 1 -- 172.21.15.16:0/2799532952 >> 172.21.15.16:0/2799532952 conn(0x7f45a40d0950 msgr2=0x7f45a40d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.118+0000 7f45ab154640 1 -- 172.21.15.16:0/2799532952 shutdown_connections 2023-11-13T10:56:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.118+0000 7f45ab154640 1 -- 172.21.15.16:0/2799532952 wait complete. 2023-11-13T10:56:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.120+0000 7f45ab154640 1 Processor -- start 2023-11-13T10:56:31.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.120+0000 7f45ab154640 1 -- start start 2023-11-13T10:56:31.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.121+0000 7f45ab154640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4590004d00 0x7f45a4150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.121+0000 7f45ab154640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a40cd0a0 0x7f45a40c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.121+0000 7f45ab154640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a406dbe0 0x7f45a406e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.121+0000 7f45ab154640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c5660 con 0x7f45a40cd0a0 2023-11-13T10:56:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.121+0000 7f45ab154640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a406e5a0 con 0x7f45a406dbe0 2023-11-13T10:56:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.121+0000 7f45ab154640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c84c0 con 0x7f4590004d00 2023-11-13T10:56:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45a9951640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a40cd0a0 0x7f45a40c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45a9951640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a40cd0a0 0x7f45a40c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58268/0 (socket says 172.21.15.16:58268) 2023-11-13T10:56:31.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45a9951640 1 -- 172.21.15.16:0/1412916597 learned_addr learned my addr 172.21.15.16:0/1412916597 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:31.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45aa152640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4590004d00 0x7f45a4150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45aa953640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a406dbe0 0x7f45a406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45a9951640 1 -- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4590004d00 msgr2=0x7f45a4150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45a9951640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4590004d00 0x7f45a4150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45a9951640 1 -- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a406dbe0 msgr2=0x7f45a406e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45a9951640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a406dbe0 0x7f45a406e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45a9951640 1 -- 172.21.15.16:0/1412916597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40c4ae0 con 0x7f45a40cd0a0 2023-11-13T10:56:31.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.122+0000 7f45a9951640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a40cd0a0 0x7f45a40c42b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f459400e550 tx=0x7f459400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:31.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.123+0000 7f459affd640 1 -- 172.21.15.16:0/1412916597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940052e0 con 0x7f45a40cd0a0 2023-11-13T10:56:31.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.123+0000 7f459affd640 1 -- 172.21.15.16:0/1412916597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4594005440 con 0x7f45a40cd0a0 2023-11-13T10:56:31.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.123+0000 7f459affd640 1 -- 172.21.15.16:0/1412916597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594003810 con 0x7f45a40cd0a0 2023-11-13T10:56:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.123+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a40c4dc0 con 0x7f45a40cd0a0 2023-11-13T10:56:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.123+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a40c5380 con 0x7f45a40cd0a0 2023-11-13T10:56:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.124+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a40c8600 con 0x7f45a40cd0a0 2023-11-13T10:56:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.125+0000 7f459affd640 1 -- 172.21.15.16:0/1412916597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4594003970 con 0x7f45a40cd0a0 2023-11-13T10:56:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.126+0000 7f459affd640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f458806ae40 0x7f458806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.126+0000 7f459affd640 1 -- 172.21.15.16:0/1412916597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f4594089630 con 0x7f45a40cd0a0 2023-11-13T10:56:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.127+0000 7f45aa152640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f458806ae40 0x7f458806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.128+0000 7f45aa152640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f458806ae40 0x7f458806d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f45a00015f0 tx=0x7f45a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.131+0000 7f459affd640 1 -- 172.21.15.16:0/1412916597 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4594054410 con 0x7f45a40cd0a0 2023-11-13T10:56:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.323+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45a40cdd30 con 0x7f45a40cd0a0 2023-11-13T10:56:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.324+0000 7f459affd640 1 -- 172.21.15.16:0/1412916597 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4594057a30 con 0x7f45a40cd0a0 2023-11-13T10:56:31.330 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:31.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f458806ae40 msgr2=0x7f458806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f458806ae40 0x7f458806d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f45a00015f0 tx=0x7f45a0009040 comp rx=0 tx=0).stop 2023-11-13T10:56:31.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a40cd0a0 msgr2=0x7f45a40c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a40cd0a0 0x7f45a40c42b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f459400e550 tx=0x7f459400ea10 comp rx=0 tx=0).stop 2023-11-13T10:56:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 shutdown_connections 2023-11-13T10:56:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f458806ae40 0x7f458806d300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45a406dbe0 0x7f45a406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a40cd0a0 0x7f45a40c42b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 --2- 172.21.15.16:0/1412916597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4590004d00 0x7f45a4150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 >> 172.21.15.16:0/1412916597 conn(0x7f45a40d0950 msgr2=0x7f45a415b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 shutdown_connections 2023-11-13T10:56:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.327+0000 7f45ab154640 1 -- 172.21.15.16:0/1412916597 wait complete. 2023-11-13T10:56:31.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.576+0000 7fdde55e2640 1 Processor -- start 2023-11-13T10:56:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.576+0000 7fdde55e2640 1 -- start start 2023-11-13T10:56:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.578+0000 7fdde55e2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdde015bcc0 0x7fdde00cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.578+0000 7fdde55e2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde006d010 con 0x7fdde00ce090 2023-11-13T10:56:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.578+0000 7fdde55e2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde006b860 con 0x7fdde015b8e0 2023-11-13T10:56:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.578+0000 7fdde55e2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde00ce540 con 0x7fdde015bcc0 2023-11-13T10:56:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.578+0000 7fdde4de1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdde00ce090 0x7fdde00cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49394/0 (socket says 172.21.15.16:49394) 2023-11-13T10:56:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.578+0000 7fdde4de1640 1 -- 172.21.15.16:0/3143917904 learned_addr learned my addr 172.21.15.16:0/3143917904 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.578+0000 7fdddf7fe640 1 --2- 172.21.15.16:0/3143917904 >> v2:172.21.15.184:3300/0 conn(0x7fdde015bcc0 0x7fdde00cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821568833 0 0) 0x7fdde006d010 con 0x7fdde00ce090 2023-11-13T10:56:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc8003520 con 0x7fdde00ce090 2023-11-13T10:56:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2630050352 0 0) 0x7fddc8003520 con 0x7fdde00ce090 2023-11-13T10:56:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 >> v2:172.21.15.184:3300/0 conn(0x7fdde015bcc0 msgr2=0x7fdde00cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:31.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 --2- 172.21.15.16:0/3143917904 >> v2:172.21.15.184:3300/0 conn(0x7fdde015bcc0 0x7fdde00cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 >> v1:172.21.15.184:6789/0 conn(0x7fdde015b8e0 legacy=0x7fdde006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde006bb40 con 0x7fdde00ce090 2023-11-13T10:56:31.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fddd8001670 con 0x7fdde00ce090 2023-11-13T10:56:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 >> v1:172.21.15.16:6789/0 conn(0x7fdde00ce090 legacy=0x7fdde00cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8003970 0x7fddc8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc80047c0 0x7fddc8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc8005180 0x7fddc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00ce540 con 0x7fddc8003970 2023-11-13T10:56:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc800c020 con 0x7fddc8005180 2023-11-13T10:56:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.579+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc800c1e0 con 0x7fddc80047c0 2023-11-13T10:56:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdddffff640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8003970 0x7fddc8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdddf7fe640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc80047c0 0x7fddc8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdde4de1640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc8005180 0x7fddc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdddffff640 1 -- 172.21.15.16:0/3143917904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc80047c0 msgr2=0x7fddc8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdddffff640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc80047c0 0x7fddc8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdddffff640 1 -- 172.21.15.16:0/3143917904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc8005180 msgr2=0x7fddc800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdddffff640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc8005180 0x7fddc800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdddffff640 1 -- 172.21.15.16:0/3143917904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc8003520 con 0x7fddc8003970 2023-11-13T10:56:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdde4de1640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fddc8005180 0x7fddc800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.580+0000 7fdde4de1640 1 -- 172.21.15.16:0/3143917904 reap_dead start 2023-11-13T10:56:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.581+0000 7fdddffff640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8003970 0x7fddc8003d90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fddd0001e50 tx=0x7fddc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704e2542f3a9c9cf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0003b70 con 0x7fddc8003970 2023-11-13T10:56:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd0003cd0 con 0x7fddc8003970 2023-11-13T10:56:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdddeffd640 1 -- 172.21.15.16:0/3143917904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0005310 con 0x7fddc8003970 2023-11-13T10:56:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdde55e2640 1 -- 172.21.15.16:0/3143917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8003970 msgr2=0x7fddc8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdde55e2640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8003970 0x7fddc8003d90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fddd0001e50 tx=0x7fddc80057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdde55e2640 1 -- 172.21.15.16:0/3143917904 shutdown_connections 2023-11-13T10:56:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdde55e2640 1 --2- 172.21.15.16:0/3143917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8003970 0x7fddc8003d90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdde55e2640 1 -- 172.21.15.16:0/3143917904 >> 172.21.15.16:0/3143917904 conn(0x7fdde00b0970 msgr2=0x7fdde00b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdde55e2640 1 -- 172.21.15.16:0/3143917904 shutdown_connections 2023-11-13T10:56:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.582+0000 7fdde55e2640 1 -- 172.21.15.16:0/3143917904 wait complete. 2023-11-13T10:56:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.584+0000 7fdde55e2640 1 Processor -- start 2023-11-13T10:56:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.584+0000 7fdde55e2640 1 -- start start 2023-11-13T10:56:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.585+0000 7fdde55e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc8003970 0x7fdde01f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.585+0000 7fdde55e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0153e10 0x7fdde0155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.585+0000 7fdde55e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde0157050 0x7fdde01574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.585+0000 7fdde55e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde0157a10 con 0x7fdde0153e10 2023-11-13T10:56:31.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.585+0000 7fdde55e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde0155e60 con 0x7fdde0157050 2023-11-13T10:56:31.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.585+0000 7fdde55e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde01f14a0 con 0x7fddc8003970 2023-11-13T10:56:31.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.585+0000 7fdddf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0153e10 0x7fdde0155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0153e10 0x7fdde0155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58282/0 (socket says 172.21.15.16:58282) 2023-11-13T10:56:31.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddf7fe640 1 -- 172.21.15.16:0/3258626666 learned_addr learned my addr 172.21.15.16:0/3258626666 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:31.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddffff640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc8003970 0x7fdde01f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdde4de1640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde0157050 0x7fdde01574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddf7fe640 1 -- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc8003970 msgr2=0x7fdde01f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddf7fe640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc8003970 0x7fdde01f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddf7fe640 1 -- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde0157050 msgr2=0x7fdde01574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddf7fe640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde0157050 0x7fdde01574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddf7fe640 1 -- 172.21.15.16:0/3258626666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0154290 con 0x7fdde0153e10 2023-11-13T10:56:31.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdde4de1640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde0157050 0x7fdde01574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:31.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddffff640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc8003970 0x7fdde01f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:31.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.586+0000 7fdddf7fe640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0153e10 0x7fdde0155920 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fddd4011510 tx=0x7fddd40119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.587+0000 7fdddcff9640 1 -- 172.21.15.16:0/3258626666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4015870 con 0x7fdde0153e10 2023-11-13T10:56:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.587+0000 7fdddcff9640 1 -- 172.21.15.16:0/3258626666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd40159d0 con 0x7fdde0153e10 2023-11-13T10:56:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.587+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde01f0b70 con 0x7fdde0153e10 2023-11-13T10:56:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.587+0000 7fdddcff9640 1 -- 172.21.15.16:0/3258626666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4015d40 con 0x7fdde0153e10 2023-11-13T10:56:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.587+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde01f1130 con 0x7fdde0153e10 2023-11-13T10:56:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.588+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde004e4d0 con 0x7fdde0153e10 2023-11-13T10:56:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.589+0000 7fdddcff9640 1 -- 172.21.15.16:0/3258626666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fddd4013400 con 0x7fdde0153e10 2023-11-13T10:56:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.590+0000 7fdddcff9640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fddcc06ae40 0x7fddcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.590+0000 7fdddcff9640 1 -- 172.21.15.16:0/3258626666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fddd408a070 con 0x7fdde0153e10 2023-11-13T10:56:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.590+0000 7fdddffff640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fddcc06ae40 0x7fddcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.592+0000 7fdddffff640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fddcc06ae40 0x7fddcc06d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fddd0034000 tx=0x7fddd0009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.594+0000 7fdddcff9640 1 -- 172.21.15.16:0/3258626666 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fddd40bce90 con 0x7fdde0153e10 2023-11-13T10:56:31.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.790+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdde01f15e0 con 0x7fdde0153e10 2023-11-13T10:56:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.791+0000 7fdddcff9640 1 -- 172.21.15.16:0/3258626666 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fddd4054dd0 con 0x7fdde0153e10 2023-11-13T10:56:31.796 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.793+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fddcc06ae40 msgr2=0x7fddcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.793+0000 7fdde55e2640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fddcc06ae40 0x7fddcc06d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fddd0034000 tx=0x7fddd0009030 comp rx=0 tx=0).stop 2023-11-13T10:56:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0153e10 msgr2=0x7fdde0155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0153e10 0x7fdde0155920 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fddd4011510 tx=0x7fddd40119d0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 shutdown_connections 2023-11-13T10:56:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fddcc06ae40 0x7fddcc06d300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdde0157050 0x7fdde01574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0153e10 0x7fdde0155920 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 --2- 172.21.15.16:0/3258626666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fddc8003970 0x7fdde01f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 >> 172.21.15.16:0/3258626666 conn(0x7fdde00b0970 msgr2=0x7fdde006c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 shutdown_connections 2023-11-13T10:56:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:31.794+0000 7fdde55e2640 1 -- 172.21.15.16:0/3258626666 wait complete. 2023-11-13T10:56:31.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.039+0000 7fef2e21a640 1 Processor -- start 2023-11-13T10:56:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.039+0000 7fef2e21a640 1 -- start start 2023-11-13T10:56:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.041+0000 7fef2e21a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef28155410 0x7fef280ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.041+0000 7fef2e21a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fef28071b50 0x7fef280cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.041+0000 7fef2e21a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2815b500 con 0x7fef280ce990 2023-11-13T10:56:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.041+0000 7fef2e21a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2815b7e0 con 0x7fef28071b50 2023-11-13T10:56:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.041+0000 7fef2e21a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2815bac0 con 0x7fef28155410 2023-11-13T10:56:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.041+0000 7fef26ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fef280ce990 0x7fef28071440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49402/0 (socket says 172.21.15.16:49402) 2023-11-13T10:56:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.041+0000 7fef26ffd640 1 -- 172.21.15.16:0/352653743 learned_addr learned my addr 172.21.15.16:0/352653743 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef277fe640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef28155410 0x7fef280ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef27fff640 1 --2- 172.21.15.16:0/352653743 >> v2:172.21.15.184:3300/0 conn(0x7fef28071b50 0x7fef280cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504343748 0 0) 0x7fef2815b500 con 0x7fef280ce990 2023-11-13T10:56:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef10003520 con 0x7fef280ce990 2023-11-13T10:56:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1655579263 0 0) 0x7fef10003520 con 0x7fef280ce990 2023-11-13T10:56:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef28155410 msgr2=0x7fef280ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef28155410 0x7fef280ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 >> v2:172.21.15.184:3300/0 conn(0x7fef28071b50 msgr2=0x7fef280cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 --2- 172.21.15.16:0/352653743 >> v2:172.21.15.184:3300/0 conn(0x7fef28071b50 0x7fef280cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2815bda0 con 0x7fef280ce990 2023-11-13T10:56:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fef1c001670 con 0x7fef280ce990 2023-11-13T10:56:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 >> v1:172.21.15.16:6789/0 conn(0x7fef280ce990 legacy=0x7fef28071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef28155410 0x7fef10003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.042+0000 7fef267fc640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef10003ef0 0x7fef100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef267fc640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef10004d00 0x7fef10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2815b7e0 con 0x7fef10004d00 2023-11-13T10:56:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2815bac0 con 0x7fef10003ef0 2023-11-13T10:56:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1000b980 con 0x7fef28155410 2023-11-13T10:56:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef27fff640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef10004d00 0x7fef10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef277fe640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef28155410 0x7fef10003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef27fff640 1 -- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef28155410 msgr2=0x7fef10003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef27fff640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef28155410 0x7fef10003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef27fff640 1 -- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef10003ef0 msgr2=0x7fef100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef27fff640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef10003ef0 0x7fef100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.043+0000 7fef27fff640 1 -- 172.21.15.16:0/352653743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10003520 con 0x7fef10004d00 2023-11-13T10:56:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.044+0000 7fef27fff640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef10004d00 0x7fef10005180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef20009fc0 tx=0x7fef100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4400816bea99644 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.045+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2002cc40 con 0x7fef10004d00 2023-11-13T10:56:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.045+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2000f040 con 0x7fef10004d00 2023-11-13T10:56:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.045+0000 7fef267fc640 1 -- 172.21.15.16:0/352653743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20034440 con 0x7fef10004d00 2023-11-13T10:56:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.045+0000 7fef2e21a640 1 -- 172.21.15.16:0/352653743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef10004d00 msgr2=0x7fef10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.045+0000 7fef2e21a640 1 --2- 172.21.15.16:0/352653743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef10004d00 0x7fef10005180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef20009fc0 tx=0x7fef100057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.045+0000 7fef27fff640 1 -- 172.21.15.16:0/352653743 reap_dead start 2023-11-13T10:56:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.046+0000 7fef2e21a640 1 -- 172.21.15.16:0/352653743 shutdown_connections 2023-11-13T10:56:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.046+0000 7fef2e21a640 1 -- 172.21.15.16:0/352653743 >> 172.21.15.16:0/352653743 conn(0x7fef2806a190 msgr2=0x7fef2806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.046+0000 7fef2e21a640 1 -- 172.21.15.16:0/352653743 shutdown_connections 2023-11-13T10:56:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.046+0000 7fef2e21a640 1 -- 172.21.15.16:0/352653743 wait complete. 2023-11-13T10:56:32.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.047+0000 7fef2e21a640 1 Processor -- start 2023-11-13T10:56:32.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.048+0000 7fef2e21a640 1 -- start start 2023-11-13T10:56:32.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.048+0000 7fef2e21a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef10004d00 0x7fef2806bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef2e21a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2814c580 0x7fef2815e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef2e21a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef2815edd0 0x7fef280ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef2e21a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef281571a0 con 0x7fef2814c580 2023-11-13T10:56:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef2e21a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef28152460 con 0x7fef10004d00 2023-11-13T10:56:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef2e21a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef28152740 con 0x7fef2815edd0 2023-11-13T10:56:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2814c580 0x7fef2815e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2814c580 0x7fef2815e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58288/0 (socket says 172.21.15.16:58288) 2023-11-13T10:56:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef26ffd640 1 -- 172.21.15.16:0/1270427278 learned_addr learned my addr 172.21.15.16:0/1270427278 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef26ffd640 1 -- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef2815edd0 msgr2=0x7fef280ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef277fe640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef10004d00 0x7fef2806bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef27fff640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef2815edd0 0x7fef280ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef26ffd640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef2815edd0 0x7fef280ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef26ffd640 1 -- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef10004d00 msgr2=0x7fef2806bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef26ffd640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef10004d00 0x7fef2806bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.049+0000 7fef26ffd640 1 -- 172.21.15.16:0/1270427278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef28156900 con 0x7fef2814c580 2023-11-13T10:56:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.050+0000 7fef26ffd640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2814c580 0x7fef2815e890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00c7a0 tx=0x7fef1c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.050+0000 7fef0ffff640 1 -- 172.21.15.16:0/1270427278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0074e0 con 0x7fef2814c580 2023-11-13T10:56:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.050+0000 7fef0ffff640 1 -- 172.21.15.16:0/1270427278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c007640 con 0x7fef2814c580 2023-11-13T10:56:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.050+0000 7fef0ffff640 1 -- 172.21.15.16:0/1270427278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0049e0 con 0x7fef2814c580 2023-11-13T10:56:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.050+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef28156be0 con 0x7fef2814c580 2023-11-13T10:56:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.050+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2814ca00 con 0x7fef2814c580 2023-11-13T10:56:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.051+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef28067090 con 0x7fef2814c580 2023-11-13T10:56:32.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.052+0000 7fef0ffff640 1 -- 172.21.15.16:0/1270427278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef1c004b40 con 0x7fef2814c580 2023-11-13T10:56:32.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.054+0000 7fef0ffff640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fef1406ae70 0x7fef1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.054+0000 7fef0ffff640 1 -- 172.21.15.16:0/1270427278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fef1c088eb0 con 0x7fef2814c580 2023-11-13T10:56:32.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.054+0000 7fef277fe640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fef1406ae70 0x7fef1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.056+0000 7fef277fe640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fef1406ae70 0x7fef1406d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef18009910 tx=0x7fef18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:32.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.058+0000 7fef0ffff640 1 -- 172.21.15.16:0/1270427278 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef1c053b80 con 0x7fef2814c580 2023-11-13T10:56:32.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.258+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef280ae1d0 con 0x7fef2814c580 2023-11-13T10:56:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.261+0000 7fef0ffff640 1 -- 172.21.15.16:0/1270427278 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fef1c018020 con 0x7fef2814c580 2023-11-13T10:56:32.266 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:32.266 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.263+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fef1406ae70 msgr2=0x7fef1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.263+0000 7fef2e21a640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fef1406ae70 0x7fef1406d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef18009910 tx=0x7fef18008040 comp rx=0 tx=0).stop 2023-11-13T10:56:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.263+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2814c580 msgr2=0x7fef2815e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.263+0000 7fef2e21a640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2814c580 0x7fef2815e890 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00c7a0 tx=0x7fef1c00cc60 comp rx=0 tx=0).stop 2023-11-13T10:56:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.264+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 shutdown_connections 2023-11-13T10:56:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.264+0000 7fef2e21a640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fef1406ae70 0x7fef1406d330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.264+0000 7fef2e21a640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef2815edd0 0x7fef280ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.264+0000 7fef2e21a640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2814c580 0x7fef2815e890 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.264+0000 7fef2e21a640 1 --2- 172.21.15.16:0/1270427278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef10004d00 0x7fef2806bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.264+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 >> 172.21.15.16:0/1270427278 conn(0x7fef2806a190 msgr2=0x7fef280b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.264+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 shutdown_connections 2023-11-13T10:56:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.264+0000 7fef2e21a640 1 -- 172.21.15.16:0/1270427278 wait complete. 2023-11-13T10:56:32.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:32.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.517+0000 7fc0fdb61640 1 Processor -- start 2023-11-13T10:56:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.518+0000 7fc0fdb61640 1 -- start start 2023-11-13T10:56:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.519+0000 7fc0fdb61640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc0f806c1c0 0x7fc0f80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.519+0000 7fc0fdb61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.519+0000 7fc0fdb61640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f80cede0 con 0x7fc0f80d1690 2023-11-13T10:56:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.519+0000 7fc0fdb61640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f80d0120 con 0x7fc0f806c1c0 2023-11-13T10:56:32.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.519+0000 7fc0fdb61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f80ce410 con 0x7fc0f806b4b0 2023-11-13T10:56:32.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.520+0000 7fc0f6ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.520+0000 7fc0f6ffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:50398/0 (socket says 172.21.15.16:50398) 2023-11-13T10:56:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.520+0000 7fc0f6ffd640 1 -- 172.21.15.16:0/283427497 learned_addr learned my addr 172.21.15.16:0/283427497 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.520+0000 7fc0f77fe640 1 --2- 172.21.15.16:0/283427497 >> v2:172.21.15.184:3300/0 conn(0x7fc0f806c1c0 0x7fc0f80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.521+0000 7fc0f77fe640 1 -- 172.21.15.16:0/283427497 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 msgr2=0x7fc0f80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:32.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.521+0000 7fc0f77fe640 1 --2- 172.21.15.16:0/283427497 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.521+0000 7fc0f77fe640 1 -- 172.21.15.16:0/283427497 >> v1:172.21.15.184:6789/0 conn(0x7fc0f80d1690 legacy=0x7fc0f815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.521+0000 7fc0f77fe640 1 -- 172.21.15.16:0/283427497 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f80cf080 con 0x7fc0f806c1c0 2023-11-13T10:56:32.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.521+0000 7fc0f6ffd640 1 --2- 172.21.15.16:0/283427497 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:56:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.522+0000 7fc0f77fe640 1 --2- 172.21.15.16:0/283427497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f806c1c0 0x7fc0f80cd420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009d90 tx=0x7fc0e802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f485cf8b27fc3b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.523+0000 7fc0f67fc640 1 -- 172.21.15.16:0/283427497 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e802ccf0 con 0x7fc0f806c1c0 2023-11-13T10:56:32.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.523+0000 7fc0f67fc640 1 -- 172.21.15.16:0/283427497 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e802ce50 con 0x7fc0f806c1c0 2023-11-13T10:56:32.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.523+0000 7fc0f67fc640 1 -- 172.21.15.16:0/283427497 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8034660 con 0x7fc0f806c1c0 2023-11-13T10:56:32.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.523+0000 7fc0fdb61640 1 -- 172.21.15.16:0/283427497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f806c1c0 msgr2=0x7fc0f80cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.523+0000 7fc0fdb61640 1 --2- 172.21.15.16:0/283427497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f806c1c0 0x7fc0f80cd420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009d90 tx=0x7fc0e802bf40 comp rx=0 tx=0).stop 2023-11-13T10:56:32.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.524+0000 7fc0fdb61640 1 -- 172.21.15.16:0/283427497 shutdown_connections 2023-11-13T10:56:32.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.524+0000 7fc0fdb61640 1 --2- 172.21.15.16:0/283427497 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.524+0000 7fc0fdb61640 1 --2- 172.21.15.16:0/283427497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f806c1c0 0x7fc0f80cd420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.524+0000 7fc0fdb61640 1 -- 172.21.15.16:0/283427497 >> 172.21.15.16:0/283427497 conn(0x7fc0f80d0ba0 msgr2=0x7fc0f80d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:32.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.524+0000 7fc0fdb61640 1 -- 172.21.15.16:0/283427497 shutdown_connections 2023-11-13T10:56:32.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.524+0000 7fc0fdb61640 1 -- 172.21.15.16:0/283427497 wait complete. 2023-11-13T10:56:32.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.526+0000 7fc0fdb61640 1 Processor -- start 2023-11-13T10:56:32.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.526+0000 7fc0fdb61640 1 -- start start 2023-11-13T10:56:32.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0fdb61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0fdb61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f806c1c0 0x7fc0f81f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0fdb61640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80d1690 0x7fc0f815f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0fdb61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f80c9cd0 con 0x7fc0f806c1c0 2023-11-13T10:56:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0fdb61640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f80c9330 con 0x7fc0f80d1690 2023-11-13T10:56:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0fdb61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f80c9610 con 0x7fc0f806b4b0 2023-11-13T10:56:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f806c1c0 0x7fc0f81f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f806c1c0 0x7fc0f81f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58298/0 (socket says 172.21.15.16:58298) 2023-11-13T10:56:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0f6ffd640 1 -- 172.21.15.16:0/1360567458 learned_addr learned my addr 172.21.15.16:0/1360567458 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.527+0000 7fc0f77fe640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0f7fff640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80d1690 0x7fc0f815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0f6ffd640 1 -- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 msgr2=0x7fc0f80b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0f6ffd640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0f6ffd640 1 -- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80d1690 msgr2=0x7fc0f815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0f6ffd640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80d1690 0x7fc0f815f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0f6ffd640 1 -- 172.21.15.16:0/1360567458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e00094d0 con 0x7fc0f806c1c0 2023-11-13T10:56:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0f6ffd640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f806c1c0 0x7fc0f81f2fa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000e9d0 tx=0x7fc0e000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:32.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0effff640 1 -- 172.21.15.16:0/1360567458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e000ca60 con 0x7fc0f806c1c0 2023-11-13T10:56:32.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0effff640 1 -- 172.21.15.16:0/1360567458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e000cbc0 con 0x7fc0f806c1c0 2023-11-13T10:56:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0effff640 1 -- 172.21.15.16:0/1360567458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0013470 con 0x7fc0f806c1c0 2023-11-13T10:56:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.528+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e8009a70 con 0x7fc0f806c1c0 2023-11-13T10:56:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.529+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f8156190 con 0x7fc0f806c1c0 2023-11-13T10:56:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.530+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f804e5b0 con 0x7fc0f806c1c0 2023-11-13T10:56:32.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.531+0000 7fc0effff640 1 -- 172.21.15.16:0/1360567458 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0e00135d0 con 0x7fc0f806c1c0 2023-11-13T10:56:32.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.531+0000 7fc0effff640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc0d806ae40 0x7fc0d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.532+0000 7fc0effff640 1 -- 172.21.15.16:0/1360567458 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fc0e0089cd0 con 0x7fc0f806c1c0 2023-11-13T10:56:32.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.532+0000 7fc0f77fe640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc0d806ae40 0x7fc0d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:32.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.533+0000 7fc0f77fe640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc0d806ae40 0x7fc0d806d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8036000 tx=0x7fc0e8030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:32.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.536+0000 7fc0effff640 1 -- 172.21.15.16:0/1360567458 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0e0054a30 con 0x7fc0f806c1c0 2023-11-13T10:56:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.728+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0f80b6b20 con 0x7fc0f806c1c0 2023-11-13T10:56:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.730+0000 7fc0effff640 1 -- 172.21.15.16:0/1360567458 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0e00170a0 con 0x7fc0f806c1c0 2023-11-13T10:56:32.734 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc0d806ae40 msgr2=0x7fc0d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc0d806ae40 0x7fc0d806d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8036000 tx=0x7fc0e8030000 comp rx=0 tx=0).stop 2023-11-13T10:56:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f806c1c0 msgr2=0x7fc0f81f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f806c1c0 0x7fc0f81f2fa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000e9d0 tx=0x7fc0e000ee90 comp rx=0 tx=0).stop 2023-11-13T10:56:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 shutdown_connections 2023-11-13T10:56:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc0d806ae40 0x7fc0d806d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0f80d1690 0x7fc0f815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f806c1c0 0x7fc0f81f2fa0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 --2- 172.21.15.16:0/1360567458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0f806b4b0 0x7fc0f80b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 >> 172.21.15.16:0/1360567458 conn(0x7fc0f80d0ba0 msgr2=0x7fc0f806e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.732+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 shutdown_connections 2023-11-13T10:56:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.733+0000 7fc0fdb61640 1 -- 172.21.15.16:0/1360567458 wait complete. 2023-11-13T10:56:32.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.989+0000 7f4a6376c640 1 Processor -- start 2023-11-13T10:56:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.989+0000 7f4a6376c640 1 -- start start 2023-11-13T10:56:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.991+0000 7f4a6376c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0ce1d0 0x7f4a5c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.991+0000 7f4a6376c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c15d8d0 con 0x7f4a5c0ce5b0 2023-11-13T10:56:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.991+0000 7f4a6376c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c15af00 con 0x7f4a5c15e390 2023-11-13T10:56:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.991+0000 7f4a6376c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c15db90 con 0x7f4a5c0ce1d0 2023-11-13T10:56:32.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.992+0000 7f4a60d07640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4a5c0ce5b0 0x7f4a5c15a130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46924/0 (socket says 172.21.15.16:46924) 2023-11-13T10:56:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.992+0000 7f4a60d07640 1 -- 172.21.15.16:0/2263626623 learned_addr learned my addr 172.21.15.16:0/2263626623 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.992+0000 7f4a61508640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0ce1d0 0x7f4a5c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.992+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111067173 0 0) 0x7f4a5c15d8d0 con 0x7f4a5c0ce5b0 2023-11-13T10:56:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.992+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a48005730 con 0x7f4a5c0ce5b0 2023-11-13T10:56:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2290804697 0 0) 0x7f4a48005730 con 0x7f4a5c0ce5b0 2023-11-13T10:56:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0ce1d0 msgr2=0x7f4a5c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0ce1d0 0x7f4a5c0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 >> v1:172.21.15.184:6789/0 conn(0x7f4a5c15e390 legacy=0x7f4a5c0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c15de70 con 0x7f4a5c0ce5b0 2023-11-13T10:56:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f4a4c001670 con 0x7f4a5c0ce5b0 2023-11-13T10:56:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 >> v1:172.21.15.16:6789/0 conn(0x7f4a5c0ce5b0 legacy=0x7f4a5c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0ce1d0 0x7f4a48005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a480060f0 0x7f4a480069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a48006f10 0x7f4a48007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c15db90 con 0x7f4a48006f10 2023-11-13T10:56:33.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4800dbf0 con 0x7f4a480060f0 2023-11-13T10:56:33.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4800ddb0 con 0x7f4a5c0ce1d0 2023-11-13T10:56:33.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.993+0000 7f4a61d09640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a48006f10 0x7f4a48007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.994+0000 7f4a61508640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0ce1d0 0x7f4a48005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.994+0000 7f4a60d07640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a480060f0 0x7f4a480069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.994+0000 7f4a61d09640 1 -- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0ce1d0 msgr2=0x7f4a48005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.994+0000 7f4a61d09640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0ce1d0 0x7f4a48005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.994+0000 7f4a61d09640 1 -- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a480060f0 msgr2=0x7f4a480069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.994+0000 7f4a61d09640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a480060f0 0x7f4a480069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.994+0000 7f4a61d09640 1 -- 172.21.15.16:0/2263626623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a48005730 con 0x7f4a48006f10 2023-11-13T10:56:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.995+0000 7f4a61d09640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a48006f10 0x7f4a48007390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a54001f60 tx=0x7f4a48007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad56b4a77988036 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.995+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5402d6f0 con 0x7f4a48006f10 2023-11-13T10:56:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.995+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5402d870 con 0x7f4a48006f10 2023-11-13T10:56:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.995+0000 7f4a53fff640 1 -- 172.21.15.16:0/2263626623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5402dc60 con 0x7f4a48006f10 2023-11-13T10:56:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.996+0000 7f4a6376c640 1 -- 172.21.15.16:0/2263626623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a48006f10 msgr2=0x7f4a48007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.996+0000 7f4a6376c640 1 --2- 172.21.15.16:0/2263626623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a48006f10 0x7f4a48007390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a54001f60 tx=0x7f4a48007a00 comp rx=0 tx=0).stop 2023-11-13T10:56:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.996+0000 7f4a61d09640 1 -- 172.21.15.16:0/2263626623 reap_dead start 2023-11-13T10:56:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.996+0000 7f4a6376c640 1 -- 172.21.15.16:0/2263626623 shutdown_connections 2023-11-13T10:56:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.996+0000 7f4a6376c640 1 -- 172.21.15.16:0/2263626623 >> 172.21.15.16:0/2263626623 conn(0x7f4a5c06a190 msgr2=0x7f4a5c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.996+0000 7f4a6376c640 1 -- 172.21.15.16:0/2263626623 shutdown_connections 2023-11-13T10:56:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.996+0000 7f4a6376c640 1 -- 172.21.15.16:0/2263626623 wait complete. 2023-11-13T10:56:33.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.998+0000 7f4a6376c640 1 Processor -- start 2023-11-13T10:56:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.998+0000 7f4a6376c640 1 -- start start 2023-11-13T10:56:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.999+0000 7f4a6376c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a48006f10 0x7f4a5c154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.999+0000 7f4a6376c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c1484b0 0x7f4a5c0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.999+0000 7f4a6376c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0cd980 0x7f4a5c1550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.999+0000 7f4a6376c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c153f50 con 0x7f4a5c1484b0 2023-11-13T10:56:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.999+0000 7f4a6376c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c14f890 con 0x7f4a48006f10 2023-11-13T10:56:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.999+0000 7f4a6376c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c0cde00 con 0x7f4a5c0cd980 2023-11-13T10:56:33.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.999+0000 7f4a60d07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c1484b0 0x7f4a5c0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.999+0000 7f4a60d07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c1484b0 0x7f4a5c0cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58802/0 (socket says 172.21.15.16:58802) 2023-11-13T10:56:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:32.999+0000 7f4a60d07640 1 -- 172.21.15.16:0/3150552177 learned_addr learned my addr 172.21.15.16:0/3150552177 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a61d09640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0cd980 0x7f4a5c1550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a61508640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a48006f10 0x7f4a5c154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a60d07640 1 -- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0cd980 msgr2=0x7f4a5c1550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a60d07640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0cd980 0x7f4a5c1550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a60d07640 1 -- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a48006f10 msgr2=0x7f4a5c154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a60d07640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a48006f10 0x7f4a5c154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a60d07640 1 -- 172.21.15.16:0/3150552177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c1548b0 con 0x7f4a5c1484b0 2023-11-13T10:56:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a61d09640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0cd980 0x7f4a5c1550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a60d07640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c1484b0 0x7f4a5c0cd440 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00e800 tx=0x7f4a4c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.000+0000 7f4a51ffb640 1 -- 172.21.15.16:0/3150552177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00c8d0 con 0x7f4a5c1484b0 2023-11-13T10:56:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.001+0000 7f4a51ffb640 1 -- 172.21.15.16:0/3150552177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4c00ca30 con 0x7f4a5c1484b0 2023-11-13T10:56:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.001+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c1535b0 con 0x7f4a5c1484b0 2023-11-13T10:56:33.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.001+0000 7f4a51ffb640 1 -- 172.21.15.16:0/3150552177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00cde0 con 0x7f4a5c1484b0 2023-11-13T10:56:33.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.001+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c153b70 con 0x7f4a5c1484b0 2023-11-13T10:56:33.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.002+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5c072520 con 0x7f4a5c1484b0 2023-11-13T10:56:33.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.003+0000 7f4a51ffb640 1 -- 172.21.15.16:0/3150552177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a4c01d460 con 0x7f4a5c1484b0 2023-11-13T10:56:33.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.004+0000 7f4a51ffb640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4a4006ae40 0x7f4a4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.004+0000 7f4a51ffb640 1 -- 172.21.15.16:0/3150552177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f4a4c089a50 con 0x7f4a5c1484b0 2023-11-13T10:56:33.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.005+0000 7f4a61508640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4a4006ae40 0x7f4a4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.006+0000 7f4a61508640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4a4006ae40 0x7f4a4006d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009950 tx=0x7f4a58008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:33.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.008+0000 7f4a51ffb640 1 -- 172.21.15.16:0/3150552177 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a4c054720 con 0x7f4a5c1484b0 2023-11-13T10:56:33.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.201+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a5c04e5b0 con 0x7f4a5c1484b0 2023-11-13T10:56:33.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.202+0000 7f4a51ffb640 1 -- 172.21.15.16:0/3150552177 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a4c057d40 con 0x7f4a5c1484b0 2023-11-13T10:56:33.206 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4a4006ae40 msgr2=0x7f4a4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4a4006ae40 0x7f4a4006d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009950 tx=0x7f4a58008040 comp rx=0 tx=0).stop 2023-11-13T10:56:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c1484b0 msgr2=0x7f4a5c0cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c1484b0 0x7f4a5c0cd440 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00e800 tx=0x7f4a4c00ecc0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 shutdown_connections 2023-11-13T10:56:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4a4006ae40 0x7f4a4006d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a5c0cd980 0x7f4a5c1550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c1484b0 0x7f4a5c0cd440 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 --2- 172.21.15.16:0/3150552177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a48006f10 0x7f4a5c154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 >> 172.21.15.16:0/3150552177 conn(0x7f4a5c06a190 msgr2=0x7f4a5c159700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 shutdown_connections 2023-11-13T10:56:33.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.204+0000 7f4a6376c640 1 -- 172.21.15.16:0/3150552177 wait complete. 2023-11-13T10:56:33.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.455+0000 7fbed89b6640 1 Processor -- start 2023-11-13T10:56:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.455+0000 7fbed89b6640 1 -- start start 2023-11-13T10:56:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.457+0000 7fbed89b6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbed406bcb0 0x7fbed406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.457+0000 7fbed89b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40b2180 0x7fbed40cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.457+0000 7fbed89b6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed40d0220 con 0x7fbed40d1600 2023-11-13T10:56:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.457+0000 7fbed89b6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed406c5d0 con 0x7fbed406bcb0 2023-11-13T10:56:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.457+0000 7fbed89b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed40cc140 con 0x7fbed40b2180 2023-11-13T10:56:33.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.457+0000 7fbed259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbed406bcb0 0x7fbed406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.457+0000 7fbed259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbed406bcb0 0x7fbed406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58804/0 (socket says 172.21.15.16:58804) 2023-11-13T10:56:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.457+0000 7fbed259c640 1 -- 172.21.15.16:0/2807434582 learned_addr learned my addr 172.21.15.16:0/2807434582 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.458+0000 7fbed2d9d640 1 --2- 172.21.15.16:0/2807434582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40b2180 0x7fbed40cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.458+0000 7fbed259c640 1 -- 172.21.15.16:0/2807434582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40b2180 msgr2=0x7fbed40cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.458+0000 7fbed259c640 1 --2- 172.21.15.16:0/2807434582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40b2180 0x7fbed40cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.458+0000 7fbed259c640 1 -- 172.21.15.16:0/2807434582 >> v1:172.21.15.184:6789/0 conn(0x7fbed40d1600 legacy=0x7fbed40cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.458+0000 7fbed259c640 1 -- 172.21.15.16:0/2807434582 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed40b1780 con 0x7fbed406bcb0 2023-11-13T10:56:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.459+0000 7fbed259c640 1 --2- 172.21.15.16:0/2807434582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 0x7fbed406c090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009870 tx=0x7fbec402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148aeacb064e1667 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:33.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed159a640 1 -- 172.21.15.16:0/2807434582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4031640 con 0x7fbed406bcb0 2023-11-13T10:56:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed159a640 1 -- 172.21.15.16:0/2807434582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec40317a0 con 0x7fbed406bcb0 2023-11-13T10:56:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed159a640 1 -- 172.21.15.16:0/2807434582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4031b30 con 0x7fbed406bcb0 2023-11-13T10:56:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed89b6640 1 -- 172.21.15.16:0/2807434582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 msgr2=0x7fbed406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed89b6640 1 --2- 172.21.15.16:0/2807434582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 0x7fbed406c090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009870 tx=0x7fbec402b8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed89b6640 1 -- 172.21.15.16:0/2807434582 shutdown_connections 2023-11-13T10:56:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed89b6640 1 --2- 172.21.15.16:0/2807434582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40b2180 0x7fbed40cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed89b6640 1 --2- 172.21.15.16:0/2807434582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 0x7fbed406c090 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed89b6640 1 -- 172.21.15.16:0/2807434582 >> 172.21.15.16:0/2807434582 conn(0x7fbed40d0ae0 msgr2=0x7fbed40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:33.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.460+0000 7fbed89b6640 1 -- 172.21.15.16:0/2807434582 shutdown_connections 2023-11-13T10:56:33.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.461+0000 7fbed89b6640 1 -- 172.21.15.16:0/2807434582 wait complete. 2023-11-13T10:56:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.462+0000 7fbed89b6640 1 Processor -- start 2023-11-13T10:56:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.463+0000 7fbed89b6640 1 -- start start 2023-11-13T10:56:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.463+0000 7fbed89b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 0x7fbed4159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed89b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbed40b2180 0x7fbed40b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed89b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40d1600 0x7fbed41594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed89b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed40b3be0 con 0x7fbed406bcb0 2023-11-13T10:56:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed89b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed40b3ec0 con 0x7fbed40b2180 2023-11-13T10:56:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed89b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed40b41a0 con 0x7fbed40d1600 2023-11-13T10:56:33.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 0x7fbed4159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 0x7fbed4159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58818/0 (socket says 172.21.15.16:58818) 2023-11-13T10:56:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed259c640 1 -- 172.21.15.16:0/2877534674 learned_addr learned my addr 172.21.15.16:0/2877534674 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed1d9b640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbed40b2180 0x7fbed40b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed2d9d640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40d1600 0x7fbed41594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed259c640 1 -- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40d1600 msgr2=0x7fbed41594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.464+0000 7fbed259c640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40d1600 0x7fbed41594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.465+0000 7fbed259c640 1 -- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbed40b2180 msgr2=0x7fbed40b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.465+0000 7fbed259c640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbed40b2180 0x7fbed40b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.465+0000 7fbed259c640 1 -- 172.21.15.16:0/2877534674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec40094d0 con 0x7fbed406bcb0 2023-11-13T10:56:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.465+0000 7fbed259c640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 0x7fbed4159cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbec4000c00 tx=0x7fbec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.465+0000 7fbec2ffd640 1 -- 172.21.15.16:0/2877534674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec403ac60 con 0x7fbed406bcb0 2023-11-13T10:56:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.465+0000 7fbec2ffd640 1 -- 172.21.15.16:0/2877534674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec403ade0 con 0x7fbed406bcb0 2023-11-13T10:56:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.465+0000 7fbec2ffd640 1 -- 172.21.15.16:0/2877534674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4044630 con 0x7fbed406bcb0 2023-11-13T10:56:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.465+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed40b4480 con 0x7fbed406bcb0 2023-11-13T10:56:33.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.465+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed4159a00 con 0x7fbed406bcb0 2023-11-13T10:56:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.467+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed409eb10 con 0x7fbed406bcb0 2023-11-13T10:56:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.468+0000 7fbec2ffd640 1 -- 172.21.15.16:0/2877534674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbec402f050 con 0x7fbed406bcb0 2023-11-13T10:56:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.469+0000 7fbec2ffd640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbeb406ae40 0x7fbeb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.469+0000 7fbec2ffd640 1 -- 172.21.15.16:0/2877534674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fbec40aba90 con 0x7fbed406bcb0 2023-11-13T10:56:33.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.469+0000 7fbed1d9b640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbeb406ae40 0x7fbeb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.471+0000 7fbed1d9b640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbeb406ae40 0x7fbeb406d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbed415dca0 tx=0x7fbec8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:33.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.473+0000 7fbec2ffd640 1 -- 172.21.15.16:0/2877534674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbec40767e0 con 0x7fbed406bcb0 2023-11-13T10:56:33.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.667+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbed404e5b0 con 0x7fbed406bcb0 2023-11-13T10:56:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.668+0000 7fbec2ffd640 1 -- 172.21.15.16:0/2877534674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbec40350a0 con 0x7fbed406bcb0 2023-11-13T10:56:33.673 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:33.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbeb406ae40 msgr2=0x7fbeb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbeb406ae40 0x7fbeb406d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbed415dca0 tx=0x7fbec8009040 comp rx=0 tx=0).stop 2023-11-13T10:56:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 msgr2=0x7fbed4159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 0x7fbed4159cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbec4000c00 tx=0x7fbec40046c0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 shutdown_connections 2023-11-13T10:56:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbeb406ae40 0x7fbeb406d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbed40d1600 0x7fbed41594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbed40b2180 0x7fbed40b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 --2- 172.21.15.16:0/2877534674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed406bcb0 0x7fbed4159cc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 >> 172.21.15.16:0/2877534674 conn(0x7fbed40d0ae0 msgr2=0x7fbed40cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 shutdown_connections 2023-11-13T10:56:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.671+0000 7fbed89b6640 1 -- 172.21.15.16:0/2877534674 wait complete. 2023-11-13T10:56:33.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.929+0000 7fd1c759e640 1 Processor -- start 2023-11-13T10:56:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.929+0000 7fd1c759e640 1 -- start start 2023-11-13T10:56:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.930+0000 7fd1c759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1c815ce70 0x7fd1c80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.930+0000 7fd1c759e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd1c80cdbb0 0x7fd1c80d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c80cac40 con 0x7fd1c80d1850 2023-11-13T10:56:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c80ce0e0 con 0x7fd1c815ce70 2023-11-13T10:56:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c759e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c80ce3c0 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1c815ce70 0x7fd1c80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1c815ce70 0x7fd1c80cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58832/0 (socket says 172.21.15.16:58832) 2023-11-13T10:56:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c659c640 1 -- 172.21.15.16:0/2605178519 learned_addr learned my addr 172.21.15.16:0/2605178519 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c6d9d640 1 --1- 172.21.15.16:0/2605178519 >> v1:172.21.15.16:6789/0 conn(0x7fd1c80d1850 0x7fd1c80cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46936/0 (socket says 172.21.15.16:46936) 2023-11-13T10:56:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c5d9b640 1 --2- 172.21.15.16:0/2605178519 >> v2:172.21.15.184:3300/0 conn(0x7fd1c80cdbb0 0x7fd1c80d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c659c640 1 -- 172.21.15.16:0/2605178519 >> v2:172.21.15.184:3300/0 conn(0x7fd1c80cdbb0 msgr2=0x7fd1c80d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c659c640 1 --2- 172.21.15.16:0/2605178519 >> v2:172.21.15.184:3300/0 conn(0x7fd1c80cdbb0 0x7fd1c80d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c659c640 1 -- 172.21.15.16:0/2605178519 >> v1:172.21.15.16:6789/0 conn(0x7fd1c80d1850 legacy=0x7fd1c80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.931+0000 7fd1c659c640 1 -- 172.21.15.16:0/2605178519 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80ce6a0 con 0x7fd1c815ce70 2023-11-13T10:56:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.933+0000 7fd1c659c640 1 --2- 172.21.15.16:0/2605178519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815ce70 0x7fd1c80cd670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80d0680 tx=0x7fd1b802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d947d9611304e867 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.933+0000 7fd1c559a640 1 -- 172.21.15.16:0/2605178519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b802cb10 con 0x7fd1c815ce70 2023-11-13T10:56:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.933+0000 7fd1c559a640 1 -- 172.21.15.16:0/2605178519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b802cc70 con 0x7fd1c815ce70 2023-11-13T10:56:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.933+0000 7fd1c559a640 1 -- 172.21.15.16:0/2605178519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80343f0 con 0x7fd1c815ce70 2023-11-13T10:56:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.933+0000 7fd1c759e640 1 -- 172.21.15.16:0/2605178519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815ce70 msgr2=0x7fd1c80cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.933+0000 7fd1c759e640 1 --2- 172.21.15.16:0/2605178519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815ce70 0x7fd1c80cd670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80d0680 tx=0x7fd1b802bf70 comp rx=0 tx=0).stop 2023-11-13T10:56:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.934+0000 7fd1c759e640 1 -- 172.21.15.16:0/2605178519 shutdown_connections 2023-11-13T10:56:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.934+0000 7fd1c759e640 1 --2- 172.21.15.16:0/2605178519 >> v2:172.21.15.184:3300/0 conn(0x7fd1c80cdbb0 0x7fd1c80d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.934+0000 7fd1c759e640 1 --2- 172.21.15.16:0/2605178519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815ce70 0x7fd1c80cd670 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.934+0000 7fd1c759e640 1 -- 172.21.15.16:0/2605178519 >> 172.21.15.16:0/2605178519 conn(0x7fd1c808c580 msgr2=0x7fd1c809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.934+0000 7fd1c759e640 1 -- 172.21.15.16:0/2605178519 shutdown_connections 2023-11-13T10:56:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.934+0000 7fd1c759e640 1 -- 172.21.15.16:0/2605178519 wait complete. 2023-11-13T10:56:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.935+0000 7fd1c759e640 1 Processor -- start 2023-11-13T10:56:33.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.936+0000 7fd1c759e640 1 -- start start 2023-11-13T10:56:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80cdbb0 0x7fd1c81e6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c759e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c80d1850 0x7fd1c81e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c815ce70 0x7fd1c80b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c81f5f30 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c759e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c81f61d0 con 0x7fd1c80d1850 2023-11-13T10:56:33.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c81f64b0 con 0x7fd1c815ce70 2023-11-13T10:56:33.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80cdbb0 0x7fd1c81e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80cdbb0 0x7fd1c81e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58840/0 (socket says 172.21.15.16:58840) 2023-11-13T10:56:33.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c659c640 1 -- 172.21.15.16:0/4107955723 learned_addr learned my addr 172.21.15.16:0/4107955723 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:33.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c659c640 1 -- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c815ce70 msgr2=0x7fd1c80b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c659c640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c815ce70 0x7fd1c80b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c659c640 1 -- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c80d1850 msgr2=0x7fd1c81e6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:33.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c5d9b640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c80d1850 0x7fd1c81e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c659c640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c80d1850 0x7fd1c81e6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:33.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.937+0000 7fd1c659c640 1 -- 172.21.15.16:0/4107955723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8009b30 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.938+0000 7fd1c659c640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80cdbb0 0x7fd1c81e6590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8000f80 tx=0x7fd1b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:33.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.938+0000 7fd1beffd640 1 -- 172.21.15.16:0/4107955723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8009510 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.938+0000 7fd1beffd640 1 -- 172.21.15.16:0/4107955723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b803e450 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.938+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c81f6790 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.938+0000 7fd1beffd640 1 -- 172.21.15.16:0/4107955723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b803e800 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.938+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c815c320 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.940+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c81f6c50 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.941+0000 7fd1beffd640 1 -- 172.21.15.16:0/4107955723 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1b803e960 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.941+0000 7fd1beffd640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae70 0x7fd1a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.942+0000 7fd1beffd640 1 -- 172.21.15.16:0/4107955723 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fd1b80ac7d0 con 0x7fd1c80cdbb0 2023-11-13T10:56:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.942+0000 7fd1c5d9b640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae70 0x7fd1a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.943+0000 7fd1c5d9b640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae70 0x7fd1a806d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0006fd0 tx=0x7fd1b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:33.946+0000 7fd1beffd640 1 -- 172.21.15.16:0/4107955723 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1b80774a0 con 0x7fd1c80cdbb0 2023-11-13T10:56:34.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.143+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd1c804e4d0 con 0x7fd1c80cdbb0 2023-11-13T10:56:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.145+0000 7fd1beffd640 1 -- 172.21.15.16:0/4107955723 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd1b807aac0 con 0x7fd1c80cdbb0 2023-11-13T10:56:34.149 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.147+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae70 msgr2=0x7fd1a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.147+0000 7fd1c759e640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae70 0x7fd1a806d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0006fd0 tx=0x7fd1b0008040 comp rx=0 tx=0).stop 2023-11-13T10:56:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.147+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80cdbb0 msgr2=0x7fd1c81e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.147+0000 7fd1c759e640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80cdbb0 0x7fd1c81e6590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8000f80 tx=0x7fd1b8004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:34.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.148+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 shutdown_connections 2023-11-13T10:56:34.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.148+0000 7fd1c759e640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1a806ae70 0x7fd1a806d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.148+0000 7fd1c759e640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1c815ce70 0x7fd1c80b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.148+0000 7fd1c759e640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1c80d1850 0x7fd1c81e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.148+0000 7fd1c759e640 1 --2- 172.21.15.16:0/4107955723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80cdbb0 0x7fd1c81e6590 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.148+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 >> 172.21.15.16:0/4107955723 conn(0x7fd1c808c580 msgr2=0x7fd1c80cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.148+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 shutdown_connections 2023-11-13T10:56:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.148+0000 7fd1c759e640 1 -- 172.21.15.16:0/4107955723 wait complete. 2023-11-13T10:56:34.191 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2023-11-13T10:56:34.192 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-13T10:56:34.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.427+0000 7f41e5187640 1 Processor -- start 2023-11-13T10:56:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.427+0000 7f41e5187640 1 -- start start 2023-11-13T10:56:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.429+0000 7f41e5187640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41e00cb190 0x7f41e015b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.429+0000 7f41e5187640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 0x7f41e006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.429+0000 7f41e5187640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f41e006a190 0x7f41e006a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.429+0000 7f41e5187640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e0158c20 con 0x7f41e00cb190 2023-11-13T10:56:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.429+0000 7f41e5187640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e00b1830 con 0x7f41e006a190 2023-11-13T10:56:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.429+0000 7f41e5187640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e006ab10 con 0x7f41e00716b0 2023-11-13T10:56:34.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.429+0000 7f41dffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41e00cb190 0x7f41e015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:34.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41dffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41e00cb190 0x7f41e015b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58846/0 (socket says 172.21.15.16:58846) 2023-11-13T10:56:34.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41dffff640 1 -- 172.21.15.16:0/3602749202 learned_addr learned my addr 172.21.15.16:0/3602749202 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:34.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41df7fe640 1 --2- 172.21.15.16:0/3602749202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 0x7f41e006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:34.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41e4986640 1 --2- 172.21.15.16:0/3602749202 >> v2:172.21.15.184:3300/0 conn(0x7f41e006a190 0x7f41e006a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:34.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41dffff640 1 -- 172.21.15.16:0/3602749202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 msgr2=0x7f41e006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:34.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41dffff640 1 --2- 172.21.15.16:0/3602749202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 0x7f41e006f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41dffff640 1 -- 172.21.15.16:0/3602749202 >> v2:172.21.15.184:3300/0 conn(0x7f41e006a190 msgr2=0x7f41e006a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:34.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41dffff640 1 --2- 172.21.15.16:0/3602749202 >> v2:172.21.15.184:3300/0 conn(0x7f41e006a190 0x7f41e006a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41dffff640 1 -- 172.21.15.16:0/3602749202 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e015b780 con 0x7f41e00cb190 2023-11-13T10:56:34.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.430+0000 7f41e4986640 1 --2- 172.21.15.16:0/3602749202 >> v2:172.21.15.184:3300/0 conn(0x7f41e006a190 0x7f41e006a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:34.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.432+0000 7f41dffff640 1 --2- 172.21.15.16:0/3602749202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 0x7f41e015b190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f41e006dc70 tx=0x7f41d002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dee9327bc456883 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:34.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.432+0000 7f41deffd640 1 -- 172.21.15.16:0/3602749202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0031860 con 0x7f41e00cb190 2023-11-13T10:56:34.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.432+0000 7f41deffd640 1 -- 172.21.15.16:0/3602749202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d00319c0 con 0x7f41e00cb190 2023-11-13T10:56:34.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.432+0000 7f41deffd640 1 -- 172.21.15.16:0/3602749202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0031d50 con 0x7f41e00cb190 2023-11-13T10:56:34.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.433+0000 7f41e5187640 1 -- 172.21.15.16:0/3602749202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 msgr2=0x7f41e015b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.433+0000 7f41e5187640 1 --2- 172.21.15.16:0/3602749202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 0x7f41e015b190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f41e006dc70 tx=0x7f41d002b900 comp rx=0 tx=0).stop 2023-11-13T10:56:34.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.433+0000 7f41e5187640 1 -- 172.21.15.16:0/3602749202 shutdown_connections 2023-11-13T10:56:34.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.433+0000 7f41e5187640 1 --2- 172.21.15.16:0/3602749202 >> v2:172.21.15.184:3300/0 conn(0x7f41e006a190 0x7f41e006a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.433+0000 7f41e5187640 1 --2- 172.21.15.16:0/3602749202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 0x7f41e006f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.433+0000 7f41e5187640 1 --2- 172.21.15.16:0/3602749202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 0x7f41e015b190 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.433+0000 7f41e5187640 1 -- 172.21.15.16:0/3602749202 >> 172.21.15.16:0/3602749202 conn(0x7f41e00b04a0 msgr2=0x7f41e00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.433+0000 7f41e5187640 1 -- 172.21.15.16:0/3602749202 shutdown_connections 2023-11-13T10:56:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.433+0000 7f41e5187640 1 -- 172.21.15.16:0/3602749202 wait complete. 2023-11-13T10:56:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.435+0000 7f41e5187640 1 Processor -- start 2023-11-13T10:56:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.435+0000 7f41e5187640 1 -- start start 2023-11-13T10:56:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.436+0000 7f41e5187640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e006a190 0x7f41e0144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.436+0000 7f41e5187640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 0x7f41e00c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.436+0000 7f41e5187640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 0x7f41e0159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.436+0000 7f41e5187640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0145330 con 0x7f41e00cb190 2023-11-13T10:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.436+0000 7f41e5187640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0147880 con 0x7f41e006a190 2023-11-13T10:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.436+0000 7f41e5187640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0147b60 con 0x7f41e00716b0 2023-11-13T10:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.436+0000 7f41e4986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 0x7f41e0159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.436+0000 7f41e4986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 0x7f41e0159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58852/0 (socket says 172.21.15.16:58852) 2023-11-13T10:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.436+0000 7f41e4986640 1 -- 172.21.15.16:0/640579090 learned_addr learned my addr 172.21.15.16:0/640579090 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.437+0000 7f41df7fe640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 0x7f41e00c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.437+0000 7f41e4986640 1 -- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 msgr2=0x7f41e00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.437+0000 7f41e4986640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 0x7f41e00c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.437+0000 7f41e4986640 1 -- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e006a190 msgr2=0x7f41e0144df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.437+0000 7f41e4986640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e006a190 0x7f41e0144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.437+0000 7f41e4986640 1 -- 172.21.15.16:0/640579090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d00094d0 con 0x7f41e00cb190 2023-11-13T10:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.437+0000 7f41df7fe640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 0x7f41e00c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.437+0000 7f41e4986640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 0x7f41e0159fc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41d800cef0 tx=0x7f41d800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.438+0000 7f41dcff9640 1 -- 172.21.15.16:0/640579090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d800d840 con 0x7f41e00cb190 2023-11-13T10:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.438+0000 7f41dcff9640 1 -- 172.21.15.16:0/640579090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d800d9a0 con 0x7f41e00cb190 2023-11-13T10:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.438+0000 7f41dcff9640 1 -- 172.21.15.16:0/640579090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d800dd50 con 0x7f41e00cb190 2023-11-13T10:56:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.438+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e0147e40 con 0x7f41e00cb190 2023-11-13T10:56:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.438+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e0147490 con 0x7f41e00cb190 2023-11-13T10:56:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.439+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e0148360 con 0x7f41e00cb190 2023-11-13T10:56:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.440+0000 7f41dcff9640 1 -- 172.21.15.16:0/640579090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f41d801e3b0 con 0x7f41e00cb190 2023-11-13T10:56:34.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.441+0000 7f41dcff9640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f41cc06ae40 0x7f41cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.441+0000 7f41dcff9640 1 -- 172.21.15.16:0/640579090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f41d808a3c0 con 0x7f41e00cb190 2023-11-13T10:56:34.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.442+0000 7f41dffff640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f41cc06ae40 0x7f41cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:34.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.443+0000 7f41dffff640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f41cc06ae40 0x7f41cc06d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41d0036000 tx=0x7f41d0030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:34.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.446+0000 7f41dcff9640 1 -- 172.21.15.16:0/640579090 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41e0148360 con 0x7f41e00cb190 2023-11-13T10:56:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.640+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41e0148360 con 0x7f41e00cb190 2023-11-13T10:56:34.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.643+0000 7f41dcff9640 1 -- 172.21.15.16:0/640579090 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f41e0148360 con 0x7f41e00cb190 2023-11-13T10:56:34.650 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:34.650 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f41cc06ae40 msgr2=0x7f41cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f41cc06ae40 0x7f41cc06d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41d0036000 tx=0x7f41d0030000 comp rx=0 tx=0).stop 2023-11-13T10:56:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 msgr2=0x7f41e0159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 0x7f41e0159fc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41d800cef0 tx=0x7f41d800a710 comp rx=0 tx=0).stop 2023-11-13T10:56:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 shutdown_connections 2023-11-13T10:56:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f41cc06ae40 0x7f41cc06d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e00cb190 0x7f41e0159fc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41e00716b0 0x7f41e00c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 --2- 172.21.15.16:0/640579090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41e006a190 0x7f41e0144df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 >> 172.21.15.16:0/640579090 conn(0x7f41e00b04a0 msgr2=0x7f41e00b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:34.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 shutdown_connections 2023-11-13T10:56:34.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.646+0000 7f41e5187640 1 -- 172.21.15.16:0/640579090 wait complete. 2023-11-13T10:56:34.672 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:34.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.917+0000 7f173dfe9640 1 Processor -- start 2023-11-13T10:56:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.917+0000 7f173dfe9640 1 -- start start 2023-11-13T10:56:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.919+0000 7f173dfe9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1738159bd0 0x7f173806f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.919+0000 7f173dfe9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f17380b2290 0x7f17380ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.919+0000 7f173dfe9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17380ccab0 con 0x7f17380b2720 2023-11-13T10:56:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.919+0000 7f173dfe9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17380ccd30 con 0x7f1738159bd0 2023-11-13T10:56:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.919+0000 7f173dfe9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17380cd010 con 0x7f17380b2290 2023-11-13T10:56:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.920+0000 7f173cfe7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1738159bd0 0x7f173806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.920+0000 7f173cfe7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1738159bd0 0x7f173806f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58856/0 (socket says 172.21.15.16:58856) 2023-11-13T10:56:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.920+0000 7f173cfe7640 1 -- 172.21.15.16:0/2280354818 learned_addr learned my addr 172.21.15.16:0/2280354818 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.920+0000 7f173cfe7640 1 -- 172.21.15.16:0/2280354818 >> v2:172.21.15.184:3300/0 conn(0x7f17380b2290 msgr2=0x7f17380ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.920+0000 7f173cfe7640 1 --2- 172.21.15.16:0/2280354818 >> v2:172.21.15.184:3300/0 conn(0x7f17380b2290 0x7f17380ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.920+0000 7f173cfe7640 1 -- 172.21.15.16:0/2280354818 >> v1:172.21.15.184:6789/0 conn(0x7f17380b2720 legacy=0x7f17380cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.920+0000 7f173cfe7640 1 -- 172.21.15.16:0/2280354818 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17380cd2f0 con 0x7f1738159bd0 2023-11-13T10:56:34.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.922+0000 7f173cfe7640 1 --2- 172.21.15.16:0/2280354818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1738159bd0 0x7f173806f940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f17380cbca0 tx=0x7f172c02da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c5e8c8f1557345 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:34.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.923+0000 7f17377fe640 1 -- 172.21.15.16:0/2280354818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c02f850 con 0x7f1738159bd0 2023-11-13T10:56:34.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.923+0000 7f17377fe640 1 -- 172.21.15.16:0/2280354818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c02f9b0 con 0x7f1738159bd0 2023-11-13T10:56:34.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.923+0000 7f17377fe640 1 -- 172.21.15.16:0/2280354818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c02fd40 con 0x7f1738159bd0 2023-11-13T10:56:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.924+0000 7f173dfe9640 1 -- 172.21.15.16:0/2280354818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1738159bd0 msgr2=0x7f173806f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.924+0000 7f173dfe9640 1 --2- 172.21.15.16:0/2280354818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1738159bd0 0x7f173806f940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f17380cbca0 tx=0x7f172c02da90 comp rx=0 tx=0).stop 2023-11-13T10:56:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.924+0000 7f173dfe9640 1 -- 172.21.15.16:0/2280354818 shutdown_connections 2023-11-13T10:56:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.924+0000 7f173dfe9640 1 --2- 172.21.15.16:0/2280354818 >> v2:172.21.15.184:3300/0 conn(0x7f17380b2290 0x7f17380ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.924+0000 7f173dfe9640 1 --2- 172.21.15.16:0/2280354818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1738159bd0 0x7f173806f940 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.924+0000 7f173dfe9640 1 -- 172.21.15.16:0/2280354818 >> 172.21.15.16:0/2280354818 conn(0x7f17380b0970 msgr2=0x7f17380b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:34.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.924+0000 7f173dfe9640 1 -- 172.21.15.16:0/2280354818 shutdown_connections 2023-11-13T10:56:34.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.924+0000 7f173dfe9640 1 -- 172.21.15.16:0/2280354818 wait complete. 2023-11-13T10:56:34.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.926+0000 7f173dfe9640 1 Processor -- start 2023-11-13T10:56:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.926+0000 7f173dfe9640 1 -- start start 2023-11-13T10:56:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.927+0000 7f173dfe9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17380b2290 0x7f1738153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.927+0000 7f173dfe9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17380b2720 0x7f17381f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.927+0000 7f173dfe9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17380ac8e0 0x7f17380acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.927+0000 7f173dfe9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1738157920 con 0x7f17380b2720 2023-11-13T10:56:34.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.927+0000 7f173dfe9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17380ad2a0 con 0x7f17380ac8e0 2023-11-13T10:56:34.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.927+0000 7f173dfe9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1738158510 con 0x7f17380b2290 2023-11-13T10:56:34.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.928+0000 7f1737fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17380b2720 0x7f17381f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:34.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.928+0000 7f1737fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17380b2720 0x7f17381f58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58864/0 (socket says 172.21.15.16:58864) 2023-11-13T10:56:34.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.928+0000 7f1737fff640 1 -- 172.21.15.16:0/945439178 learned_addr learned my addr 172.21.15.16:0/945439178 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:34.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.928+0000 7f1737fff640 1 -- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17380b2290 msgr2=0x7f1738153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.928+0000 7f1737fff640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17380b2290 0x7f1738153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.928+0000 7f1737fff640 1 -- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17380ac8e0 msgr2=0x7f17380acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:34.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.928+0000 7f1737fff640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17380ac8e0 0x7f17380acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:34.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.928+0000 7f1737fff640 1 -- 172.21.15.16:0/945439178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c0094d0 con 0x7f17380b2720 2023-11-13T10:56:34.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.929+0000 7f1737fff640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17380b2720 0x7f17381f58b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f172400b4a0 tx=0x7f172400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:34.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.929+0000 7f17357fa640 1 -- 172.21.15.16:0/945439178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724010610 con 0x7f17380b2720 2023-11-13T10:56:34.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.929+0000 7f17357fa640 1 -- 172.21.15.16:0/945439178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1724010770 con 0x7f17380b2720 2023-11-13T10:56:34.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.929+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1738156ca0 con 0x7f17380b2720 2023-11-13T10:56:34.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.929+0000 7f17357fa640 1 -- 172.21.15.16:0/945439178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724010b20 con 0x7f17380b2720 2023-11-13T10:56:34.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.929+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1738157260 con 0x7f17380b2720 2023-11-13T10:56:34.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.930+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17380675b0 con 0x7f17380b2720 2023-11-13T10:56:34.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.931+0000 7f17357fa640 1 -- 172.21.15.16:0/945439178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1724010c80 con 0x7f17380b2720 2023-11-13T10:56:34.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.933+0000 7f17357fa640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f171c06ae40 0x7f171c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.933+0000 7f17357fa640 1 -- 172.21.15.16:0/945439178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f17240895c0 con 0x7f17380b2720 2023-11-13T10:56:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.934+0000 7f173cfe7640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f171c06ae40 0x7f171c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.935+0000 7f173cfe7640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f171c06ae40 0x7f171c06d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f172c038000 tx=0x7f172c032000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:34.937+0000 7f17357fa640 1 -- 172.21.15.16:0/945439178 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1724054290 con 0x7f17380b2720 2023-11-13T10:56:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.131+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f173804e4d0 con 0x7f17380b2720 2023-11-13T10:56:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.131+0000 7f17357fa640 1 -- 172.21.15.16:0/945439178 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17240578b0 con 0x7f17380b2720 2023-11-13T10:56:35.137 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:35.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f171c06ae40 msgr2=0x7f171c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f171c06ae40 0x7f171c06d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f172c038000 tx=0x7f172c032000 comp rx=0 tx=0).stop 2023-11-13T10:56:35.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17380b2720 msgr2=0x7f17381f58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17380b2720 0x7f17381f58b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f172400b4a0 tx=0x7f172400b960 comp rx=0 tx=0).stop 2023-11-13T10:56:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 shutdown_connections 2023-11-13T10:56:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f171c06ae40 0x7f171c06d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17380ac8e0 0x7f17380acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17380b2720 0x7f17381f58b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 --2- 172.21.15.16:0/945439178 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17380b2290 0x7f1738153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 >> 172.21.15.16:0/945439178 conn(0x7f17380b0970 msgr2=0x7f17380cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:35.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 shutdown_connections 2023-11-13T10:56:35.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.134+0000 7f173dfe9640 1 -- 172.21.15.16:0/945439178 wait complete. 2023-11-13T10:56:35.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.377+0000 7fdb36c7c640 1 Processor -- start 2023-11-13T10:56:35.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.378+0000 7fdb36c7c640 1 -- start start 2023-11-13T10:56:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.379+0000 7fdb36c7c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb300ce550 0x7fdb300ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.379+0000 7fdb36c7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb300b19d0 0x7fdb300b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.379+0000 7fdb36c7c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb300ceef0 con 0x7fdb30144570 2023-11-13T10:56:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.379+0000 7fdb36c7c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb30144950 con 0x7fdb300b19d0 2023-11-13T10:56:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.379+0000 7fdb36c7c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300ce300 con 0x7fdb300ce550 2023-11-13T10:56:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.380+0000 7fdb34a18640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdb30144570 0x7fdb3015f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46948/0 (socket says 172.21.15.16:46948) 2023-11-13T10:56:35.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.380+0000 7fdb34a18640 1 -- 172.21.15.16:0/1289983114 learned_addr learned my addr 172.21.15.16:0/1289983114 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:35.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.380+0000 7fdb35219640 1 --2- 172.21.15.16:0/1289983114 >> v2:172.21.15.16:3300/0 conn(0x7fdb300b19d0 0x7fdb300b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.380+0000 7fdb2ffff640 1 --2- 172.21.15.16:0/1289983114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb300ce550 0x7fdb300ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.380+0000 7fdb2f7fe640 1 -- 172.21.15.16:0/1289983114 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853604419 0 0) 0x7fdb300ceef0 con 0x7fdb30144570 2023-11-13T10:56:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.380+0000 7fdb35219640 1 -- 172.21.15.16:0/1289983114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb300ce550 msgr2=0x7fdb300ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.380+0000 7fdb35219640 1 --2- 172.21.15.16:0/1289983114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb300ce550 0x7fdb300ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.380+0000 7fdb35219640 1 -- 172.21.15.16:0/1289983114 >> v1:172.21.15.16:6789/0 conn(0x7fdb30144570 legacy=0x7fdb3015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.380+0000 7fdb35219640 1 -- 172.21.15.16:0/1289983114 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300cd260 con 0x7fdb300b19d0 2023-11-13T10:56:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.381+0000 7fdb35219640 1 --2- 172.21.15.16:0/1289983114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 0x7fdb300b1e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009d90 tx=0x7fdb2802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322182a2a389a131 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.382+0000 7fdb2f7fe640 1 -- 172.21.15.16:0/1289983114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2802cc50 con 0x7fdb300b19d0 2023-11-13T10:56:35.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.382+0000 7fdb2f7fe640 1 -- 172.21.15.16:0/1289983114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2802cdb0 con 0x7fdb300b19d0 2023-11-13T10:56:35.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.382+0000 7fdb2f7fe640 1 -- 172.21.15.16:0/1289983114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28034660 con 0x7fdb300b19d0 2023-11-13T10:56:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.382+0000 7fdb36c7c640 1 -- 172.21.15.16:0/1289983114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 msgr2=0x7fdb300b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.382+0000 7fdb36c7c640 1 --2- 172.21.15.16:0/1289983114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 0x7fdb300b1e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009d90 tx=0x7fdb2802bea0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.383+0000 7fdb36c7c640 1 -- 172.21.15.16:0/1289983114 shutdown_connections 2023-11-13T10:56:35.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.383+0000 7fdb36c7c640 1 --2- 172.21.15.16:0/1289983114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 0x7fdb300b1e60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.383+0000 7fdb36c7c640 1 --2- 172.21.15.16:0/1289983114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb300ce550 0x7fdb300ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.383+0000 7fdb36c7c640 1 -- 172.21.15.16:0/1289983114 >> 172.21.15.16:0/1289983114 conn(0x7fdb3006a190 msgr2=0x7fdb3006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:35.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.383+0000 7fdb36c7c640 1 -- 172.21.15.16:0/1289983114 shutdown_connections 2023-11-13T10:56:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.383+0000 7fdb36c7c640 1 -- 172.21.15.16:0/1289983114 wait complete. 2023-11-13T10:56:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.384+0000 7fdb36c7c640 1 Processor -- start 2023-11-13T10:56:35.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.385+0000 7fdb36c7c640 1 -- start start 2023-11-13T10:56:35.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb36c7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 0x7fdb30159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb36c7c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb300ce550 0x7fdb30153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb36c7c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb30144570 0x7fdb3013f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb36c7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3006fad0 con 0x7fdb300b19d0 2023-11-13T10:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb36c7c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30155e80 con 0x7fdb300ce550 2023-11-13T10:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb36c7c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30156160 con 0x7fdb30144570 2023-11-13T10:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb34a18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 0x7fdb30159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb34a18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 0x7fdb30159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58892/0 (socket says 172.21.15.16:58892) 2023-11-13T10:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb34a18640 1 -- 172.21.15.16:0/2032572551 learned_addr learned my addr 172.21.15.16:0/2032572551 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb35219640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb30144570 0x7fdb3013f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb34a18640 1 -- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb30144570 msgr2=0x7fdb3013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb34a18640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb30144570 0x7fdb3013f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb34a18640 1 -- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb300ce550 msgr2=0x7fdb30153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb34a18640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb300ce550 0x7fdb30153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.386+0000 7fdb34a18640 1 -- 172.21.15.16:0/2032572551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28009a70 con 0x7fdb300b19d0 2023-11-13T10:56:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.387+0000 7fdb34a18640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 0x7fdb30159120 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c430 tx=0x7fdb2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.387+0000 7fdb2d7fa640 1 -- 172.21.15.16:0/2032572551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240074e0 con 0x7fdb300b19d0 2023-11-13T10:56:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.387+0000 7fdb2d7fa640 1 -- 172.21.15.16:0/2032572551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb24007660 con 0x7fdb300b19d0 2023-11-13T10:56:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.387+0000 7fdb2d7fa640 1 -- 172.21.15.16:0/2032572551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24004a20 con 0x7fdb300b19d0 2023-11-13T10:56:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.387+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb30156440 con 0x7fdb300b19d0 2023-11-13T10:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.387+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb30159660 con 0x7fdb300b19d0 2023-11-13T10:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.388+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb30072520 con 0x7fdb300b19d0 2023-11-13T10:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.389+0000 7fdb2d7fa640 1 -- 172.21.15.16:0/2032572551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb24004b80 con 0x7fdb300b19d0 2023-11-13T10:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.390+0000 7fdb2d7fa640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb1406ae40 0x7fdb1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.390+0000 7fdb2d7fa640 1 -- 172.21.15.16:0/2032572551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fdb240897f0 con 0x7fdb300b19d0 2023-11-13T10:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.391+0000 7fdb2ffff640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb1406ae40 0x7fdb1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.392+0000 7fdb2ffff640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb1406ae40 0x7fdb1406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdb300cc030 tx=0x7fdb1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.395+0000 7fdb2d7fa640 1 -- 172.21.15.16:0/2032572551 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb24054550 con 0x7fdb300b19d0 2023-11-13T10:56:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.589+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb3004e5b0 con 0x7fdb300b19d0 2023-11-13T10:56:35.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.590+0000 7fdb2d7fa640 1 -- 172.21.15.16:0/2032572551 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb24057b70 con 0x7fdb300b19d0 2023-11-13T10:56:35.595 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.593+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb1406ae40 msgr2=0x7fdb1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.593+0000 7fdb36c7c640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb1406ae40 0x7fdb1406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdb300cc030 tx=0x7fdb1c008040 comp rx=0 tx=0).stop 2023-11-13T10:56:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.593+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 msgr2=0x7fdb30159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.593+0000 7fdb36c7c640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 0x7fdb30159120 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c430 tx=0x7fdb2400c8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.594+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 shutdown_connections 2023-11-13T10:56:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.594+0000 7fdb36c7c640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdb1406ae40 0x7fdb1406d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.594+0000 7fdb36c7c640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb30144570 0x7fdb3013f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.594+0000 7fdb36c7c640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb300ce550 0x7fdb30153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.594+0000 7fdb36c7c640 1 --2- 172.21.15.16:0/2032572551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb300b19d0 0x7fdb30159120 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.594+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 >> 172.21.15.16:0/2032572551 conn(0x7fdb3006a190 msgr2=0x7fdb3015b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.594+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 shutdown_connections 2023-11-13T10:56:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.594+0000 7fdb36c7c640 1 -- 172.21.15.16:0/2032572551 wait complete. 2023-11-13T10:56:35.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.848+0000 7fdda8765640 1 Processor -- start 2023-11-13T10:56:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.848+0000 7fdda8765640 1 -- start start 2023-11-13T10:56:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.849+0000 7fdda8765640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdda01604b0 0x7fdda00ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.849+0000 7fdda8765640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda0160890 0x7fdda00d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.849+0000 7fdda8765640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda00cd910 con 0x7fdda00d0b00 2023-11-13T10:56:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.849+0000 7fdda8765640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda00cdb40 con 0x7fdda01604b0 2023-11-13T10:56:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.849+0000 7fdda8765640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00cde20 con 0x7fdda0160890 2023-11-13T10:56:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.850+0000 7fdda5d00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda0160890 0x7fdda00d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.850+0000 7fdda5d00640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda0160890 0x7fdda00d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:52310/0 (socket says 172.21.15.16:52310) 2023-11-13T10:56:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.850+0000 7fdda5d00640 1 -- 172.21.15.16:0/3588779262 learned_addr learned my addr 172.21.15.16:0/3588779262 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.850+0000 7fdda6501640 1 --2- 172.21.15.16:0/3588779262 >> v2:172.21.15.184:3300/0 conn(0x7fdda01604b0 0x7fdda00ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.851+0000 7fdda5d00640 1 -- 172.21.15.16:0/3588779262 >> v2:172.21.15.184:3300/0 conn(0x7fdda01604b0 msgr2=0x7fdda00ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.851+0000 7fdda5d00640 1 --2- 172.21.15.16:0/3588779262 >> v2:172.21.15.184:3300/0 conn(0x7fdda01604b0 0x7fdda00ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.851+0000 7fdda54ff640 1 -- 172.21.15.16:0/3588779262 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171251957 0 0) 0x7fdda00cd910 con 0x7fdda00d0b00 2023-11-13T10:56:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.851+0000 7fdda5d00640 1 -- 172.21.15.16:0/3588779262 >> v1:172.21.15.184:6789/0 conn(0x7fdda00d0b00 legacy=0x7fdda00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.851+0000 7fdda5d00640 1 -- 172.21.15.16:0/3588779262 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda00ce100 con 0x7fdda0160890 2023-11-13T10:56:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.852+0000 7fdda5d00640 1 --2- 172.21.15.16:0/3588779262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda0160890 0x7fdda00d1300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009870 tx=0x7fdd9002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d35dff7932ea6eec server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda54ff640 1 -- 172.21.15.16:0/3588779262 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9002f840 con 0x7fdda0160890 2023-11-13T10:56:35.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda54ff640 1 -- 172.21.15.16:0/3588779262 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9002f9a0 con 0x7fdda0160890 2023-11-13T10:56:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda54ff640 1 -- 172.21.15.16:0/3588779262 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9002fd30 con 0x7fdda0160890 2023-11-13T10:56:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda8765640 1 -- 172.21.15.16:0/3588779262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda0160890 msgr2=0x7fdda00d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda8765640 1 --2- 172.21.15.16:0/3588779262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda0160890 0x7fdda00d1300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009870 tx=0x7fdd9002ba20 comp rx=0 tx=0).stop 2023-11-13T10:56:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda8765640 1 -- 172.21.15.16:0/3588779262 shutdown_connections 2023-11-13T10:56:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda8765640 1 --2- 172.21.15.16:0/3588779262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda0160890 0x7fdda00d1300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda8765640 1 --2- 172.21.15.16:0/3588779262 >> v2:172.21.15.184:3300/0 conn(0x7fdda01604b0 0x7fdda00ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda8765640 1 -- 172.21.15.16:0/3588779262 >> 172.21.15.16:0/3588779262 conn(0x7fdda00d2d20 msgr2=0x7fdda00d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda8765640 1 -- 172.21.15.16:0/3588779262 shutdown_connections 2023-11-13T10:56:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.853+0000 7fdda8765640 1 -- 172.21.15.16:0/3588779262 wait complete. 2023-11-13T10:56:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.855+0000 7fdda8765640 1 Processor -- start 2023-11-13T10:56:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.855+0000 7fdda8765640 1 -- start start 2023-11-13T10:56:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.856+0000 7fdda8765640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda00d0b00 0x7fdda00b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.856+0000 7fdda8765640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdda01604b0 0x7fdda00b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.856+0000 7fdda8765640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0160890 0x7fdda014a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.856+0000 7fdda8765640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00b7960 con 0x7fdda0160890 2023-11-13T10:56:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.856+0000 7fdda8765640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00b6d50 con 0x7fdda01604b0 2023-11-13T10:56:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.856+0000 7fdda8765640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00b7030 con 0x7fdda00d0b00 2023-11-13T10:56:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6d02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0160890 0x7fdda014a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6d02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0160890 0x7fdda014a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58906/0 (socket says 172.21.15.16:58906) 2023-11-13T10:56:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6d02640 1 -- 172.21.15.16:0/2611949435 learned_addr learned my addr 172.21.15.16:0/2611949435 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6501640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda00d0b00 0x7fdda00b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda5d00640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdda01604b0 0x7fdda00b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6d02640 1 -- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda00d0b00 msgr2=0x7fdda00b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6d02640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda00d0b00 0x7fdda00b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6d02640 1 -- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdda01604b0 msgr2=0x7fdda00b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6d02640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdda01604b0 0x7fdda00b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6d02640 1 -- 172.21.15.16:0/2611949435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd900094d0 con 0x7fdda0160890 2023-11-13T10:56:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.857+0000 7fdda6d02640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0160890 0x7fdda014a650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c430 tx=0x7fdd9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.858+0000 7fdd96ffd640 1 -- 172.21.15.16:0/2611949435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd980035f0 con 0x7fdda0160890 2023-11-13T10:56:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.858+0000 7fdd96ffd640 1 -- 172.21.15.16:0/2611949435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd98003770 con 0x7fdda0160890 2023-11-13T10:56:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.858+0000 7fdd96ffd640 1 -- 172.21.15.16:0/2611949435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98003b20 con 0x7fdda0160890 2023-11-13T10:56:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.858+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda00b7310 con 0x7fdda0160890 2023-11-13T10:56:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.858+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda00b69e0 con 0x7fdda0160890 2023-11-13T10:56:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.859+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda00b7ad0 con 0x7fdda0160890 2023-11-13T10:56:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.860+0000 7fdd96ffd640 1 -- 172.21.15.16:0/2611949435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd98003c80 con 0x7fdda0160890 2023-11-13T10:56:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.861+0000 7fdd96ffd640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd8406ae40 0x7fdd8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:35.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.861+0000 7fdd96ffd640 1 -- 172.21.15.16:0/2611949435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fdd98088fe0 con 0x7fdda0160890 2023-11-13T10:56:35.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.862+0000 7fdda6501640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd8406ae40 0x7fdd8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:35.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.863+0000 7fdda6501640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd8406ae40 0x7fdd8406d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0074a0 tx=0x7fdd9c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:35.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:35.866+0000 7fdd96ffd640 1 -- 172.21.15.16:0/2611949435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd98053cf0 con 0x7fdda0160890 2023-11-13T10:56:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.060+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdda00a0ca0 con 0x7fdda0160890 2023-11-13T10:56:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.061+0000 7fdd96ffd640 1 -- 172.21.15.16:0/2611949435 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd98017020 con 0x7fdda0160890 2023-11-13T10:56:36.067 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd8406ae40 msgr2=0x7fdd8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd8406ae40 0x7fdd8406d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0074a0 tx=0x7fdd9c00b040 comp rx=0 tx=0).stop 2023-11-13T10:56:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0160890 msgr2=0x7fdda014a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0160890 0x7fdda014a650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c430 tx=0x7fdd9800c8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 shutdown_connections 2023-11-13T10:56:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd8406ae40 0x7fdd8406d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0160890 0x7fdda014a650 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdda01604b0 0x7fdda00b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 --2- 172.21.15.16:0/2611949435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdda00d0b00 0x7fdda00b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 >> 172.21.15.16:0/2611949435 conn(0x7fdda00d2d20 msgr2=0x7fdda015eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 shutdown_connections 2023-11-13T10:56:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.064+0000 7fdda8765640 1 -- 172.21.15.16:0/2611949435 wait complete. 2023-11-13T10:56:36.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.315+0000 7f129ffe9640 1 Processor -- start 2023-11-13T10:56:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.315+0000 7f129ffe9640 1 -- start start 2023-11-13T10:56:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.317+0000 7f129ffe9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129815cf90 0x7f12980b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.317+0000 7f129ffe9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f129806c3b0 0x7f12980b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.317+0000 7f129ffe9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f12980b39c0 0x7f12980ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.317+0000 7f129ffe9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12980b3e50 con 0x7f129806c3b0 2023-11-13T10:56:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.317+0000 7f129ffe9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12980b2e00 con 0x7f12980b39c0 2023-11-13T10:56:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.317+0000 7f129ffe9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12980b21f0 con 0x7f129815cf90 2023-11-13T10:56:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.317+0000 7f129d584640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f129806c3b0 0x7f12980b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.317+0000 7f129d584640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f129806c3b0 0x7f12980b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58920/0 (socket says 172.21.15.16:58920) 2023-11-13T10:56:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.317+0000 7f129d584640 1 -- 172.21.15.16:0/3525582440 learned_addr learned my addr 172.21.15.16:0/3525582440 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:36.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.318+0000 7f129dd85640 1 --2- 172.21.15.16:0/3525582440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129815cf90 0x7f12980b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.318+0000 7f129e586640 1 --2- 172.21.15.16:0/3525582440 >> v2:172.21.15.184:3300/0 conn(0x7f12980b39c0 0x7f12980ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.318+0000 7f129d584640 1 -- 172.21.15.16:0/3525582440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129815cf90 msgr2=0x7f12980b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.318+0000 7f129d584640 1 --2- 172.21.15.16:0/3525582440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129815cf90 0x7f12980b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.318+0000 7f129d584640 1 -- 172.21.15.16:0/3525582440 >> v2:172.21.15.184:3300/0 conn(0x7f12980b39c0 msgr2=0x7f12980ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.318+0000 7f129d584640 1 --2- 172.21.15.16:0/3525582440 >> v2:172.21.15.184:3300/0 conn(0x7f12980b39c0 0x7f12980ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.318+0000 7f129d584640 1 -- 172.21.15.16:0/3525582440 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980b24d0 con 0x7f129806c3b0 2023-11-13T10:56:36.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.318+0000 7f129e586640 1 --2- 172.21.15.16:0/3525582440 >> v2:172.21.15.184:3300/0 conn(0x7f12980b39c0 0x7f12980ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.319+0000 7f129d584640 1 --2- 172.21.15.16:0/3525582440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 0x7f12980b3480 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f129806dbb0 tx=0x7f128802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a32c25c0c22a5b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.319+0000 7f129cd83640 1 -- 172.21.15.16:0/3525582440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288031860 con 0x7f129806c3b0 2023-11-13T10:56:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129cd83640 1 -- 172.21.15.16:0/3525582440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12880319c0 con 0x7f129806c3b0 2023-11-13T10:56:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129cd83640 1 -- 172.21.15.16:0/3525582440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288031d50 con 0x7f129806c3b0 2023-11-13T10:56:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129ffe9640 1 -- 172.21.15.16:0/3525582440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 msgr2=0x7f12980b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129ffe9640 1 --2- 172.21.15.16:0/3525582440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 0x7f12980b3480 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f129806dbb0 tx=0x7f128802b900 comp rx=0 tx=0).stop 2023-11-13T10:56:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129ffe9640 1 -- 172.21.15.16:0/3525582440 shutdown_connections 2023-11-13T10:56:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129ffe9640 1 --2- 172.21.15.16:0/3525582440 >> v2:172.21.15.184:3300/0 conn(0x7f12980b39c0 0x7f12980ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129ffe9640 1 --2- 172.21.15.16:0/3525582440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 0x7f12980b3480 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129ffe9640 1 --2- 172.21.15.16:0/3525582440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129815cf90 0x7f12980b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129ffe9640 1 -- 172.21.15.16:0/3525582440 >> 172.21.15.16:0/3525582440 conn(0x7f129806a190 msgr2=0x7f129806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129ffe9640 1 -- 172.21.15.16:0/3525582440 shutdown_connections 2023-11-13T10:56:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.320+0000 7f129ffe9640 1 -- 172.21.15.16:0/3525582440 wait complete. 2023-11-13T10:56:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.322+0000 7f129ffe9640 1 Processor -- start 2023-11-13T10:56:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.322+0000 7f129ffe9640 1 -- start start 2023-11-13T10:56:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.323+0000 7f129ffe9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 0x7f129813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.323+0000 7f129ffe9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12980b39c0 0x7f129813cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.323+0000 7f129ffe9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129815cf90 0x7f1298140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.323+0000 7f129ffe9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12981f3260 con 0x7f129806c3b0 2023-11-13T10:56:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.323+0000 7f129ffe9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12980ccf50 con 0x7f129815cf90 2023-11-13T10:56:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.323+0000 7f129ffe9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12980cd0e0 con 0x7f12980b39c0 2023-11-13T10:56:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.323+0000 7f129dd85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 0x7f129813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.323+0000 7f129dd85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 0x7f129813d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58928/0 (socket says 172.21.15.16:58928) 2023-11-13T10:56:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.323+0000 7f129dd85640 1 -- 172.21.15.16:0/2077078915 learned_addr learned my addr 172.21.15.16:0/2077078915 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:36.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f129d584640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12980b39c0 0x7f129813cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f129dd85640 1 -- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12980b39c0 msgr2=0x7f129813cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f129dd85640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12980b39c0 0x7f129813cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f129dd85640 1 -- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129815cf90 msgr2=0x7f1298140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f129e586640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129815cf90 0x7f1298140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f129dd85640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129815cf90 0x7f1298140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f129dd85640 1 -- 172.21.15.16:0/2077078915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12880094d0 con 0x7f129806c3b0 2023-11-13T10:56:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f129d584640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12980b39c0 0x7f129813cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f129dd85640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 0x7f129813d660 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f129400d5f0 tx=0x7f129400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.324+0000 7f128e7fc640 1 -- 172.21.15.16:0/2077078915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12940125f0 con 0x7f129806c3b0 2023-11-13T10:56:36.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.325+0000 7f128e7fc640 1 -- 172.21.15.16:0/2077078915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1294012750 con 0x7f129806c3b0 2023-11-13T10:56:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.325+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1298140640 con 0x7f129806c3b0 2023-11-13T10:56:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.325+0000 7f128e7fc640 1 -- 172.21.15.16:0/2077078915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294012b00 con 0x7f129806c3b0 2023-11-13T10:56:36.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.325+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1298151360 con 0x7f129806c3b0 2023-11-13T10:56:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.326+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1298072520 con 0x7f129806c3b0 2023-11-13T10:56:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.327+0000 7f128e7fc640 1 -- 172.21.15.16:0/2077078915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1294012c60 con 0x7f129806c3b0 2023-11-13T10:56:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.328+0000 7f128e7fc640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f127c06ae40 0x7f127c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.328+0000 7f128e7fc640 1 -- 172.21.15.16:0/2077078915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f1294089e50 con 0x7f129806c3b0 2023-11-13T10:56:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.328+0000 7f129d584640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f127c06ae40 0x7f127c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.329+0000 7f129d584640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f127c06ae40 0x7f127c06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1288036000 tx=0x7f128803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.332+0000 7f128e7fc640 1 -- 172.21.15.16:0/2077078915 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1294054b20 con 0x7f129806c3b0 2023-11-13T10:56:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.524+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f129815a1d0 con 0x7f129806c3b0 2023-11-13T10:56:36.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.525+0000 7f128e7fc640 1 -- 172.21.15.16:0/2077078915 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1294058140 con 0x7f129806c3b0 2023-11-13T10:56:36.532 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f127c06ae40 msgr2=0x7f127c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f127c06ae40 0x7f127c06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1288036000 tx=0x7f128803c040 comp rx=0 tx=0).stop 2023-11-13T10:56:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 msgr2=0x7f129813d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 0x7f129813d660 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f129400d5f0 tx=0x7f129400dab0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 shutdown_connections 2023-11-13T10:56:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f127c06ae40 0x7f127c06d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129815cf90 0x7f1298140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12980b39c0 0x7f129813cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 --2- 172.21.15.16:0/2077078915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129806c3b0 0x7f129813d660 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 >> 172.21.15.16:0/2077078915 conn(0x7f129806a190 msgr2=0x7f129806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 shutdown_connections 2023-11-13T10:56:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.528+0000 7f129ffe9640 1 -- 172.21.15.16:0/2077078915 wait complete. 2023-11-13T10:56:36.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2023-11-13T10:56:36.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.772+0000 7fd681b05640 1 Processor -- start 2023-11-13T10:56:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.773+0000 7fd681b05640 1 -- start start 2023-11-13T10:56:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.774+0000 7fd681b05640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd67c070650 0x7fd67c06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.774+0000 7fd681b05640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67c0b3a80 con 0x7fd67c06fb60 2023-11-13T10:56:36.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.774+0000 7fd681b05640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67c161880 con 0x7fd67c0ae030 2023-11-13T10:56:36.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.774+0000 7fd681b05640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67c161b60 con 0x7fd67c070650 2023-11-13T10:56:36.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.775+0000 7fd67b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd67c06fb60 0x7fd67c06ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46964/0 (socket says 172.21.15.16:46964) 2023-11-13T10:56:36.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.775+0000 7fd67b7fe640 1 -- 172.21.15.16:0/2898270899 learned_addr learned my addr 172.21.15.16:0/2898270899 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:36.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.775+0000 7fd67bfff640 1 --2- 172.21.15.16:0/2898270899 >> v2:172.21.15.16:3300/0 conn(0x7fd67c070650 0x7fd67c06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.775+0000 7fd67bfff640 1 -- 172.21.15.16:0/2898270899 >> v1:172.21.15.184:6789/0 conn(0x7fd67c0ae030 legacy=0x7fd67c06c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.775+0000 7fd67bfff640 1 -- 172.21.15.16:0/2898270899 >> v1:172.21.15.16:6789/0 conn(0x7fd67c06fb60 legacy=0x7fd67c06ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.775+0000 7fd67bfff640 1 -- 172.21.15.16:0/2898270899 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c161e40 con 0x7fd67c070650 2023-11-13T10:56:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.776+0000 7fd67bfff640 1 --2- 172.21.15.16:0/2898270899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 0x7fd67c06c770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd6740097a0 tx=0x7fd67402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12451b32a4e5087d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd67a7fc640 1 -- 172.21.15.16:0/2898270899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674030710 con 0x7fd67c070650 2023-11-13T10:56:36.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd67a7fc640 1 -- 172.21.15.16:0/2898270899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67402e050 con 0x7fd67c070650 2023-11-13T10:56:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd67a7fc640 1 -- 172.21.15.16:0/2898270899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674030a90 con 0x7fd67c070650 2023-11-13T10:56:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd681b05640 1 -- 172.21.15.16:0/2898270899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 msgr2=0x7fd67c06c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd681b05640 1 --2- 172.21.15.16:0/2898270899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 0x7fd67c06c770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd6740097a0 tx=0x7fd67402b710 comp rx=0 tx=0).stop 2023-11-13T10:56:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd681b05640 1 -- 172.21.15.16:0/2898270899 shutdown_connections 2023-11-13T10:56:36.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd681b05640 1 --2- 172.21.15.16:0/2898270899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 0x7fd67c06c770 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd681b05640 1 -- 172.21.15.16:0/2898270899 >> 172.21.15.16:0/2898270899 conn(0x7fd67c06a190 msgr2=0x7fd67c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:36.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd681b05640 1 -- 172.21.15.16:0/2898270899 shutdown_connections 2023-11-13T10:56:36.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.777+0000 7fd681b05640 1 -- 172.21.15.16:0/2898270899 wait complete. 2023-11-13T10:56:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.779+0000 7fd681b05640 1 Processor -- start 2023-11-13T10:56:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.779+0000 7fd681b05640 1 -- start start 2023-11-13T10:56:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.780+0000 7fd681b05640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd67c06fb60 0x7fd67c146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.780+0000 7fd681b05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 0x7fd67c145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.780+0000 7fd681b05640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd67c0ae030 0x7fd67c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.780+0000 7fd681b05640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c1ea960 con 0x7fd67c070650 2023-11-13T10:56:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.780+0000 7fd681b05640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c158800 con 0x7fd67c06fb60 2023-11-13T10:56:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.780+0000 7fd681b05640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c146050 con 0x7fd67c0ae030 2023-11-13T10:56:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.780+0000 7fd67affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 0x7fd67c145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.781+0000 7fd67affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 0x7fd67c145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58944/0 (socket says 172.21.15.16:58944) 2023-11-13T10:56:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.781+0000 7fd67affd640 1 -- 172.21.15.16:0/2535249707 learned_addr learned my addr 172.21.15.16:0/2535249707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.781+0000 7fd67affd640 1 -- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd67c0ae030 msgr2=0x7fd67c157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.781+0000 7fd67affd640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd67c0ae030 0x7fd67c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.781+0000 7fd67affd640 1 -- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd67c06fb60 msgr2=0x7fd67c146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.781+0000 7fd67b7fe640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd67c06fb60 0x7fd67c146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.781+0000 7fd67affd640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd67c06fb60 0x7fd67c146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.781+0000 7fd67affd640 1 -- 172.21.15.16:0/2535249707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd674009400 con 0x7fd67c070650 2023-11-13T10:56:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.781+0000 7fd67affd640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 0x7fd67c145b10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd67000c3f0 tx=0x7fd67000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.782+0000 7fd663fff640 1 -- 172.21.15.16:0/2535249707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6700052d0 con 0x7fd67c070650 2023-11-13T10:56:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.782+0000 7fd663fff640 1 -- 172.21.15.16:0/2535249707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd670005430 con 0x7fd67c070650 2023-11-13T10:56:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.782+0000 7fd663fff640 1 -- 172.21.15.16:0/2535249707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670007260 con 0x7fd67c070650 2023-11-13T10:56:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.782+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67c157950 con 0x7fd67c070650 2023-11-13T10:56:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.782+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67c157ed0 con 0x7fd67c070650 2023-11-13T10:56:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.783+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67c1589b0 con 0x7fd67c070650 2023-11-13T10:56:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.784+0000 7fd663fff640 1 -- 172.21.15.16:0/2535249707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6700073c0 con 0x7fd67c070650 2023-11-13T10:56:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.785+0000 7fd663fff640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd66806ae40 0x7fd66806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.785+0000 7fd663fff640 1 -- 172.21.15.16:0/2535249707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fd670088c50 con 0x7fd67c070650 2023-11-13T10:56:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.786+0000 7fd67b7fe640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd66806ae40 0x7fd66806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.787+0000 7fd67b7fe640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd66806ae40 0x7fd66806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0012e0 tx=0x7fd66c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:36.789+0000 7fd663fff640 1 -- 172.21.15.16:0/2535249707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd67c1589b0 con 0x7fd67c070650 2023-11-13T10:56:37.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.054+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7fd67c1589b0 con 0x7fd67c070650 2023-11-13T10:56:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.127+0000 7fd663fff640 1 -- 172.21.15.16:0/2535249707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v86) v1 ==== 151+0+0 (secure 0 0 0) 0x7fd67c1589b0 con 0x7fd67c070650 2023-11-13T10:56:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd66806ae40 msgr2=0x7fd66806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd66806ae40 0x7fd66806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0012e0 tx=0x7fd66c009040 comp rx=0 tx=0).stop 2023-11-13T10:56:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 msgr2=0x7fd67c145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 0x7fd67c145b10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd67000c3f0 tx=0x7fd67000c8b0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 shutdown_connections 2023-11-13T10:56:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd66806ae40 0x7fd66806d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd67c0ae030 0x7fd67c157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd67c070650 0x7fd67c145b10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 --2- 172.21.15.16:0/2535249707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd67c06fb60 0x7fd67c146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.130+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 >> 172.21.15.16:0/2535249707 conn(0x7fd67c06a190 msgr2=0x7fd67c15f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.131+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 shutdown_connections 2023-11-13T10:56:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.131+0000 7fd681b05640 1 -- 172.21.15.16:0/2535249707 wait complete. 2023-11-13T10:56:37.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-13T10:56:37.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.383+0000 7efc0f59e640 1 Processor -- start 2023-11-13T10:56:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.384+0000 7efc0f59e640 1 -- start start 2023-11-13T10:56:37.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.385+0000 7efc0f59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc10160240 0x7efc100d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.385+0000 7efc0f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc10160620 0x7efc100cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.385+0000 7efc0f59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc100d3700 0x7efc100cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.385+0000 7efc0f59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc100c6670 con 0x7efc10160620 2023-11-13T10:56:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.385+0000 7efc0f59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc100ce440 con 0x7efc10160240 2023-11-13T10:56:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.385+0000 7efc0f59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc100ce650 con 0x7efc100d3700 2023-11-13T10:56:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.386+0000 7efc0dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc10160620 0x7efc100cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.386+0000 7efc0dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc10160620 0x7efc100cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58948/0 (socket says 172.21.15.16:58948) 2023-11-13T10:56:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.386+0000 7efc0dd9b640 1 -- 172.21.15.16:0/3639273055 learned_addr learned my addr 172.21.15.16:0/3639273055 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.386+0000 7efc0e59c640 1 --2- 172.21.15.16:0/3639273055 >> v2:172.21.15.184:3300/0 conn(0x7efc10160240 0x7efc100d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:37.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.386+0000 7efc0dd9b640 1 -- 172.21.15.16:0/3639273055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc100d3700 msgr2=0x7efc100cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.386+0000 7efc0dd9b640 1 --2- 172.21.15.16:0/3639273055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc100d3700 0x7efc100cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.386+0000 7efc0dd9b640 1 -- 172.21.15.16:0/3639273055 >> v2:172.21.15.184:3300/0 conn(0x7efc10160240 msgr2=0x7efc100d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.386+0000 7efc0dd9b640 1 --2- 172.21.15.16:0/3639273055 >> v2:172.21.15.184:3300/0 conn(0x7efc10160240 0x7efc100d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.387+0000 7efc0dd9b640 1 -- 172.21.15.16:0/3639273055 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc100ce930 con 0x7efc10160620 2023-11-13T10:56:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.387+0000 7efc0e59c640 1 --2- 172.21.15.16:0/3639273055 >> v2:172.21.15.184:3300/0 conn(0x7efc10160240 0x7efc100d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.388+0000 7efc0dd9b640 1 --2- 172.21.15.16:0/3639273055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160620 0x7efc100cf3c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efc1006a5d0 tx=0x7efbf802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ff2677a47295ee server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.388+0000 7efc0d59a640 1 -- 172.21.15.16:0/3639273055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8031860 con 0x7efc10160620 2023-11-13T10:56:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.388+0000 7efc0d59a640 1 -- 172.21.15.16:0/3639273055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf80319c0 con 0x7efc10160620 2023-11-13T10:56:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.388+0000 7efc0d59a640 1 -- 172.21.15.16:0/3639273055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8031d50 con 0x7efc10160620 2023-11-13T10:56:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.389+0000 7efc0f59e640 1 -- 172.21.15.16:0/3639273055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160620 msgr2=0x7efc100cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.389+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3639273055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160620 0x7efc100cf3c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efc1006a5d0 tx=0x7efbf802b900 comp rx=0 tx=0).stop 2023-11-13T10:56:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.389+0000 7efc0f59e640 1 -- 172.21.15.16:0/3639273055 shutdown_connections 2023-11-13T10:56:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.389+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3639273055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc100d3700 0x7efc100cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.389+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3639273055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160620 0x7efc100cf3c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.389+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3639273055 >> v2:172.21.15.184:3300/0 conn(0x7efc10160240 0x7efc100d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.389+0000 7efc0f59e640 1 -- 172.21.15.16:0/3639273055 >> 172.21.15.16:0/3639273055 conn(0x7efc1009be00 msgr2=0x7efc100d2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.389+0000 7efc0f59e640 1 -- 172.21.15.16:0/3639273055 shutdown_connections 2023-11-13T10:56:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.389+0000 7efc0f59e640 1 -- 172.21.15.16:0/3639273055 wait complete. 2023-11-13T10:56:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.391+0000 7efc0f59e640 1 Processor -- start 2023-11-13T10:56:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.391+0000 7efc0f59e640 1 -- start start 2023-11-13T10:56:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.392+0000 7efc0f59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc100d3700 0x7efc1015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.392+0000 7efc0f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160240 0x7efc100b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.392+0000 7efc0f59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc10160620 0x7efc100b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.392+0000 7efc0f59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc100b2cb0 con 0x7efc10160240 2023-11-13T10:56:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.392+0000 7efc0f59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc100b5690 con 0x7efc10160620 2023-11-13T10:56:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.392+0000 7efc0f59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc100b5970 con 0x7efc100d3700 2023-11-13T10:56:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.392+0000 7efc0dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160240 0x7efc100b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.392+0000 7efc0dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160240 0x7efc100b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58962/0 (socket says 172.21.15.16:58962) 2023-11-13T10:56:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.392+0000 7efc0dd9b640 1 -- 172.21.15.16:0/3670880238 learned_addr learned my addr 172.21.15.16:0/3670880238 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0e59c640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc100d3700 0x7efc1015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0dd9b640 1 -- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc100d3700 msgr2=0x7efc1015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0ed9d640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc10160620 0x7efc100b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0dd9b640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc100d3700 0x7efc1015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0dd9b640 1 -- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc10160620 msgr2=0x7efc100b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0dd9b640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc10160620 0x7efc100b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0dd9b640 1 -- 172.21.15.16:0/3670880238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf80094d0 con 0x7efc10160240 2023-11-13T10:56:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0ed9d640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc10160620 0x7efc100b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0dd9b640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160240 0x7efc100b2770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efbf80090b0 tx=0x7efbf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc06ffd640 1 -- 172.21.15.16:0/3670880238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf803adf0 con 0x7efc10160240 2023-11-13T10:56:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc06ffd640 1 -- 172.21.15.16:0/3670880238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf80443f0 con 0x7efc10160240 2023-11-13T10:56:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc06ffd640 1 -- 172.21.15.16:0/3670880238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf80447a0 con 0x7efc10160240 2023-11-13T10:56:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.393+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc100b1230 con 0x7efc10160240 2023-11-13T10:56:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.394+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc100b3170 con 0x7efc10160240 2023-11-13T10:56:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.395+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc100b5b20 con 0x7efc10160240 2023-11-13T10:56:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.396+0000 7efc06ffd640 1 -- 172.21.15.16:0/3670880238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efbf804d450 con 0x7efc10160240 2023-11-13T10:56:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.396+0000 7efc06ffd640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbf006ae40 0x7efbf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.397+0000 7efc06ffd640 1 -- 172.21.15.16:0/3670880238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7efbf80abea0 con 0x7efc10160240 2023-11-13T10:56:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.397+0000 7efc0e59c640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbf006ae40 0x7efbf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.398+0000 7efc0e59c640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbf006ae40 0x7efbf006d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efc10073bd0 tx=0x7efc000084f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.401+0000 7efc06ffd640 1 -- 172.21.15.16:0/3670880238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc100b5b20 con 0x7efc10160240 2023-11-13T10:56:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.598+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efc100b5ae0 con 0x7efc10160240 2023-11-13T10:56:37.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.605+0000 7efc06ffd640 1 -- 172.21.15.16:0/3670880238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7efbf8035020 con 0x7efc10160240 2023-11-13T10:56:37.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbf006ae40 msgr2=0x7efbf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:37.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbf006ae40 0x7efbf006d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efc10073bd0 tx=0x7efc000084f0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160240 msgr2=0x7efc100b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160240 0x7efc100b2770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efbf80090b0 tx=0x7efbf80046c0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 shutdown_connections 2023-11-13T10:56:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbf006ae40 0x7efbf006d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc10160620 0x7efc100b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc10160240 0x7efc100b2770 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 --2- 172.21.15.16:0/3670880238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc100d3700 0x7efc1015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 >> 172.21.15.16:0/3670880238 conn(0x7efc1009be00 msgr2=0x7efc100cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.608+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 shutdown_connections 2023-11-13T10:56:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:37.609+0000 7efc0f59e640 1 -- 172.21.15.16:0/3670880238 wait complete. 2023-11-13T10:56:37.633 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-13T10:56:37.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T10:56:37.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T10:56:37.875 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-13T10:56:37.875 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:56:37.876 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:37.948 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:56:37.949 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:38.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.233+0000 7fed52bd2640 1 Processor -- start 2023-11-13T10:56:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.234+0000 7fed52bd2640 1 -- start start 2023-11-13T10:56:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.235+0000 7fed52bd2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed4c15cfc0 0x7fed4c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.235+0000 7fed52bd2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fed4c0d2ae0 0x7fed4c0d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.235+0000 7fed52bd2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4c0b3cf0 con 0x7fed4c0b29d0 2023-11-13T10:56:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.235+0000 7fed52bd2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4c0b3550 con 0x7fed4c15cfc0 2023-11-13T10:56:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.235+0000 7fed52bd2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4c0d3480 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed51bd0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed4c15cfc0 0x7fed4c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed523d1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fed4c0b29d0 0x7fed4c0b2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46970/0 (socket says 172.21.15.16:46970) 2023-11-13T10:56:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed523d1640 1 -- 172.21.15.16:0/924903907 learned_addr learned my addr 172.21.15.16:0/924903907 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed513cf640 1 --2- 172.21.15.16:0/924903907 >> v2:172.21.15.184:3300/0 conn(0x7fed4c0d2ae0 0x7fed4c0d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed51bd0640 1 -- 172.21.15.16:0/924903907 >> v2:172.21.15.184:3300/0 conn(0x7fed4c0d2ae0 msgr2=0x7fed4c0d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed51bd0640 1 --2- 172.21.15.16:0/924903907 >> v2:172.21.15.184:3300/0 conn(0x7fed4c0d2ae0 0x7fed4c0d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed51bd0640 1 -- 172.21.15.16:0/924903907 >> v1:172.21.15.16:6789/0 conn(0x7fed4c0b29d0 legacy=0x7fed4c0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed51bd0640 1 -- 172.21.15.16:0/924903907 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c0b3730 con 0x7fed4c15cfc0 2023-11-13T10:56:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed50bce640 1 -- 172.21.15.16:0/924903907 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198499491 0 0) 0x7fed4c0b3cf0 con 0x7fed4c0b29d0 2023-11-13T10:56:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.236+0000 7fed513cf640 1 --2- 172.21.15.16:0/924903907 >> v2:172.21.15.184:3300/0 conn(0x7fed4c0d2ae0 0x7fed4c0d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.237+0000 7fed51bd0640 1 --2- 172.21.15.16:0/924903907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c15cfc0 0x7fed4c0cd770 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed4c08ce30 tx=0x7fed4802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b07c22828ad26d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed50bce640 1 -- 172.21.15.16:0/924903907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4802e070 con 0x7fed4c15cfc0 2023-11-13T10:56:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed50bce640 1 -- 172.21.15.16:0/924903907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed480315a0 con 0x7fed4c15cfc0 2023-11-13T10:56:38.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed50bce640 1 -- 172.21.15.16:0/924903907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48031930 con 0x7fed4c15cfc0 2023-11-13T10:56:38.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed52bd2640 1 -- 172.21.15.16:0/924903907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c15cfc0 msgr2=0x7fed4c0cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed52bd2640 1 --2- 172.21.15.16:0/924903907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c15cfc0 0x7fed4c0cd770 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed4c08ce30 tx=0x7fed4802b840 comp rx=0 tx=0).stop 2023-11-13T10:56:38.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed52bd2640 1 -- 172.21.15.16:0/924903907 shutdown_connections 2023-11-13T10:56:38.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed52bd2640 1 --2- 172.21.15.16:0/924903907 >> v2:172.21.15.184:3300/0 conn(0x7fed4c0d2ae0 0x7fed4c0d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed52bd2640 1 --2- 172.21.15.16:0/924903907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c15cfc0 0x7fed4c0cd770 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed52bd2640 1 -- 172.21.15.16:0/924903907 >> 172.21.15.16:0/924903907 conn(0x7fed4c08c570 msgr2=0x7fed4c09be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:38.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed52bd2640 1 -- 172.21.15.16:0/924903907 shutdown_connections 2023-11-13T10:56:38.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.238+0000 7fed52bd2640 1 -- 172.21.15.16:0/924903907 wait complete. 2023-11-13T10:56:38.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.240+0000 7fed52bd2640 1 Processor -- start 2023-11-13T10:56:38.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.240+0000 7fed52bd2640 1 -- start start 2023-11-13T10:56:38.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.241+0000 7fed52bd2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c0b29d0 0x7fed4c0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.241+0000 7fed52bd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c0d2ae0 0x7fed4c146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.241+0000 7fed52bd2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed4c15cfc0 0x7fed4c1471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.241+0000 7fed52bd2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c0c90a0 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.241+0000 7fed52bd2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c0c8110 con 0x7fed4c15cfc0 2023-11-13T10:56:38.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.241+0000 7fed52bd2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c15d4d0 con 0x7fed4c0b29d0 2023-11-13T10:56:38.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.242+0000 7fed513cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c0d2ae0 0x7fed4c146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.242+0000 7fed513cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c0d2ae0 0x7fed4c146670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58990/0 (socket says 172.21.15.16:58990) 2023-11-13T10:56:38.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.242+0000 7fed513cf640 1 -- 172.21.15.16:0/3967974575 learned_addr learned my addr 172.21.15.16:0/3967974575 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:38.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.242+0000 7fed513cf640 1 -- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c0b29d0 msgr2=0x7fed4c0c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.242+0000 7fed513cf640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c0b29d0 0x7fed4c0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.242+0000 7fed513cf640 1 -- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed4c15cfc0 msgr2=0x7fed4c1471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.242+0000 7fed513cf640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed4c15cfc0 0x7fed4c1471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.242+0000 7fed513cf640 1 -- 172.21.15.16:0/3967974575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48009400 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.242+0000 7fed513cf640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c0d2ae0 0x7fed4c146670 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00c7f0 tx=0x7fed3c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:38.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.243+0000 7fed427fc640 1 -- 172.21.15.16:0/3967974575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c007a30 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.243+0000 7fed427fc640 1 -- 172.21.15.16:0/3967974575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c007b90 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.243+0000 7fed427fc640 1 -- 172.21.15.16:0/3967974575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00ce40 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.243+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4c15d7b0 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.243+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4c0c8640 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.244+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4c15d9d0 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.245+0000 7fed427fc640 1 -- 172.21.15.16:0/3967974575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed3c0113f0 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.246+0000 7fed427fc640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fed3006ae40 0x7fed3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.246+0000 7fed427fc640 1 -- 172.21.15.16:0/3967974575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fed3c089b50 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.247+0000 7fed51bd0640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fed3006ae40 0x7fed3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.248+0000 7fed51bd0640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fed3006ae40 0x7fed3006d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed48036000 tx=0x7fed48030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.251+0000 7fed427fc640 1 -- 172.21.15.16:0/3967974575 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed3c0548b0 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.449+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed4c0c8830 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.452+0000 7fed427fc640 1 -- 172.21.15.16:0/3967974575 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fed3c01f090 con 0x7fed4c0d2ae0 2023-11-13T10:56:38.459 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:38.459 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:38.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fed3006ae40 msgr2=0x7fed3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fed3006ae40 0x7fed3006d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed48036000 tx=0x7fed48030000 comp rx=0 tx=0).stop 2023-11-13T10:56:38.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c0d2ae0 msgr2=0x7fed4c146670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c0d2ae0 0x7fed4c146670 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00c7f0 tx=0x7fed3c00cbf0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 shutdown_connections 2023-11-13T10:56:38.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fed3006ae40 0x7fed3006d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed4c15cfc0 0x7fed4c1471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed4c0d2ae0 0x7fed4c146670 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 --2- 172.21.15.16:0/3967974575 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c0b29d0 0x7fed4c0c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 >> 172.21.15.16:0/3967974575 conn(0x7fed4c08c570 msgr2=0x7fed4c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.455+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 shutdown_connections 2023-11-13T10:56:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.456+0000 7fed52bd2640 1 -- 172.21.15.16:0/3967974575 wait complete. 2023-11-13T10:56:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:38.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:38.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.707+0000 7f20d9580640 1 Processor -- start 2023-11-13T10:56:38.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.707+0000 7f20d9580640 1 -- start start 2023-11-13T10:56:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.708+0000 7f20d9580640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 0x7f20d40d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.708+0000 7f20d9580640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20d40d2d00 0x7f20d40c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d9580640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d40ce0e0 con 0x7f20d415ce70 2023-11-13T10:56:38.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d9580640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d40ce3c0 con 0x7f20d40d2d00 2023-11-13T10:56:38.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d9580640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40ce6a0 con 0x7f20d40d1850 2023-11-13T10:56:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d8d7f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20d40d2d00 0x7f20d40c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d8d7f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20d40d2d00 0x7f20d40c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58992/0 (socket says 172.21.15.16:58992) 2023-11-13T10:56:38.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d8d7f640 1 -- 172.21.15.16:0/3054951511 learned_addr learned my addr 172.21.15.16:0/3054951511 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:38.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d37fe640 1 --2- 172.21.15.16:0/3054951511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 0x7f20d40d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d8d7f640 1 -- 172.21.15.16:0/3054951511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 msgr2=0x7f20d40d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:38.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d8d7f640 1 --2- 172.21.15.16:0/3054951511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 0x7f20d40d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d8d7f640 1 -- 172.21.15.16:0/3054951511 >> v1:172.21.15.184:6789/0 conn(0x7f20d415ce70 legacy=0x7f20d40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:38.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d8d7f640 1 -- 172.21.15.16:0/3054951511 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d40ce980 con 0x7f20d40d2d00 2023-11-13T10:56:38.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.709+0000 7f20d37fe640 1 --2- 172.21.15.16:0/3054951511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 0x7f20d40d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.711+0000 7f20d8d7f640 1 --2- 172.21.15.16:0/3054951511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 0x7f20d40c9c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009870 tx=0x7f20cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d470108cc12201 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.711+0000 7f20d2ffd640 1 -- 172.21.15.16:0/3054951511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc031640 con 0x7f20d40d2d00 2023-11-13T10:56:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.711+0000 7f20d2ffd640 1 -- 172.21.15.16:0/3054951511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc0317a0 con 0x7f20d40d2d00 2023-11-13T10:56:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.711+0000 7f20d2ffd640 1 -- 172.21.15.16:0/3054951511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc031b30 con 0x7f20d40d2d00 2023-11-13T10:56:38.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.712+0000 7f20d9580640 1 -- 172.21.15.16:0/3054951511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 msgr2=0x7f20d40c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.712+0000 7f20d9580640 1 --2- 172.21.15.16:0/3054951511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 0x7f20d40c9c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009870 tx=0x7f20cc02b8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.712+0000 7f20d9580640 1 -- 172.21.15.16:0/3054951511 shutdown_connections 2023-11-13T10:56:38.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.712+0000 7f20d9580640 1 --2- 172.21.15.16:0/3054951511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 0x7f20d40c9c30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.712+0000 7f20d9580640 1 --2- 172.21.15.16:0/3054951511 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 0x7f20d40d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.712+0000 7f20d9580640 1 -- 172.21.15.16:0/3054951511 >> 172.21.15.16:0/3054951511 conn(0x7f20d408c580 msgr2=0x7f20d409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:38.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.712+0000 7f20d9580640 1 -- 172.21.15.16:0/3054951511 shutdown_connections 2023-11-13T10:56:38.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.712+0000 7f20d9580640 1 -- 172.21.15.16:0/3054951511 wait complete. 2023-11-13T10:56:38.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.714+0000 7f20d9580640 1 Processor -- start 2023-11-13T10:56:38.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.714+0000 7f20d9580640 1 -- start start 2023-11-13T10:56:38.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.715+0000 7f20d9580640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 0x7f20d41f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.715+0000 7f20d9580640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 0x7f20d41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.715+0000 7f20d9580640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20d415ce70 0x7f20d40b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.715+0000 7f20d9580640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41f6cb0 con 0x7f20d40d2d00 2023-11-13T10:56:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.715+0000 7f20d9580640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d415d950 con 0x7f20d415ce70 2023-11-13T10:56:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.715+0000 7f20d9580640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d415dc30 con 0x7f20d40d1850 2023-11-13T10:56:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.715+0000 7f20d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 0x7f20d41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.715+0000 7f20d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 0x7f20d41522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59008/0 (socket says 172.21.15.16:59008) 2023-11-13T10:56:38.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.715+0000 7f20d37fe640 1 -- 172.21.15.16:0/3108912193 learned_addr learned my addr 172.21.15.16:0/3108912193 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:38.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d8d7f640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20d415ce70 0x7f20d40b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d3fff640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 0x7f20d41f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d37fe640 1 -- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 msgr2=0x7f20d41f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d37fe640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 0x7f20d41f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d37fe640 1 -- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20d415ce70 msgr2=0x7f20d40b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d37fe640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20d415ce70 0x7f20d40b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d37fe640 1 -- 172.21.15.16:0/3108912193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc0094d0 con 0x7f20d40d2d00 2023-11-13T10:56:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d37fe640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 0x7f20d41522a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00c7f0 tx=0x7f20bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d0ff9640 1 -- 172.21.15.16:0/3108912193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc007820 con 0x7f20d40d2d00 2023-11-13T10:56:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d0ff9640 1 -- 172.21.15.16:0/3108912193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc007980 con 0x7f20d40d2d00 2023-11-13T10:56:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d0ff9640 1 -- 172.21.15.16:0/3108912193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc007cf0 con 0x7f20d40d2d00 2023-11-13T10:56:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.716+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d415df10 con 0x7f20d40d2d00 2023-11-13T10:56:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.717+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d415e4d0 con 0x7f20d40d2d00 2023-11-13T10:56:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.718+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d4160630 con 0x7f20d40d2d00 2023-11-13T10:56:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.719+0000 7f20d0ff9640 1 -- 172.21.15.16:0/3108912193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20bc0113f0 con 0x7f20d40d2d00 2023-11-13T10:56:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.720+0000 7f20d0ff9640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20b406ae40 0x7f20b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.720+0000 7f20d0ff9640 1 -- 172.21.15.16:0/3108912193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f20bc089d30 con 0x7f20d40d2d00 2023-11-13T10:56:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.721+0000 7f20d3fff640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20b406ae40 0x7f20b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.722+0000 7f20d3fff640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20b406ae40 0x7f20b406d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005520 tx=0x7f20c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.724+0000 7f20d0ff9640 1 -- 172.21.15.16:0/3108912193 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20bc054a00 con 0x7f20d40d2d00 2023-11-13T10:56:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.924+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20d404e4d0 con 0x7f20d40d2d00 2023-11-13T10:56:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.925+0000 7f20d0ff9640 1 -- 172.21.15.16:0/3108912193 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20bc01f0a0 con 0x7f20d40d2d00 2023-11-13T10:56:38.930 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:38.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.928+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20b406ae40 msgr2=0x7f20b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.928+0000 7f20d9580640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20b406ae40 0x7f20b406d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005520 tx=0x7f20c4009040 comp rx=0 tx=0).stop 2023-11-13T10:56:38.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.928+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 msgr2=0x7f20d41522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:38.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.928+0000 7f20d9580640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 0x7f20d41522a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00c7f0 tx=0x7f20bc00ccb0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.928+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 shutdown_connections 2023-11-13T10:56:38.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.928+0000 7f20d9580640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20b406ae40 0x7f20b406d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.928+0000 7f20d9580640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20d415ce70 0x7f20d40b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.929+0000 7f20d9580640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d40d2d00 0x7f20d41522a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.929+0000 7f20d9580640 1 --2- 172.21.15.16:0/3108912193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20d40d1850 0x7f20d41f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:38.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.929+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 >> 172.21.15.16:0/3108912193 conn(0x7f20d408c580 msgr2=0x7f20d40b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:38.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.929+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 shutdown_connections 2023-11-13T10:56:38.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:38.929+0000 7f20d9580640 1 -- 172.21.15.16:0/3108912193 wait complete. 2023-11-13T10:56:38.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:39.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.164+0000 7f05d781a640 1 Processor -- start 2023-11-13T10:56:39.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.165+0000 7f05d781a640 1 -- start start 2023-11-13T10:56:39.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.166+0000 7f05d781a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05d0144570 0x7f05d015f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.166+0000 7f05d781a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05d00ce550 0x7f05d00ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.166+0000 7f05d781a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d00ceef0 con 0x7f05d006bb30 2023-11-13T10:56:39.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.166+0000 7f05d781a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d0144950 con 0x7f05d00ce550 2023-11-13T10:56:39.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.166+0000 7f05d781a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d015f740 con 0x7f05d0144570 2023-11-13T10:56:39.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.167+0000 7f05d4db5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05d00ce550 0x7f05d00ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:39.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.167+0000 7f05d4db5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05d00ce550 0x7f05d00ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59024/0 (socket says 172.21.15.16:59024) 2023-11-13T10:56:39.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.167+0000 7f05d4db5640 1 -- 172.21.15.16:0/1702558506 learned_addr learned my addr 172.21.15.16:0/1702558506 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:39.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.167+0000 7f05d4db5640 1 -- 172.21.15.16:0/1702558506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05d0144570 msgr2=0x7f05d015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:39.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.167+0000 7f05d4db5640 1 --2- 172.21.15.16:0/1702558506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05d0144570 0x7f05d015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.167+0000 7f05d4db5640 1 -- 172.21.15.16:0/1702558506 >> v1:172.21.15.184:6789/0 conn(0x7f05d006bb30 legacy=0x7f05d006bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:39.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.167+0000 7f05d4db5640 1 -- 172.21.15.16:0/1702558506 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d00ce300 con 0x7f05d00ce550 2023-11-13T10:56:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.168+0000 7f05d4db5640 1 --2- 172.21.15.16:0/1702558506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d00ce550 0x7f05d00ce9b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009870 tx=0x7f05c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e17cade3fb29b77 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.168+0000 7f05c7fff640 1 -- 172.21.15.16:0/1702558506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0031640 con 0x7f05d00ce550 2023-11-13T10:56:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05c7fff640 1 -- 172.21.15.16:0/1702558506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c00317a0 con 0x7f05d00ce550 2023-11-13T10:56:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05c7fff640 1 -- 172.21.15.16:0/1702558506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0031b30 con 0x7f05d00ce550 2023-11-13T10:56:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05d781a640 1 -- 172.21.15.16:0/1702558506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d00ce550 msgr2=0x7f05d00ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05d781a640 1 --2- 172.21.15.16:0/1702558506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d00ce550 0x7f05d00ce9b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009870 tx=0x7f05c002b8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05d781a640 1 -- 172.21.15.16:0/1702558506 shutdown_connections 2023-11-13T10:56:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05d781a640 1 --2- 172.21.15.16:0/1702558506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d00ce550 0x7f05d00ce9b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05d781a640 1 --2- 172.21.15.16:0/1702558506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05d0144570 0x7f05d015f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05d781a640 1 -- 172.21.15.16:0/1702558506 >> 172.21.15.16:0/1702558506 conn(0x7f05d006a190 msgr2=0x7f05d006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05d781a640 1 -- 172.21.15.16:0/1702558506 shutdown_connections 2023-11-13T10:56:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.169+0000 7f05d781a640 1 -- 172.21.15.16:0/1702558506 wait complete. 2023-11-13T10:56:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.171+0000 7f05d781a640 1 Processor -- start 2023-11-13T10:56:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.171+0000 7f05d781a640 1 -- start start 2023-11-13T10:56:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.172+0000 7f05d781a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05d006bb30 0x7f05d013d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.172+0000 7f05d781a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05d00ce550 0x7f05d013a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.172+0000 7f05d781a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d0144570 0x7f05d00af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.172+0000 7f05d781a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d013db60 con 0x7f05d0144570 2023-11-13T10:56:39.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.172+0000 7f05d781a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d013de40 con 0x7f05d00ce550 2023-11-13T10:56:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.172+0000 7f05d781a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d0141c00 con 0x7f05d006bb30 2023-11-13T10:56:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.172+0000 7f05d5db7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d0144570 0x7f05d00af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.172+0000 7f05d5db7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d0144570 0x7f05d00af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59036/0 (socket says 172.21.15.16:59036) 2023-11-13T10:56:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.172+0000 7f05d5db7640 1 -- 172.21.15.16:0/958932539 learned_addr learned my addr 172.21.15.16:0/958932539 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05d55b6640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05d006bb30 0x7f05d013d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05d4db5640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05d00ce550 0x7f05d013a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05d5db7640 1 -- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05d006bb30 msgr2=0x7f05d013d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05d5db7640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05d006bb30 0x7f05d013d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05d5db7640 1 -- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05d00ce550 msgr2=0x7f05d013a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05d5db7640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05d00ce550 0x7f05d013a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05d5db7640 1 -- 172.21.15.16:0/958932539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c00094d0 con 0x7f05d0144570 2023-11-13T10:56:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05d4db5640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05d00ce550 0x7f05d013a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05d5db7640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d0144570 0x7f05d00af0f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f05c800c780 tx=0x7f05c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05c5ffb640 1 -- 172.21.15.16:0/958932539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8004b60 con 0x7f05d0144570 2023-11-13T10:56:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.173+0000 7f05c5ffb640 1 -- 172.21.15.16:0/958932539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c800ce90 con 0x7f05d0144570 2023-11-13T10:56:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.174+0000 7f05c5ffb640 1 -- 172.21.15.16:0/958932539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c800a850 con 0x7f05d0144570 2023-11-13T10:56:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.174+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d0141ee0 con 0x7f05d0144570 2023-11-13T10:56:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.174+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d00af630 con 0x7f05d0144570 2023-11-13T10:56:39.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.175+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d01420d0 con 0x7f05d0144570 2023-11-13T10:56:39.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.176+0000 7f05c5ffb640 1 -- 172.21.15.16:0/958932539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f05c800a9b0 con 0x7f05d0144570 2023-11-13T10:56:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.177+0000 7f05c5ffb640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f05b406ae90 0x7f05b406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.177+0000 7f05c5ffb640 1 -- 172.21.15.16:0/958932539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f05c8089a60 con 0x7f05d0144570 2023-11-13T10:56:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.178+0000 7f05d55b6640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f05b406ae90 0x7f05b406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.179+0000 7f05d55b6640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f05b406ae90 0x7f05b406d350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00b810 tx=0x7f05cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.182+0000 7f05c5ffb640 1 -- 172.21.15.16:0/958932539 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05c8054840 con 0x7f05d0144570 2023-11-13T10:56:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.377+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05d0072520 con 0x7f05d0144570 2023-11-13T10:56:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.378+0000 7f05c5ffb640 1 -- 172.21.15.16:0/958932539 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05c8057e60 con 0x7f05d0144570 2023-11-13T10:56:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.380+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f05b406ae90 msgr2=0x7f05b406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.380+0000 7f05d781a640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f05b406ae90 0x7f05b406d350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00b810 tx=0x7f05cc008040 comp rx=0 tx=0).stop 2023-11-13T10:56:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d0144570 msgr2=0x7f05d00af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d0144570 0x7f05d00af0f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f05c800c780 tx=0x7f05c800cc40 comp rx=0 tx=0).stop 2023-11-13T10:56:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 shutdown_connections 2023-11-13T10:56:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f05b406ae90 0x7f05b406d350 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d0144570 0x7f05d00af0f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05d00ce550 0x7f05d013a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 --2- 172.21.15.16:0/958932539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05d006bb30 0x7f05d013d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 >> 172.21.15.16:0/958932539 conn(0x7f05d006a190 msgr2=0x7f05d00afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 shutdown_connections 2023-11-13T10:56:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.381+0000 7f05d781a640 1 -- 172.21.15.16:0/958932539 wait complete. 2023-11-13T10:56:39.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:39.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.631+0000 7fe343810640 1 Processor -- start 2023-11-13T10:56:39.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.631+0000 7fe343810640 1 -- start start 2023-11-13T10:56:39.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.632+0000 7fe343810640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe33c0cfdd0 0x7fe33c0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe343810640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe33c0d1bc0 0x7fe33c0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe343810640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33c0ce990 con 0x7fe33c15d280 2023-11-13T10:56:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe343810640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33c0cf510 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe343810640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c0cec70 con 0x7fe33c0d1bc0 2023-11-13T10:56:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe340dab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe33c0cfdd0 0x7fe33c0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe340dab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe33c0cfdd0 0x7fe33c0b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59042/0 (socket says 172.21.15.16:59042) 2023-11-13T10:56:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe340dab640 1 -- 172.21.15.16:0/523465443 learned_addr learned my addr 172.21.15.16:0/523465443 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe341dad640 1 --2- 172.21.15.16:0/523465443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe33c0d1bc0 0x7fe33c0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe340dab640 1 -- 172.21.15.16:0/523465443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe33c0d1bc0 msgr2=0x7fe33c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe340dab640 1 --2- 172.21.15.16:0/523465443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe33c0d1bc0 0x7fe33c0b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe340dab640 1 -- 172.21.15.16:0/523465443 >> v1:172.21.15.184:6789/0 conn(0x7fe33c15d280 legacy=0x7fe33c0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.633+0000 7fe340dab640 1 -- 172.21.15.16:0/523465443 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c0cef50 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.635+0000 7fe340dab640 1 --2- 172.21.15.16:0/523465443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 0x7fe33c0b29d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b22c07f0097f5b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.635+0000 7fe333fff640 1 -- 172.21.15.16:0/523465443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c031640 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.635+0000 7fe333fff640 1 -- 172.21.15.16:0/523465443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c0317a0 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.635+0000 7fe333fff640 1 -- 172.21.15.16:0/523465443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c031b30 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.635+0000 7fe343810640 1 -- 172.21.15.16:0/523465443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 msgr2=0x7fe33c0b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.635+0000 7fe343810640 1 --2- 172.21.15.16:0/523465443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 0x7fe33c0b29d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c02b8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.636+0000 7fe343810640 1 -- 172.21.15.16:0/523465443 shutdown_connections 2023-11-13T10:56:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.636+0000 7fe343810640 1 --2- 172.21.15.16:0/523465443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe33c0d1bc0 0x7fe33c0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.636+0000 7fe343810640 1 --2- 172.21.15.16:0/523465443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 0x7fe33c0b29d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.636+0000 7fe343810640 1 -- 172.21.15.16:0/523465443 >> 172.21.15.16:0/523465443 conn(0x7fe33c0d2cb0 msgr2=0x7fe33c0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.636+0000 7fe343810640 1 -- 172.21.15.16:0/523465443 shutdown_connections 2023-11-13T10:56:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.636+0000 7fe343810640 1 -- 172.21.15.16:0/523465443 wait complete. 2023-11-13T10:56:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.637+0000 7fe343810640 1 Processor -- start 2023-11-13T10:56:39.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.638+0000 7fe343810640 1 -- start start 2023-11-13T10:56:39.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.638+0000 7fe343810640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 0x7fe33c158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.638+0000 7fe343810640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe33c0d1bc0 0x7fe33c09eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe343810640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe33c15d280 0x7fe33c09efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe343810640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c0ca830 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe343810640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c0cab10 con 0x7fe33c0d1bc0 2023-11-13T10:56:39.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe343810640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c0cadf0 con 0x7fe33c15d280 2023-11-13T10:56:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe3415ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 0x7fe33c158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe3415ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 0x7fe33c158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59050/0 (socket says 172.21.15.16:59050) 2023-11-13T10:56:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe3415ac640 1 -- 172.21.15.16:0/3015759088 learned_addr learned my addr 172.21.15.16:0/3015759088 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:39.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe3415ac640 1 -- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe33c15d280 msgr2=0x7fe33c09efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe3415ac640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe33c15d280 0x7fe33c09efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe3415ac640 1 -- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe33c0d1bc0 msgr2=0x7fe33c09eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe3415ac640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe33c0d1bc0 0x7fe33c09eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe3415ac640 1 -- 172.21.15.16:0/3015759088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c0094d0 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.639+0000 7fe3415ac640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 0x7fe33c158b10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b3f0 tx=0x7fe33800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:39.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.640+0000 7fe331ffb640 1 -- 172.21.15.16:0/3015759088 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338011830 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.640+0000 7fe331ffb640 1 -- 172.21.15.16:0/3015759088 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe338011990 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.640+0000 7fe331ffb640 1 -- 172.21.15.16:0/3015759088 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338011d40 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.640+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33c0cb0d0 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.640+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33c159050 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.641+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33c0cb8d0 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.642+0000 7fe331ffb640 1 -- 172.21.15.16:0/3015759088 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe33801a400 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.643+0000 7fe331ffb640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:39.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.643+0000 7fe331ffb640 1 -- 172.21.15.16:0/3015759088 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fe3380897b0 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.643+0000 7fe340dab640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:39.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.645+0000 7fe340dab640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe32c036000 tx=0x7fe32c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:39.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.647+0000 7fe331ffb640 1 -- 172.21.15.16:0/3015759088 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe338054500 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.834+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe33c0cb5f0 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.835+0000 7fe331ffb640 1 -- 172.21.15.16:0/3015759088 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe33c0cb5f0 con 0x7fe33c0cfdd0 2023-11-13T10:56:39.840 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:39.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 msgr2=0x7fe32006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe32c036000 tx=0x7fe32c030000 comp rx=0 tx=0).stop 2023-11-13T10:56:39.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 msgr2=0x7fe33c158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:39.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 0x7fe33c158b10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe33800b3f0 tx=0x7fe33800b8b0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 shutdown_connections 2023-11-13T10:56:39.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe32006ae40 0x7fe32006d300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe33c15d280 0x7fe33c09efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe33c0d1bc0 0x7fe33c09eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 --2- 172.21.15.16:0/3015759088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c0cfdd0 0x7fe33c158b10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:39.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 >> 172.21.15.16:0/3015759088 conn(0x7fe33c0d2cb0 msgr2=0x7fe33c0d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:39.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 shutdown_connections 2023-11-13T10:56:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:39.838+0000 7fe343810640 1 -- 172.21.15.16:0/3015759088 wait complete. 2023-11-13T10:56:39.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.079+0000 7f306b59e640 1 Processor -- start 2023-11-13T10:56:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.079+0000 7f306b59e640 1 -- start start 2023-11-13T10:56:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.081+0000 7f306b59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f306c0d1840 0x7f306c0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.081+0000 7f306b59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0d3a80 con 0x7f306c15cf60 2023-11-13T10:56:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.081+0000 7f306b59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0d37a0 con 0x7f306c0b1a50 2023-11-13T10:56:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.081+0000 7f306b59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c0af8b0 con 0x7f306c0d1840 2023-11-13T10:56:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.081+0000 7f306a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f306c15cf60 0x7f306c0d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46986/0 (socket says 172.21.15.16:46986) 2023-11-13T10:56:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.081+0000 7f306a59c640 1 -- 172.21.15.16:0/393922549 learned_addr learned my addr 172.21.15.16:0/393922549 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.082+0000 7f3069d9b640 1 --2- 172.21.15.16:0/393922549 >> v2:172.21.15.16:3300/0 conn(0x7f306c0d1840 0x7f306c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.082+0000 7f306959a640 1 -- 172.21.15.16:0/393922549 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654755402 0 0) 0x7f306c0d3a80 con 0x7f306c15cf60 2023-11-13T10:56:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.082+0000 7f306959a640 1 -- 172.21.15.16:0/393922549 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3058003520 con 0x7f306c15cf60 2023-11-13T10:56:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.082+0000 7f3069d9b640 1 -- 172.21.15.16:0/393922549 >> v1:172.21.15.184:6789/0 conn(0x7f306c0b1a50 legacy=0x7f306c0b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.082+0000 7f3069d9b640 1 -- 172.21.15.16:0/393922549 >> v1:172.21.15.16:6789/0 conn(0x7f306c15cf60 legacy=0x7f306c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.082+0000 7f3069d9b640 1 -- 172.21.15.16:0/393922549 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c0afb90 con 0x7f306c0d1840 2023-11-13T10:56:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.083+0000 7f3069d9b640 1 --2- 172.21.15.16:0/393922549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0d1840 0x7f306c0cfb90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30540097a0 tx=0x7f305402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31225f444ab04d00 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.084+0000 7f306959a640 1 -- 172.21.15.16:0/393922549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305402e070 con 0x7f306c0d1840 2023-11-13T10:56:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.084+0000 7f306959a640 1 -- 172.21.15.16:0/393922549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30540315a0 con 0x7f306c0d1840 2023-11-13T10:56:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.084+0000 7f306959a640 1 -- 172.21.15.16:0/393922549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054031930 con 0x7f306c0d1840 2023-11-13T10:56:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.084+0000 7f306b59e640 1 -- 172.21.15.16:0/393922549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0d1840 msgr2=0x7f306c0cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.084+0000 7f306b59e640 1 --2- 172.21.15.16:0/393922549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0d1840 0x7f306c0cfb90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30540097a0 tx=0x7f305402b840 comp rx=0 tx=0).stop 2023-11-13T10:56:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.084+0000 7f306b59e640 1 -- 172.21.15.16:0/393922549 shutdown_connections 2023-11-13T10:56:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.084+0000 7f306b59e640 1 --2- 172.21.15.16:0/393922549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0d1840 0x7f306c0cfb90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.084+0000 7f306b59e640 1 -- 172.21.15.16:0/393922549 >> 172.21.15.16:0/393922549 conn(0x7f306c0d2b50 msgr2=0x7f306c0d2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.084+0000 7f306b59e640 1 -- 172.21.15.16:0/393922549 shutdown_connections 2023-11-13T10:56:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.085+0000 7f306b59e640 1 -- 172.21.15.16:0/393922549 wait complete. 2023-11-13T10:56:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.086+0000 7f306b59e640 1 Processor -- start 2023-11-13T10:56:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.086+0000 7f306b59e640 1 -- start start 2023-11-13T10:56:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.087+0000 7f306b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0b1a50 0x7f306c0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.087+0000 7f306b59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c0d1840 0x7f306c1f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.087+0000 7f306b59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c15cf60 0x7f306c0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c1f0370 con 0x7f306c0b1a50 2023-11-13T10:56:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306b59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0ca2b0 con 0x7f306c0d1840 2023-11-13T10:56:40.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306b59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0ca590 con 0x7f306c15cf60 2023-11-13T10:56:40.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0b1a50 0x7f306c0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:40.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0b1a50 0x7f306c0c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59076/0 (socket says 172.21.15.16:59076) 2023-11-13T10:56:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306a59c640 1 -- 172.21.15.16:0/2101439996 learned_addr learned my addr 172.21.15.16:0/2101439996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306a59c640 1 -- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c15cf60 msgr2=0x7f306c0cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306a59c640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c15cf60 0x7f306c0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306a59c640 1 -- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c0d1840 msgr2=0x7f306c1f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306a59c640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c0d1840 0x7f306c1f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.088+0000 7f306a59c640 1 -- 172.21.15.16:0/2101439996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3054009400 con 0x7f306c0b1a50 2023-11-13T10:56:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.089+0000 7f306a59c640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0b1a50 0x7f306c0c9c20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f305c00c410 tx=0x7f305c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.089+0000 7f3062ffd640 1 -- 172.21.15.16:0/2101439996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c005ce0 con 0x7f306c0b1a50 2023-11-13T10:56:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.089+0000 7f3062ffd640 1 -- 172.21.15.16:0/2101439996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c005e40 con 0x7f306c0b1a50 2023-11-13T10:56:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.089+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c0ca870 con 0x7f306c0b1a50 2023-11-13T10:56:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.089+0000 7f3062ffd640 1 -- 172.21.15.16:0/2101439996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c0072f0 con 0x7f306c0b1a50 2023-11-13T10:56:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.089+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c0cae30 con 0x7f306c0b1a50 2023-11-13T10:56:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.090+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c0cafc0 con 0x7f306c0b1a50 2023-11-13T10:56:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.091+0000 7f3062ffd640 1 -- 172.21.15.16:0/2101439996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f305c007510 con 0x7f306c0b1a50 2023-11-13T10:56:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.092+0000 7f3062ffd640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f304c06ae40 0x7f304c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.092+0000 7f3062ffd640 1 -- 172.21.15.16:0/2101439996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f305c0890c0 con 0x7f306c0b1a50 2023-11-13T10:56:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.093+0000 7f3069d9b640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f304c06ae40 0x7f304c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.094+0000 7f3069d9b640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f304c06ae40 0x7f304c06d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3054036000 tx=0x7f3054030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:40.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.096+0000 7f3062ffd640 1 -- 172.21.15.16:0/2101439996 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f305c053e50 con 0x7f306c0b1a50 2023-11-13T10:56:40.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.287+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f306c04e4d0 con 0x7f306c0b1a50 2023-11-13T10:56:40.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.287+0000 7f3062ffd640 1 -- 172.21.15.16:0/2101439996 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f305c057470 con 0x7f306c0b1a50 2023-11-13T10:56:40.295 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f304c06ae40 msgr2=0x7f304c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f304c06ae40 0x7f304c06d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3054036000 tx=0x7f3054030000 comp rx=0 tx=0).stop 2023-11-13T10:56:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0b1a50 msgr2=0x7f306c0c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0b1a50 0x7f306c0c9c20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f305c00c410 tx=0x7f305c00c8d0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 shutdown_connections 2023-11-13T10:56:40.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f304c06ae40 0x7f304c06d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c15cf60 0x7f306c0cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c0d1840 0x7f306c1f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 --2- 172.21.15.16:0/2101439996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f306c0b1a50 0x7f306c0c9c20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 >> 172.21.15.16:0/2101439996 conn(0x7f306c0d2b50 msgr2=0x7f306c0d1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 shutdown_connections 2023-11-13T10:56:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.290+0000 7f306b59e640 1 -- 172.21.15.16:0/2101439996 wait complete. 2023-11-13T10:56:40.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:56:40.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.532+0000 7f3f3277c640 1 Processor -- start 2023-11-13T10:56:40.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.532+0000 7f3f3277c640 1 -- start start 2023-11-13T10:56:40.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.534+0000 7f3f3277c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f2c06a190 0x7f3f2c06a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.534+0000 7f3f3277c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c06eba0 con 0x7f3f2c15be90 2023-11-13T10:56:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.534+0000 7f3f3277c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c06e5e0 con 0x7f3f2c144510 2023-11-13T10:56:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.534+0000 7f3f3277c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2c15c310 con 0x7f3f2c06a190 2023-11-13T10:56:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.534+0000 7f3f30d19640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f2c06a190 0x7f3f2c06a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:40.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.534+0000 7f3f2b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3f2c15be90 0x7f3f2c0cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46990/0 (socket says 172.21.15.16:46990) 2023-11-13T10:56:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.534+0000 7f3f2b7fe640 1 -- 172.21.15.16:0/2719262189 learned_addr learned my addr 172.21.15.16:0/2719262189 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.535+0000 7f3f30d19640 1 -- 172.21.15.16:0/2719262189 >> v1:172.21.15.184:6789/0 conn(0x7f3f2c144510 legacy=0x7f3f2c0719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.535+0000 7f3f30d19640 1 -- 172.21.15.16:0/2719262189 >> v1:172.21.15.16:6789/0 conn(0x7f3f2c15be90 legacy=0x7f3f2c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.535+0000 7f3f30d19640 1 -- 172.21.15.16:0/2719262189 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c06e8c0 con 0x7f3f2c06a190 2023-11-13T10:56:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.536+0000 7f3f30d19640 1 --2- 172.21.15.16:0/2719262189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c06a190 0x7f3f2c06a600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f240097a0 tx=0x7f3f2402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f097d1f4bc569719 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.536+0000 7f3f2affd640 1 -- 172.21.15.16:0/2719262189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24031830 con 0x7f3f2c06a190 2023-11-13T10:56:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.536+0000 7f3f2affd640 1 -- 172.21.15.16:0/2719262189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f24031990 con 0x7f3f2c06a190 2023-11-13T10:56:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.536+0000 7f3f2affd640 1 -- 172.21.15.16:0/2719262189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24031d20 con 0x7f3f2c06a190 2023-11-13T10:56:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.537+0000 7f3f3277c640 1 -- 172.21.15.16:0/2719262189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c06a190 msgr2=0x7f3f2c06a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.537+0000 7f3f3277c640 1 --2- 172.21.15.16:0/2719262189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c06a190 0x7f3f2c06a600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f240097a0 tx=0x7f3f2402b8d0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.537+0000 7f3f3277c640 1 -- 172.21.15.16:0/2719262189 shutdown_connections 2023-11-13T10:56:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.537+0000 7f3f3277c640 1 --2- 172.21.15.16:0/2719262189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c06a190 0x7f3f2c06a600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.537+0000 7f3f3277c640 1 -- 172.21.15.16:0/2719262189 >> 172.21.15.16:0/2719262189 conn(0x7f3f2c06b2a0 msgr2=0x7f3f2c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.537+0000 7f3f3277c640 1 -- 172.21.15.16:0/2719262189 shutdown_connections 2023-11-13T10:56:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.537+0000 7f3f3277c640 1 -- 172.21.15.16:0/2719262189 wait complete. 2023-11-13T10:56:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.538+0000 7f3f3277c640 1 Processor -- start 2023-11-13T10:56:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.539+0000 7f3f3277c640 1 -- start start 2023-11-13T10:56:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.539+0000 7f3f3277c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f2c06a190 0x7f3f2c14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.539+0000 7f3f3277c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f2c144510 0x7f3f2c13ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f3277c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c15be90 0x7f3f2c15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f3277c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0c5590 con 0x7f3f2c15be90 2023-11-13T10:56:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f3277c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c14f2e0 con 0x7f3f2c06a190 2023-11-13T10:56:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f3277c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c14f5c0 con 0x7f3f2c144510 2023-11-13T10:56:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f30d19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c15be90 0x7f3f2c15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f30d19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c15be90 0x7f3f2c15d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59088/0 (socket says 172.21.15.16:59088) 2023-11-13T10:56:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f30d19640 1 -- 172.21.15.16:0/2545413168 learned_addr learned my addr 172.21.15.16:0/2545413168 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f30d19640 1 -- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f2c144510 msgr2=0x7f3f2c13ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f30d19640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f2c144510 0x7f3f2c13ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f30d19640 1 -- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f2c06a190 msgr2=0x7f3f2c14ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f2bfff640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f2c06a190 0x7f3f2c14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f30d19640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f2c06a190 0x7f3f2c14ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.540+0000 7f3f30d19640 1 -- 172.21.15.16:0/2545413168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f24009400 con 0x7f3f2c15be90 2023-11-13T10:56:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.541+0000 7f3f30d19640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c15be90 0x7f3f2c15d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f240097a0 tx=0x7f3f240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.541+0000 7f3f28ff9640 1 -- 172.21.15.16:0/2545413168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2403a9e0 con 0x7f3f2c15be90 2023-11-13T10:56:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.541+0000 7f3f28ff9640 1 -- 172.21.15.16:0/2545413168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2403ab40 con 0x7f3f2c15be90 2023-11-13T10:56:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.541+0000 7f3f28ff9640 1 -- 172.21.15.16:0/2545413168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240443f0 con 0x7f3f2c15be90 2023-11-13T10:56:40.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.541+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2c14f8a0 con 0x7f3f2c15be90 2023-11-13T10:56:40.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.541+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c14fe60 con 0x7f3f2c15be90 2023-11-13T10:56:40.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.542+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2c150050 con 0x7f3f2c15be90 2023-11-13T10:56:40.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.543+0000 7f3f28ff9640 1 -- 172.21.15.16:0/2545413168 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f2404d450 con 0x7f3f2c15be90 2023-11-13T10:56:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.544+0000 7f3f28ff9640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f0c06ae90 0x7f3f0c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.544+0000 7f3f28ff9640 1 -- 172.21.15.16:0/2545413168 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f3f240abb80 con 0x7f3f2c15be90 2023-11-13T10:56:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.545+0000 7f3f2bfff640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f0c06ae90 0x7f3f0c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.546+0000 7f3f2bfff640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f0c06ae90 0x7f3f0c06d350 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c007fd0 tx=0x7f3f1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.548+0000 7f3f28ff9640 1 -- 172.21.15.16:0/2545413168 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f240768d0 con 0x7f3f2c15be90 2023-11-13T10:56:40.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.745+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3f2c0677a0 con 0x7f3f2c15be90 2023-11-13T10:56:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.746+0000 7f3f28ff9640 1 -- 172.21.15.16:0/2545413168 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3f24035080 con 0x7f3f2c15be90 2023-11-13T10:56:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:56:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f0c06ae90 msgr2=0x7f3f0c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f0c06ae90 0x7f3f0c06d350 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c007fd0 tx=0x7f3f1c009040 comp rx=0 tx=0).stop 2023-11-13T10:56:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c15be90 msgr2=0x7f3f2c15d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:40.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c15be90 0x7f3f2c15d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f240097a0 tx=0x7f3f240046c0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 shutdown_connections 2023-11-13T10:56:40.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f0c06ae90 0x7f3f0c06d350 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c15be90 0x7f3f2c15d9c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f2c144510 0x7f3f2c13ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 --2- 172.21.15.16:0/2545413168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f2c06a190 0x7f3f2c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 >> 172.21.15.16:0/2545413168 conn(0x7f3f2c06b2a0 msgr2=0x7f3f2c0cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 shutdown_connections 2023-11-13T10:56:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:40.748+0000 7f3f3277c640 1 -- 172.21.15.16:0/2545413168 wait complete. 2023-11-13T10:56:40.772 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:56:40.773 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:40.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:41.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.018+0000 7fb39e18c640 1 Processor -- start 2023-11-13T10:56:41.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.019+0000 7fb39e18c640 1 -- start start 2023-11-13T10:56:41.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.020+0000 7fb39e18c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3980cb190 0x7fb39815b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.020+0000 7fb39e18c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3980716b0 0x7fb39806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.020+0000 7fb39e18c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb398158c20 con 0x7fb39806a190 2023-11-13T10:56:41.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.020+0000 7fb39e18c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3980b1830 con 0x7fb3980cb190 2023-11-13T10:56:41.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.020+0000 7fb39e18c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39815b780 con 0x7fb3980716b0 2023-11-13T10:56:41.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.021+0000 7fb39d98b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb39806a190 0x7fb3980b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46996/0 (socket says 172.21.15.16:46996) 2023-11-13T10:56:41.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.021+0000 7fb39d98b640 1 -- 172.21.15.16:0/1096168601 learned_addr learned my addr 172.21.15.16:0/1096168601 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:41.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.021+0000 7fb39d18a640 1 --2- 172.21.15.16:0/1096168601 >> v2:172.21.15.184:3300/0 conn(0x7fb3980cb190 0x7fb39815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.021+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643334301 0 0) 0x7fb398158c20 con 0x7fb39806a190 2023-11-13T10:56:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.021+0000 7fb39c989640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3980716b0 0x7fb39806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.021+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb384003520 con 0x7fb39806a190 2023-11-13T10:56:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.021+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1086789724 0 0) 0x7fb384003520 con 0x7fb39806a190 2023-11-13T10:56:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3980716b0 msgr2=0x7fb39806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3980716b0 0x7fb39806f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 >> v2:172.21.15.184:3300/0 conn(0x7fb3980cb190 msgr2=0x7fb39815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 --2- 172.21.15.16:0/1096168601 >> v2:172.21.15.184:3300/0 conn(0x7fb3980cb190 0x7fb39815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39815ba60 con 0x7fb39806a190 2023-11-13T10:56:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fb3900019f0 con 0x7fb39806a190 2023-11-13T10:56:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 >> v1:172.21.15.16:6789/0 conn(0x7fb39806a190 legacy=0x7fb3980b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb384003970 0x7fb384003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb384004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3980716b0 0x7fb384005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3980b1830 con 0x7fb3840047c0 2023-11-13T10:56:41.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39815b780 con 0x7fb384003970 2023-11-13T10:56:41.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38400b980 con 0x7fb3980716b0 2023-11-13T10:56:41.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb39c989640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb384004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb39c989640 1 -- 172.21.15.16:0/1096168601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3980716b0 msgr2=0x7fb384005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:41.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb39c989640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3980716b0 0x7fb384005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb39c989640 1 -- 172.21.15.16:0/1096168601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb384003970 msgr2=0x7fb384003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:41.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb39c989640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb384003970 0x7fb384003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.022+0000 7fb39c989640 1 -- 172.21.15.16:0/1096168601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb384003520 con 0x7fb3840047c0 2023-11-13T10:56:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.023+0000 7fb39c989640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb384004c40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb388009ac0 tx=0x7fb3840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff171e8e915dcd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38802c710 con 0x7fb3840047c0 2023-11-13T10:56:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38802c870 con 0x7fb3840047c0 2023-11-13T10:56:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb38ffff640 1 -- 172.21.15.16:0/1096168601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38802cc00 con 0x7fb3840047c0 2023-11-13T10:56:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb39e18c640 1 -- 172.21.15.16:0/1096168601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 msgr2=0x7fb384004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb39e18c640 1 --2- 172.21.15.16:0/1096168601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb384004c40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb388009ac0 tx=0x7fb3840057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb39d98b640 1 -- 172.21.15.16:0/1096168601 reap_dead start 2023-11-13T10:56:41.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb39e18c640 1 -- 172.21.15.16:0/1096168601 shutdown_connections 2023-11-13T10:56:41.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb39e18c640 1 -- 172.21.15.16:0/1096168601 >> 172.21.15.16:0/1096168601 conn(0x7fb3980b04a0 msgr2=0x7fb3980b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:41.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb39e18c640 1 -- 172.21.15.16:0/1096168601 shutdown_connections 2023-11-13T10:56:41.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.024+0000 7fb39e18c640 1 -- 172.21.15.16:0/1096168601 wait complete. 2023-11-13T10:56:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.026+0000 7fb39e18c640 1 Processor -- start 2023-11-13T10:56:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.026+0000 7fb39e18c640 1 -- start start 2023-11-13T10:56:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.027+0000 7fb39e18c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb39806cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.027+0000 7fb39e18c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb398159d00 0x7fb39814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.027+0000 7fb39e18c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815a180 0x7fb39814c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.027+0000 7fb39e18c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39815a9b0 con 0x7fb3840047c0 2023-11-13T10:56:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.027+0000 7fb39e18c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981f65d0 con 0x7fb398159d00 2023-11-13T10:56:41.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.027+0000 7fb39e18c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981f6870 con 0x7fb39815a180 2023-11-13T10:56:41.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d18a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb39806cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d18a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb39806cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59106/0 (socket says 172.21.15.16:59106) 2023-11-13T10:56:41.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d18a640 1 -- 172.21.15.16:0/3542159539 learned_addr learned my addr 172.21.15.16:0/3542159539 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d98b640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815a180 0x7fb39814c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39c989640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb398159d00 0x7fb39814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d18a640 1 -- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815a180 msgr2=0x7fb39814c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d18a640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815a180 0x7fb39814c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d18a640 1 -- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb398159d00 msgr2=0x7fb39814bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d18a640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb398159d00 0x7fb39814bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d18a640 1 -- 172.21.15.16:0/3542159539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3981f6b50 con 0x7fb3840047c0 2023-11-13T10:56:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39c989640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb398159d00 0x7fb39814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.028+0000 7fb39d18a640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb39806cc90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb39400e860 tx=0x7fb39400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.029+0000 7fb38dffb640 1 -- 172.21.15.16:0/3542159539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394009600 con 0x7fb3840047c0 2023-11-13T10:56:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.029+0000 7fb38dffb640 1 -- 172.21.15.16:0/3542159539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39400c940 con 0x7fb3840047c0 2023-11-13T10:56:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.029+0000 7fb38dffb640 1 -- 172.21.15.16:0/3542159539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39400ccd0 con 0x7fb3840047c0 2023-11-13T10:56:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.029+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3981f6e30 con 0x7fb3840047c0 2023-11-13T10:56:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.029+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3981f73f0 con 0x7fb3840047c0 2023-11-13T10:56:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.030+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39804e4d0 con 0x7fb3840047c0 2023-11-13T10:56:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.031+0000 7fb38dffb640 1 -- 172.21.15.16:0/3542159539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb394013470 con 0x7fb3840047c0 2023-11-13T10:56:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.032+0000 7fb38dffb640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb37c06ae70 0x7fb37c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.032+0000 7fb38dffb640 1 -- 172.21.15.16:0/3542159539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fb394089bc0 con 0x7fb3840047c0 2023-11-13T10:56:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.033+0000 7fb39c989640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb37c06ae70 0x7fb37c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.034+0000 7fb39c989640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb37c06ae70 0x7fb37c06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb388036000 tx=0x7fb388030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.037+0000 7fb38dffb640 1 -- 172.21.15.16:0/3542159539 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb394058320 con 0x7fb3840047c0 2023-11-13T10:56:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.230+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3980b3100 con 0x7fb3840047c0 2023-11-13T10:56:41.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.233+0000 7fb38dffb640 1 -- 172.21.15.16:0/3542159539 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fb394057eb0 con 0x7fb3840047c0 2023-11-13T10:56:41.241 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:41.242 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:41.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb37c06ae70 msgr2=0x7fb37c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb37c06ae70 0x7fb37c06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb388036000 tx=0x7fb388030000 comp rx=0 tx=0).stop 2023-11-13T10:56:41.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 msgr2=0x7fb39806cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb39806cc90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb39400e860 tx=0x7fb39400ed20 comp rx=0 tx=0).stop 2023-11-13T10:56:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 shutdown_connections 2023-11-13T10:56:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb37c06ae70 0x7fb37c06d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb39815a180 0x7fb39814c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb398159d00 0x7fb39814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 --2- 172.21.15.16:0/3542159539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3840047c0 0x7fb39806cc90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 >> 172.21.15.16:0/3542159539 conn(0x7fb3980b04a0 msgr2=0x7fb3980b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.236+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 shutdown_connections 2023-11-13T10:56:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.237+0000 7fb39e18c640 1 -- 172.21.15.16:0/3542159539 wait complete. 2023-11-13T10:56:41.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:41.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:41.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.477+0000 7f61d6175640 1 Processor -- start 2023-11-13T10:56:41.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.477+0000 7f61d6175640 1 -- start start 2023-11-13T10:56:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.479+0000 7f61d6175640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61d015b8e0 0x7f61d006c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.479+0000 7f61d6175640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61d00ce260 0x7f61d00cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.479+0000 7f61d6175640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d006b860 con 0x7f61d015bcc0 2023-11-13T10:56:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.479+0000 7f61d6175640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d006bb40 con 0x7f61d00ce260 2023-11-13T10:56:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.479+0000 7f61d6175640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d006be20 con 0x7f61d015b8e0 2023-11-13T10:56:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.480+0000 7f61d5974640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61d00ce260 0x7f61d00cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.480+0000 7f61d5974640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61d00ce260 0x7f61d00cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59118/0 (socket says 172.21.15.16:59118) 2023-11-13T10:56:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.480+0000 7f61d5974640 1 -- 172.21.15.16:0/793062772 learned_addr learned my addr 172.21.15.16:0/793062772 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.480+0000 7f61d5173640 1 --2- 172.21.15.16:0/793062772 >> v2:172.21.15.184:3300/0 conn(0x7f61d015b8e0 0x7f61d006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.480+0000 7f61d5974640 1 -- 172.21.15.16:0/793062772 >> v2:172.21.15.184:3300/0 conn(0x7f61d015b8e0 msgr2=0x7f61d006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.480+0000 7f61d5974640 1 --2- 172.21.15.16:0/793062772 >> v2:172.21.15.184:3300/0 conn(0x7f61d015b8e0 0x7f61d006c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.480+0000 7f61d5974640 1 -- 172.21.15.16:0/793062772 >> v1:172.21.15.184:6789/0 conn(0x7f61d015bcc0 legacy=0x7f61d00cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.480+0000 7f61d5974640 1 -- 172.21.15.16:0/793062772 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d006c100 con 0x7f61d00ce260 2023-11-13T10:56:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.481+0000 7f61d5974640 1 --2- 172.21.15.16:0/793062772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 0x7f61d00cbae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f61d0071d70 tx=0x7f61c802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765d0333cf890fb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.482+0000 7f61c7fff640 1 -- 172.21.15.16:0/793062772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c802cb20 con 0x7f61d00ce260 2023-11-13T10:56:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.482+0000 7f61c7fff640 1 -- 172.21.15.16:0/793062772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c802cc80 con 0x7f61d00ce260 2023-11-13T10:56:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.482+0000 7f61c7fff640 1 -- 172.21.15.16:0/793062772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c80343f0 con 0x7f61d00ce260 2023-11-13T10:56:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.482+0000 7f61d6175640 1 -- 172.21.15.16:0/793062772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 msgr2=0x7f61d00cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.482+0000 7f61d6175640 1 --2- 172.21.15.16:0/793062772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 0x7f61d00cbae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f61d0071d70 tx=0x7f61c802bf80 comp rx=0 tx=0).stop 2023-11-13T10:56:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.482+0000 7f61d6175640 1 -- 172.21.15.16:0/793062772 shutdown_connections 2023-11-13T10:56:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.482+0000 7f61d6175640 1 --2- 172.21.15.16:0/793062772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 0x7f61d00cbae0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.482+0000 7f61d6175640 1 --2- 172.21.15.16:0/793062772 >> v2:172.21.15.184:3300/0 conn(0x7f61d015b8e0 0x7f61d006c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.482+0000 7f61d6175640 1 -- 172.21.15.16:0/793062772 >> 172.21.15.16:0/793062772 conn(0x7f61d00b0970 msgr2=0x7f61d00b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.483+0000 7f61d6175640 1 -- 172.21.15.16:0/793062772 shutdown_connections 2023-11-13T10:56:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.483+0000 7f61d6175640 1 -- 172.21.15.16:0/793062772 wait complete. 2023-11-13T10:56:41.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.484+0000 7f61d6175640 1 Processor -- start 2023-11-13T10:56:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.484+0000 7f61d6175640 1 -- start start 2023-11-13T10:56:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.485+0000 7f61d6175640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 0x7f61d0150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.485+0000 7f61d6175640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61d015b8e0 0x7f61d01f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.485+0000 7f61d6175640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61d00713c0 0x7f61d0151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.485+0000 7f61d6175640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d01f6c60 con 0x7f61d00ce260 2023-11-13T10:56:41.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.485+0000 7f61d6175640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d006f9d0 con 0x7f61d015b8e0 2023-11-13T10:56:41.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.485+0000 7f61d6175640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0151780 con 0x7f61d00713c0 2023-11-13T10:56:41.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.486+0000 7f61d5173640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 0x7f61d0150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.486+0000 7f61d5173640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 0x7f61d0150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59122/0 (socket says 172.21.15.16:59122) 2023-11-13T10:56:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.486+0000 7f61d5173640 1 -- 172.21.15.16:0/1007257548 learned_addr learned my addr 172.21.15.16:0/1007257548 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.486+0000 7f61d5173640 1 -- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61d00713c0 msgr2=0x7f61d0151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.486+0000 7f61d5173640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61d00713c0 0x7f61d0151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.486+0000 7f61d5173640 1 -- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61d015b8e0 msgr2=0x7f61d01f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.486+0000 7f61d5173640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61d015b8e0 0x7f61d01f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.486+0000 7f61d5173640 1 -- 172.21.15.16:0/1007257548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8009b30 con 0x7f61d00ce260 2023-11-13T10:56:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.486+0000 7f61d5173640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 0x7f61d0150590 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e9b0 tx=0x7f61cc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.487+0000 7f61c5ffb640 1 -- 172.21.15.16:0/1007257548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00c9a0 con 0x7f61d00ce260 2023-11-13T10:56:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.487+0000 7f61c5ffb640 1 -- 172.21.15.16:0/1007257548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc00cb00 con 0x7f61d00ce260 2023-11-13T10:56:41.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.487+0000 7f61c5ffb640 1 -- 172.21.15.16:0/1007257548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc013470 con 0x7f61d00ce260 2023-11-13T10:56:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.487+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d00acdd0 con 0x7f61d00ce260 2023-11-13T10:56:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.487+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d00ad090 con 0x7f61d00ce260 2023-11-13T10:56:41.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.488+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d0150ad0 con 0x7f61d00ce260 2023-11-13T10:56:41.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.489+0000 7f61c5ffb640 1 -- 172.21.15.16:0/1007257548 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61cc0135d0 con 0x7f61d00ce260 2023-11-13T10:56:41.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.490+0000 7f61c5ffb640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f61b406ae70 0x7f61b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.490+0000 7f61c5ffb640 1 -- 172.21.15.16:0/1007257548 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f61cc089c20 con 0x7f61d00ce260 2023-11-13T10:56:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.491+0000 7f61d4972640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f61b406ae70 0x7f61b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.492+0000 7f61d4972640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f61b406ae70 0x7f61b406d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61c0000f60 tx=0x7f61c00079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:41.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.495+0000 7f61c5ffb640 1 -- 172.21.15.16:0/1007257548 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61cc0548f0 con 0x7f61d00ce260 2023-11-13T10:56:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.691+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61d00cbff0 con 0x7f61d00ce260 2023-11-13T10:56:41.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.692+0000 7f61c5ffb640 1 -- 172.21.15.16:0/1007257548 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61cc0170a0 con 0x7f61d00ce260 2023-11-13T10:56:41.697 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f61b406ae70 msgr2=0x7f61b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f61b406ae70 0x7f61b406d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61c0000f60 tx=0x7f61c00079b0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 msgr2=0x7f61d0150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 0x7f61d0150590 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e9b0 tx=0x7f61cc00ee70 comp rx=0 tx=0).stop 2023-11-13T10:56:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 shutdown_connections 2023-11-13T10:56:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f61b406ae70 0x7f61b406d330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61d00713c0 0x7f61d0151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61d015b8e0 0x7f61d01f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 --2- 172.21.15.16:0/1007257548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d00ce260 0x7f61d0150590 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 >> 172.21.15.16:0/1007257548 conn(0x7f61d00b0970 msgr2=0x7f61d015c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 shutdown_connections 2023-11-13T10:56:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.695+0000 7f61d6175640 1 -- 172.21.15.16:0/1007257548 wait complete. 2023-11-13T10:56:41.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:41.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.947+0000 7fb984ea6640 1 Processor -- start 2023-11-13T10:56:41.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.947+0000 7fb984ea6640 1 -- start start 2023-11-13T10:56:41.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.948+0000 7fb984ea6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb9800d2ed0 0x7fb980167a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.948+0000 7fb984ea6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9800cac40 con 0x7fb98015ce70 2023-11-13T10:56:41.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.949+0000 7fb984ea6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9800ce0e0 con 0x7fb9800d1850 2023-11-13T10:56:41.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.949+0000 7fb984ea6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9800ce3c0 con 0x7fb9800d2ed0 2023-11-13T10:56:41.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.949+0000 7fb97f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb98015ce70 0x7fb9800cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46998/0 (socket says 172.21.15.16:46998) 2023-11-13T10:56:41.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.949+0000 7fb97f7fe640 1 -- 172.21.15.16:0/3491972268 learned_addr learned my addr 172.21.15.16:0/3491972268 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:41.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.949+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810300357 0 0) 0x7fb9800cac40 con 0x7fb98015ce70 2023-11-13T10:56:41.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.949+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb96c003520 con 0x7fb98015ce70 2023-11-13T10:56:41.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.949+0000 7fb97ffff640 1 --2- 172.21.15.16:0/3491972268 >> v2:172.21.15.184:3300/0 conn(0x7fb9800d2ed0 0x7fb980167a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 313371426 0 0) 0x7fb96c003520 con 0x7fb98015ce70 2023-11-13T10:56:41.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 >> v2:172.21.15.184:3300/0 conn(0x7fb9800d2ed0 msgr2=0x7fb980167a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:41.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 --2- 172.21.15.16:0/3491972268 >> v2:172.21.15.184:3300/0 conn(0x7fb9800d2ed0 0x7fb980167a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 >> v1:172.21.15.184:6789/0 conn(0x7fb9800d1850 legacy=0x7fb9800d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9800ce6a0 con 0x7fb98015ce70 2023-11-13T10:56:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fb9700019f0 con 0x7fb98015ce70 2023-11-13T10:56:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 >> v1:172.21.15.16:6789/0 conn(0x7fb98015ce70 legacy=0x7fb9800cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb96c003970 0x7fb96c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb96c0047c0 0x7fb96c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c005180 0x7fb96c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800ce3c0 con 0x7fb96c005180 2023-11-13T10:56:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c00c0d0 con 0x7fb96c003970 2023-11-13T10:56:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c00c290 con 0x7fb96c0047c0 2023-11-13T10:56:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.950+0000 7fb97ffff640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c005180 0x7fb96c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.951+0000 7fb97f7fe640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb96c003970 0x7fb96c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.951+0000 7fb97effd640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb96c0047c0 0x7fb96c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.951+0000 7fb97ffff640 1 -- 172.21.15.16:0/3491972268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb96c0047c0 msgr2=0x7fb96c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.951+0000 7fb97ffff640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb96c0047c0 0x7fb96c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.951+0000 7fb97ffff640 1 -- 172.21.15.16:0/3491972268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb96c003970 msgr2=0x7fb96c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.951+0000 7fb97ffff640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb96c003970 0x7fb96c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.951+0000 7fb97ffff640 1 -- 172.21.15.16:0/3491972268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c003520 con 0x7fb96c005180 2023-11-13T10:56:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.952+0000 7fb97ffff640 1 -- 172.21.15.16:0/3491972268 reap_dead start 2023-11-13T10:56:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.952+0000 7fb97ffff640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c005180 0x7fb96c00b980 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb97800ba30 tx=0x7fb96c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd48fcf850972ca server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.952+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978004020 con 0x7fb96c005180 2023-11-13T10:56:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.952+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9780041c0 con 0x7fb96c005180 2023-11-13T10:56:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.952+0000 7fb97e7fc640 1 -- 172.21.15.16:0/3491972268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978003610 con 0x7fb96c005180 2023-11-13T10:56:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.953+0000 7fb984ea6640 1 -- 172.21.15.16:0/3491972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c005180 msgr2=0x7fb96c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.953+0000 7fb984ea6640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c005180 0x7fb96c00b980 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb97800ba30 tx=0x7fb96c0057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.953+0000 7fb984ea6640 1 -- 172.21.15.16:0/3491972268 shutdown_connections 2023-11-13T10:56:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.953+0000 7fb984ea6640 1 --2- 172.21.15.16:0/3491972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c005180 0x7fb96c00b980 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.953+0000 7fb984ea6640 1 -- 172.21.15.16:0/3491972268 >> 172.21.15.16:0/3491972268 conn(0x7fb98008c580 msgr2=0x7fb98009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.953+0000 7fb984ea6640 1 -- 172.21.15.16:0/3491972268 shutdown_connections 2023-11-13T10:56:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.953+0000 7fb984ea6640 1 -- 172.21.15.16:0/3491972268 wait complete. 2023-11-13T10:56:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.955+0000 7fb984ea6640 1 Processor -- start 2023-11-13T10:56:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.955+0000 7fb984ea6640 1 -- start start 2023-11-13T10:56:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.956+0000 7fb984ea6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb96c005180 0x7fb9800b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.956+0000 7fb984ea6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9801e5ff0 0x7fb980148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.956+0000 7fb984ea6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9801f57b0 0x7fb9800b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.956+0000 7fb984ea6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801e67b0 con 0x7fb9801f57b0 2023-11-13T10:56:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.956+0000 7fb984ea6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800b8810 con 0x7fb9801e5ff0 2023-11-13T10:56:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.956+0000 7fb984ea6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800c7f90 con 0x7fb96c005180 2023-11-13T10:56:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.956+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9801f57b0 0x7fb9800b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.956+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9801f57b0 0x7fb9800b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59130/0 (socket says 172.21.15.16:59130) 2023-11-13T10:56:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.956+0000 7fb97ffff640 1 -- 172.21.15.16:0/594278586 learned_addr learned my addr 172.21.15.16:0/594278586 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.957+0000 7fb97f7fe640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb96c005180 0x7fb9800b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.957+0000 7fb97effd640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9801e5ff0 0x7fb980148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.957+0000 7fb97ffff640 1 -- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb96c005180 msgr2=0x7fb9800b82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.957+0000 7fb97ffff640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb96c005180 0x7fb9800b82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.957+0000 7fb97ffff640 1 -- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9801e5ff0 msgr2=0x7fb980148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.957+0000 7fb97ffff640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9801e5ff0 0x7fb980148ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.957+0000 7fb97ffff640 1 -- 172.21.15.16:0/594278586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9801e64d0 con 0x7fb9801f57b0 2023-11-13T10:56:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.957+0000 7fb97ffff640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9801f57b0 0x7fb9800b47a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb97800bb20 tx=0x7fb978004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.957+0000 7fb977fff640 1 -- 172.21.15.16:0/594278586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978032480 con 0x7fb9801f57b0 2023-11-13T10:56:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.958+0000 7fb977fff640 1 -- 172.21.15.16:0/594278586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb978032600 con 0x7fb9801f57b0 2023-11-13T10:56:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.958+0000 7fb977fff640 1 -- 172.21.15.16:0/594278586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9780329b0 con 0x7fb9801f57b0 2023-11-13T10:56:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.958+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9800c8230 con 0x7fb9801f57b0 2023-11-13T10:56:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.958+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9800c87f0 con 0x7fb9801f57b0 2023-11-13T10:56:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.959+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb980160630 con 0x7fb9801f57b0 2023-11-13T10:56:42.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.960+0000 7fb977fff640 1 -- 172.21.15.16:0/594278586 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb978042020 con 0x7fb9801f57b0 2023-11-13T10:56:42.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.960+0000 7fb977fff640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb96006ae70 0x7fb96006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.961+0000 7fb977fff640 1 -- 172.21.15.16:0/594278586 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fb9780abda0 con 0x7fb9801f57b0 2023-11-13T10:56:42.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.961+0000 7fb97f7fe640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb96006ae70 0x7fb96006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.962+0000 7fb97f7fe640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb96006ae70 0x7fb96006d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb9700013d0 tx=0x7fb970009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:42.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:41.965+0000 7fb977fff640 1 -- 172.21.15.16:0/594278586 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9780769e0 con 0x7fb9801f57b0 2023-11-13T10:56:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.161+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb98004e4d0 con 0x7fb9801f57b0 2023-11-13T10:56:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.162+0000 7fb977fff640 1 -- 172.21.15.16:0/594278586 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb97804a020 con 0x7fb9801f57b0 2023-11-13T10:56:42.167 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:42.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb96006ae70 msgr2=0x7fb96006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb96006ae70 0x7fb96006d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb9700013d0 tx=0x7fb970009040 comp rx=0 tx=0).stop 2023-11-13T10:56:42.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9801f57b0 msgr2=0x7fb9800b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9801f57b0 0x7fb9800b47a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb97800bb20 tx=0x7fb978004730 comp rx=0 tx=0).stop 2023-11-13T10:56:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 shutdown_connections 2023-11-13T10:56:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb96006ae70 0x7fb96006d330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9801f57b0 0x7fb9800b47a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9801e5ff0 0x7fb980148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 --2- 172.21.15.16:0/594278586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb96c005180 0x7fb9800b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 >> 172.21.15.16:0/594278586 conn(0x7fb98008c580 msgr2=0x7fb9800cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 shutdown_connections 2023-11-13T10:56:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.165+0000 7fb984ea6640 1 -- 172.21.15.16:0/594278586 wait complete. 2023-11-13T10:56:42.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:42.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.422+0000 7fcffa935640 1 Processor -- start 2023-11-13T10:56:42.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.422+0000 7fcffa935640 1 -- start start 2023-11-13T10:56:42.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.423+0000 7fcffa935640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcff40d1850 0x7fcff40d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.423+0000 7fcffa935640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff40ce0e0 con 0x7fcff40d2d00 2023-11-13T10:56:42.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.423+0000 7fcffa935640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff40ce3c0 con 0x7fcff415ce70 2023-11-13T10:56:42.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.423+0000 7fcffa935640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff40ce6a0 con 0x7fcff40d1850 2023-11-13T10:56:42.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.424+0000 7fcffa134640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcff40d2d00 0x7fcff40c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47010/0 (socket says 172.21.15.16:47010) 2023-11-13T10:56:42.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.424+0000 7fcffa134640 1 -- 172.21.15.16:0/2787068943 learned_addr learned my addr 172.21.15.16:0/2787068943 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:42.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.424+0000 7fcff9132640 1 --2- 172.21.15.16:0/2787068943 >> v2:172.21.15.184:3300/0 conn(0x7fcff40d1850 0x7fcff40d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.424+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225063816 0 0) 0x7fcff40ce0e0 con 0x7fcff40d2d00 2023-11-13T10:56:42.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.424+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfdc003520 con 0x7fcff40d2d00 2023-11-13T10:56:42.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4046151858 0 0) 0x7fcfdc003520 con 0x7fcff40d2d00 2023-11-13T10:56:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 >> v2:172.21.15.184:3300/0 conn(0x7fcff40d1850 msgr2=0x7fcff40d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 --2- 172.21.15.16:0/2787068943 >> v2:172.21.15.184:3300/0 conn(0x7fcff40d1850 0x7fcff40d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 >> v1:172.21.15.184:6789/0 conn(0x7fcff415ce70 legacy=0x7fcff40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff40ce980 con 0x7fcff40d2d00 2023-11-13T10:56:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fcfec001670 con 0x7fcff40d2d00 2023-11-13T10:56:42.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 >> v1:172.21.15.16:6789/0 conn(0x7fcff40d2d00 legacy=0x7fcff40c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfdc003970 0x7fcfdc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc0047c0 0x7fcfdc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc005180 0x7fcfdc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40ce6a0 con 0x7fcfdc0047c0 2023-11-13T10:56:42.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc00c020 con 0x7fcfdc005180 2023-11-13T10:56:42.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc00c1e0 con 0x7fcfdc003970 2023-11-13T10:56:42.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.425+0000 7fcff9132640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc0047c0 0x7fcfdc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.426+0000 7fcffa134640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc005180 0x7fcfdc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.426+0000 7fcff9132640 1 -- 172.21.15.16:0/2787068943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfdc003970 msgr2=0x7fcfdc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.426+0000 7fcff9132640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfdc003970 0x7fcfdc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.426+0000 7fcff9132640 1 -- 172.21.15.16:0/2787068943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc005180 msgr2=0x7fcfdc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.426+0000 7fcff9132640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc005180 0x7fcfdc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.426+0000 7fcff9132640 1 -- 172.21.15.16:0/2787068943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc003520 con 0x7fcfdc0047c0 2023-11-13T10:56:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.426+0000 7fcffa134640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc005180 0x7fcfdc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.426+0000 7fcffa134640 1 -- 172.21.15.16:0/2787068943 reap_dead start 2023-11-13T10:56:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.427+0000 7fcff9132640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc0047c0 0x7fcfdc004c40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009ac0 tx=0x7fcfdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb5d2628b39f85f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.427+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe802c6b0 con 0x7fcfdc0047c0 2023-11-13T10:56:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.427+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe802c810 con 0x7fcfdc0047c0 2023-11-13T10:56:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.427+0000 7fcff8931640 1 -- 172.21.15.16:0/2787068943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe802cba0 con 0x7fcfdc0047c0 2023-11-13T10:56:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.428+0000 7fcffa935640 1 -- 172.21.15.16:0/2787068943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc0047c0 msgr2=0x7fcfdc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.428+0000 7fcffa935640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc0047c0 0x7fcfdc004c40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009ac0 tx=0x7fcfdc0057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.428+0000 7fcffa935640 1 -- 172.21.15.16:0/2787068943 shutdown_connections 2023-11-13T10:56:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.428+0000 7fcffa935640 1 --2- 172.21.15.16:0/2787068943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc0047c0 0x7fcfdc004c40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.428+0000 7fcffa935640 1 -- 172.21.15.16:0/2787068943 >> 172.21.15.16:0/2787068943 conn(0x7fcff408c580 msgr2=0x7fcff409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:42.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.428+0000 7fcffa935640 1 -- 172.21.15.16:0/2787068943 shutdown_connections 2023-11-13T10:56:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.428+0000 7fcffa935640 1 -- 172.21.15.16:0/2787068943 wait complete. 2023-11-13T10:56:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.430+0000 7fcffa935640 1 Processor -- start 2023-11-13T10:56:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.430+0000 7fcffa935640 1 -- start start 2023-11-13T10:56:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.431+0000 7fcffa935640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc0047c0 0x7fcff415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.431+0000 7fcffa935640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff4155eb0 0x7fcff4156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.431+0000 7fcffa935640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff4142c80 0x7fcff4143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.431+0000 7fcffa935640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff4156870 con 0x7fcff4155eb0 2023-11-13T10:56:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.431+0000 7fcffa935640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40c91c0 con 0x7fcfdc0047c0 2023-11-13T10:56:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.431+0000 7fcffa935640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff415e670 con 0x7fcff4142c80 2023-11-13T10:56:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.431+0000 7fcff9132640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff4155eb0 0x7fcff4156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.431+0000 7fcff9132640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff4155eb0 0x7fcff4156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59138/0 (socket says 172.21.15.16:59138) 2023-11-13T10:56:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcff9132640 1 -- 172.21.15.16:0/834232398 learned_addr learned my addr 172.21.15.16:0/834232398 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcff9933640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc0047c0 0x7fcff415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcffa134640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff4142c80 0x7fcff4143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcff9132640 1 -- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff4142c80 msgr2=0x7fcff4143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcff9132640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff4142c80 0x7fcff4143130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcff9132640 1 -- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc0047c0 msgr2=0x7fcff415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcff9132640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc0047c0 0x7fcff415f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcff9132640 1 -- 172.21.15.16:0/834232398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff415e950 con 0x7fcff4155eb0 2023-11-13T10:56:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcffa134640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff4142c80 0x7fcff4143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.432+0000 7fcff9132640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff4155eb0 0x7fcff4156330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009ac0 tx=0x7fcfe8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.433+0000 7fcff27fc640 1 -- 172.21.15.16:0/834232398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8003810 con 0x7fcff4155eb0 2023-11-13T10:56:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.433+0000 7fcff27fc640 1 -- 172.21.15.16:0/834232398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe8003970 con 0x7fcff4155eb0 2023-11-13T10:56:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.433+0000 7fcff27fc640 1 -- 172.21.15.16:0/834232398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8003d00 con 0x7fcff4155eb0 2023-11-13T10:56:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.433+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff415ec30 con 0x7fcff4155eb0 2023-11-13T10:56:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.433+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff415f1f0 con 0x7fcff4155eb0 2023-11-13T10:56:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.434+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff415f380 con 0x7fcff4155eb0 2023-11-13T10:56:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.435+0000 7fcff27fc640 1 -- 172.21.15.16:0/834232398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcfe8040450 con 0x7fcff4155eb0 2023-11-13T10:56:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.436+0000 7fcff27fc640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcfe006ae70 0x7fcfe006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.436+0000 7fcff27fc640 1 -- 172.21.15.16:0/834232398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fcfe80ab9d0 con 0x7fcff4155eb0 2023-11-13T10:56:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.437+0000 7fcff9933640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcfe006ae70 0x7fcfe006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.438+0000 7fcff9933640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcfe006ae70 0x7fcfe006d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4001df0 tx=0x7fcfe4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.441+0000 7fcff27fc640 1 -- 172.21.15.16:0/834232398 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfe80766a0 con 0x7fcff4155eb0 2023-11-13T10:56:42.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.635+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcff404e4d0 con 0x7fcff4155eb0 2023-11-13T10:56:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.636+0000 7fcff27fc640 1 -- 172.21.15.16:0/834232398 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcfe8003e60 con 0x7fcff4155eb0 2023-11-13T10:56:42.640 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcfe006ae70 msgr2=0x7fcfe006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcfe006ae70 0x7fcfe006d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4001df0 tx=0x7fcfe4009040 comp rx=0 tx=0).stop 2023-11-13T10:56:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff4155eb0 msgr2=0x7fcff4156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff4155eb0 0x7fcff4156330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009ac0 tx=0x7fcfe8004730 comp rx=0 tx=0).stop 2023-11-13T10:56:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 shutdown_connections 2023-11-13T10:56:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcfe006ae70 0x7fcfe006d330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcff4142c80 0x7fcff4143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff4155eb0 0x7fcff4156330 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 --2- 172.21.15.16:0/834232398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfdc0047c0 0x7fcff415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 >> 172.21.15.16:0/834232398 conn(0x7fcff408c580 msgr2=0x7fcff40d24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 shutdown_connections 2023-11-13T10:56:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.638+0000 7fcffa935640 1 -- 172.21.15.16:0/834232398 wait complete. 2023-11-13T10:56:42.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.898+0000 7f6c3bea2640 1 Processor -- start 2023-11-13T10:56:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.898+0000 7f6c3bea2640 1 -- start start 2023-11-13T10:56:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.900+0000 7f6c3bea2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c3415cf90 0x7f6c340b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.900+0000 7f6c3bea2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6c340b3b90 0x7f6c34070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.900+0000 7f6c3bea2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c340b2e00 con 0x7f6c3406c3b0 2023-11-13T10:56:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.900+0000 7f6c3bea2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c340b21f0 con 0x7f6c3415cf90 2023-11-13T10:56:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.900+0000 7f6c3bea2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c340b24d0 con 0x7f6c340b3b90 2023-11-13T10:56:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.900+0000 7f6c3943d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f6c3406c3b0 0x7f6c340b3480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:40956/0 (socket says 172.21.15.16:40956) 2023-11-13T10:56:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.900+0000 7f6c3943d640 1 -- 172.21.15.16:0/2013595942 learned_addr learned my addr 172.21.15.16:0/2013595942 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.901+0000 7f6c39c3e640 1 --2- 172.21.15.16:0/2013595942 >> v2:172.21.15.16:3300/0 conn(0x7f6c3415cf90 0x7f6c340b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.901+0000 7f6c3a43f640 1 --2- 172.21.15.16:0/2013595942 >> v2:172.21.15.184:3300/0 conn(0x7f6c340b3b90 0x7f6c34070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.901+0000 7f6c39c3e640 1 -- 172.21.15.16:0/2013595942 >> v2:172.21.15.184:3300/0 conn(0x7f6c340b3b90 msgr2=0x7f6c34070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.901+0000 7f6c39c3e640 1 --2- 172.21.15.16:0/2013595942 >> v2:172.21.15.184:3300/0 conn(0x7f6c340b3b90 0x7f6c34070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.901+0000 7f6c39c3e640 1 -- 172.21.15.16:0/2013595942 >> v1:172.21.15.184:6789/0 conn(0x7f6c3406c3b0 legacy=0x7f6c340b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.901+0000 7f6c39c3e640 1 -- 172.21.15.16:0/2013595942 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340b27b0 con 0x7f6c3415cf90 2023-11-13T10:56:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.903+0000 7f6c39c3e640 1 --2- 172.21.15.16:0/2013595942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3415cf90 0x7f6c340b1c10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6c340adae0 tx=0x7f6c3002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e38abcc3c4dbc74 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.903+0000 7f6c38c3c640 1 -- 172.21.15.16:0/2013595942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3002cc80 con 0x7f6c3415cf90 2023-11-13T10:56:42.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.903+0000 7f6c38c3c640 1 -- 172.21.15.16:0/2013595942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3002cde0 con 0x7f6c3415cf90 2023-11-13T10:56:42.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.903+0000 7f6c38c3c640 1 -- 172.21.15.16:0/2013595942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30034660 con 0x7f6c3415cf90 2023-11-13T10:56:42.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.903+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2013595942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3415cf90 msgr2=0x7f6c340b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.903+0000 7f6c3bea2640 1 --2- 172.21.15.16:0/2013595942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3415cf90 0x7f6c340b1c10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6c340adae0 tx=0x7f6c3002bed0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.904+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2013595942 shutdown_connections 2023-11-13T10:56:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.904+0000 7f6c3bea2640 1 --2- 172.21.15.16:0/2013595942 >> v2:172.21.15.184:3300/0 conn(0x7f6c340b3b90 0x7f6c34070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.904+0000 7f6c3bea2640 1 --2- 172.21.15.16:0/2013595942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3415cf90 0x7f6c340b1c10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.904+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2013595942 >> 172.21.15.16:0/2013595942 conn(0x7f6c3406a190 msgr2=0x7f6c3406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.904+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2013595942 shutdown_connections 2023-11-13T10:56:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.904+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2013595942 wait complete. 2023-11-13T10:56:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.906+0000 7f6c3bea2640 1 Processor -- start 2023-11-13T10:56:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.906+0000 7f6c3bea2640 1 -- start start 2023-11-13T10:56:42.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c3bea2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3406c3b0 0x7f6c34145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c3bea2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c340b3b90 0x7f6c341e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c3bea2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c3415cf90 0x7f6c341e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c3bea2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34147160 con 0x7f6c3406c3b0 2023-11-13T10:56:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c3bea2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341e5f50 con 0x7f6c340b3b90 2023-11-13T10:56:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c3bea2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341464e0 con 0x7f6c3415cf90 2023-11-13T10:56:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c39c3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3406c3b0 0x7f6c34145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c39c3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3406c3b0 0x7f6c34145bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46490/0 (socket says 172.21.15.16:46490) 2023-11-13T10:56:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c39c3e640 1 -- 172.21.15.16:0/2130066496 learned_addr learned my addr 172.21.15.16:0/2130066496 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.907+0000 7f6c3a43f640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c3415cf90 0x7f6c341e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c3943d640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c340b3b90 0x7f6c341e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c39c3e640 1 -- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c3415cf90 msgr2=0x7f6c341e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c39c3e640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c3415cf90 0x7f6c341e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c39c3e640 1 -- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c340b3b90 msgr2=0x7f6c341e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:42.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c39c3e640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c340b3b90 0x7f6c341e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c39c3e640 1 -- 172.21.15.16:0/2130066496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30009a70 con 0x7f6c3406c3b0 2023-11-13T10:56:42.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c3943d640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c340b3b90 0x7f6c341e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c39c3e640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3406c3b0 0x7f6c34145bf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6c30037040 tx=0x7f6c30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:42.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c2a7fc640 1 -- 172.21.15.16:0/2130066496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30040650 con 0x7f6c3406c3b0 2023-11-13T10:56:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c2a7fc640 1 -- 172.21.15.16:0/2130066496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c300407b0 con 0x7f6c3406c3b0 2023-11-13T10:56:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c2a7fc640 1 -- 172.21.15.16:0/2130066496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30040b60 con 0x7f6c3406c3b0 2023-11-13T10:56:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.908+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c341467c0 con 0x7f6c3406c3b0 2023-11-13T10:56:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.909+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c34146d80 con 0x7f6c3406c3b0 2023-11-13T10:56:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.910+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c34072520 con 0x7f6c3406c3b0 2023-11-13T10:56:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.911+0000 7f6c2a7fc640 1 -- 172.21.15.16:0/2130066496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c30040cc0 con 0x7f6c3406c3b0 2023-11-13T10:56:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.912+0000 7f6c2a7fc640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6c1806ae40 0x7f6c1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.912+0000 7f6c2a7fc640 1 -- 172.21.15.16:0/2130066496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f6c300ac6b0 con 0x7f6c3406c3b0 2023-11-13T10:56:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.913+0000 7f6c3943d640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6c1806ae40 0x7f6c1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.914+0000 7f6c3943d640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6c1806ae40 0x7f6c1806d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6c340b3b30 tx=0x7f6c24009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:42.917+0000 7f6c2a7fc640 1 -- 172.21.15.16:0/2130066496 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c300774c0 con 0x7f6c3406c3b0 2023-11-13T10:56:43.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.113+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6c340adea0 con 0x7f6c3406c3b0 2023-11-13T10:56:43.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.114+0000 7f6c2a7fc640 1 -- 172.21.15.16:0/2130066496 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6c30009450 con 0x7f6c3406c3b0 2023-11-13T10:56:43.119 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6c1806ae40 msgr2=0x7f6c1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6c1806ae40 0x7f6c1806d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6c340b3b30 tx=0x7f6c24009040 comp rx=0 tx=0).stop 2023-11-13T10:56:43.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3406c3b0 msgr2=0x7f6c34145bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:43.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3406c3b0 0x7f6c34145bf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6c30037040 tx=0x7f6c30004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:43.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 shutdown_connections 2023-11-13T10:56:43.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6c1806ae40 0x7f6c1806d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c3415cf90 0x7f6c341e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c340b3b90 0x7f6c341e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 --2- 172.21.15.16:0/2130066496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3406c3b0 0x7f6c34145bf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.117+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 >> 172.21.15.16:0/2130066496 conn(0x7f6c3406a190 msgr2=0x7f6c340ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:43.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.118+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 shutdown_connections 2023-11-13T10:56:43.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.118+0000 7f6c3bea2640 1 -- 172.21.15.16:0/2130066496 wait complete. 2023-11-13T10:56:43.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:56:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.338+0000 7f185aa6d640 1 Processor -- start 2023-11-13T10:56:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.339+0000 7f185aa6d640 1 -- start start 2023-11-13T10:56:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.340+0000 7f185aa6d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f18540b30e0 0x7f18540d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.340+0000 7f185aa6d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540b3cf0 con 0x7f18540d2ae0 2023-11-13T10:56:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.340+0000 7f185aa6d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540d3480 con 0x7f185415cfc0 2023-11-13T10:56:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.340+0000 7f185aa6d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18540b3550 con 0x7f18540b30e0 2023-11-13T10:56:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f185926a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f18540d2ae0 0x7f18540b29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35354/0 (socket says 172.21.15.16:35354) 2023-11-13T10:56:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f185926a640 1 -- 172.21.15.16:0/2002093971 learned_addr learned my addr 172.21.15.16:0/2002093971 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f185a26c640 1 --2- 172.21.15.16:0/2002093971 >> v2:172.21.15.184:3300/0 conn(0x7f18540b30e0 0x7f18540d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801043845 0 0) 0x7f18540b3cf0 con 0x7f18540d2ae0 2023-11-13T10:56:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1840003520 con 0x7f18540d2ae0 2023-11-13T10:56:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3651127551 0 0) 0x7f1840003520 con 0x7f18540d2ae0 2023-11-13T10:56:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 >> v2:172.21.15.184:3300/0 conn(0x7f18540b30e0 msgr2=0x7f18540d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f1858a69640 1 --2- 172.21.15.16:0/2002093971 >> v2:172.21.15.184:3300/0 conn(0x7f18540b30e0 0x7f18540d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 >> v1:172.21.15.184:6789/0 conn(0x7f185415cfc0 legacy=0x7f18540cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:43.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540cde80 con 0x7f18540d2ae0 2023-11-13T10:56:43.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.341+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f1844001670 con 0x7f18540d2ae0 2023-11-13T10:56:43.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 >> v1:172.21.15.16:6789/0 conn(0x7f18540d2ae0 legacy=0x7f18540b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:43.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f1858a69640 1 --2- 172.21.15.16:0/2002093971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18400039c0 0x7f1840003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:43.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f1858a69640 1 --2- 172.21.15.16:0/2002093971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f1840004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:43.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f1858a69640 1 --2- 172.21.15.16:0/2002093971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18400052a0 0x7f184000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:43.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540b3550 con 0x7f18400047c0 2023-11-13T10:56:43.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184000c1c0 con 0x7f18400039c0 2023-11-13T10:56:43.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184000c380 con 0x7f18400052a0 2023-11-13T10:56:43.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f185a26c640 1 --2- 172.21.15.16:0/2002093971 >> v2:172.21.15.184:3300/0 conn(0x7f18540b30e0 0x7f18540d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:56:43.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f185926a640 1 --2- 172.21.15.16:0/2002093971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f1840004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:43.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f185926a640 1 -- 172.21.15.16:0/2002093971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18400052a0 msgr2=0x7f184000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:43.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f185926a640 1 --2- 172.21.15.16:0/2002093971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18400052a0 0x7f184000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f185926a640 1 -- 172.21.15.16:0/2002093971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18400039c0 msgr2=0x7f1840003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:43.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f185926a640 1 --2- 172.21.15.16:0/2002093971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18400039c0 0x7f1840003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f185926a640 1 -- 172.21.15.16:0/2002093971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1840003520 con 0x7f18400047c0 2023-11-13T10:56:43.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.342+0000 7f185a26c640 1 -- 172.21.15.16:0/2002093971 reap_dead start 2023-11-13T10:56:43.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.343+0000 7f185926a640 1 --2- 172.21.15.16:0/2002093971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f1840004c40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f184400c0e0 tx=0x7f18400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b843ffb48f8411e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.344+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844005ad0 con 0x7f18400047c0 2023-11-13T10:56:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.344+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1844005c30 con 0x7f18400047c0 2023-11-13T10:56:43.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.344+0000 7f1858a69640 1 -- 172.21.15.16:0/2002093971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184402ca50 con 0x7f18400047c0 2023-11-13T10:56:43.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.344+0000 7f185aa6d640 1 -- 172.21.15.16:0/2002093971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 msgr2=0x7f1840004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:43.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.344+0000 7f185aa6d640 1 --2- 172.21.15.16:0/2002093971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f1840004c40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f184400c0e0 tx=0x7f18400057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.344+0000 7f185aa6d640 1 -- 172.21.15.16:0/2002093971 shutdown_connections 2023-11-13T10:56:43.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.344+0000 7f185aa6d640 1 --2- 172.21.15.16:0/2002093971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f1840004c40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.344+0000 7f185aa6d640 1 -- 172.21.15.16:0/2002093971 >> 172.21.15.16:0/2002093971 conn(0x7f185408c570 msgr2=0x7f185409be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.344+0000 7f185aa6d640 1 -- 172.21.15.16:0/2002093971 shutdown_connections 2023-11-13T10:56:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.345+0000 7f185aa6d640 1 -- 172.21.15.16:0/2002093971 wait complete. 2023-11-13T10:56:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.346+0000 7f185aa6d640 1 Processor -- start 2023-11-13T10:56:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.346+0000 7f185aa6d640 1 -- start start 2023-11-13T10:56:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.347+0000 7f185aa6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f18540b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.347+0000 7f185aa6d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18540c9160 0x7f18540b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.347+0000 7f185aa6d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1854158ec0 0x7f18540caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.347+0000 7f185aa6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540b6190 con 0x7f18400047c0 2023-11-13T10:56:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.347+0000 7f185aa6d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540b6450 con 0x7f18540c9160 2023-11-13T10:56:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.347+0000 7f185aa6d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18540b6730 con 0x7f1854158ec0 2023-11-13T10:56:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.347+0000 7f1859a6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f18540b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f1859a6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f18540b74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46508/0 (socket says 172.21.15.16:46508) 2023-11-13T10:56:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f1859a6b640 1 -- 172.21.15.16:0/1477702133 learned_addr learned my addr 172.21.15.16:0/1477702133 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f1859a6b640 1 -- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1854158ec0 msgr2=0x7f18540caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f1859a6b640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1854158ec0 0x7f18540caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f185926a640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18540c9160 0x7f18540b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f1859a6b640 1 -- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18540c9160 msgr2=0x7f18540b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f1859a6b640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18540c9160 0x7f18540b79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f1859a6b640 1 -- 172.21.15.16:0/1477702133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540b6a10 con 0x7f18400047c0 2023-11-13T10:56:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f1859a6b640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f18540b74b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1850001eb0 tx=0x7f185000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f184a7fc640 1 -- 172.21.15.16:0/1477702133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18500052d0 con 0x7f18400047c0 2023-11-13T10:56:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f184a7fc640 1 -- 172.21.15.16:0/1477702133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1850005450 con 0x7f18400047c0 2023-11-13T10:56:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f184a7fc640 1 -- 172.21.15.16:0/1477702133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850012870 con 0x7f18400047c0 2023-11-13T10:56:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.348+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18540b6cf0 con 0x7f18400047c0 2023-11-13T10:56:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.349+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18540cb1e0 con 0x7f18400047c0 2023-11-13T10:56:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.350+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18540b1ba0 con 0x7f18400047c0 2023-11-13T10:56:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.350+0000 7f184a7fc640 1 -- 172.21.15.16:0/1477702133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18500129d0 con 0x7f18400047c0 2023-11-13T10:56:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.351+0000 7f184a7fc640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f183806ae70 0x7f183806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.351+0000 7f184a7fc640 1 -- 172.21.15.16:0/1477702133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f1850089500 con 0x7f18400047c0 2023-11-13T10:56:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.352+0000 7f185926a640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f183806ae70 0x7f183806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.353+0000 7f185926a640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f183806ae70 0x7f183806d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1844034000 tx=0x7f1844009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.356+0000 7f184a7fc640 1 -- 172.21.15.16:0/1477702133 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18500541d0 con 0x7f18400047c0 2023-11-13T10:56:43.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.561+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f185404e4d0 con 0x7f18400047c0 2023-11-13T10:56:43.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.562+0000 7f184a7fc640 1 -- 172.21.15.16:0/1477702133 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f18500577f0 con 0x7f18400047c0 2023-11-13T10:56:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:56:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f183806ae70 msgr2=0x7f183806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f183806ae70 0x7f183806d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1844034000 tx=0x7f1844009030 comp rx=0 tx=0).stop 2023-11-13T10:56:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 msgr2=0x7f18540b74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f18540b74b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1850001eb0 tx=0x7f185000c960 comp rx=0 tx=0).stop 2023-11-13T10:56:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 shutdown_connections 2023-11-13T10:56:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f183806ae70 0x7f183806d330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1854158ec0 0x7f18540caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18540c9160 0x7f18540b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 --2- 172.21.15.16:0/1477702133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18400047c0 0x7f18540b74b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.564+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 >> 172.21.15.16:0/1477702133 conn(0x7f185408c570 msgr2=0x7f18540b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.565+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 shutdown_connections 2023-11-13T10:56:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:43.565+0000 7f185aa6d640 1 -- 172.21.15.16:0/1477702133 wait complete. 2023-11-13T10:56:43.589 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-13T10:56:43.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:43.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-13T10:56:43.683 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T10:56:43.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T10:56:43.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:43.685 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-13T10:56:43.741 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:56:43.742 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:43.811 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:56:43.811 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:43.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.099+0000 7f0ec0831640 1 Processor -- start 2023-11-13T10:56:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.100+0000 7f0ec0831640 1 -- start start 2023-11-13T10:56:44.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.101+0000 7f0ec0831640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ebc15b190 0x7f0ebc06f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.101+0000 7f0ec0831640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 0x7f0ebc06a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.101+0000 7f0ec0831640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ebc158c20 con 0x7f0ebc0cb190 2023-11-13T10:56:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.101+0000 7f0ec0831640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ebc06ab40 con 0x7f0ebc15b190 2023-11-13T10:56:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.101+0000 7f0ec0831640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0b1830 con 0x7f0ebc06a190 2023-11-13T10:56:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.102+0000 7f0ebad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0ebc0cb190 0x7f0ebc0716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35364/0 (socket says 172.21.15.16:35364) 2023-11-13T10:56:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.102+0000 7f0ebad9d640 1 -- 172.21.15.16:0/2164374742 learned_addr learned my addr 172.21.15.16:0/2164374742 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.102+0000 7f0eba59c640 1 --2- 172.21.15.16:0/2164374742 >> v2:172.21.15.16:3300/0 conn(0x7f0ebc15b190 0x7f0ebc06f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.102+0000 7f0ebb59e640 1 --2- 172.21.15.16:0/2164374742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 0x7f0ebc06a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.102+0000 7f0eb9d9b640 1 -- 172.21.15.16:0/2164374742 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467906453 0 0) 0x7f0ebc158c20 con 0x7f0ebc0cb190 2023-11-13T10:56:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.102+0000 7f0eb9d9b640 1 -- 172.21.15.16:0/2164374742 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea4003520 con 0x7f0ebc0cb190 2023-11-13T10:56:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.103+0000 7f0eba59c640 1 -- 172.21.15.16:0/2164374742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 msgr2=0x7f0ebc06a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.103+0000 7f0eba59c640 1 --2- 172.21.15.16:0/2164374742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 0x7f0ebc06a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.103+0000 7f0eba59c640 1 -- 172.21.15.16:0/2164374742 >> v1:172.21.15.16:6789/0 conn(0x7f0ebc0cb190 legacy=0x7f0ebc0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.103+0000 7f0eba59c640 1 -- 172.21.15.16:0/2164374742 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc15b780 con 0x7f0ebc15b190 2023-11-13T10:56:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.104+0000 7f0eba59c640 1 --2- 172.21.15.16:0/2164374742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 0x7f0ebc06f970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00097a0 tx=0x7f0eb002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd3b2f427c54ae3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.104+0000 7f0eb9d9b640 1 -- 172.21.15.16:0/2164374742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0031780 con 0x7f0ebc15b190 2023-11-13T10:56:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.104+0000 7f0eb9d9b640 1 -- 172.21.15.16:0/2164374742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb00318e0 con 0x7f0ebc15b190 2023-11-13T10:56:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.104+0000 7f0eb9d9b640 1 -- 172.21.15.16:0/2164374742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0031c70 con 0x7f0ebc15b190 2023-11-13T10:56:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.105+0000 7f0ec0831640 1 -- 172.21.15.16:0/2164374742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 msgr2=0x7f0ebc06f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.105+0000 7f0ec0831640 1 --2- 172.21.15.16:0/2164374742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 0x7f0ebc06f970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00097a0 tx=0x7f0eb002b820 comp rx=0 tx=0).stop 2023-11-13T10:56:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.105+0000 7f0ec0831640 1 -- 172.21.15.16:0/2164374742 shutdown_connections 2023-11-13T10:56:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.105+0000 7f0ec0831640 1 --2- 172.21.15.16:0/2164374742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 0x7f0ebc06a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.105+0000 7f0ec0831640 1 --2- 172.21.15.16:0/2164374742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 0x7f0ebc06f970 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.105+0000 7f0ec0831640 1 -- 172.21.15.16:0/2164374742 >> 172.21.15.16:0/2164374742 conn(0x7f0ebc0b04a0 msgr2=0x7f0ebc0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.105+0000 7f0ec0831640 1 -- 172.21.15.16:0/2164374742 shutdown_connections 2023-11-13T10:56:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.105+0000 7f0ec0831640 1 -- 172.21.15.16:0/2164374742 wait complete. 2023-11-13T10:56:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.107+0000 7f0ec0831640 1 Processor -- start 2023-11-13T10:56:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.107+0000 7f0ec0831640 1 -- start start 2023-11-13T10:56:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.108+0000 7f0ec0831640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 0x7f0ebc146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.108+0000 7f0ec0831640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ebc0cb190 0x7f0ebc149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.108+0000 7f0ec0831640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 0x7f0ebc1f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.108+0000 7f0ec0831640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0af8e0 con 0x7f0ebc15b190 2023-11-13T10:56:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.108+0000 7f0ec0831640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc1f62a0 con 0x7f0ebc0cb190 2023-11-13T10:56:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.108+0000 7f0ec0831640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc06d9e0 con 0x7f0ebc06a190 2023-11-13T10:56:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.108+0000 7f0ebb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 0x7f0ebc1f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.108+0000 7f0ebb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 0x7f0ebc1f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46532/0 (socket says 172.21.15.16:46532) 2023-11-13T10:56:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.108+0000 7f0ebb59e640 1 -- 172.21.15.16:0/2030707964 learned_addr learned my addr 172.21.15.16:0/2030707964 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.109+0000 7f0ebad9d640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 0x7f0ebc146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.109+0000 7f0eba59c640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ebc0cb190 0x7f0ebc149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.109+0000 7f0ebb59e640 1 -- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 msgr2=0x7f0ebc146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.109+0000 7f0ebb59e640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 0x7f0ebc146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.109+0000 7f0ebb59e640 1 -- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ebc0cb190 msgr2=0x7f0ebc149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.109+0000 7f0ebb59e640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ebc0cb190 0x7f0ebc149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.109+0000 7f0ebb59e640 1 -- 172.21.15.16:0/2030707964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb0009400 con 0x7f0ebc15b190 2023-11-13T10:56:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.109+0000 7f0ebb59e640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 0x7f0ebc1f66c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4007e40 tx=0x7f0eb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.110+0000 7f0ea37fe640 1 -- 172.21.15.16:0/2030707964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4014620 con 0x7f0ebc15b190 2023-11-13T10:56:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.110+0000 7f0ea37fe640 1 -- 172.21.15.16:0/2030707964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb400ee90 con 0x7f0ebc15b190 2023-11-13T10:56:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.110+0000 7f0ea37fe640 1 -- 172.21.15.16:0/2030707964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40149e0 con 0x7f0ebc15b190 2023-11-13T10:56:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.110+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ebc149cd0 con 0x7f0ebc15b190 2023-11-13T10:56:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.110+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ebc0af600 con 0x7f0ebc15b190 2023-11-13T10:56:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.111+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ebc1f63e0 con 0x7f0ebc15b190 2023-11-13T10:56:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.112+0000 7f0ea37fe640 1 -- 172.21.15.16:0/2030707964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0eb4014b40 con 0x7f0ebc15b190 2023-11-13T10:56:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.113+0000 7f0ea37fe640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0ea806ae40 0x7f0ea806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.113+0000 7f0ea37fe640 1 -- 172.21.15.16:0/2030707964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f0eb408a410 con 0x7f0ebc15b190 2023-11-13T10:56:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.114+0000 7f0ebad9d640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0ea806ae40 0x7f0ea806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.115+0000 7f0ebad9d640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0ea806ae40 0x7f0ea806d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0eac005880 tx=0x7f0eac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.117+0000 7f0ea37fe640 1 -- 172.21.15.16:0/2030707964 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ebc1f63e0 con 0x7f0ebc15b190 2023-11-13T10:56:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.315+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ebc1f63e0 con 0x7f0ebc15b190 2023-11-13T10:56:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.317+0000 7f0ea37fe640 1 -- 172.21.15.16:0/2030707964 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f0ebc1f63e0 con 0x7f0ebc15b190 2023-11-13T10:56:44.322 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:44.323 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0ea806ae40 msgr2=0x7f0ea806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0ea806ae40 0x7f0ea806d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0eac005880 tx=0x7f0eac009040 comp rx=0 tx=0).stop 2023-11-13T10:56:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 msgr2=0x7f0ebc1f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 0x7f0ebc1f66c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4007e40 tx=0x7f0eb400e4b0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 shutdown_connections 2023-11-13T10:56:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0ea806ae40 0x7f0ea806d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc15b190 0x7f0ebc1f66c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ebc0cb190 0x7f0ebc149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 --2- 172.21.15.16:0/2030707964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ebc06a190 0x7f0ebc146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 >> 172.21.15.16:0/2030707964 conn(0x7f0ebc0b04a0 msgr2=0x7f0ebc1572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 shutdown_connections 2023-11-13T10:56:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.320+0000 7f0ec0831640 1 -- 172.21.15.16:0/2030707964 wait complete. 2023-11-13T10:56:44.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:44.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.571+0000 7f50e1602640 1 Processor -- start 2023-11-13T10:56:44.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.571+0000 7f50e1602640 1 -- start start 2023-11-13T10:56:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.572+0000 7f50e1602640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc0d1850 0x7f50dc0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.572+0000 7f50e1602640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f50dc0d2d00 0x7f50dc0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.572+0000 7f50e1602640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50dc0ce0e0 con 0x7f50dc15ce70 2023-11-13T10:56:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.572+0000 7f50e1602640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50dc0ce3c0 con 0x7f50dc0d2d00 2023-11-13T10:56:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.572+0000 7f50e1602640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0ce6a0 con 0x7f50dc0d1850 2023-11-13T10:56:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.572+0000 7f50dbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f50dc15ce70 0x7f50dc0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35374/0 (socket says 172.21.15.16:35374) 2023-11-13T10:56:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.572+0000 7f50dbfff640 1 -- 172.21.15.16:0/656075379 learned_addr learned my addr 172.21.15.16:0/656075379 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.573+0000 7f50e0e01640 1 --2- 172.21.15.16:0/656075379 >> v2:172.21.15.16:3300/0 conn(0x7f50dc0d2d00 0x7f50dc0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.573+0000 7f50db7fe640 1 --2- 172.21.15.16:0/656075379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc0d1850 0x7f50dc0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.573+0000 7f50daffd640 1 -- 172.21.15.16:0/656075379 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255557261 0 0) 0x7f50dc0ce0e0 con 0x7f50dc15ce70 2023-11-13T10:56:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.573+0000 7f50daffd640 1 -- 172.21.15.16:0/656075379 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c8003520 con 0x7f50dc15ce70 2023-11-13T10:56:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.573+0000 7f50e0e01640 1 -- 172.21.15.16:0/656075379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc0d1850 msgr2=0x7f50dc0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.573+0000 7f50e0e01640 1 --2- 172.21.15.16:0/656075379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc0d1850 0x7f50dc0d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.573+0000 7f50e0e01640 1 -- 172.21.15.16:0/656075379 >> v1:172.21.15.16:6789/0 conn(0x7f50dc15ce70 legacy=0x7f50dc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.573+0000 7f50e0e01640 1 -- 172.21.15.16:0/656075379 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc0ce980 con 0x7f50dc0d2d00 2023-11-13T10:56:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.573+0000 7f50db7fe640 1 --2- 172.21.15.16:0/656075379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc0d1850 0x7f50dc0d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:56:44.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.575+0000 7f50e0e01640 1 --2- 172.21.15.16:0/656075379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d2d00 0x7f50dc0c9c30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f50d40097a0 tx=0x7f50d402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f06a0b15c28bd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.575+0000 7f50daffd640 1 -- 172.21.15.16:0/656075379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d402d040 con 0x7f50dc0d2d00 2023-11-13T10:56:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.575+0000 7f50daffd640 1 -- 172.21.15.16:0/656075379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d4031810 con 0x7f50dc0d2d00 2023-11-13T10:56:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.575+0000 7f50daffd640 1 -- 172.21.15.16:0/656075379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4031ba0 con 0x7f50dc0d2d00 2023-11-13T10:56:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.575+0000 7f50e1602640 1 -- 172.21.15.16:0/656075379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d2d00 msgr2=0x7f50dc0c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.575+0000 7f50e1602640 1 --2- 172.21.15.16:0/656075379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d2d00 0x7f50dc0c9c30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f50d40097a0 tx=0x7f50d402b8b0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.576+0000 7f50e1602640 1 -- 172.21.15.16:0/656075379 shutdown_connections 2023-11-13T10:56:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.576+0000 7f50e1602640 1 --2- 172.21.15.16:0/656075379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d2d00 0x7f50dc0c9c30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.576+0000 7f50e1602640 1 --2- 172.21.15.16:0/656075379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc0d1850 0x7f50dc0d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.576+0000 7f50e1602640 1 -- 172.21.15.16:0/656075379 >> 172.21.15.16:0/656075379 conn(0x7f50dc08c580 msgr2=0x7f50dc09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.576+0000 7f50e1602640 1 -- 172.21.15.16:0/656075379 shutdown_connections 2023-11-13T10:56:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.576+0000 7f50e1602640 1 -- 172.21.15.16:0/656075379 wait complete. 2023-11-13T10:56:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.577+0000 7f50e1602640 1 Processor -- start 2023-11-13T10:56:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.578+0000 7f50e1602640 1 -- start start 2023-11-13T10:56:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50e1602640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d1850 0x7f50dc15b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50e1602640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50dc0d2d00 0x7f50dc09ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50e1602640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc15ce70 0x7f50dc06b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50e1602640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0b6100 con 0x7f50dc0d1850 2023-11-13T10:56:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50e1602640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc15b870 con 0x7f50dc0d2d00 2023-11-13T10:56:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50e1602640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc09d460 con 0x7f50dc15ce70 2023-11-13T10:56:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d1850 0x7f50dc15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d1850 0x7f50dc15b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46550/0 (socket says 172.21.15.16:46550) 2023-11-13T10:56:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50dbfff640 1 -- 172.21.15.16:0/2862758144 learned_addr learned my addr 172.21.15.16:0/2862758144 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50e0e01640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc15ce70 0x7f50dc06b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50dbfff640 1 -- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc15ce70 msgr2=0x7f50dc06b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50dbfff640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc15ce70 0x7f50dc06b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50dbfff640 1 -- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50dc0d2d00 msgr2=0x7f50dc09ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50db7fe640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50dc0d2d00 0x7f50dc09ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50dbfff640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50dc0d2d00 0x7f50dc09ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.579+0000 7f50dbfff640 1 -- 172.21.15.16:0/2862758144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d4009400 con 0x7f50dc0d1850 2023-11-13T10:56:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.580+0000 7f50dbfff640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d1850 0x7f50dc15b330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00cbf0 tx=0x7f50cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.580+0000 7f50d8ff9640 1 -- 172.21.15.16:0/2862758144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00ad50 con 0x7f50dc0d1850 2023-11-13T10:56:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.580+0000 7f50d8ff9640 1 -- 172.21.15.16:0/2862758144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc012400 con 0x7f50dc0d1850 2023-11-13T10:56:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.580+0000 7f50d8ff9640 1 -- 172.21.15.16:0/2862758144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc012790 con 0x7f50dc0d1850 2023-11-13T10:56:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.580+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc0b57d0 con 0x7f50dc0d1850 2023-11-13T10:56:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.580+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc0b5d90 con 0x7f50dc0d1850 2023-11-13T10:56:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.581+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50dc04e4d0 con 0x7f50dc0d1850 2023-11-13T10:56:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.583+0000 7f50d8ff9640 1 -- 172.21.15.16:0/2862758144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50cc007070 con 0x7f50dc0d1850 2023-11-13T10:56:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.584+0000 7f50d8ff9640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50bc06ae40 0x7f50bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.584+0000 7f50d8ff9640 1 -- 172.21.15.16:0/2862758144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f50cc089a00 con 0x7f50dc0d1850 2023-11-13T10:56:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.584+0000 7f50db7fe640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50bc06ae40 0x7f50bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.586+0000 7f50db7fe640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50bc06ae40 0x7f50bc06d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f50c4006fd0 tx=0x7f50c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.588+0000 7f50d8ff9640 1 -- 172.21.15.16:0/2862758144 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50cc054750 con 0x7f50dc0d1850 2023-11-13T10:56:44.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.781+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50dc0b4dd0 con 0x7f50dc0d1850 2023-11-13T10:56:44.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.782+0000 7f50d8ff9640 1 -- 172.21.15.16:0/2862758144 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50cc017020 con 0x7f50dc0d1850 2023-11-13T10:56:44.787 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50bc06ae40 msgr2=0x7f50bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50bc06ae40 0x7f50bc06d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f50c4006fd0 tx=0x7f50c4008040 comp rx=0 tx=0).stop 2023-11-13T10:56:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d1850 msgr2=0x7f50dc15b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d1850 0x7f50dc15b330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00cbf0 tx=0x7f50cc00a430 comp rx=0 tx=0).stop 2023-11-13T10:56:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 shutdown_connections 2023-11-13T10:56:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50bc06ae40 0x7f50bc06d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50dc15ce70 0x7f50dc06b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50dc0d2d00 0x7f50dc09ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 --2- 172.21.15.16:0/2862758144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50dc0d1850 0x7f50dc15b330 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 >> 172.21.15.16:0/2862758144 conn(0x7f50dc08c580 msgr2=0x7f50dc15fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 shutdown_connections 2023-11-13T10:56:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:44.785+0000 7f50e1602640 1 -- 172.21.15.16:0/2862758144 wait complete. 2023-11-13T10:56:44.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.044+0000 7feb0db73640 1 Processor -- start 2023-11-13T10:56:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.044+0000 7feb0db73640 1 -- start start 2023-11-13T10:56:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.046+0000 7feb0db73640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb0815ce70 0x7feb080cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.046+0000 7feb0db73640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 0x7feb080d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.046+0000 7feb0db73640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb080cac40 con 0x7feb080d1850 2023-11-13T10:56:45.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.046+0000 7feb0db73640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb080ce0e0 con 0x7feb0815ce70 2023-11-13T10:56:45.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.046+0000 7feb0db73640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb080ce3c0 con 0x7feb080cdbb0 2023-11-13T10:56:45.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.046+0000 7feb0cb71640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb0815ce70 0x7feb080cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.046+0000 7feb0cb71640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb0815ce70 0x7feb080cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46560/0 (socket says 172.21.15.16:46560) 2023-11-13T10:56:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.046+0000 7feb0cb71640 1 -- 172.21.15.16:0/472072401 learned_addr learned my addr 172.21.15.16:0/472072401 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.047+0000 7feb07fff640 1 --2- 172.21.15.16:0/472072401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 0x7feb080d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.047+0000 7feb0cb71640 1 -- 172.21.15.16:0/472072401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 msgr2=0x7feb080d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.047+0000 7feb0cb71640 1 --2- 172.21.15.16:0/472072401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 0x7feb080d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.047+0000 7feb0cb71640 1 -- 172.21.15.16:0/472072401 >> v1:172.21.15.184:6789/0 conn(0x7feb080d1850 legacy=0x7feb080cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.047+0000 7feb0cb71640 1 -- 172.21.15.16:0/472072401 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb080ce6a0 con 0x7feb0815ce70 2023-11-13T10:56:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.047+0000 7feb07fff640 1 --2- 172.21.15.16:0/472072401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 0x7feb080d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.048+0000 7feb0cb71640 1 --2- 172.21.15.16:0/472072401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 0x7feb080cd670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feafc0098a0 tx=0x7feafc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d81eebb04f3493 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.048+0000 7feb077fe640 1 -- 172.21.15.16:0/472072401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc031640 con 0x7feb0815ce70 2023-11-13T10:56:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.049+0000 7feb077fe640 1 -- 172.21.15.16:0/472072401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc0317a0 con 0x7feb0815ce70 2023-11-13T10:56:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.049+0000 7feb077fe640 1 -- 172.21.15.16:0/472072401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc031b30 con 0x7feb0815ce70 2023-11-13T10:56:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.049+0000 7feb0db73640 1 -- 172.21.15.16:0/472072401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 msgr2=0x7feb080cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.049+0000 7feb0db73640 1 --2- 172.21.15.16:0/472072401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 0x7feb080cd670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feafc0098a0 tx=0x7feafc02b8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.049+0000 7feb0db73640 1 -- 172.21.15.16:0/472072401 shutdown_connections 2023-11-13T10:56:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.049+0000 7feb0db73640 1 --2- 172.21.15.16:0/472072401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 0x7feb080d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.049+0000 7feb0db73640 1 --2- 172.21.15.16:0/472072401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 0x7feb080cd670 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.049+0000 7feb0db73640 1 -- 172.21.15.16:0/472072401 >> 172.21.15.16:0/472072401 conn(0x7feb0808c580 msgr2=0x7feb0809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.050+0000 7feb0db73640 1 -- 172.21.15.16:0/472072401 shutdown_connections 2023-11-13T10:56:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.050+0000 7feb0db73640 1 -- 172.21.15.16:0/472072401 wait complete. 2023-11-13T10:56:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.052+0000 7feb0db73640 1 Processor -- start 2023-11-13T10:56:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.052+0000 7feb0db73640 1 -- start start 2023-11-13T10:56:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.053+0000 7feb0db73640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 0x7feb08155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0db73640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb080d1850 0x7feb081520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0db73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 0x7feb080b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0db73640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb08156480 con 0x7feb0815ce70 2023-11-13T10:56:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0db73640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb08152630 con 0x7feb080d1850 2023-11-13T10:56:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0db73640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb08152a60 con 0x7feb080cdbb0 2023-11-13T10:56:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0d372640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 0x7feb080b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0d372640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 0x7feb080b74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46574/0 (socket says 172.21.15.16:46574) 2023-11-13T10:56:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0d372640 1 -- 172.21.15.16:0/264904990 learned_addr learned my addr 172.21.15.16:0/264904990 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0cb71640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 0x7feb08155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb07fff640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb080d1850 0x7feb081520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0d372640 1 -- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 msgr2=0x7feb08155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0d372640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 0x7feb08155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0d372640 1 -- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb080d1850 msgr2=0x7feb081520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0d372640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb080d1850 0x7feb081520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.054+0000 7feb0d372640 1 -- 172.21.15.16:0/264904990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc0094d0 con 0x7feb0815ce70 2023-11-13T10:56:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.055+0000 7feb07fff640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb080d1850 0x7feb081520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.055+0000 7feb0d372640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 0x7feb080b74b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7feb0000c700 tx=0x7feb0000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.055+0000 7feb057fa640 1 -- 172.21.15.16:0/264904990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0000aa50 con 0x7feb0815ce70 2023-11-13T10:56:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.055+0000 7feb057fa640 1 -- 172.21.15.16:0/264904990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0000abb0 con 0x7feb0815ce70 2023-11-13T10:56:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.055+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb08152d40 con 0x7feb0815ce70 2023-11-13T10:56:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.055+0000 7feb057fa640 1 -- 172.21.15.16:0/264904990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0000ce10 con 0x7feb0815ce70 2023-11-13T10:56:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.055+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb08153300 con 0x7feb0815ce70 2023-11-13T10:56:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.056+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0804e4d0 con 0x7feb0815ce70 2023-11-13T10:56:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.057+0000 7feb057fa640 1 -- 172.21.15.16:0/264904990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb00012420 con 0x7feb0815ce70 2023-11-13T10:56:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.058+0000 7feb057fa640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feaec06ae40 0x7feaec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.058+0000 7feb057fa640 1 -- 172.21.15.16:0/264904990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7feb00089af0 con 0x7feb0815ce70 2023-11-13T10:56:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.059+0000 7feb0cb71640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feaec06ae40 0x7feaec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.060+0000 7feb0cb71640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feaec06ae40 0x7feaec06d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feafc036000 tx=0x7feafc03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.062+0000 7feb057fa640 1 -- 172.21.15.16:0/264904990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb00054850 con 0x7feb0815ce70 2023-11-13T10:56:45.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.257+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb0815c670 con 0x7feb0815ce70 2023-11-13T10:56:45.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.258+0000 7feb057fa640 1 -- 172.21.15.16:0/264904990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb000160a0 con 0x7feb0815ce70 2023-11-13T10:56:45.263 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feaec06ae40 msgr2=0x7feaec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feaec06ae40 0x7feaec06d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feafc036000 tx=0x7feafc03c040 comp rx=0 tx=0).stop 2023-11-13T10:56:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 msgr2=0x7feb080b74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 0x7feb080b74b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7feb0000c700 tx=0x7feb0000cbc0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 shutdown_connections 2023-11-13T10:56:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feaec06ae40 0x7feaec06d300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0815ce70 0x7feb080b74b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb080d1850 0x7feb081520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 --2- 172.21.15.16:0/264904990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb080cdbb0 0x7feb08155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 >> 172.21.15.16:0/264904990 conn(0x7feb0808c580 msgr2=0x7feb080cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 shutdown_connections 2023-11-13T10:56:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.261+0000 7feb0db73640 1 -- 172.21.15.16:0/264904990 wait complete. 2023-11-13T10:56:45.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.508+0000 7fcc9d227640 1 Processor -- start 2023-11-13T10:56:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.509+0000 7fcc9d227640 1 -- start start 2023-11-13T10:56:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.510+0000 7fcc9d227640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcc9815ce70 0x7fcc980cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.510+0000 7fcc9d227640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc980ce0e0 con 0x7fcc980d1850 2023-11-13T10:56:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.510+0000 7fcc9d227640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc980ce3c0 con 0x7fcc980cdbb0 2023-11-13T10:56:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.510+0000 7fcc9d227640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc980ce6a0 con 0x7fcc9815ce70 2023-11-13T10:56:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.511+0000 7fcc9ca26640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcc980d1850 0x7fcc980c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35390/0 (socket says 172.21.15.16:35390) 2023-11-13T10:56:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.511+0000 7fcc9ca26640 1 -- 172.21.15.16:0/961693142 learned_addr learned my addr 172.21.15.16:0/961693142 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:45.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.511+0000 7fcc97fff640 1 --2- 172.21.15.16:0/961693142 >> v2:172.21.15.184:3300/0 conn(0x7fcc9815ce70 0x7fcc980cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.511+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875459708 0 0) 0x7fcc980ce0e0 con 0x7fcc980d1850 2023-11-13T10:56:45.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.511+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc80003520 con 0x7fcc980d1850 2023-11-13T10:56:45.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2862178940 0 0) 0x7fcc80003520 con 0x7fcc980d1850 2023-11-13T10:56:45.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 >> v2:172.21.15.184:3300/0 conn(0x7fcc9815ce70 msgr2=0x7fcc980cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:45.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 --2- 172.21.15.16:0/961693142 >> v2:172.21.15.184:3300/0 conn(0x7fcc9815ce70 0x7fcc980cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 >> v1:172.21.15.184:6789/0 conn(0x7fcc980cdbb0 legacy=0x7fcc980d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc980ce980 con 0x7fcc980d1850 2023-11-13T10:56:45.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fcc900019f0 con 0x7fcc980d1850 2023-11-13T10:56:45.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 >> v1:172.21.15.16:6789/0 conn(0x7fcc980d1850 legacy=0x7fcc980c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc80003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc800047c0 0x7fcc80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc80005180 0x7fcc8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980ce6a0 con 0x7fcc80003970 2023-11-13T10:56:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8000bfd0 con 0x7fcc80005180 2023-11-13T10:56:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8000c190 con 0x7fcc800047c0 2023-11-13T10:56:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc97fff640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc80003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc977fe640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc800047c0 0x7fcc80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc97fff640 1 -- 172.21.15.16:0/961693142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc800047c0 msgr2=0x7fcc80004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc97fff640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc800047c0 0x7fcc80004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc97fff640 1 -- 172.21.15.16:0/961693142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc80005180 msgr2=0x7fcc8000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:45.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc97fff640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc80005180 0x7fcc8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc97fff640 1 -- 172.21.15.16:0/961693142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80003520 con 0x7fcc80003970 2023-11-13T10:56:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.512+0000 7fcc9ca26640 1 -- 172.21.15.16:0/961693142 reap_dead start 2023-11-13T10:56:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.513+0000 7fcc97fff640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc80003d50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009ac0 tx=0x7fcc800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f53fb1067ca145 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8802c710 con 0x7fcc80003970 2023-11-13T10:56:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8802c870 con 0x7fcc80003970 2023-11-13T10:56:45.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc96ffd640 1 -- 172.21.15.16:0/961693142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8802cc00 con 0x7fcc80003970 2023-11-13T10:56:45.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc9d227640 1 -- 172.21.15.16:0/961693142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 msgr2=0x7fcc80003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc9d227640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc80003d50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009ac0 tx=0x7fcc800057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc9d227640 1 -- 172.21.15.16:0/961693142 shutdown_connections 2023-11-13T10:56:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc9d227640 1 --2- 172.21.15.16:0/961693142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc80003d50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc9d227640 1 -- 172.21.15.16:0/961693142 >> 172.21.15.16:0/961693142 conn(0x7fcc9808c580 msgr2=0x7fcc9809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc9d227640 1 -- 172.21.15.16:0/961693142 shutdown_connections 2023-11-13T10:56:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.514+0000 7fcc9d227640 1 -- 172.21.15.16:0/961693142 wait complete. 2023-11-13T10:56:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.516+0000 7fcc9d227640 1 Processor -- start 2023-11-13T10:56:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.516+0000 7fcc9d227640 1 -- start start 2023-11-13T10:56:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.517+0000 7fcc9d227640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc98152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.517+0000 7fcc9d227640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc98160020 0x7fcc981566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.517+0000 7fcc9d227640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc980b5760 0x7fcc980c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.517+0000 7fcc9d227640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98156c00 con 0x7fcc80003970 2023-11-13T10:56:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.517+0000 7fcc9d227640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9815f790 con 0x7fcc98160020 2023-11-13T10:56:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.517+0000 7fcc9d227640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9815fa70 con 0x7fcc980b5760 2023-11-13T10:56:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.517+0000 7fcc97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc98152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.517+0000 7fcc97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc98152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46596/0 (socket says 172.21.15.16:46596) 2023-11-13T10:56:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.517+0000 7fcc97fff640 1 -- 172.21.15.16:0/3941039083 learned_addr learned my addr 172.21.15.16:0/3941039083 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.518+0000 7fcc9ca26640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc980b5760 0x7fcc980c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.518+0000 7fcc977fe640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc98160020 0x7fcc981566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.518+0000 7fcc97fff640 1 -- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc980b5760 msgr2=0x7fcc980c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.518+0000 7fcc97fff640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc980b5760 0x7fcc980c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.518+0000 7fcc97fff640 1 -- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc98160020 msgr2=0x7fcc981566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.518+0000 7fcc97fff640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc98160020 0x7fcc981566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.518+0000 7fcc97fff640 1 -- 172.21.15.16:0/3941039083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc980c8500 con 0x7fcc80003970 2023-11-13T10:56:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.518+0000 7fcc97fff640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc98152210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcc8802c2b0 tx=0x7fcc88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.518+0000 7fcc94ff9640 1 -- 172.21.15.16:0/3941039083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88003810 con 0x7fcc80003970 2023-11-13T10:56:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.519+0000 7fcc94ff9640 1 -- 172.21.15.16:0/3941039083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8802de90 con 0x7fcc80003970 2023-11-13T10:56:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.519+0000 7fcc94ff9640 1 -- 172.21.15.16:0/3941039083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88003b70 con 0x7fcc80003970 2023-11-13T10:56:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.519+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc980b5be0 con 0x7fcc80003970 2023-11-13T10:56:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.519+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc980c8f60 con 0x7fcc80003970 2023-11-13T10:56:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.520+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc980c8690 con 0x7fcc80003970 2023-11-13T10:56:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.521+0000 7fcc94ff9640 1 -- 172.21.15.16:0/3941039083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc88003cd0 con 0x7fcc80003970 2023-11-13T10:56:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.522+0000 7fcc94ff9640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc8406ae70 0x7fcc8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.522+0000 7fcc94ff9640 1 -- 172.21.15.16:0/3941039083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fcc880ab770 con 0x7fcc80003970 2023-11-13T10:56:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.522+0000 7fcc977fe640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc8406ae70 0x7fcc8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.523+0000 7fcc977fe640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc8406ae70 0x7fcc8406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcc980b88d0 tx=0x7fcc8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.526+0000 7fcc94ff9640 1 -- 172.21.15.16:0/3941039083 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc880764d0 con 0x7fcc80003970 2023-11-13T10:56:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.719+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc9804e4d0 con 0x7fcc80003970 2023-11-13T10:56:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.720+0000 7fcc94ff9640 1 -- 172.21.15.16:0/3941039083 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc88079af0 con 0x7fcc80003970 2023-11-13T10:56:45.725 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc8406ae70 msgr2=0x7fcc8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc8406ae70 0x7fcc8406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcc980b88d0 tx=0x7fcc8c009040 comp rx=0 tx=0).stop 2023-11-13T10:56:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 msgr2=0x7fcc98152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc98152210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcc8802c2b0 tx=0x7fcc88004730 comp rx=0 tx=0).stop 2023-11-13T10:56:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 shutdown_connections 2023-11-13T10:56:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcc8406ae70 0x7fcc8406d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc980b5760 0x7fcc980c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc98160020 0x7fcc981566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 --2- 172.21.15.16:0/3941039083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc80003970 0x7fcc98152210 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 >> 172.21.15.16:0/3941039083 conn(0x7fcc9808c580 msgr2=0x7fcc980ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 shutdown_connections 2023-11-13T10:56:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.723+0000 7fcc9d227640 1 -- 172.21.15.16:0/3941039083 wait complete. 2023-11-13T10:56:45.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.967+0000 7fccdf0ad640 1 Processor -- start 2023-11-13T10:56:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.967+0000 7fccdf0ad640 1 -- start start 2023-11-13T10:56:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.968+0000 7fccdf0ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fccd80cdc70 0x7fccd806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.968+0000 7fccdf0ad640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fccd80cfd10 0x7fccd806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.969+0000 7fccdf0ad640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd806c710 con 0x7fccd80ccad0 2023-11-13T10:56:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.969+0000 7fccdf0ad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd806b9b0 con 0x7fccd80cdc70 2023-11-13T10:56:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.969+0000 7fccdf0ad640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd80cef50 con 0x7fccd80cfd10 2023-11-13T10:56:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.969+0000 7fccdd8aa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fccd80cdc70 0x7fccd806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.969+0000 7fccdd8aa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fccd80cdc70 0x7fccd806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46604/0 (socket says 172.21.15.16:46604) 2023-11-13T10:56:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.969+0000 7fccdd8aa640 1 -- 172.21.15.16:0/2529379434 learned_addr learned my addr 172.21.15.16:0/2529379434 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.969+0000 7fccde8ac640 1 --2- 172.21.15.16:0/2529379434 >> v2:172.21.15.184:3300/0 conn(0x7fccd80cfd10 0x7fccd806c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.969+0000 7fccdd8aa640 1 -- 172.21.15.16:0/2529379434 >> v2:172.21.15.184:3300/0 conn(0x7fccd80cfd10 msgr2=0x7fccd806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.970+0000 7fccdd8aa640 1 --2- 172.21.15.16:0/2529379434 >> v2:172.21.15.184:3300/0 conn(0x7fccd80cfd10 0x7fccd806c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.970+0000 7fccdd8aa640 1 -- 172.21.15.16:0/2529379434 >> v1:172.21.15.16:6789/0 conn(0x7fccd80ccad0 legacy=0x7fccd806b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.970+0000 7fccdd8aa640 1 -- 172.21.15.16:0/2529379434 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80cf230 con 0x7fccd80cdc70 2023-11-13T10:56:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.970+0000 7fccdd0a9640 1 -- 172.21.15.16:0/2529379434 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752713226 0 0) 0x7fccd806c710 con 0x7fccd80ccad0 2023-11-13T10:56:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.970+0000 7fccde8ac640 1 --2- 172.21.15.16:0/2529379434 >> v2:172.21.15.184:3300/0 conn(0x7fccd80cfd10 0x7fccd806c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.971+0000 7fccdd8aa640 1 --2- 172.21.15.16:0/2529379434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 0x7fccd806bc90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fccd8070510 tx=0x7fccc402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728f6705ff2d977d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.971+0000 7fccdd0a9640 1 -- 172.21.15.16:0/2529379434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4030860 con 0x7fccd80cdc70 2023-11-13T10:56:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.971+0000 7fccdd0a9640 1 -- 172.21.15.16:0/2529379434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc40309c0 con 0x7fccd80cdc70 2023-11-13T10:56:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.971+0000 7fccdd0a9640 1 -- 172.21.15.16:0/2529379434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4030d50 con 0x7fccd80cdc70 2023-11-13T10:56:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.972+0000 7fccdf0ad640 1 -- 172.21.15.16:0/2529379434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 msgr2=0x7fccd806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.972+0000 7fccdf0ad640 1 --2- 172.21.15.16:0/2529379434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 0x7fccd806bc90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fccd8070510 tx=0x7fccc402b9a0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.972+0000 7fccdf0ad640 1 -- 172.21.15.16:0/2529379434 shutdown_connections 2023-11-13T10:56:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.972+0000 7fccdf0ad640 1 --2- 172.21.15.16:0/2529379434 >> v2:172.21.15.184:3300/0 conn(0x7fccd80cfd10 0x7fccd806c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.972+0000 7fccdf0ad640 1 --2- 172.21.15.16:0/2529379434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 0x7fccd806bc90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.972+0000 7fccdf0ad640 1 -- 172.21.15.16:0/2529379434 >> 172.21.15.16:0/2529379434 conn(0x7fccd80d0950 msgr2=0x7fccd80d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.972+0000 7fccdf0ad640 1 -- 172.21.15.16:0/2529379434 shutdown_connections 2023-11-13T10:56:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.972+0000 7fccdf0ad640 1 -- 172.21.15.16:0/2529379434 wait complete. 2023-11-13T10:56:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.974+0000 7fccdf0ad640 1 Processor -- start 2023-11-13T10:56:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.974+0000 7fccdf0ad640 1 -- start start 2023-11-13T10:56:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdf0ad640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd80ccad0 0x7fccd814e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdf0ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 0x7fccd815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdf0ad640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd80cfd10 0x7fccd815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdf0ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd814ed10 con 0x7fccd80cdc70 2023-11-13T10:56:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdf0ad640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80c60f0 con 0x7fccd80ccad0 2023-11-13T10:56:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdf0ad640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80c63d0 con 0x7fccd80cfd10 2023-11-13T10:56:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdd8aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 0x7fccd815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdd8aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 0x7fccd815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46614/0 (socket says 172.21.15.16:46614) 2023-11-13T10:56:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdd8aa640 1 -- 172.21.15.16:0/196846590 learned_addr learned my addr 172.21.15.16:0/196846590 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccde0ab640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd80ccad0 0x7fccd814e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdd8aa640 1 -- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd80cfd10 msgr2=0x7fccd815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdd8aa640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd80cfd10 0x7fccd815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdd8aa640 1 -- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd80ccad0 msgr2=0x7fccd814e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdd8aa640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd80ccad0 0x7fccd814e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.975+0000 7fccdd8aa640 1 -- 172.21.15.16:0/196846590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc40094d0 con 0x7fccd80cdc70 2023-11-13T10:56:46.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.976+0000 7fccdd8aa640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 0x7fccd815aa10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fccc402b9e0 tx=0x7fccc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.976+0000 7fccd6ffd640 1 -- 172.21.15.16:0/196846590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4030850 con 0x7fccd80cdc70 2023-11-13T10:56:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.976+0000 7fccd6ffd640 1 -- 172.21.15.16:0/196846590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc4043400 con 0x7fccd80cdc70 2023-11-13T10:56:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.976+0000 7fccd6ffd640 1 -- 172.21.15.16:0/196846590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc40437b0 con 0x7fccd80cdc70 2023-11-13T10:56:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.976+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd80c66b0 con 0x7fccd80cdc70 2023-11-13T10:56:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.976+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd80c6c70 con 0x7fccd80cdc70 2023-11-13T10:56:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.978+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd804e4d0 con 0x7fccd80cdc70 2023-11-13T10:56:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.978+0000 7fccd6ffd640 1 -- 172.21.15.16:0/196846590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fccc4043910 con 0x7fccd80cdc70 2023-11-13T10:56:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.979+0000 7fccd6ffd640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fccbc06ae40 0x7fccbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.979+0000 7fccd6ffd640 1 -- 172.21.15.16:0/196846590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fccc403e070 con 0x7fccd80cdc70 2023-11-13T10:56:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.980+0000 7fccde0ab640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fccbc06ae40 0x7fccbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.981+0000 7fccde0ab640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fccbc06ae40 0x7fccbc06d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0012e0 tx=0x7fcccc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:46.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:45.984+0000 7fccd6ffd640 1 -- 172.21.15.16:0/196846590 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fccc4077ac0 con 0x7fccd80cdc70 2023-11-13T10:56:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.170+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fccd80ce2f0 con 0x7fccd80cdc70 2023-11-13T10:56:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.171+0000 7fccd6ffd640 1 -- 172.21.15.16:0/196846590 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fccc4034020 con 0x7fccd80cdc70 2023-11-13T10:56:46.180 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.173+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fccbc06ae40 msgr2=0x7fccbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.173+0000 7fccdf0ad640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fccbc06ae40 0x7fccbc06d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0012e0 tx=0x7fcccc009040 comp rx=0 tx=0).stop 2023-11-13T10:56:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.173+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 msgr2=0x7fccd815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.173+0000 7fccdf0ad640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 0x7fccd815aa10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fccc402b9e0 tx=0x7fccc4004730 comp rx=0 tx=0).stop 2023-11-13T10:56:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.174+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 shutdown_connections 2023-11-13T10:56:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.174+0000 7fccdf0ad640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fccbc06ae40 0x7fccbc06d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.174+0000 7fccdf0ad640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccd80cfd10 0x7fccd815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.174+0000 7fccdf0ad640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccd80cdc70 0x7fccd815aa10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.174+0000 7fccdf0ad640 1 --2- 172.21.15.16:0/196846590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd80ccad0 0x7fccd814e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.174+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 >> 172.21.15.16:0/196846590 conn(0x7fccd80d0950 msgr2=0x7fccd806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.174+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 shutdown_connections 2023-11-13T10:56:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.174+0000 7fccdf0ad640 1 -- 172.21.15.16:0/196846590 wait complete. 2023-11-13T10:56:46.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:56:46.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.418+0000 7f6dfe259640 1 Processor -- start 2023-11-13T10:56:46.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.419+0000 7f6dfe259640 1 -- start start 2023-11-13T10:56:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.420+0000 7f6dfe259640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df806a8a0 0x7f6df80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.420+0000 7f6dfe259640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df806fc50 con 0x7f6df815cc90 2023-11-13T10:56:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.420+0000 7f6dfe259640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df806ff30 con 0x7f6df8070cf0 2023-11-13T10:56:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.420+0000 7f6dfe259640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8070210 con 0x7f6df806a8a0 2023-11-13T10:56:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6dfd257640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6df815cc90 0x7f6df806a190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35412/0 (socket says 172.21.15.16:35412) 2023-11-13T10:56:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6dfd257640 1 -- 172.21.15.16:0/78455680 learned_addr learned my addr 172.21.15.16:0/78455680 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6dfca56640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df806a8a0 0x7f6df80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143440209 0 0) 0x7f6df806fc50 con 0x7f6df815cc90 2023-11-13T10:56:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6de4003520 con 0x7f6df815cc90 2023-11-13T10:56:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1662414560 0 0) 0x7f6de4003520 con 0x7f6df815cc90 2023-11-13T10:56:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df806a8a0 msgr2=0x7f6df80b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6deffff640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df806a8a0 0x7f6df80b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 >> v1:172.21.15.184:6789/0 conn(0x7f6df8070cf0 legacy=0x7f6df80b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.421+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df80704f0 con 0x7f6df815cc90 2023-11-13T10:56:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f6df4001670 con 0x7f6df815cc90 2023-11-13T10:56:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 >> v1:172.21.15.16:6789/0 conn(0x7f6df815cc90 legacy=0x7f6df806a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6deffff640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6de40039a0 0x7f6de4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6deffff640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df806a8a0 0x7f6de40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6deffff640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6de4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8070210 con 0x7f6de4004d00 2023-11-13T10:56:46.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de400bb60 con 0x7f6de40039a0 2023-11-13T10:56:46.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de400bd20 con 0x7f6df806a8a0 2023-11-13T10:56:46.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6dfda58640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6de4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6dfca56640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df806a8a0 0x7f6de40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6dfd257640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6de40039a0 0x7f6de4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6dfda58640 1 -- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df806a8a0 msgr2=0x7f6de40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6dfda58640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df806a8a0 0x7f6de40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6dfda58640 1 -- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6de40039a0 msgr2=0x7f6de4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:46.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6dfda58640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6de40039a0 0x7f6de4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.422+0000 7f6dfda58640 1 -- 172.21.15.16:0/78455680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4003520 con 0x7f6de4004d00 2023-11-13T10:56:46.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.423+0000 7f6dfda58640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6de4005180 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6df0001e50 tx=0x7f6de40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1118f7f1381851da server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:46.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.424+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0003b70 con 0x7f6de4004d00 2023-11-13T10:56:46.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.424+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df0003cd0 con 0x7f6de4004d00 2023-11-13T10:56:46.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.424+0000 7f6deffff640 1 -- 172.21.15.16:0/78455680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0005310 con 0x7f6de4004d00 2023-11-13T10:56:46.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.424+0000 7f6dfe259640 1 -- 172.21.15.16:0/78455680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 msgr2=0x7f6de4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.424+0000 7f6dfe259640 1 --2- 172.21.15.16:0/78455680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6de4005180 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6df0001e50 tx=0x7f6de40057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.424+0000 7f6dfda58640 1 -- 172.21.15.16:0/78455680 reap_dead start 2023-11-13T10:56:46.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.425+0000 7f6dfe259640 1 -- 172.21.15.16:0/78455680 shutdown_connections 2023-11-13T10:56:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.425+0000 7f6dfe259640 1 -- 172.21.15.16:0/78455680 >> 172.21.15.16:0/78455680 conn(0x7f6df80b04a0 msgr2=0x7f6df80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.425+0000 7f6dfe259640 1 -- 172.21.15.16:0/78455680 shutdown_connections 2023-11-13T10:56:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.425+0000 7f6dfe259640 1 -- 172.21.15.16:0/78455680 wait complete. 2023-11-13T10:56:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.426+0000 7f6dfe259640 1 Processor -- start 2023-11-13T10:56:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.427+0000 7f6dfe259640 1 -- start start 2023-11-13T10:56:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfe259640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6df813f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfe259640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6df8141d20 0x7f6df8140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfe259640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df8140770 0x7f6df8158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfe259640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8141900 con 0x7f6de4004d00 2023-11-13T10:56:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfe259640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8140c80 con 0x7f6df8141d20 2023-11-13T10:56:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfe259640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8140f60 con 0x7f6df8140770 2023-11-13T10:56:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfd257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6df813f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfd257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6df813f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46644/0 (socket says 172.21.15.16:46644) 2023-11-13T10:56:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfd257640 1 -- 172.21.15.16:0/1307441964 learned_addr learned my addr 172.21.15.16:0/1307441964 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfca56640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6df8141d20 0x7f6df8140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfd257640 1 -- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df8140770 msgr2=0x7f6df8158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfd257640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df8140770 0x7f6df8158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfd257640 1 -- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6df8141d20 msgr2=0x7f6df8140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfd257640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6df8141d20 0x7f6df8140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.428+0000 7f6dfd257640 1 -- 172.21.15.16:0/1307441964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8141240 con 0x7f6de4004d00 2023-11-13T10:56:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.429+0000 7f6dfd257640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6df813f8c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6df400e800 tx=0x7f6df400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.429+0000 7f6dedffb640 1 -- 172.21.15.16:0/1307441964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df400c7f0 con 0x7f6de4004d00 2023-11-13T10:56:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.429+0000 7f6dedffb640 1 -- 172.21.15.16:0/1307441964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df400c950 con 0x7f6de4004d00 2023-11-13T10:56:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.429+0000 7f6dedffb640 1 -- 172.21.15.16:0/1307441964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df400cd00 con 0x7f6de4004d00 2023-11-13T10:56:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.429+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df8141520 con 0x7f6de4004d00 2023-11-13T10:56:46.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.429+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df81534e0 con 0x7f6de4004d00 2023-11-13T10:56:46.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.430+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df804e4d0 con 0x7f6de4004d00 2023-11-13T10:56:46.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.431+0000 7f6dedffb640 1 -- 172.21.15.16:0/1307441964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6df401d460 con 0x7f6de4004d00 2023-11-13T10:56:46.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.432+0000 7f6dedffb640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ddc06ae70 0x7f6ddc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.432+0000 7f6dedffb640 1 -- 172.21.15.16:0/1307441964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f6df4089980 con 0x7f6de4004d00 2023-11-13T10:56:46.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.433+0000 7f6dfca56640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ddc06ae70 0x7f6ddc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.433+0000 7f6dfca56640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ddc06ae70 0x7f6ddc06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6df806d060 tx=0x7f6de8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:46.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.436+0000 7f6dedffb640 1 -- 172.21.15.16:0/1307441964 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6df4054650 con 0x7f6de4004d00 2023-11-13T10:56:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.641+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6df815bd40 con 0x7f6de4004d00 2023-11-13T10:56:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.641+0000 7f6dedffb640 1 -- 172.21.15.16:0/1307441964 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6df400ce60 con 0x7f6de4004d00 2023-11-13T10:56:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:56:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ddc06ae70 msgr2=0x7f6ddc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ddc06ae70 0x7f6ddc06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6df806d060 tx=0x7f6de8008040 comp rx=0 tx=0).stop 2023-11-13T10:56:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 msgr2=0x7f6df813f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6df813f8c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6df400e800 tx=0x7f6df400ecc0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 shutdown_connections 2023-11-13T10:56:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ddc06ae70 0x7f6ddc06d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6df8140770 0x7f6df8158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6df8141d20 0x7f6df8140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 --2- 172.21.15.16:0/1307441964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de4004d00 0x7f6df813f8c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 >> 172.21.15.16:0/1307441964 conn(0x7f6df80b04a0 msgr2=0x7f6df80b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.644+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 shutdown_connections 2023-11-13T10:56:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.645+0000 7f6dfe259640 1 -- 172.21.15.16:0/1307441964 wait complete. 2023-11-13T10:56:46.667 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:56:46.668 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:46.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:46.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:56:46.966 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:56:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.963+0000 7f68be0b4640 1 Processor -- start 2023-11-13T10:56:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.963+0000 7f68be0b4640 1 -- start start 2023-11-13T10:56:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.965+0000 7f68be0b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b8146480 0x7f68b8071390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.965+0000 7f68be0b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68b80b1d80 0x7f68b806cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.965+0000 7f68be0b4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b815e400 con 0x7f68b815dd30 2023-11-13T10:56:46.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.965+0000 7f68be0b4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b80b3310 con 0x7f68b80b1d80 2023-11-13T10:56:46.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.965+0000 7f68be0b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b8071a50 con 0x7f68b8146480 2023-11-13T10:56:46.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.965+0000 7f68bc8b1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f68b815dd30 0x7f68b80b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35422/0 (socket says 172.21.15.16:35422) 2023-11-13T10:56:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.965+0000 7f68bc8b1640 1 -- 172.21.15.16:0/813689582 learned_addr learned my addr 172.21.15.16:0/813689582 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.965+0000 7f68bd8b3640 1 --2- 172.21.15.16:0/813689582 >> v2:172.21.15.16:3300/0 conn(0x7f68b80b1d80 0x7f68b806cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.965+0000 7f68bd0b2640 1 --2- 172.21.15.16:0/813689582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b8146480 0x7f68b8071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.966+0000 7f68affff640 1 -- 172.21.15.16:0/813689582 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132287371 0 0) 0x7f68b815e400 con 0x7f68b815dd30 2023-11-13T10:56:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.966+0000 7f68affff640 1 -- 172.21.15.16:0/813689582 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68a4003520 con 0x7f68b815dd30 2023-11-13T10:56:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.966+0000 7f68bd8b3640 1 -- 172.21.15.16:0/813689582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b8146480 msgr2=0x7f68b8071390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.966+0000 7f68bd8b3640 1 --2- 172.21.15.16:0/813689582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b8146480 0x7f68b8071390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.966+0000 7f68bd8b3640 1 -- 172.21.15.16:0/813689582 >> v1:172.21.15.16:6789/0 conn(0x7f68b815dd30 legacy=0x7f68b80b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.966+0000 7f68bd8b3640 1 -- 172.21.15.16:0/813689582 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b80b29e0 con 0x7f68b80b1d80 2023-11-13T10:56:46.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.967+0000 7f68bd8b3640 1 --2- 172.21.15.16:0/813689582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 0x7f68b806cc20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009d90 tx=0x7f68b002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b158c71bb1a00ec server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:46.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.968+0000 7f68affff640 1 -- 172.21.15.16:0/813689582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b002cc60 con 0x7f68b80b1d80 2023-11-13T10:56:46.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.968+0000 7f68affff640 1 -- 172.21.15.16:0/813689582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b002cdc0 con 0x7f68b80b1d80 2023-11-13T10:56:46.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.968+0000 7f68affff640 1 -- 172.21.15.16:0/813689582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0034660 con 0x7f68b80b1d80 2023-11-13T10:56:46.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.968+0000 7f68be0b4640 1 -- 172.21.15.16:0/813689582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 msgr2=0x7f68b806cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.968+0000 7f68be0b4640 1 --2- 172.21.15.16:0/813689582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 0x7f68b806cc20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009d90 tx=0x7f68b002beb0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.968+0000 7f68be0b4640 1 -- 172.21.15.16:0/813689582 shutdown_connections 2023-11-13T10:56:46.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.968+0000 7f68be0b4640 1 --2- 172.21.15.16:0/813689582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 0x7f68b806cc20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.968+0000 7f68be0b4640 1 --2- 172.21.15.16:0/813689582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b8146480 0x7f68b8071390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.968+0000 7f68be0b4640 1 -- 172.21.15.16:0/813689582 >> 172.21.15.16:0/813689582 conn(0x7f68b806b2a0 msgr2=0x7f68b806b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:46.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.969+0000 7f68be0b4640 1 -- 172.21.15.16:0/813689582 shutdown_connections 2023-11-13T10:56:46.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.969+0000 7f68be0b4640 1 -- 172.21.15.16:0/813689582 wait complete. 2023-11-13T10:56:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.970+0000 7f68be0b4640 1 Processor -- start 2023-11-13T10:56:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.971+0000 7f68be0b4640 1 -- start start 2023-11-13T10:56:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.971+0000 7f68be0b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 0x7f68b81f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.971+0000 7f68be0b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8146480 0x7f68b815c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68be0b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b815dd30 0x7f68b815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68be0b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b81f2f30 con 0x7f68b80b1d80 2023-11-13T10:56:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68be0b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b815ca70 con 0x7f68b8146480 2023-11-13T10:56:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68be0b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b815eb90 con 0x7f68b815dd30 2023-11-13T10:56:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd0b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 0x7f68b81f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd0b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 0x7f68b81f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46666/0 (socket says 172.21.15.16:46666) 2023-11-13T10:56:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd0b2640 1 -- 172.21.15.16:0/2663352124 learned_addr learned my addr 172.21.15.16:0/2663352124 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd8b3640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b815dd30 0x7f68b815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd0b2640 1 -- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b815dd30 msgr2=0x7f68b815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bc8b1640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8146480 0x7f68b815c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd0b2640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b815dd30 0x7f68b815fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd0b2640 1 -- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8146480 msgr2=0x7f68b815c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd0b2640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8146480 0x7f68b815c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd0b2640 1 -- 172.21.15.16:0/2663352124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b0009a70 con 0x7f68b80b1d80 2023-11-13T10:56:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.972+0000 7f68bd8b3640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b815dd30 0x7f68b815fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.973+0000 7f68bd0b2640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 0x7f68b81f29f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68b400d800 tx=0x7f68b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.973+0000 7f68adffb640 1 -- 172.21.15.16:0/2663352124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b400b6c0 con 0x7f68b80b1d80 2023-11-13T10:56:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.973+0000 7f68adffb640 1 -- 172.21.15.16:0/2663352124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b400b820 con 0x7f68b80b1d80 2023-11-13T10:56:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.973+0000 7f68adffb640 1 -- 172.21.15.16:0/2663352124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b400bbb0 con 0x7f68b80b1d80 2023-11-13T10:56:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.973+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b815ee70 con 0x7f68b80b1d80 2023-11-13T10:56:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.973+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b815f430 con 0x7f68b80b1d80 2023-11-13T10:56:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.974+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b804e4d0 con 0x7f68b80b1d80 2023-11-13T10:56:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.975+0000 7f68adffb640 1 -- 172.21.15.16:0/2663352124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68b400bd10 con 0x7f68b80b1d80 2023-11-13T10:56:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.976+0000 7f68adffb640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f689c06ae40 0x7f689c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.976+0000 7f68adffb640 1 -- 172.21.15.16:0/2663352124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f68b4089f90 con 0x7f68b80b1d80 2023-11-13T10:56:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.977+0000 7f68bc8b1640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f689c06ae40 0x7f689c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.978+0000 7f68bc8b1640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f689c06ae40 0x7f689c06d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68b81606d0 tx=0x7f68a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:46.980+0000 7f68adffb640 1 -- 172.21.15.16:0/2663352124 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68b4054c60 con 0x7f68b80b1d80 2023-11-13T10:56:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.177+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68b8072c50 con 0x7f68b80b1d80 2023-11-13T10:56:47.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.184+0000 7f68adffb640 1 -- 172.21.15.16:0/2663352124 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f68b4058280 con 0x7f68b80b1d80 2023-11-13T10:56:47.189 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:47.190 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.186+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f689c06ae40 msgr2=0x7f689c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.186+0000 7f68be0b4640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f689c06ae40 0x7f689c06d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68b81606d0 tx=0x7f68a8009040 comp rx=0 tx=0).stop 2023-11-13T10:56:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.186+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 msgr2=0x7f68b81f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.186+0000 7f68be0b4640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 0x7f68b81f29f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68b400d800 tx=0x7f68b400dcc0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.187+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 shutdown_connections 2023-11-13T10:56:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.187+0000 7f68be0b4640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f689c06ae40 0x7f689c06d300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.187+0000 7f68be0b4640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68b815dd30 0x7f68b815fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.187+0000 7f68be0b4640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68b8146480 0x7f68b815c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.187+0000 7f68be0b4640 1 --2- 172.21.15.16:0/2663352124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68b80b1d80 0x7f68b81f29f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.187+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 >> 172.21.15.16:0/2663352124 conn(0x7f68b806b2a0 msgr2=0x7f68b806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.187+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 shutdown_connections 2023-11-13T10:56:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.187+0000 7f68be0b4640 1 -- 172.21.15.16:0/2663352124 wait complete. 2023-11-13T10:56:47.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:47.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.445+0000 7f7ebb59e640 1 Processor -- start 2023-11-13T10:56:47.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.445+0000 7f7ebb59e640 1 -- start start 2023-11-13T10:56:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.446+0000 7f7ebb59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ebc159e50 0x7f7ebc0b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7ebb59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7ebc0cf200 0x7f7ebc070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc070f90 0x7f7ebc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7ebb59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ebc071400 con 0x7f7ebc159e50 2023-11-13T10:56:47.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7ebb59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ebc06cb80 con 0x7f7ebc0cf200 2023-11-13T10:56:47.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7ebb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc06c980 con 0x7f7ebc070f90 2023-11-13T10:56:47.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7eba59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ebc159e50 0x7f7ebc0b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7eba59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ebc159e50 0x7f7ebc0b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46674/0 (socket says 172.21.15.16:46674) 2023-11-13T10:56:47.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7eba59c640 1 -- 172.21.15.16:0/356795773 learned_addr learned my addr 172.21.15.16:0/356795773 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:47.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7eb9d9b640 1 --2- 172.21.15.16:0/356795773 >> v2:172.21.15.184:3300/0 conn(0x7f7ebc0cf200 0x7f7ebc070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.447+0000 7f7ebad9d640 1 --2- 172.21.15.16:0/356795773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc070f90 0x7f7ebc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.448+0000 7f7eba59c640 1 -- 172.21.15.16:0/356795773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc070f90 msgr2=0x7f7ebc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:47.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.448+0000 7f7eba59c640 1 --2- 172.21.15.16:0/356795773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc070f90 0x7f7ebc0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.448+0000 7f7eba59c640 1 -- 172.21.15.16:0/356795773 >> v2:172.21.15.184:3300/0 conn(0x7f7ebc0cf200 msgr2=0x7f7ebc070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:47.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.448+0000 7f7eba59c640 1 --2- 172.21.15.16:0/356795773 >> v2:172.21.15.184:3300/0 conn(0x7f7ebc0cf200 0x7f7ebc070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.448+0000 7f7eba59c640 1 -- 172.21.15.16:0/356795773 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc071620 con 0x7f7ebc159e50 2023-11-13T10:56:47.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.448+0000 7f7eb9d9b640 1 --2- 172.21.15.16:0/356795773 >> v2:172.21.15.184:3300/0 conn(0x7f7ebc0cf200 0x7f7ebc070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:47.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.449+0000 7f7eba59c640 1 --2- 172.21.15.16:0/356795773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc159e50 0x7f7ebc0b32f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc06c860 tx=0x7f7eac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb471fae07c7ff96 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.449+0000 7f7eb959a640 1 -- 172.21.15.16:0/356795773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac02cb00 con 0x7f7ebc159e50 2023-11-13T10:56:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.449+0000 7f7eb959a640 1 -- 172.21.15.16:0/356795773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac02cc60 con 0x7f7ebc159e50 2023-11-13T10:56:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.449+0000 7f7eb959a640 1 -- 172.21.15.16:0/356795773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0343f0 con 0x7f7ebc159e50 2023-11-13T10:56:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.450+0000 7f7ebb59e640 1 -- 172.21.15.16:0/356795773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc159e50 msgr2=0x7f7ebc0b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.450+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/356795773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc159e50 0x7f7ebc0b32f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc06c860 tx=0x7f7eac02bf60 comp rx=0 tx=0).stop 2023-11-13T10:56:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.450+0000 7f7ebb59e640 1 -- 172.21.15.16:0/356795773 shutdown_connections 2023-11-13T10:56:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.450+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/356795773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc070f90 0x7f7ebc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.450+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/356795773 >> v2:172.21.15.184:3300/0 conn(0x7f7ebc0cf200 0x7f7ebc070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.450+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/356795773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc159e50 0x7f7ebc0b32f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.450+0000 7f7ebb59e640 1 -- 172.21.15.16:0/356795773 >> 172.21.15.16:0/356795773 conn(0x7f7ebc0cc3c0 msgr2=0x7f7ebc0cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.451+0000 7f7ebb59e640 1 -- 172.21.15.16:0/356795773 shutdown_connections 2023-11-13T10:56:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.451+0000 7f7ebb59e640 1 -- 172.21.15.16:0/356795773 wait complete. 2023-11-13T10:56:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.453+0000 7f7ebb59e640 1 Processor -- start 2023-11-13T10:56:47.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.453+0000 7f7ebb59e640 1 -- start start 2023-11-13T10:56:47.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.454+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc070f90 0x7f7ebc145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.454+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc0cf200 0x7f7ebc131d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.454+0000 7f7ebb59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ebc159e50 0x7f7ebc132250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.454+0000 7f7ebb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc1486c0 con 0x7f7ebc070f90 2023-11-13T10:56:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.454+0000 7f7ebb59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc147d20 con 0x7f7ebc159e50 2023-11-13T10:56:47.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7ebb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc148000 con 0x7f7ebc0cf200 2023-11-13T10:56:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc070f90 0x7f7ebc145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc070f90 0x7f7ebc145540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46684/0 (socket says 172.21.15.16:46684) 2023-11-13T10:56:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eba59c640 1 -- 172.21.15.16:0/2492565577 learned_addr learned my addr 172.21.15.16:0/2492565577 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eba59c640 1 -- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc0cf200 msgr2=0x7f7ebc131d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7ebad9d640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ebc159e50 0x7f7ebc132250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eb9d9b640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc0cf200 0x7f7ebc131d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eba59c640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc0cf200 0x7f7ebc131d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eba59c640 1 -- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ebc159e50 msgr2=0x7f7ebc132250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eba59c640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ebc159e50 0x7f7ebc132250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eba59c640 1 -- 172.21.15.16:0/2492565577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac009b30 con 0x7f7ebc070f90 2023-11-13T10:56:47.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7eb9d9b640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc0cf200 0x7f7ebc131d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.455+0000 7f7ebad9d640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ebc159e50 0x7f7ebc132250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.456+0000 7f7eba59c640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc070f90 0x7f7ebc145540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000f80 tx=0x7f7eac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.456+0000 7f7eaaffd640 1 -- 172.21.15.16:0/2492565577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac03e450 con 0x7f7ebc070f90 2023-11-13T10:56:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.456+0000 7f7eaaffd640 1 -- 172.21.15.16:0/2492565577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac03e5b0 con 0x7f7ebc070f90 2023-11-13T10:56:47.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.456+0000 7f7eaaffd640 1 -- 172.21.15.16:0/2492565577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac03e960 con 0x7f7ebc070f90 2023-11-13T10:56:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.456+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc1482e0 con 0x7f7ebc070f90 2023-11-13T10:56:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.456+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc132790 con 0x7f7ebc070f90 2023-11-13T10:56:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.457+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc0b4590 con 0x7f7ebc070f90 2023-11-13T10:56:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.458+0000 7f7eaaffd640 1 -- 172.21.15.16:0/2492565577 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7eac03eac0 con 0x7f7ebc070f90 2023-11-13T10:56:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.459+0000 7f7eaaffd640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e9c06ae40 0x7f7e9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.459+0000 7f7eaaffd640 1 -- 172.21.15.16:0/2492565577 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f7eac0ac8b0 con 0x7f7ebc070f90 2023-11-13T10:56:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.460+0000 7f7eb9d9b640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e9c06ae40 0x7f7e9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.461+0000 7f7eb9d9b640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e9c06ae40 0x7f7e9c06d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006fd0 tx=0x7f7eb0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.463+0000 7f7eaaffd640 1 -- 172.21.15.16:0/2492565577 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7eac077580 con 0x7f7ebc070f90 2023-11-13T10:56:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.659+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ebc04e4d0 con 0x7f7ebc070f90 2023-11-13T10:56:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.660+0000 7f7eaaffd640 1 -- 172.21.15.16:0/2492565577 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7eac07aba0 con 0x7f7ebc070f90 2023-11-13T10:56:47.665 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:47.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.663+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e9c06ae40 msgr2=0x7f7e9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.663+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e9c06ae40 0x7f7e9c06d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006fd0 tx=0x7f7eb0008040 comp rx=0 tx=0).stop 2023-11-13T10:56:47.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.663+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc070f90 msgr2=0x7f7ebc145540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.663+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc070f90 0x7f7ebc145540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000f80 tx=0x7f7eac004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:47.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.664+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 shutdown_connections 2023-11-13T10:56:47.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.664+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7e9c06ae40 0x7f7e9c06d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.664+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ebc159e50 0x7f7ebc132250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.664+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ebc0cf200 0x7f7ebc131d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.664+0000 7f7ebb59e640 1 --2- 172.21.15.16:0/2492565577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ebc070f90 0x7f7ebc145540 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.664+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 >> 172.21.15.16:0/2492565577 conn(0x7f7ebc0cc3c0 msgr2=0x7f7ebc06bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:47.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.664+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 shutdown_connections 2023-11-13T10:56:47.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.664+0000 7f7ebb59e640 1 -- 172.21.15.16:0/2492565577 wait complete. 2023-11-13T10:56:47.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.921+0000 7f0dff59e640 1 Processor -- start 2023-11-13T10:56:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.921+0000 7f0dff59e640 1 -- start start 2023-11-13T10:56:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.923+0000 7f0dff59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 0x7f0e0006c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.923+0000 7f0dff59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e000ce260 0x7f0e000cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.923+0000 7f0dff59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0006b860 con 0x7f0e0015bcc0 2023-11-13T10:56:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.923+0000 7f0dff59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0006bb40 con 0x7f0e000ce260 2023-11-13T10:56:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.923+0000 7f0dff59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0006be20 con 0x7f0e0015b8e0 2023-11-13T10:56:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.923+0000 7f0df7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0e0015bcc0 0x7f0e000cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35438/0 (socket says 172.21.15.16:35438) 2023-11-13T10:56:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.923+0000 7f0df7fff640 1 -- 172.21.15.16:0/2414919889 learned_addr learned my addr 172.21.15.16:0/2414919889 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.923+0000 7f0dfed9d640 1 --2- 172.21.15.16:0/2414919889 >> v2:172.21.15.16:3300/0 conn(0x7f0e000ce260 0x7f0e000cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.923+0000 7f0dfe59c640 1 --2- 172.21.15.16:0/2414919889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 0x7f0e0006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.924+0000 7f0dfdd9b640 1 -- 172.21.15.16:0/2414919889 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799164167 0 0) 0x7f0e0006b860 con 0x7f0e0015bcc0 2023-11-13T10:56:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.924+0000 7f0dfed9d640 1 -- 172.21.15.16:0/2414919889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 msgr2=0x7f0e0006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.924+0000 7f0dfed9d640 1 --2- 172.21.15.16:0/2414919889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 0x7f0e0006c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.924+0000 7f0dfed9d640 1 -- 172.21.15.16:0/2414919889 >> v1:172.21.15.16:6789/0 conn(0x7f0e0015bcc0 legacy=0x7f0e000cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.924+0000 7f0dfed9d640 1 -- 172.21.15.16:0/2414919889 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0006c100 con 0x7f0e000ce260 2023-11-13T10:56:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.925+0000 7f0dfed9d640 1 --2- 172.21.15.16:0/2414919889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000ce260 0x7f0e000cbae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009dc0 tx=0x7f0df802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85594124b824d660 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.925+0000 7f0dfdd9b640 1 -- 172.21.15.16:0/2414919889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df802cc40 con 0x7f0e000ce260 2023-11-13T10:56:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.925+0000 7f0dfdd9b640 1 -- 172.21.15.16:0/2414919889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df802cda0 con 0x7f0e000ce260 2023-11-13T10:56:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.925+0000 7f0dfdd9b640 1 -- 172.21.15.16:0/2414919889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8034660 con 0x7f0e000ce260 2023-11-13T10:56:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.926+0000 7f0dff59e640 1 -- 172.21.15.16:0/2414919889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000ce260 msgr2=0x7f0e000cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.926+0000 7f0dff59e640 1 --2- 172.21.15.16:0/2414919889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000ce260 0x7f0e000cbae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009dc0 tx=0x7f0df802be90 comp rx=0 tx=0).stop 2023-11-13T10:56:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.926+0000 7f0dff59e640 1 -- 172.21.15.16:0/2414919889 shutdown_connections 2023-11-13T10:56:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.926+0000 7f0dff59e640 1 --2- 172.21.15.16:0/2414919889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000ce260 0x7f0e000cbae0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.926+0000 7f0dff59e640 1 --2- 172.21.15.16:0/2414919889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 0x7f0e0006c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.926+0000 7f0dff59e640 1 -- 172.21.15.16:0/2414919889 >> 172.21.15.16:0/2414919889 conn(0x7f0e000b0970 msgr2=0x7f0e000b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.926+0000 7f0dff59e640 1 -- 172.21.15.16:0/2414919889 shutdown_connections 2023-11-13T10:56:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.926+0000 7f0dff59e640 1 -- 172.21.15.16:0/2414919889 wait complete. 2023-11-13T10:56:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.928+0000 7f0dff59e640 1 Processor -- start 2023-11-13T10:56:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.928+0000 7f0dff59e640 1 -- start start 2023-11-13T10:56:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.929+0000 7f0dff59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e000ce260 0x7f0e0014a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.929+0000 7f0dff59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 0x7f0e001f0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.929+0000 7f0dff59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000acc10 0x7f0e000ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.929+0000 7f0dff59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000a86a0 con 0x7f0e000acc10 2023-11-13T10:56:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.929+0000 7f0dff59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000a8980 con 0x7f0e000ce260 2023-11-13T10:56:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.929+0000 7f0dff59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000a8c60 con 0x7f0e0015b8e0 2023-11-13T10:56:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.929+0000 7f0dfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000acc10 0x7f0e000ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.929+0000 7f0dfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000acc10 0x7f0e000ad0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46700/0 (socket says 172.21.15.16:46700) 2023-11-13T10:56:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.929+0000 7f0dfed9d640 1 -- 172.21.15.16:0/3250864240 learned_addr learned my addr 172.21.15.16:0/3250864240 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.930+0000 7f0dfe59c640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e000ce260 0x7f0e0014a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.930+0000 7f0df7fff640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 0x7f0e001f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.930+0000 7f0dfed9d640 1 -- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 msgr2=0x7f0e001f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.930+0000 7f0dfed9d640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 0x7f0e001f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.930+0000 7f0dfed9d640 1 -- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e000ce260 msgr2=0x7f0e0014a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.930+0000 7f0dfed9d640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e000ce260 0x7f0e0014a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.930+0000 7f0dfed9d640 1 -- 172.21.15.16:0/3250864240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df8009a70 con 0x7f0e000acc10 2023-11-13T10:56:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.930+0000 7f0dfed9d640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000acc10 0x7f0e000ad0c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009ee0 tx=0x7f0df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.931+0000 7f0df6ffd640 1 -- 172.21.15.16:0/3250864240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df803e610 con 0x7f0e000acc10 2023-11-13T10:56:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.931+0000 7f0df6ffd640 1 -- 172.21.15.16:0/3250864240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df803e770 con 0x7f0e000acc10 2023-11-13T10:56:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.931+0000 7f0df6ffd640 1 -- 172.21.15.16:0/3250864240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df803eb20 con 0x7f0e000acc10 2023-11-13T10:56:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.931+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e000a8f40 con 0x7f0e000acc10 2023-11-13T10:56:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.931+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e000ad600 con 0x7f0e000acc10 2023-11-13T10:56:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.932+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e000a9460 con 0x7f0e000acc10 2023-11-13T10:56:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.933+0000 7f0df6ffd640 1 -- 172.21.15.16:0/3250864240 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0df803b050 con 0x7f0e000acc10 2023-11-13T10:56:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.934+0000 7f0df6ffd640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0de006ae40 0x7f0de006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.934+0000 7f0df6ffd640 1 -- 172.21.15.16:0/3250864240 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f0df80ac120 con 0x7f0e000acc10 2023-11-13T10:56:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.934+0000 7f0dfe59c640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0de006ae40 0x7f0de006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.936+0000 7f0dfe59c640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0de006ae40 0x7f0de006d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0df0006fd0 tx=0x7f0df0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:47.938+0000 7f0df6ffd640 1 -- 172.21.15.16:0/3250864240 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e000a9460 con 0x7f0e000acc10 2023-11-13T10:56:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.135+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e000a9460 con 0x7f0e000acc10 2023-11-13T10:56:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.136+0000 7f0df6ffd640 1 -- 172.21.15.16:0/3250864240 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e000a9460 con 0x7f0e000acc10 2023-11-13T10:56:48.141 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0de006ae40 msgr2=0x7f0de006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0de006ae40 0x7f0de006d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0df0006fd0 tx=0x7f0df0008040 comp rx=0 tx=0).stop 2023-11-13T10:56:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000acc10 msgr2=0x7f0e000ad0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000acc10 0x7f0e000ad0c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009ee0 tx=0x7f0df8004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 shutdown_connections 2023-11-13T10:56:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0de006ae40 0x7f0de006d300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e000acc10 0x7f0e000ad0c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0e0015b8e0 0x7f0e001f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 --2- 172.21.15.16:0/3250864240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e000ce260 0x7f0e0014a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 >> 172.21.15.16:0/3250864240 conn(0x7f0e000b0970 msgr2=0x7f0e000b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 shutdown_connections 2023-11-13T10:56:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.139+0000 7f0dff59e640 1 -- 172.21.15.16:0/3250864240 wait complete. 2023-11-13T10:56:48.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.392+0000 7f94fffff640 1 Processor -- start 2023-11-13T10:56:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.392+0000 7f94fffff640 1 -- start start 2023-11-13T10:56:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.394+0000 7f94fffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f950006b6c0 0x7f95000ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.394+0000 7f94fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95000ace20 0x7f950006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.394+0000 7f94fffff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95000ad270 con 0x7f95000715b0 2023-11-13T10:56:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.394+0000 7f94fffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950006c920 con 0x7f95000ace20 2023-11-13T10:56:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.394+0000 7f94fffff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950006bca0 con 0x7f950006b6c0 2023-11-13T10:56:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.394+0000 7f94ff7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95000ace20 0x7f950006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.394+0000 7f94ff7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95000ace20 0x7f950006fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46706/0 (socket says 172.21.15.16:46706) 2023-11-13T10:56:48.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.394+0000 7f94ff7fe640 1 -- 172.21.15.16:0/1012119767 learned_addr learned my addr 172.21.15.16:0/1012119767 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.395+0000 7f94fe7fc640 1 --2- 172.21.15.16:0/1012119767 >> v2:172.21.15.184:3300/0 conn(0x7f950006b6c0 0x7f95000ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.395+0000 7f94ff7fe640 1 -- 172.21.15.16:0/1012119767 >> v2:172.21.15.184:3300/0 conn(0x7f950006b6c0 msgr2=0x7f95000ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.395+0000 7f94ff7fe640 1 --2- 172.21.15.16:0/1012119767 >> v2:172.21.15.184:3300/0 conn(0x7f950006b6c0 0x7f95000ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.395+0000 7f94ff7fe640 1 -- 172.21.15.16:0/1012119767 >> v1:172.21.15.184:6789/0 conn(0x7f95000715b0 legacy=0x7f95000b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.395+0000 7f94ff7fe640 1 -- 172.21.15.16:0/1012119767 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950006bf80 con 0x7f95000ace20 2023-11-13T10:56:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.395+0000 7f94fe7fc640 1 --2- 172.21.15.16:0/1012119767 >> v2:172.21.15.184:3300/0 conn(0x7f950006b6c0 0x7f95000ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.396+0000 7f94ff7fe640 1 --2- 172.21.15.16:0/1012119767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 0x7f950006fb50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f950006dc00 tx=0x7f94f802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce97b293586f787 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.396+0000 7f94fdffb640 1 -- 172.21.15.16:0/1012119767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f802c9f0 con 0x7f95000ace20 2023-11-13T10:56:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.396+0000 7f94fdffb640 1 -- 172.21.15.16:0/1012119767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f802cb50 con 0x7f95000ace20 2023-11-13T10:56:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.396+0000 7f94fdffb640 1 -- 172.21.15.16:0/1012119767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f80343f0 con 0x7f95000ace20 2023-11-13T10:56:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.397+0000 7f94fffff640 1 -- 172.21.15.16:0/1012119767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 msgr2=0x7f950006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.397+0000 7f94fffff640 1 --2- 172.21.15.16:0/1012119767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 0x7f950006fb50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f950006dc00 tx=0x7f94f802be50 comp rx=0 tx=0).stop 2023-11-13T10:56:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.397+0000 7f94fffff640 1 -- 172.21.15.16:0/1012119767 shutdown_connections 2023-11-13T10:56:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.397+0000 7f94fffff640 1 --2- 172.21.15.16:0/1012119767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 0x7f950006fb50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.397+0000 7f94fffff640 1 --2- 172.21.15.16:0/1012119767 >> v2:172.21.15.184:3300/0 conn(0x7f950006b6c0 0x7f95000ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.397+0000 7f94fffff640 1 -- 172.21.15.16:0/1012119767 >> 172.21.15.16:0/1012119767 conn(0x7f95000b0970 msgr2=0x7f95000b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.397+0000 7f94fffff640 1 -- 172.21.15.16:0/1012119767 shutdown_connections 2023-11-13T10:56:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.397+0000 7f94fffff640 1 -- 172.21.15.16:0/1012119767 wait complete. 2023-11-13T10:56:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.399+0000 7f94fffff640 1 Processor -- start 2023-11-13T10:56:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.399+0000 7f94fffff640 1 -- start start 2023-11-13T10:56:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.400+0000 7f94fffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950006b6c0 0x7f95001f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.400+0000 7f94fffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000715b0 0x7f950015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.400+0000 7f94fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 0x7f950015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.400+0000 7f94fffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95001f60f0 con 0x7f95000ace20 2023-11-13T10:56:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.400+0000 7f94fffff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95001f63d0 con 0x7f95000715b0 2023-11-13T10:56:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.400+0000 7f94fffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95001f66b0 con 0x7f950006b6c0 2023-11-13T10:56:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 0x7f950015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 0x7f950015e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46714/0 (socket says 172.21.15.16:46714) 2023-11-13T10:56:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94ff7fe640 1 -- 172.21.15.16:0/2432640100 learned_addr learned my addr 172.21.15.16:0/2432640100 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94fe7fc640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000715b0 0x7f950015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94feffd640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950006b6c0 0x7f95001f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94ff7fe640 1 -- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950006b6c0 msgr2=0x7f95001f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94ff7fe640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950006b6c0 0x7f95001f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94ff7fe640 1 -- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000715b0 msgr2=0x7f950015e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94ff7fe640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000715b0 0x7f950015e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94ff7fe640 1 -- 172.21.15.16:0/2432640100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8009b30 con 0x7f95000ace20 2023-11-13T10:56:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.401+0000 7f94ff7fe640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 0x7f950015e7f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94f8036040 tx=0x7f94f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.402+0000 7f94e77fe640 1 -- 172.21.15.16:0/2432640100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f803e450 con 0x7f95000ace20 2023-11-13T10:56:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.402+0000 7f94e77fe640 1 -- 172.21.15.16:0/2432640100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f803e5d0 con 0x7f95000ace20 2023-11-13T10:56:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.402+0000 7f94e77fe640 1 -- 172.21.15.16:0/2432640100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f803e980 con 0x7f95000ace20 2023-11-13T10:56:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.402+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95001f6990 con 0x7f95000ace20 2023-11-13T10:56:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.402+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9500133d70 con 0x7f95000ace20 2023-11-13T10:56:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.403+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950004e4d0 con 0x7f95000ace20 2023-11-13T10:56:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.404+0000 7f94e77fe640 1 -- 172.21.15.16:0/2432640100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94f803eae0 con 0x7f95000ace20 2023-11-13T10:56:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.405+0000 7f94e77fe640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94ec06ae40 0x7f94ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.405+0000 7f94e77fe640 1 -- 172.21.15.16:0/2432640100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f94f80ac750 con 0x7f95000ace20 2023-11-13T10:56:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.406+0000 7f94feffd640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94ec06ae40 0x7f94ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.407+0000 7f94feffd640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94ec06ae40 0x7f94ec06d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006fd0 tx=0x7f94f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.410+0000 7f94e77fe640 1 -- 172.21.15.16:0/2432640100 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94f8077540 con 0x7f95000ace20 2023-11-13T10:56:48.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.605+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f95000ad270 con 0x7f95000ace20 2023-11-13T10:56:48.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.606+0000 7f94e77fe640 1 -- 172.21.15.16:0/2432640100 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95000ad270 con 0x7f95000ace20 2023-11-13T10:56:48.611 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94ec06ae40 msgr2=0x7f94ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94ec06ae40 0x7f94ec06d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006fd0 tx=0x7f94f0008040 comp rx=0 tx=0).stop 2023-11-13T10:56:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 msgr2=0x7f950015e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 0x7f950015e7f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94f8036040 tx=0x7f94f8004a40 comp rx=0 tx=0).stop 2023-11-13T10:56:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 shutdown_connections 2023-11-13T10:56:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94ec06ae40 0x7f94ec06d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95000ace20 0x7f950015e7f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95000715b0 0x7f950015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 --2- 172.21.15.16:0/2432640100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f950006b6c0 0x7f95001f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 >> 172.21.15.16:0/2432640100 conn(0x7f95000b0970 msgr2=0x7f95000b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:48.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 shutdown_connections 2023-11-13T10:56:48.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.609+0000 7f94fffff640 1 -- 172.21.15.16:0/2432640100 wait complete. 2023-11-13T10:56:48.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.878+0000 7f7d5b59e640 1 Processor -- start 2023-11-13T10:56:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.878+0000 7f7d5b59e640 1 -- start start 2023-11-13T10:56:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.879+0000 7f7d5b59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c15ce70 0x7f7d5c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.879+0000 7f7d5b59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d5c0cdbb0 0x7f7d5c0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.879+0000 7f7d5b59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7d5c0d1850 0x7f7d5c0d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.879+0000 7f7d5b59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c0d3240 con 0x7f7d5c0cdbb0 2023-11-13T10:56:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.879+0000 7f7d5b59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c0ce0e0 con 0x7f7d5c0d1850 2023-11-13T10:56:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.879+0000 7f7d5b59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0ce3c0 con 0x7f7d5c15ce70 2023-11-13T10:56:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.880+0000 7f7d59d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d5c0cdbb0 0x7f7d5c0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.880+0000 7f7d59d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d5c0cdbb0 0x7f7d5c0d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46734/0 (socket says 172.21.15.16:46734) 2023-11-13T10:56:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.880+0000 7f7d5ad9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7d5c0d1850 0x7f7d5c0d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.880+0000 7f7d59d9b640 1 -- 172.21.15.16:0/1918477100 learned_addr learned my addr 172.21.15.16:0/1918477100 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:48.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.880+0000 7f7d5a59c640 1 --2- 172.21.15.16:0/1918477100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c15ce70 0x7f7d5c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.881+0000 7f7d59d9b640 1 -- 172.21.15.16:0/1918477100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c15ce70 msgr2=0x7f7d5c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.881+0000 7f7d59d9b640 1 --2- 172.21.15.16:0/1918477100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c15ce70 0x7f7d5c0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.881+0000 7f7d59d9b640 1 -- 172.21.15.16:0/1918477100 >> v2:172.21.15.184:3300/0 conn(0x7f7d5c0d1850 msgr2=0x7f7d5c0d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.881+0000 7f7d59d9b640 1 --2- 172.21.15.16:0/1918477100 >> v2:172.21.15.184:3300/0 conn(0x7f7d5c0d1850 0x7f7d5c0d2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.881+0000 7f7d59d9b640 1 -- 172.21.15.16:0/1918477100 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c0ce6a0 con 0x7f7d5c0cdbb0 2023-11-13T10:56:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.882+0000 7f7d59d9b640 1 --2- 172.21.15.16:0/1918477100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c0cdbb0 0x7f7d5c0d27c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009870 tx=0x7f7d4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9074afd977e2059 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.883+0000 7f7d5959a640 1 -- 172.21.15.16:0/1918477100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c031640 con 0x7f7d5c0cdbb0 2023-11-13T10:56:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.883+0000 7f7d5959a640 1 -- 172.21.15.16:0/1918477100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c0317a0 con 0x7f7d5c0cdbb0 2023-11-13T10:56:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.883+0000 7f7d5959a640 1 -- 172.21.15.16:0/1918477100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c031b30 con 0x7f7d5c0cdbb0 2023-11-13T10:56:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.884+0000 7f7d5b59e640 1 -- 172.21.15.16:0/1918477100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c0cdbb0 msgr2=0x7f7d5c0d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.884+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/1918477100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c0cdbb0 0x7f7d5c0d27c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009870 tx=0x7f7d4c02b8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.884+0000 7f7d5b59e640 1 -- 172.21.15.16:0/1918477100 shutdown_connections 2023-11-13T10:56:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.884+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/1918477100 >> v2:172.21.15.184:3300/0 conn(0x7f7d5c0d1850 0x7f7d5c0d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.884+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/1918477100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c0cdbb0 0x7f7d5c0d27c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.884+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/1918477100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c15ce70 0x7f7d5c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.884+0000 7f7d5b59e640 1 -- 172.21.15.16:0/1918477100 >> 172.21.15.16:0/1918477100 conn(0x7f7d5c08c580 msgr2=0x7f7d5c09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.884+0000 7f7d5b59e640 1 -- 172.21.15.16:0/1918477100 shutdown_connections 2023-11-13T10:56:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.884+0000 7f7d5b59e640 1 -- 172.21.15.16:0/1918477100 wait complete. 2023-11-13T10:56:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.886+0000 7f7d5b59e640 1 Processor -- start 2023-11-13T10:56:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.887+0000 7f7d5b59e640 1 -- start start 2023-11-13T10:56:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.888+0000 7f7d5b59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c0cdbb0 0x7f7d5c152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.888+0000 7f7d5b59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c0d1850 0x7f7d5c153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.888+0000 7f7d5b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c15ce70 0x7f7d5c154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.888+0000 7f7d5b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c15d3d0 con 0x7f7d5c15ce70 2023-11-13T10:56:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.888+0000 7f7d5b59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c15d990 con 0x7f7d5c0cdbb0 2023-11-13T10:56:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.888+0000 7f7d5b59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1525a0 con 0x7f7d5c0d1850 2023-11-13T10:56:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.888+0000 7f7d5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c15ce70 0x7f7d5c154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.888+0000 7f7d5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c15ce70 0x7f7d5c154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46746/0 (socket says 172.21.15.16:46746) 2023-11-13T10:56:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.888+0000 7f7d5ad9d640 1 -- 172.21.15.16:0/3775453077 learned_addr learned my addr 172.21.15.16:0/3775453077 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.889+0000 7f7d5a59c640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c0cdbb0 0x7f7d5c152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.889+0000 7f7d59d9b640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c0d1850 0x7f7d5c153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.889+0000 7f7d5ad9d640 1 -- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c0d1850 msgr2=0x7f7d5c153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.889+0000 7f7d5ad9d640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c0d1850 0x7f7d5c153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.889+0000 7f7d5ad9d640 1 -- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c0cdbb0 msgr2=0x7f7d5c152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.889+0000 7f7d5ad9d640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c0cdbb0 0x7f7d5c152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.889+0000 7f7d5ad9d640 1 -- 172.21.15.16:0/3775453077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c0094d0 con 0x7f7d5c15ce70 2023-11-13T10:56:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.889+0000 7f7d5a59c640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c0cdbb0 0x7f7d5c152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:56:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.889+0000 7f7d5ad9d640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c15ce70 0x7f7d5c154640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009e30 tx=0x7f7d5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.890+0000 7f7d52ffd640 1 -- 172.21.15.16:0/3775453077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54009ca0 con 0x7f7d5c15ce70 2023-11-13T10:56:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.890+0000 7f7d52ffd640 1 -- 172.21.15.16:0/3775453077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5400f040 con 0x7f7d5c15ce70 2023-11-13T10:56:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.890+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c153cc0 con 0x7f7d5c15ce70 2023-11-13T10:56:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.890+0000 7f7d52ffd640 1 -- 172.21.15.16:0/3775453077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d540146c0 con 0x7f7d5c15ce70 2023-11-13T10:56:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.890+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c1600f0 con 0x7f7d5c15ce70 2023-11-13T10:56:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.891+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c04e4d0 con 0x7f7d5c15ce70 2023-11-13T10:56:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.893+0000 7f7d52ffd640 1 -- 172.21.15.16:0/3775453077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d54014820 con 0x7f7d5c15ce70 2023-11-13T10:56:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.894+0000 7f7d52ffd640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7d3c06ae40 0x7f7d3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.894+0000 7f7d52ffd640 1 -- 172.21.15.16:0/3775453077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f7d5408a950 con 0x7f7d5c15ce70 2023-11-13T10:56:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.894+0000 7f7d5a59c640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7d3c06ae40 0x7f7d3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.895+0000 7f7d5a59c640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7d3c06ae40 0x7f7d3c06d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009a70 tx=0x7f7d44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:48.899+0000 7f7d52ffd640 1 -- 172.21.15.16:0/3775453077 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d54055620 con 0x7f7d5c15ce70 2023-11-13T10:56:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.095+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7d5c15dad0 con 0x7f7d5c15ce70 2023-11-13T10:56:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.096+0000 7f7d52ffd640 1 -- 172.21.15.16:0/3775453077 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7d540180a0 con 0x7f7d5c15ce70 2023-11-13T10:56:49.101 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:49.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7d3c06ae40 msgr2=0x7f7d3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:49.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7d3c06ae40 0x7f7d3c06d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009a70 tx=0x7f7d44008040 comp rx=0 tx=0).stop 2023-11-13T10:56:49.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c15ce70 msgr2=0x7f7d5c154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c15ce70 0x7f7d5c154640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009e30 tx=0x7f7d5400c490 comp rx=0 tx=0).stop 2023-11-13T10:56:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 shutdown_connections 2023-11-13T10:56:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7d3c06ae40 0x7f7d3c06d300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c15ce70 0x7f7d5c154640 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7d5c0d1850 0x7f7d5c153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 --2- 172.21.15.16:0/3775453077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d5c0cdbb0 0x7f7d5c152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 >> 172.21.15.16:0/3775453077 conn(0x7f7d5c08c580 msgr2=0x7f7d5c06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.099+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 shutdown_connections 2023-11-13T10:56:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.100+0000 7f7d5b59e640 1 -- 172.21.15.16:0/3775453077 wait complete. 2023-11-13T10:56:49.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:56:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.349+0000 7f3fdbfff640 1 Processor -- start 2023-11-13T10:56:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.349+0000 7f3fdbfff640 1 -- start start 2023-11-13T10:56:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.351+0000 7f3fdbfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc06f970 0x7f3fdc162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.351+0000 7f3fdbfff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc0b1840 con 0x7f3fdc161060 2023-11-13T10:56:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.351+0000 7f3fdbfff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc071600 con 0x7f3fdc06a190 2023-11-13T10:56:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.351+0000 7f3fdbfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0718e0 con 0x7f3fdc06f970 2023-11-13T10:56:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.351+0000 7f3fdaffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3fdc161060 0x7f3fdc1621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35454/0 (socket says 172.21.15.16:35454) 2023-11-13T10:56:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.351+0000 7f3fdaffd640 1 -- 172.21.15.16:0/799947592 learned_addr learned my addr 172.21.15.16:0/799947592 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fda7fc640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc06f970 0x7f3fdc162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688861433 0 0) 0x7f3fdc0b1840 con 0x7f3fdc161060 2023-11-13T10:56:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc4003520 con 0x7f3fdc161060 2023-11-13T10:56:49.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3820468537 0 0) 0x7f3fc4003520 con 0x7f3fdc161060 2023-11-13T10:56:49.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc06f970 msgr2=0x7f3fdc162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:49.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc06f970 0x7f3fdc162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 >> v1:172.21.15.184:6789/0 conn(0x7f3fdc06a190 legacy=0x7f3fdc0b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:49.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc071bc0 con 0x7f3fdc161060 2023-11-13T10:56:49.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f3fcc001670 con 0x7f3fdc161060 2023-11-13T10:56:49.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 >> v1:172.21.15.16:6789/0 conn(0x7f3fdc161060 legacy=0x7f3fdc1621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:49.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc40039a0 0x7f3fc4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:49.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc06f970 0x7f3fc40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc4004d00 0x7f3fc4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0718e0 con 0x7f3fc40039a0 2023-11-13T10:56:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc400ba30 con 0x7f3fc4004d00 2023-11-13T10:56:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.352+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc400bbf0 con 0x7f3fdc06f970 2023-11-13T10:56:49.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.353+0000 7f3fdaffd640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc40039a0 0x7f3fc4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:49.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.353+0000 7f3fda7fc640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc06f970 0x7f3fc40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:49.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.353+0000 7f3fdb7fe640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc4004d00 0x7f3fc4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:49.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.353+0000 7f3fdaffd640 1 -- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc06f970 msgr2=0x7f3fc40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:49.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.353+0000 7f3fdaffd640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc06f970 0x7f3fc40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.353+0000 7f3fdaffd640 1 -- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc4004d00 msgr2=0x7f3fc4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:49.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.353+0000 7f3fdaffd640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc4004d00 0x7f3fc4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.353+0000 7f3fdaffd640 1 -- 172.21.15.16:0/799947592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc4003520 con 0x7f3fc40039a0 2023-11-13T10:56:49.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.354+0000 7f3fdaffd640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc40039a0 0x7f3fc4003dc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c210 tx=0x7f3fc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb925d2c1e04615 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:49.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.354+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc005ad0 con 0x7f3fc40039a0 2023-11-13T10:56:49.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.355+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fcc005c30 con 0x7f3fc40039a0 2023-11-13T10:56:49.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.355+0000 7f3fd9ffb640 1 -- 172.21.15.16:0/799947592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc02c890 con 0x7f3fc40039a0 2023-11-13T10:56:49.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.355+0000 7f3fdbfff640 1 -- 172.21.15.16:0/799947592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc40039a0 msgr2=0x7f3fc4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:49.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.355+0000 7f3fdbfff640 1 --2- 172.21.15.16:0/799947592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc40039a0 0x7f3fc4003dc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c210 tx=0x7f3fc40057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.355+0000 7f3fdb7fe640 1 -- 172.21.15.16:0/799947592 reap_dead start 2023-11-13T10:56:49.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.355+0000 7f3fdbfff640 1 -- 172.21.15.16:0/799947592 shutdown_connections 2023-11-13T10:56:49.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.355+0000 7f3fdbfff640 1 -- 172.21.15.16:0/799947592 >> 172.21.15.16:0/799947592 conn(0x7f3fdc0b04a0 msgr2=0x7f3fdc0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:49.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.355+0000 7f3fdbfff640 1 -- 172.21.15.16:0/799947592 shutdown_connections 2023-11-13T10:56:49.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.355+0000 7f3fdbfff640 1 -- 172.21.15.16:0/799947592 wait complete. 2023-11-13T10:56:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.357+0000 7f3fdbfff640 1 Processor -- start 2023-11-13T10:56:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.357+0000 7f3fdbfff640 1 -- start start 2023-11-13T10:56:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.358+0000 7f3fdbfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc40039a0 0x7f3fdc0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.358+0000 7f3fdbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fdc0a8e60 0x7f3fdc1f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.358+0000 7f3fdbfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc15ac90 0x7f3fdc158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.358+0000 7f3fdbfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0a95e0 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.358+0000 7f3fdbfff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0a9300 con 0x7f3fc40039a0 2023-11-13T10:56:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.358+0000 7f3fdbfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc15a1b0 con 0x7f3fdc15ac90 2023-11-13T10:56:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fda7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fdc0a8e60 0x7f3fdc1f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fda7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fdc0a8e60 0x7f3fdc1f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46758/0 (socket says 172.21.15.16:46758) 2023-11-13T10:56:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fda7fc640 1 -- 172.21.15.16:0/2607634366 learned_addr learned my addr 172.21.15.16:0/2607634366 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fdaffd640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc40039a0 0x7f3fdc0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fdb7fe640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc15ac90 0x7f3fdc158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fda7fc640 1 -- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc15ac90 msgr2=0x7f3fdc158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fda7fc640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc15ac90 0x7f3fdc158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fda7fc640 1 -- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc40039a0 msgr2=0x7f3fdc0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:49.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fda7fc640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc40039a0 0x7f3fdc0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fda7fc640 1 -- 172.21.15.16:0/2607634366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc15a490 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.359+0000 7f3fdaffd640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc40039a0 0x7f3fdc0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.360+0000 7f3fda7fc640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fdc0a8e60 0x7f3fdc1f6e70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0011600 tx=0x7f3fd0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.360+0000 7f3fc37fe640 1 -- 172.21.15.16:0/2607634366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000f920 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.360+0000 7f3fc37fe640 1 -- 172.21.15.16:0/2607634366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd000fa80 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.360+0000 7f3fc37fe640 1 -- 172.21.15.16:0/2607634366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000fdf0 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.360+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc15a770 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.360+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc0a8c60 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.361+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc04e4d0 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.363+0000 7f3fc37fe640 1 -- 172.21.15.16:0/2607634366 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3fd0015460 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.364+0000 7f3fc37fe640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3fc806ae40 0x7f3fc806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.364+0000 7f3fc37fe640 1 -- 172.21.15.16:0/2607634366 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f3fd0089d00 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.364+0000 7f3fdaffd640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3fc806ae40 0x7f3fc806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.365+0000 7f3fdaffd640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3fc806ae40 0x7f3fc806d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc034000 tx=0x7f3fcc037040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.368+0000 7f3fc37fe640 1 -- 172.21.15.16:0/2607634366 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fd0054ae0 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.576+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3fdc0b2820 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.576+0000 7f3fc37fe640 1 -- 172.21.15.16:0/2607634366 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3fd0022080 con 0x7f3fdc0a8e60 2023-11-13T10:56:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:56:49.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3fc806ae40 msgr2=0x7f3fc806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:49.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3fc806ae40 0x7f3fc806d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc034000 tx=0x7f3fcc037040 comp rx=0 tx=0).stop 2023-11-13T10:56:49.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fdc0a8e60 msgr2=0x7f3fdc1f6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:49.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fdc0a8e60 0x7f3fdc1f6e70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0011600 tx=0x7f3fd0011ac0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 shutdown_connections 2023-11-13T10:56:49.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3fc806ae40 0x7f3fc806d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3fdc15ac90 0x7f3fdc158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fdc0a8e60 0x7f3fdc1f6e70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 --2- 172.21.15.16:0/2607634366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc40039a0 0x7f3fdc0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 >> 172.21.15.16:0/2607634366 conn(0x7f3fdc0b04a0 msgr2=0x7f3fdc06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 shutdown_connections 2023-11-13T10:56:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:49.580+0000 7f3fdbfff640 1 -- 172.21.15.16:0/2607634366 wait complete. 2023-11-13T10:56:49.604 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi184.front.sepia.ceph.com... 2023-11-13T10:56:49.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:49.605 DEBUG:teuthology.orchestra.run.smithi184:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-13T10:56:49.713 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T10:56:49.713 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T10:56:49.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:49.718 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-13T10:56:49.776 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:56:49.776 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:49.843 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:56:49.843 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:49.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.115+0000 7f70cd78c640 1 Processor -- start 2023-11-13T10:56:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.115+0000 7f70cd78c640 1 -- start start 2023-11-13T10:56:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.117+0000 7f70cd78c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c815cc90 0x7f70c806a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.117+0000 7f70cd78c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f70c8071320 0x7f70c806a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.117+0000 7f70cd78c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c806fc50 con 0x7f70c8070cf0 2023-11-13T10:56:50.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.117+0000 7f70cd78c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c806ff30 con 0x7f70c8071320 2023-11-13T10:56:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.117+0000 7f70cd78c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c8070210 con 0x7f70c815cc90 2023-11-13T10:56:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.118+0000 7f70ccf8b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f70c8071320 0x7f70c806a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.118+0000 7f70c7fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c815cc90 0x7f70c806a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.118+0000 7f70ccf8b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f70c8071320 0x7f70c806a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56778/0 (socket says 172.21.15.16:56778) 2023-11-13T10:56:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.118+0000 7f70ccf8b640 1 -- 172.21.15.16:0/4056469257 learned_addr learned my addr 172.21.15.16:0/4056469257 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.119+0000 7f70ccf8b640 1 -- 172.21.15.16:0/4056469257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c815cc90 msgr2=0x7f70c806a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.119+0000 7f70ccf8b640 1 --2- 172.21.15.16:0/4056469257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c815cc90 0x7f70c806a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.119+0000 7f70ccf8b640 1 -- 172.21.15.16:0/4056469257 >> v1:172.21.15.184:6789/0 conn(0x7f70c8070cf0 legacy=0x7f70c806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.119+0000 7f70ccf8b640 1 -- 172.21.15.16:0/4056469257 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c80704f0 con 0x7f70c8071320 2023-11-13T10:56:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.120+0000 7f70ccf8b640 1 --2- 172.21.15.16:0/4056469257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c8071320 0x7f70c806a6d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009e50 tx=0x7f70c002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ebafc261115fe66 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.121+0000 7f70c6ffd640 1 -- 172.21.15.16:0/4056469257 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c002cb00 con 0x7f70c8071320 2023-11-13T10:56:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.121+0000 7f70c6ffd640 1 -- 172.21.15.16:0/4056469257 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c002cc60 con 0x7f70c8071320 2023-11-13T10:56:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.121+0000 7f70c6ffd640 1 -- 172.21.15.16:0/4056469257 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00343f0 con 0x7f70c8071320 2023-11-13T10:56:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.122+0000 7f70cd78c640 1 -- 172.21.15.16:0/4056469257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c8071320 msgr2=0x7f70c806a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.122+0000 7f70cd78c640 1 --2- 172.21.15.16:0/4056469257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c8071320 0x7f70c806a6d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009e50 tx=0x7f70c002bf60 comp rx=0 tx=0).stop 2023-11-13T10:56:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.122+0000 7f70cd78c640 1 -- 172.21.15.16:0/4056469257 shutdown_connections 2023-11-13T10:56:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.122+0000 7f70cd78c640 1 --2- 172.21.15.16:0/4056469257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c8071320 0x7f70c806a6d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.122+0000 7f70cd78c640 1 --2- 172.21.15.16:0/4056469257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c815cc90 0x7f70c806a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.122+0000 7f70cd78c640 1 -- 172.21.15.16:0/4056469257 >> 172.21.15.16:0/4056469257 conn(0x7f70c80b04a0 msgr2=0x7f70c80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.122+0000 7f70cd78c640 1 -- 172.21.15.16:0/4056469257 shutdown_connections 2023-11-13T10:56:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.122+0000 7f70cd78c640 1 -- 172.21.15.16:0/4056469257 wait complete. 2023-11-13T10:56:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.124+0000 7f70cd78c640 1 Processor -- start 2023-11-13T10:56:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.124+0000 7f70cd78c640 1 -- start start 2023-11-13T10:56:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.125+0000 7f70cd78c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8070cf0 0x7f70c814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.125+0000 7f70cd78c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70c8071320 0x7f70c806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.125+0000 7f70cd78c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815cc90 0x7f70c806bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.125+0000 7f70cd78c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c81f2950 con 0x7f70c8071320 2023-11-13T10:56:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.125+0000 7f70cd78c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c815f850 con 0x7f70c815cc90 2023-11-13T10:56:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.125+0000 7f70cd78c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c815fb30 con 0x7f70c8070cf0 2023-11-13T10:56:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.125+0000 7f70c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70c8071320 0x7f70c806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70c8071320 0x7f70c806b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46770/0 (socket says 172.21.15.16:46770) 2023-11-13T10:56:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70c77fe640 1 -- 172.21.15.16:0/1857335989 learned_addr learned my addr 172.21.15.16:0/1857335989 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70c7fff640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8070cf0 0x7f70c814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70ccf8b640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815cc90 0x7f70c806bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70c77fe640 1 -- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8070cf0 msgr2=0x7f70c814b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70c77fe640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8070cf0 0x7f70c814b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70c77fe640 1 -- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815cc90 msgr2=0x7f70c806bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70c77fe640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815cc90 0x7f70c806bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70c77fe640 1 -- 172.21.15.16:0/1857335989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c0009b30 con 0x7f70c8071320 2023-11-13T10:56:50.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.126+0000 7f70c77fe640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70c8071320 0x7f70c806b9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00c3c0 tx=0x7f70bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:50.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.127+0000 7f70c4ff9640 1 -- 172.21.15.16:0/1857335989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0052d0 con 0x7f70c8071320 2023-11-13T10:56:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.127+0000 7f70c4ff9640 1 -- 172.21.15.16:0/1857335989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70bc005450 con 0x7f70c8071320 2023-11-13T10:56:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.127+0000 7f70c4ff9640 1 -- 172.21.15.16:0/1857335989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc007260 con 0x7f70c8071320 2023-11-13T10:56:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.127+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c815fe10 con 0x7f70c8071320 2023-11-13T10:56:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.127+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c814b750 con 0x7f70c8071320 2023-11-13T10:56:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.128+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c81f2ac0 con 0x7f70c8071320 2023-11-13T10:56:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.129+0000 7f70c4ff9640 1 -- 172.21.15.16:0/1857335989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70bc0073c0 con 0x7f70c8071320 2023-11-13T10:56:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.130+0000 7f70c4ff9640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f70b406ae40 0x7f70b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.130+0000 7f70c4ff9640 1 -- 172.21.15.16:0/1857335989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f70bc088d00 con 0x7f70c8071320 2023-11-13T10:56:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.131+0000 7f70c7fff640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f70b406ae40 0x7f70b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.132+0000 7f70c7fff640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f70b406ae40 0x7f70b406d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009ad0 tx=0x7f70b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.134+0000 7f70c4ff9640 1 -- 172.21.15.16:0/1857335989 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70c81f2ac0 con 0x7f70c8071320 2023-11-13T10:56:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.329+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70c81f2a90 con 0x7f70c8071320 2023-11-13T10:56:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.332+0000 7f70c4ff9640 1 -- 172.21.15.16:0/1857335989 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f70bc053a60 con 0x7f70c8071320 2023-11-13T10:56:50.337 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:50.338 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.334+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f70b406ae40 msgr2=0x7f70b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.334+0000 7f70cd78c640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f70b406ae40 0x7f70b406d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009ad0 tx=0x7f70b8008040 comp rx=0 tx=0).stop 2023-11-13T10:56:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.334+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70c8071320 msgr2=0x7f70c806b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.334+0000 7f70cd78c640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70c8071320 0x7f70c806b9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00c3c0 tx=0x7f70bc00c880 comp rx=0 tx=0).stop 2023-11-13T10:56:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.335+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 shutdown_connections 2023-11-13T10:56:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.335+0000 7f70cd78c640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f70b406ae40 0x7f70b406d300 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.335+0000 7f70cd78c640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c815cc90 0x7f70c806bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.335+0000 7f70cd78c640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70c8071320 0x7f70c806b9d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.335+0000 7f70cd78c640 1 --2- 172.21.15.16:0/1857335989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70c8070cf0 0x7f70c814b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.335+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 >> 172.21.15.16:0/1857335989 conn(0x7f70c80b04a0 msgr2=0x7f70c80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.335+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 shutdown_connections 2023-11-13T10:56:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.335+0000 7f70cd78c640 1 -- 172.21.15.16:0/1857335989 wait complete. 2023-11-13T10:56:50.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:50.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:50.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.585+0000 7f53eb59e640 1 Processor -- start 2023-11-13T10:56:50.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.586+0000 7f53eb59e640 1 -- start start 2023-11-13T10:56:50.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.587+0000 7f53eb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec15ce70 0x7f53ec0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.587+0000 7f53eb59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53ec0cdbb0 0x7f53ec0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.587+0000 7f53eb59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ec0cac40 con 0x7f53ec0d1850 2023-11-13T10:56:50.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.587+0000 7f53eb59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec0ce0e0 con 0x7f53ec0cdbb0 2023-11-13T10:56:50.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.587+0000 7f53eb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0ce3c0 con 0x7f53ec15ce70 2023-11-13T10:56:50.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.588+0000 7f53e9d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53ec0cdbb0 0x7f53ec0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.588+0000 7f53ea59c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec15ce70 0x7f53ec0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.588+0000 7f53e9d9b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53ec0cdbb0 0x7f53ec0d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:56798/0 (socket says 172.21.15.16:56798) 2023-11-13T10:56:50.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.588+0000 7f53e9d9b640 1 -- 172.21.15.16:0/876519729 learned_addr learned my addr 172.21.15.16:0/876519729 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:50.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.588+0000 7f53ead9d640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f53ec0d1850 0x7f53ec0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:41034/0 (socket says 172.21.15.16:41034) 2023-11-13T10:56:50.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.589+0000 7f53e9d9b640 1 -- 172.21.15.16:0/876519729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec15ce70 msgr2=0x7f53ec0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:50.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.589+0000 7f53e9d9b640 1 --2- 172.21.15.16:0/876519729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec15ce70 0x7f53ec0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.589+0000 7f53e9d9b640 1 -- 172.21.15.16:0/876519729 >> v1:172.21.15.184:6789/0 conn(0x7f53ec0d1850 legacy=0x7f53ec0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.589+0000 7f53e9d9b640 1 -- 172.21.15.16:0/876519729 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0ce6a0 con 0x7f53ec0cdbb0 2023-11-13T10:56:50.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.589+0000 7f53e959a640 1 -- 172.21.15.16:0/876519729 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258942487 0 0) 0x7f53ec0cac40 con 0x7f53ec0d1850 2023-11-13T10:56:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.591+0000 7f53e9d9b640 1 --2- 172.21.15.16:0/876519729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 0x7f53ec0d27c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009e80 tx=0x7f53d402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e1e488d33e3cf23 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.591+0000 7f53e959a640 1 -- 172.21.15.16:0/876519729 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d402caf0 con 0x7f53ec0cdbb0 2023-11-13T10:56:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.591+0000 7f53e959a640 1 -- 172.21.15.16:0/876519729 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d402cc50 con 0x7f53ec0cdbb0 2023-11-13T10:56:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.591+0000 7f53e959a640 1 -- 172.21.15.16:0/876519729 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d40343f0 con 0x7f53ec0cdbb0 2023-11-13T10:56:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.592+0000 7f53eb59e640 1 -- 172.21.15.16:0/876519729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 msgr2=0x7f53ec0d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.592+0000 7f53eb59e640 1 --2- 172.21.15.16:0/876519729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 0x7f53ec0d27c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009e80 tx=0x7f53d402bf50 comp rx=0 tx=0).stop 2023-11-13T10:56:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.592+0000 7f53eb59e640 1 -- 172.21.15.16:0/876519729 shutdown_connections 2023-11-13T10:56:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.592+0000 7f53eb59e640 1 --2- 172.21.15.16:0/876519729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 0x7f53ec0d27c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.592+0000 7f53eb59e640 1 --2- 172.21.15.16:0/876519729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec15ce70 0x7f53ec0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.592+0000 7f53eb59e640 1 -- 172.21.15.16:0/876519729 >> 172.21.15.16:0/876519729 conn(0x7f53ec08c580 msgr2=0x7f53ec09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.592+0000 7f53eb59e640 1 -- 172.21.15.16:0/876519729 shutdown_connections 2023-11-13T10:56:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.592+0000 7f53eb59e640 1 -- 172.21.15.16:0/876519729 wait complete. 2023-11-13T10:56:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.594+0000 7f53eb59e640 1 Processor -- start 2023-11-13T10:56:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.594+0000 7f53eb59e640 1 -- start start 2023-11-13T10:56:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.595+0000 7f53eb59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 0x7f53ec151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.595+0000 7f53eb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec0d1850 0x7f53ec0c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.595+0000 7f53eb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ec15ce70 0x7f53ec0c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.595+0000 7f53eb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec15e5f0 con 0x7f53ec15ce70 2023-11-13T10:56:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.595+0000 7f53eb59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1f2290 con 0x7f53ec0cdbb0 2023-11-13T10:56:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.595+0000 7f53eb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1f3260 con 0x7f53ec0d1850 2023-11-13T10:56:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.595+0000 7f53ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ec15ce70 0x7f53ec0c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.595+0000 7f53ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ec15ce70 0x7f53ec0c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46774/0 (socket says 172.21.15.16:46774) 2023-11-13T10:56:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.595+0000 7f53ead9d640 1 -- 172.21.15.16:0/3570994081 learned_addr learned my addr 172.21.15.16:0/3570994081 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.596+0000 7f53ea59c640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 0x7f53ec151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.596+0000 7f53e9d9b640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec0d1850 0x7f53ec0c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.596+0000 7f53ead9d640 1 -- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec0d1850 msgr2=0x7f53ec0c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.596+0000 7f53ead9d640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec0d1850 0x7f53ec0c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.596+0000 7f53ead9d640 1 -- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 msgr2=0x7f53ec151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.596+0000 7f53ead9d640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 0x7f53ec151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.596+0000 7f53ead9d640 1 -- 172.21.15.16:0/3570994081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4009b30 con 0x7f53ec15ce70 2023-11-13T10:56:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.596+0000 7f53ea59c640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 0x7f53ec151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:56:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.596+0000 7f53ead9d640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ec15ce70 0x7f53ec0c67d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c430 tx=0x7f53e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.597+0000 7f53e2ffd640 1 -- 172.21.15.16:0/3570994081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40074e0 con 0x7f53ec15ce70 2023-11-13T10:56:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.597+0000 7f53e2ffd640 1 -- 172.21.15.16:0/3570994081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e4007660 con 0x7f53ec15ce70 2023-11-13T10:56:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.597+0000 7f53e2ffd640 1 -- 172.21.15.16:0/3570994081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4004a20 con 0x7f53ec15ce70 2023-11-13T10:56:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.597+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec15e840 con 0x7f53ec15ce70 2023-11-13T10:56:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.597+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec15ee00 con 0x7f53ec15ce70 2023-11-13T10:56:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.598+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec0675b0 con 0x7f53ec15ce70 2023-11-13T10:56:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.599+0000 7f53e2ffd640 1 -- 172.21.15.16:0/3570994081 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f53e4004b80 con 0x7f53ec15ce70 2023-11-13T10:56:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.600+0000 7f53e2ffd640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53cc06ae40 0x7f53cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.600+0000 7f53e2ffd640 1 -- 172.21.15.16:0/3570994081 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f53e40898d0 con 0x7f53ec15ce70 2023-11-13T10:56:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.601+0000 7f53ea59c640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53cc06ae40 0x7f53cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.602+0000 7f53ea59c640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53cc06ae40 0x7f53cc06d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0c84c0 tx=0x7f53dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.604+0000 7f53e2ffd640 1 -- 172.21.15.16:0/3570994081 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53e4054660 con 0x7f53ec15ce70 2023-11-13T10:56:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.798+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53ec04e4d0 con 0x7f53ec15ce70 2023-11-13T10:56:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.800+0000 7f53e2ffd640 1 -- 172.21.15.16:0/3570994081 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53e4057c80 con 0x7f53ec15ce70 2023-11-13T10:56:50.807 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53cc06ae40 msgr2=0x7f53cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53cc06ae40 0x7f53cc06d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0c84c0 tx=0x7f53dc008040 comp rx=0 tx=0).stop 2023-11-13T10:56:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ec15ce70 msgr2=0x7f53ec0c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ec15ce70 0x7f53ec0c67d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c430 tx=0x7f53e400c8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 shutdown_connections 2023-11-13T10:56:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f53cc06ae40 0x7f53cc06d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ec15ce70 0x7f53ec0c67d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53ec0d1850 0x7f53ec0c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 --2- 172.21.15.16:0/3570994081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0cdbb0 0x7f53ec151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.802+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 >> 172.21.15.16:0/3570994081 conn(0x7f53ec08c580 msgr2=0x7f53ec0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.803+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 shutdown_connections 2023-11-13T10:56:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:50.803+0000 7f53eb59e640 1 -- 172.21.15.16:0/3570994081 wait complete. 2023-11-13T10:56:50.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.056+0000 7ff5c0c52640 1 Processor -- start 2023-11-13T10:56:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.057+0000 7ff5c0c52640 1 -- start start 2023-11-13T10:56:51.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.058+0000 7ff5c0c52640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff5bc144570 0x7ff5bc15f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.058+0000 7ff5c0c52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc0ce550 0x7ff5bc0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.058+0000 7ff5c0c52640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5bc0ceef0 con 0x7ff5bc06bb30 2023-11-13T10:56:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.058+0000 7ff5c0c52640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc144950 con 0x7ff5bc144570 2023-11-13T10:56:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.058+0000 7ff5c0c52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc15f740 con 0x7ff5bc0ce550 2023-11-13T10:56:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.059+0000 7ff5bad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff5bc06bb30 0x7ff5bc06bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35468/0 (socket says 172.21.15.16:35468) 2023-11-13T10:56:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.059+0000 7ff5bad9d640 1 -- 172.21.15.16:0/3090885136 learned_addr learned my addr 172.21.15.16:0/3090885136 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.059+0000 7ff5ba59c640 1 --2- 172.21.15.16:0/3090885136 >> v2:172.21.15.16:3300/0 conn(0x7ff5bc144570 0x7ff5bc15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.059+0000 7ff5b9d9b640 1 --2- 172.21.15.16:0/3090885136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc0ce550 0x7ff5bc0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.060+0000 7ff5ba59c640 1 -- 172.21.15.16:0/3090885136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc0ce550 msgr2=0x7ff5bc0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.060+0000 7ff5ba59c640 1 --2- 172.21.15.16:0/3090885136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc0ce550 0x7ff5bc0ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.060+0000 7ff5ba59c640 1 -- 172.21.15.16:0/3090885136 >> v1:172.21.15.16:6789/0 conn(0x7ff5bc06bb30 legacy=0x7ff5bc06bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.060+0000 7ff5ba59c640 1 -- 172.21.15.16:0/3090885136 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc0ce300 con 0x7ff5bc144570 2023-11-13T10:56:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.060+0000 7ff5b959a640 1 -- 172.21.15.16:0/3090885136 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808697618 0 0) 0x7ff5bc0ceef0 con 0x7ff5bc06bb30 2023-11-13T10:56:51.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.061+0000 7ff5ba59c640 1 --2- 172.21.15.16:0/3090885136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 0x7ff5bc15f200 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0097a0 tx=0x7ff5ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc19b7a2886b1133 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.061+0000 7ff5b959a640 1 -- 172.21.15.16:0/3090885136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac031830 con 0x7ff5bc144570 2023-11-13T10:56:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.061+0000 7ff5b959a640 1 -- 172.21.15.16:0/3090885136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac031990 con 0x7ff5bc144570 2023-11-13T10:56:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.061+0000 7ff5b959a640 1 -- 172.21.15.16:0/3090885136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac031d20 con 0x7ff5bc144570 2023-11-13T10:56:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.062+0000 7ff5c0c52640 1 -- 172.21.15.16:0/3090885136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 msgr2=0x7ff5bc15f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.062+0000 7ff5c0c52640 1 --2- 172.21.15.16:0/3090885136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 0x7ff5bc15f200 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0097a0 tx=0x7ff5ac02b8d0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.062+0000 7ff5c0c52640 1 -- 172.21.15.16:0/3090885136 shutdown_connections 2023-11-13T10:56:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.062+0000 7ff5c0c52640 1 --2- 172.21.15.16:0/3090885136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc0ce550 0x7ff5bc0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.062+0000 7ff5c0c52640 1 --2- 172.21.15.16:0/3090885136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 0x7ff5bc15f200 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.062+0000 7ff5c0c52640 1 -- 172.21.15.16:0/3090885136 >> 172.21.15.16:0/3090885136 conn(0x7ff5bc06a190 msgr2=0x7ff5bc06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.062+0000 7ff5c0c52640 1 -- 172.21.15.16:0/3090885136 shutdown_connections 2023-11-13T10:56:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.062+0000 7ff5c0c52640 1 -- 172.21.15.16:0/3090885136 wait complete. 2023-11-13T10:56:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.064+0000 7ff5c0c52640 1 Processor -- start 2023-11-13T10:56:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.064+0000 7ff5c0c52640 1 -- start start 2023-11-13T10:56:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.065+0000 7ff5c0c52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc06bb30 0x7ff5bc15a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.065+0000 7ff5c0c52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5bc0ce550 0x7ff5bc139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.065+0000 7ff5c0c52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 0x7ff5bc070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.065+0000 7ff5c0c52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc15bc20 con 0x7ff5bc144570 2023-11-13T10:56:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.065+0000 7ff5c0c52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc15ad00 con 0x7ff5bc0ce550 2023-11-13T10:56:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.065+0000 7ff5c0c52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc15afa0 con 0x7ff5bc06bb30 2023-11-13T10:56:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.065+0000 7ff5bad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 0x7ff5bc070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.065+0000 7ff5bad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 0x7ff5bc070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46786/0 (socket says 172.21.15.16:46786) 2023-11-13T10:56:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.065+0000 7ff5bad9d640 1 -- 172.21.15.16:0/1583521331 learned_addr learned my addr 172.21.15.16:0/1583521331 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.066+0000 7ff5ba59c640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc06bb30 0x7ff5bc15a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.066+0000 7ff5b9d9b640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5bc0ce550 0x7ff5bc139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.066+0000 7ff5bad9d640 1 -- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc06bb30 msgr2=0x7ff5bc15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.066+0000 7ff5bad9d640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc06bb30 0x7ff5bc15a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.066+0000 7ff5bad9d640 1 -- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5bc0ce550 msgr2=0x7ff5bc139df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.066+0000 7ff5bad9d640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5bc0ce550 0x7ff5bc139df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.066+0000 7ff5bad9d640 1 -- 172.21.15.16:0/1583521331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac009400 con 0x7ff5bc144570 2023-11-13T10:56:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.066+0000 7ff5bad9d640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 0x7ff5bc070b20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400c450 tx=0x7ff5b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.067+0000 7ff5b2ffd640 1 -- 172.21.15.16:0/1583521331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40074e0 con 0x7ff5bc144570 2023-11-13T10:56:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.067+0000 7ff5b2ffd640 1 -- 172.21.15.16:0/1583521331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b4007660 con 0x7ff5bc144570 2023-11-13T10:56:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.067+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc15b280 con 0x7ff5bc144570 2023-11-13T10:56:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.067+0000 7ff5b2ffd640 1 -- 172.21.15.16:0/1583521331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4004a20 con 0x7ff5bc144570 2023-11-13T10:56:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.067+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc15b840 con 0x7ff5bc144570 2023-11-13T10:56:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.068+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc0672a0 con 0x7ff5bc144570 2023-11-13T10:56:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.069+0000 7ff5b2ffd640 1 -- 172.21.15.16:0/1583521331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5b4004b80 con 0x7ff5bc144570 2023-11-13T10:56:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.070+0000 7ff5b2ffd640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff59c06ae40 0x7ff59c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.070+0000 7ff5b2ffd640 1 -- 172.21.15.16:0/1583521331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7ff5b4088a40 con 0x7ff5bc144570 2023-11-13T10:56:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.071+0000 7ff5ba59c640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff59c06ae40 0x7ff59c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.072+0000 7ff5ba59c640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff59c06ae40 0x7ff59c06d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac036000 tx=0x7ff5ac008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.075+0000 7ff5b2ffd640 1 -- 172.21.15.16:0/1583521331 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5b40537a0 con 0x7ff5bc144570 2023-11-13T10:56:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.269+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5bc072520 con 0x7ff5bc144570 2023-11-13T10:56:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.270+0000 7ff5b2ffd640 1 -- 172.21.15.16:0/1583521331 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5b4018020 con 0x7ff5bc144570 2023-11-13T10:56:51.275 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.272+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff59c06ae40 msgr2=0x7ff59c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.272+0000 7ff5c0c52640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff59c06ae40 0x7ff59c06d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac036000 tx=0x7ff5ac008030 comp rx=0 tx=0).stop 2023-11-13T10:56:51.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.272+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 msgr2=0x7ff5bc070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.272+0000 7ff5c0c52640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 0x7ff5bc070b20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400c450 tx=0x7ff5b400c910 comp rx=0 tx=0).stop 2023-11-13T10:56:51.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.273+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 shutdown_connections 2023-11-13T10:56:51.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.273+0000 7ff5c0c52640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff59c06ae40 0x7ff59c06d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.273+0000 7ff5c0c52640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5bc144570 0x7ff5bc070b20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.273+0000 7ff5c0c52640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5bc0ce550 0x7ff5bc139df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.273+0000 7ff5c0c52640 1 --2- 172.21.15.16:0/1583521331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5bc06bb30 0x7ff5bc15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.273+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 >> 172.21.15.16:0/1583521331 conn(0x7ff5bc06a190 msgr2=0x7ff5bc15cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.273+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 shutdown_connections 2023-11-13T10:56:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.273+0000 7ff5c0c52640 1 -- 172.21.15.16:0/1583521331 wait complete. 2023-11-13T10:56:51.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:51.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.522+0000 7f0a35578640 1 Processor -- start 2023-11-13T10:56:51.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.522+0000 7f0a35578640 1 -- start start 2023-11-13T10:56:51.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.524+0000 7f0a35578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a3015d280 0x7f0a300cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.524+0000 7f0a35578640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0a300b39e0 0x7f0a300cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.524+0000 7f0a35578640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a300ce990 con 0x7f0a300d37d0 2023-11-13T10:56:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.524+0000 7f0a35578640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a300cf510 con 0x7f0a300b39e0 2023-11-13T10:56:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.524+0000 7f0a35578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300cec70 con 0x7f0a3015d280 2023-11-13T10:56:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.524+0000 7f0a2e7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0a300d37d0 0x7f0a300d0bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35476/0 (socket says 172.21.15.16:35476) 2023-11-13T10:56:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.524+0000 7f0a2e7fc640 1 -- 172.21.15.16:0/2269274439 learned_addr learned my addr 172.21.15.16:0/2269274439 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2effd640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a3015d280 0x7f0a300cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2f7fe640 1 --2- 172.21.15.16:0/2269274439 >> v2:172.21.15.184:3300/0 conn(0x7f0a300b39e0 0x7f0a300cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776284775 0 0) 0x7f0a300ce990 con 0x7f0a300d37d0 2023-11-13T10:56:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a1c003520 con 0x7f0a300d37d0 2023-11-13T10:56:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1068821809 0 0) 0x7f0a1c003520 con 0x7f0a300d37d0 2023-11-13T10:56:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a3015d280 msgr2=0x7f0a300cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a3015d280 0x7f0a300cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 >> v2:172.21.15.184:3300/0 conn(0x7f0a300b39e0 msgr2=0x7f0a300cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 --2- 172.21.15.16:0/2269274439 >> v2:172.21.15.184:3300/0 conn(0x7f0a300b39e0 0x7f0a300cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a300cef50 con 0x7f0a300d37d0 2023-11-13T10:56:51.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f0a18001670 con 0x7f0a300d37d0 2023-11-13T10:56:51.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 >> v1:172.21.15.16:6789/0 conn(0x7f0a300d37d0 legacy=0x7f0a300d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a1c003970 0x7f0a1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a3015d280 0x7f0a1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a1c004d00 0x7f0a1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300cf510 con 0x7f0a1c004d00 2023-11-13T10:56:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300cec70 con 0x7f0a1c003970 2023-11-13T10:56:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.525+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c00b980 con 0x7f0a3015d280 2023-11-13T10:56:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2f7fe640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a1c004d00 0x7f0a1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2e7fc640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a3015d280 0x7f0a1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2effd640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a1c003970 0x7f0a1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2f7fe640 1 -- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a3015d280 msgr2=0x7f0a1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2f7fe640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a3015d280 0x7f0a1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2f7fe640 1 -- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a1c003970 msgr2=0x7f0a1c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2f7fe640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a1c003970 0x7f0a1c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2f7fe640 1 -- 172.21.15.16:0/2269274439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c003520 con 0x7f0a1c004d00 2023-11-13T10:56:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2e7fc640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a3015d280 0x7f0a1c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.526+0000 7f0a2effd640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a1c003970 0x7f0a1c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.527+0000 7f0a2f7fe640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a1c004d00 0x7f0a1c005180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800ba40 tx=0x7f0a1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d71b50cb2c2ad3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.527+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28032660 con 0x7f0a1c004d00 2023-11-13T10:56:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.527+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a280327c0 con 0x7f0a1c004d00 2023-11-13T10:56:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.527+0000 7f0a2dffb640 1 -- 172.21.15.16:0/2269274439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28032b50 con 0x7f0a1c004d00 2023-11-13T10:56:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.528+0000 7f0a35578640 1 -- 172.21.15.16:0/2269274439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a1c004d00 msgr2=0x7f0a1c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.528+0000 7f0a35578640 1 --2- 172.21.15.16:0/2269274439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a1c004d00 0x7f0a1c005180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800ba40 tx=0x7f0a1c0057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.528+0000 7f0a2f7fe640 1 -- 172.21.15.16:0/2269274439 reap_dead start 2023-11-13T10:56:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.528+0000 7f0a35578640 1 -- 172.21.15.16:0/2269274439 shutdown_connections 2023-11-13T10:56:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.528+0000 7f0a35578640 1 -- 172.21.15.16:0/2269274439 >> 172.21.15.16:0/2269274439 conn(0x7f0a300d2cb0 msgr2=0x7f0a300d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.528+0000 7f0a35578640 1 -- 172.21.15.16:0/2269274439 shutdown_connections 2023-11-13T10:56:51.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.528+0000 7f0a35578640 1 -- 172.21.15.16:0/2269274439 wait complete. 2023-11-13T10:56:51.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.530+0000 7f0a35578640 1 Processor -- start 2023-11-13T10:56:51.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.530+0000 7f0a35578640 1 -- start start 2023-11-13T10:56:51.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.531+0000 7f0a35578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a1c004d00 0x7f0a30159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.531+0000 7f0a35578640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a3009ca60 0x7f0a3015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.531+0000 7f0a35578640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a3009e280 0x7f0a3009e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.531+0000 7f0a35578640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3009dbe0 con 0x7f0a3009ca60 2023-11-13T10:56:51.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.531+0000 7f0a35578640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3009ec70 con 0x7f0a3009e280 2023-11-13T10:56:51.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.531+0000 7f0a35578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3015a080 con 0x7f0a1c004d00 2023-11-13T10:56:51.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a3009ca60 0x7f0a3015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a3009ca60 0x7f0a3015a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46812/0 (socket says 172.21.15.16:46812) 2023-11-13T10:56:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2e7fc640 1 -- 172.21.15.16:0/157829438 learned_addr learned my addr 172.21.15.16:0/157829438 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2effd640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a1c004d00 0x7f0a30159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2f7fe640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a3009e280 0x7f0a3009e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2e7fc640 1 -- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a1c004d00 msgr2=0x7f0a30159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2e7fc640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a1c004d00 0x7f0a30159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2e7fc640 1 -- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a3009e280 msgr2=0x7f0a3009e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2e7fc640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a3009e280 0x7f0a3009e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2e7fc640 1 -- 172.21.15.16:0/157829438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3009cf60 con 0x7f0a3009ca60 2023-11-13T10:56:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2effd640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a1c004d00 0x7f0a30159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:51.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.532+0000 7f0a2e7fc640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a3009ca60 0x7f0a3015a3a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d8e0 tx=0x7f0a1800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:51.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.533+0000 7f0a277fe640 1 -- 172.21.15.16:0/157829438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a180074e0 con 0x7f0a3009ca60 2023-11-13T10:56:51.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.533+0000 7f0a277fe640 1 -- 172.21.15.16:0/157829438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a18007640 con 0x7f0a3009ca60 2023-11-13T10:56:51.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.533+0000 7f0a277fe640 1 -- 172.21.15.16:0/157829438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a180049e0 con 0x7f0a3009ca60 2023-11-13T10:56:51.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.533+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3009d240 con 0x7f0a3009ca60 2023-11-13T10:56:51.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.533+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3009d800 con 0x7f0a3009ca60 2023-11-13T10:56:51.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.534+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3009dd20 con 0x7f0a3009ca60 2023-11-13T10:56:51.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.535+0000 7f0a277fe640 1 -- 172.21.15.16:0/157829438 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a18004b40 con 0x7f0a3009ca60 2023-11-13T10:56:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.536+0000 7f0a277fe640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0a1006ae40 0x7f0a1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.536+0000 7f0a277fe640 1 -- 172.21.15.16:0/157829438 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f0a18089540 con 0x7f0a3009ca60 2023-11-13T10:56:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.537+0000 7f0a2effd640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0a1006ae40 0x7f0a1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.538+0000 7f0a2effd640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0a1006ae40 0x7f0a1006d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0a20007520 tx=0x7f0a2000d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.540+0000 7f0a277fe640 1 -- 172.21.15.16:0/157829438 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a3009dd20 con 0x7f0a3009ca60 2023-11-13T10:56:51.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.733+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a3009dd20 con 0x7f0a3009ca60 2023-11-13T10:56:51.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.734+0000 7f0a277fe640 1 -- 172.21.15.16:0/157829438 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a3009dd20 con 0x7f0a3009ca60 2023-11-13T10:56:51.739 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.736+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0a1006ae40 msgr2=0x7f0a1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.736+0000 7f0a35578640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0a1006ae40 0x7f0a1006d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0a20007520 tx=0x7f0a2000d040 comp rx=0 tx=0).stop 2023-11-13T10:56:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.736+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a3009ca60 msgr2=0x7f0a3015a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.736+0000 7f0a35578640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a3009ca60 0x7f0a3015a3a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d8e0 tx=0x7f0a1800dda0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.737+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 shutdown_connections 2023-11-13T10:56:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.737+0000 7f0a35578640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0a1006ae40 0x7f0a1006d300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.737+0000 7f0a35578640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a3009e280 0x7f0a3009e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.737+0000 7f0a35578640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a3009ca60 0x7f0a3015a3a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.737+0000 7f0a35578640 1 --2- 172.21.15.16:0/157829438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a1c004d00 0x7f0a30159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.737+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 >> 172.21.15.16:0/157829438 conn(0x7f0a300d2cb0 msgr2=0x7f0a300c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.737+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 shutdown_connections 2023-11-13T10:56:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.737+0000 7f0a35578640 1 -- 172.21.15.16:0/157829438 wait complete. 2023-11-13T10:56:51.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:51.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.984+0000 7fdbf8d05640 1 Processor -- start 2023-11-13T10:56:51.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.984+0000 7fdbf8d05640 1 -- start start 2023-11-13T10:56:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.986+0000 7fdbf8d05640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdbf40ce5b0 0x7fdbf415a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.986+0000 7fdbf8d05640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbf40cf0e0 0x7fdbf415a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.986+0000 7fdbf8d05640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf415d8d0 con 0x7fdbf40ce1d0 2023-11-13T10:56:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.986+0000 7fdbf8d05640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf415af00 con 0x7fdbf40cf0e0 2023-11-13T10:56:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.986+0000 7fdbf8d05640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf415db90 con 0x7fdbf40ce5b0 2023-11-13T10:56:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.987+0000 7fdbf259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdbf40ce1d0 0x7fdbf40ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35478/0 (socket says 172.21.15.16:35478) 2023-11-13T10:56:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.987+0000 7fdbf259c640 1 -- 172.21.15.16:0/994557971 learned_addr learned my addr 172.21.15.16:0/994557971 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.987+0000 7fdbf2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbf40cf0e0 0x7fdbf415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.987+0000 7fdbf1d9b640 1 --2- 172.21.15.16:0/994557971 >> v2:172.21.15.184:3300/0 conn(0x7fdbf40ce5b0 0x7fdbf415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.987+0000 7fdbf2d9d640 1 -- 172.21.15.16:0/994557971 >> v2:172.21.15.184:3300/0 conn(0x7fdbf40ce5b0 msgr2=0x7fdbf415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.987+0000 7fdbf2d9d640 1 --2- 172.21.15.16:0/994557971 >> v2:172.21.15.184:3300/0 conn(0x7fdbf40ce5b0 0x7fdbf415a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.987+0000 7fdbf2d9d640 1 -- 172.21.15.16:0/994557971 >> v1:172.21.15.16:6789/0 conn(0x7fdbf40ce1d0 legacy=0x7fdbf40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.987+0000 7fdbf2d9d640 1 -- 172.21.15.16:0/994557971 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf415de70 con 0x7fdbf40cf0e0 2023-11-13T10:56:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.987+0000 7fdbf159a640 1 -- 172.21.15.16:0/994557971 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6921087 0 0) 0x7fdbf415d8d0 con 0x7fdbf40ce1d0 2023-11-13T10:56:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.988+0000 7fdbf2d9d640 1 --2- 172.21.15.16:0/994557971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 0x7fdbf415a670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40b36e0 tx=0x7fdbec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b842c31ded278ade server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.989+0000 7fdbf159a640 1 -- 172.21.15.16:0/994557971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec02e070 con 0x7fdbf40cf0e0 2023-11-13T10:56:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.989+0000 7fdbf159a640 1 -- 172.21.15.16:0/994557971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec0315a0 con 0x7fdbf40cf0e0 2023-11-13T10:56:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.989+0000 7fdbf159a640 1 -- 172.21.15.16:0/994557971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec031930 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.989+0000 7fdbf8d05640 1 -- 172.21.15.16:0/994557971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 msgr2=0x7fdbf415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.989+0000 7fdbf8d05640 1 --2- 172.21.15.16:0/994557971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 0x7fdbf415a670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40b36e0 tx=0x7fdbec02b840 comp rx=0 tx=0).stop 2023-11-13T10:56:52.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.989+0000 7fdbf8d05640 1 -- 172.21.15.16:0/994557971 shutdown_connections 2023-11-13T10:56:52.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.989+0000 7fdbf8d05640 1 --2- 172.21.15.16:0/994557971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 0x7fdbf415a670 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.989+0000 7fdbf8d05640 1 --2- 172.21.15.16:0/994557971 >> v2:172.21.15.184:3300/0 conn(0x7fdbf40ce5b0 0x7fdbf415a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.989+0000 7fdbf8d05640 1 -- 172.21.15.16:0/994557971 >> 172.21.15.16:0/994557971 conn(0x7fdbf406a190 msgr2=0x7fdbf406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.990+0000 7fdbf8d05640 1 -- 172.21.15.16:0/994557971 shutdown_connections 2023-11-13T10:56:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.990+0000 7fdbf8d05640 1 -- 172.21.15.16:0/994557971 wait complete. 2023-11-13T10:56:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.991+0000 7fdbf8d05640 1 Processor -- start 2023-11-13T10:56:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.992+0000 7fdbf8d05640 1 -- start start 2023-11-13T10:56:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf8d05640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf40ce1d0 0x7fdbf4147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf8d05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 0x7fdbf414b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf8d05640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbf41480d0 0x7fdbf414b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf8d05640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf414f0e0 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf8d05640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf414eb20 con 0x7fdbf41480d0 2023-11-13T10:56:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf8d05640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf414ee00 con 0x7fdbf40ce1d0 2023-11-13T10:56:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 0x7fdbf414b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 0x7fdbf414b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46826/0 (socket says 172.21.15.16:46826) 2023-11-13T10:56:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf1d9b640 1 -- 172.21.15.16:0/1160970467 learned_addr learned my addr 172.21.15.16:0/1160970467 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf1d9b640 1 -- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf40ce1d0 msgr2=0x7fdbf4147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf1d9b640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf40ce1d0 0x7fdbf4147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf1d9b640 1 -- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbf41480d0 msgr2=0x7fdbf414b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf1d9b640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbf41480d0 0x7fdbf414b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.993+0000 7fdbf1d9b640 1 -- 172.21.15.16:0/1160970467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec009400 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.994+0000 7fdbf1d9b640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 0x7fdbf414b0b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800e840 tx=0x7fdbe800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.994+0000 7fdbe2ffd640 1 -- 172.21.15.16:0/1160970467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80094f0 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.994+0000 7fdbe2ffd640 1 -- 172.21.15.16:0/1160970467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe800c940 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.994+0000 7fdbe2ffd640 1 -- 172.21.15.16:0/1160970467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800ccd0 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.994+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf414ddf0 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.994+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf414e3b0 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.995+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf4072520 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.996+0000 7fdbe2ffd640 1 -- 172.21.15.16:0/1160970467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdbe8013470 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.997+0000 7fdbe2ffd640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdbd406ae40 0x7fdbd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.997+0000 7fdbe2ffd640 1 -- 172.21.15.16:0/1160970467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fdbe8089a30 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.998+0000 7fdbf259c640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdbd406ae40 0x7fdbd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:51.999+0000 7fdbf259c640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdbd406ae40 0x7fdbd406d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4007fd0 tx=0x7fdbe4007ce0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.001+0000 7fdbe2ffd640 1 -- 172.21.15.16:0/1160970467 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdbe8054790 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.195+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdbf404e5b0 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.196+0000 7fdbe2ffd640 1 -- 172.21.15.16:0/1160970467 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdbe8057db0 con 0x7fdbf40cf0e0 2023-11-13T10:56:52.203 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdbd406ae40 msgr2=0x7fdbd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdbd406ae40 0x7fdbd406d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4007fd0 tx=0x7fdbe4007ce0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 msgr2=0x7fdbf414b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 0x7fdbf414b0b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800e840 tx=0x7fdbe800ed00 comp rx=0 tx=0).stop 2023-11-13T10:56:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 shutdown_connections 2023-11-13T10:56:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdbd406ae40 0x7fdbd406d300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbf41480d0 0x7fdbf414b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbf40cf0e0 0x7fdbf414b0b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 --2- 172.21.15.16:0/1160970467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbf40ce1d0 0x7fdbf4147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 >> 172.21.15.16:0/1160970467 conn(0x7fdbf406a190 msgr2=0x7fdbf40cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 shutdown_connections 2023-11-13T10:56:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.199+0000 7fdbf8d05640 1 -- 172.21.15.16:0/1160970467 wait complete. 2023-11-13T10:56:52.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:56:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.457+0000 7fc44b202640 1 Processor -- start 2023-11-13T10:56:52.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.458+0000 7fc44b202640 1 -- start start 2023-11-13T10:56:52.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.459+0000 7fc44b202640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc44406f970 0x7fc444162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.459+0000 7fc44b202640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc44406a190 0x7fc44406a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.459+0000 7fc44b202640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4440b1840 con 0x7fc444161060 2023-11-13T10:56:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.459+0000 7fc44b202640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44406ab60 con 0x7fc44406a190 2023-11-13T10:56:52.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.459+0000 7fc44b202640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc444071600 con 0x7fc44406f970 2023-11-13T10:56:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.460+0000 7fc44aa01640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc44406a190 0x7fc44406a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.460+0000 7fc44aa01640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc44406a190 0x7fc44406a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46840/0 (socket says 172.21.15.16:46840) 2023-11-13T10:56:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.460+0000 7fc44aa01640 1 -- 172.21.15.16:0/1098405657 learned_addr learned my addr 172.21.15.16:0/1098405657 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.460+0000 7fc4499ff640 1 --2- 172.21.15.16:0/1098405657 >> v2:172.21.15.184:3300/0 conn(0x7fc44406f970 0x7fc444162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.460+0000 7fc44aa01640 1 -- 172.21.15.16:0/1098405657 >> v2:172.21.15.184:3300/0 conn(0x7fc44406f970 msgr2=0x7fc444162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.460+0000 7fc44aa01640 1 --2- 172.21.15.16:0/1098405657 >> v2:172.21.15.184:3300/0 conn(0x7fc44406f970 0x7fc444162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.460+0000 7fc44aa01640 1 -- 172.21.15.16:0/1098405657 >> v1:172.21.15.184:6789/0 conn(0x7fc444161060 legacy=0x7fc4441621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.460+0000 7fc44aa01640 1 -- 172.21.15.16:0/1098405657 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4440718e0 con 0x7fc44406a190 2023-11-13T10:56:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.460+0000 7fc4499ff640 1 --2- 172.21.15.16:0/1098405657 >> v2:172.21.15.184:3300/0 conn(0x7fc44406f970 0x7fc444162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.461+0000 7fc44aa01640 1 --2- 172.21.15.16:0/1098405657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc44406a620 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc4440ae3e0 tx=0x7fc43c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3f4d13bac05004 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:52.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.462+0000 7fc4491fe640 1 -- 172.21.15.16:0/1098405657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c02ca80 con 0x7fc44406a190 2023-11-13T10:56:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.462+0000 7fc4491fe640 1 -- 172.21.15.16:0/1098405657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c02cbe0 con 0x7fc44406a190 2023-11-13T10:56:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.462+0000 7fc4491fe640 1 -- 172.21.15.16:0/1098405657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0343f0 con 0x7fc44406a190 2023-11-13T10:56:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.462+0000 7fc44b202640 1 -- 172.21.15.16:0/1098405657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 msgr2=0x7fc44406a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.462+0000 7fc44b202640 1 --2- 172.21.15.16:0/1098405657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc44406a620 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc4440ae3e0 tx=0x7fc43c02bee0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.462+0000 7fc44b202640 1 -- 172.21.15.16:0/1098405657 shutdown_connections 2023-11-13T10:56:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.462+0000 7fc44b202640 1 --2- 172.21.15.16:0/1098405657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc44406a620 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.462+0000 7fc44b202640 1 --2- 172.21.15.16:0/1098405657 >> v2:172.21.15.184:3300/0 conn(0x7fc44406f970 0x7fc444162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.462+0000 7fc44b202640 1 -- 172.21.15.16:0/1098405657 >> 172.21.15.16:0/1098405657 conn(0x7fc4440b04a0 msgr2=0x7fc4440b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:52.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.463+0000 7fc44b202640 1 -- 172.21.15.16:0/1098405657 shutdown_connections 2023-11-13T10:56:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.463+0000 7fc44b202640 1 -- 172.21.15.16:0/1098405657 wait complete. 2023-11-13T10:56:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.464+0000 7fc44b202640 1 Processor -- start 2023-11-13T10:56:52.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.465+0000 7fc44b202640 1 -- start start 2023-11-13T10:56:52.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.465+0000 7fc44b202640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc444155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.465+0000 7fc44b202640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44406f970 0x7fc4440af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44b202640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc444161060 0x7fc4440b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44b202640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc444156280 con 0x7fc44406a190 2023-11-13T10:56:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44b202640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440afe50 con 0x7fc444161060 2023-11-13T10:56:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44b202640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc444156410 con 0x7fc44406f970 2023-11-13T10:56:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc444155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc444155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46842/0 (socket says 172.21.15.16:46842) 2023-11-13T10:56:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44a200640 1 -- 172.21.15.16:0/3966663252 learned_addr learned my addr 172.21.15.16:0/3966663252 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44a200640 1 -- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44406f970 msgr2=0x7fc4440af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44a200640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44406f970 0x7fc4440af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44a200640 1 -- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc444161060 msgr2=0x7fc4440b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44a200640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc444161060 0x7fc4440b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.466+0000 7fc44a200640 1 -- 172.21.15.16:0/3966663252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c009b30 con 0x7fc44406a190 2023-11-13T10:56:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.467+0000 7fc44a200640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc444155d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc44000b3f0 tx=0x7fc44000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.467+0000 7fc43affd640 1 -- 172.21.15.16:0/3966663252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440011830 con 0x7fc44406a190 2023-11-13T10:56:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.467+0000 7fc43affd640 1 -- 172.21.15.16:0/3966663252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc440011990 con 0x7fc44406a190 2023-11-13T10:56:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.467+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4441566f0 con 0x7fc44406a190 2023-11-13T10:56:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.467+0000 7fc43affd640 1 -- 172.21.15.16:0/3966663252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440011d40 con 0x7fc44406a190 2023-11-13T10:56:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.467+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc444156cb0 con 0x7fc44406a190 2023-11-13T10:56:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.468+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc444156ed0 con 0x7fc44406a190 2023-11-13T10:56:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.469+0000 7fc43affd640 1 -- 172.21.15.16:0/3966663252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc44001a400 con 0x7fc44406a190 2023-11-13T10:56:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.470+0000 7fc43affd640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc42806ae40 0x7fc42806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.470+0000 7fc43affd640 1 -- 172.21.15.16:0/3966663252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fc4400897a0 con 0x7fc44406a190 2023-11-13T10:56:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.471+0000 7fc4499ff640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc42806ae40 0x7fc42806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.472+0000 7fc4499ff640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc42806ae40 0x7fc42806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc434006fd0 tx=0x7fc434008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.474+0000 7fc43affd640 1 -- 172.21.15.16:0/3966663252 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc440054500 con 0x7fc44406a190 2023-11-13T10:56:52.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.682+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc44404e4d0 con 0x7fc44406a190 2023-11-13T10:56:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.683+0000 7fc43affd640 1 -- 172.21.15.16:0/3966663252 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc440057b20 con 0x7fc44406a190 2023-11-13T10:56:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:56:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc42806ae40 msgr2=0x7fc42806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc42806ae40 0x7fc42806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc434006fd0 tx=0x7fc434008040 comp rx=0 tx=0).stop 2023-11-13T10:56:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 msgr2=0x7fc444155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc444155d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc44000b3f0 tx=0x7fc44000b8b0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 shutdown_connections 2023-11-13T10:56:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc42806ae40 0x7fc42806d300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc444161060 0x7fc4440b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc44406f970 0x7fc4440af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 --2- 172.21.15.16:0/3966663252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc44406a190 0x7fc444155d40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 >> 172.21.15.16:0/3966663252 conn(0x7fc4440b04a0 msgr2=0x7fc44415cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 shutdown_connections 2023-11-13T10:56:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:52.686+0000 7fc44b202640 1 -- 172.21.15.16:0/3966663252 wait complete. 2023-11-13T10:56:52.712 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:56:52.713 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:52.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.033+0000 7f3804967640 1 Processor -- start 2023-11-13T10:56:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.033+0000 7f3804967640 1 -- start start 2023-11-13T10:56:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.035+0000 7f3804967640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f380015cfc0 0x7f38000cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.035+0000 7f3804967640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38000b30e0 0x7f38000d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.035+0000 7f3804967640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38000b3cf0 con 0x7f38000d2ae0 2023-11-13T10:56:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.035+0000 7f3804967640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38000d3480 con 0x7f38000b30e0 2023-11-13T10:56:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.035+0000 7f3804967640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38000b3730 con 0x7f380015cfc0 2023-11-13T10:56:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.035+0000 7f37f7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f38000d2ae0 0x7f38000b29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36426/0 (socket says 172.21.15.16:36426) 2023-11-13T10:56:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.035+0000 7f37fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38000b30e0 0x7f38000d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.035+0000 7f37f7fff640 1 -- 172.21.15.16:0/3847015309 learned_addr learned my addr 172.21.15.16:0/3847015309 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.036+0000 7f37ff7fe640 1 --2- 172.21.15.16:0/3847015309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f380015cfc0 0x7f38000cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.036+0000 7f37fffff640 1 -- 172.21.15.16:0/3847015309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f380015cfc0 msgr2=0x7f38000cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.036+0000 7f37fffff640 1 --2- 172.21.15.16:0/3847015309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f380015cfc0 0x7f38000cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.036+0000 7f37fffff640 1 -- 172.21.15.16:0/3847015309 >> v1:172.21.15.16:6789/0 conn(0x7f38000d2ae0 legacy=0x7f38000b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.036+0000 7f37fffff640 1 -- 172.21.15.16:0/3847015309 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38000b3a10 con 0x7f38000b30e0 2023-11-13T10:56:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.037+0000 7f37fffff640 1 --2- 172.21.15.16:0/3847015309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000b30e0 0x7f38000d2f40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009d90 tx=0x7f37f802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f2f3187ea39f0d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.038+0000 7f37feffd640 1 -- 172.21.15.16:0/3847015309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f802cbd0 con 0x7f38000b30e0 2023-11-13T10:56:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.038+0000 7f37feffd640 1 -- 172.21.15.16:0/3847015309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f802cd30 con 0x7f38000b30e0 2023-11-13T10:56:53.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.038+0000 7f37feffd640 1 -- 172.21.15.16:0/3847015309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f802cea0 con 0x7f38000b30e0 2023-11-13T10:56:53.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.038+0000 7f3804967640 1 -- 172.21.15.16:0/3847015309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000b30e0 msgr2=0x7f38000d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.038+0000 7f3804967640 1 --2- 172.21.15.16:0/3847015309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000b30e0 0x7f38000d2f40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009d90 tx=0x7f37f802be20 comp rx=0 tx=0).stop 2023-11-13T10:56:53.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.039+0000 7f3804967640 1 -- 172.21.15.16:0/3847015309 shutdown_connections 2023-11-13T10:56:53.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.039+0000 7f3804967640 1 --2- 172.21.15.16:0/3847015309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000b30e0 0x7f38000d2f40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.039+0000 7f3804967640 1 --2- 172.21.15.16:0/3847015309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f380015cfc0 0x7f38000cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.039+0000 7f3804967640 1 -- 172.21.15.16:0/3847015309 >> 172.21.15.16:0/3847015309 conn(0x7f380008c570 msgr2=0x7f380009be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:53.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.039+0000 7f3804967640 1 -- 172.21.15.16:0/3847015309 shutdown_connections 2023-11-13T10:56:53.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.039+0000 7f3804967640 1 -- 172.21.15.16:0/3847015309 wait complete. 2023-11-13T10:56:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.041+0000 7f3804967640 1 Processor -- start 2023-11-13T10:56:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.041+0000 7f3804967640 1 -- start start 2023-11-13T10:56:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f3804967640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38000b30e0 0x7f38000ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f3804967640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000d2ae0 0x7f380015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f3804967640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f380015cfc0 0x7f38000c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f3804967640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000cc460 con 0x7f38000d2ae0 2023-11-13T10:56:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f3804967640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380015ec50 con 0x7f380015cfc0 2023-11-13T10:56:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f3804967640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380015ef30 con 0x7f38000b30e0 2023-11-13T10:56:53.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000d2ae0 0x7f380015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000d2ae0 0x7f380015e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44492/0 (socket says 172.21.15.16:44492) 2023-11-13T10:56:53.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37f7fff640 1 -- 172.21.15.16:0/2150334943 learned_addr learned my addr 172.21.15.16:0/2150334943 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:53.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37ff7fe640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38000b30e0 0x7f38000ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37fffff640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f380015cfc0 0x7f38000c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37f7fff640 1 -- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38000b30e0 msgr2=0x7f38000ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37f7fff640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38000b30e0 0x7f38000ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37f7fff640 1 -- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f380015cfc0 msgr2=0x7f38000c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37f7fff640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f380015cfc0 0x7f38000c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37f7fff640 1 -- 172.21.15.16:0/2150334943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8009a70 con 0x7f38000d2ae0 2023-11-13T10:56:53.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.043+0000 7f37fffff640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f380015cfc0 0x7f38000c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.044+0000 7f37f7fff640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000d2ae0 0x7f380015e5b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c480 tx=0x7f37e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.044+0000 7f37fcff9640 1 -- 172.21.15.16:0/2150334943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8005ce0 con 0x7f38000d2ae0 2023-11-13T10:56:53.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.044+0000 7f37fcff9640 1 -- 172.21.15.16:0/2150334943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e8005e40 con 0x7f38000d2ae0 2023-11-13T10:56:53.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.044+0000 7f37fcff9640 1 -- 172.21.15.16:0/2150334943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80072f0 con 0x7f38000d2ae0 2023-11-13T10:56:53.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.044+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380015f210 con 0x7f38000d2ae0 2023-11-13T10:56:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.044+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380015f7d0 con 0x7f38000d2ae0 2023-11-13T10:56:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.045+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380015f9c0 con 0x7f38000d2ae0 2023-11-13T10:56:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.046+0000 7f37fcff9640 1 -- 172.21.15.16:0/2150334943 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37e8007510 con 0x7f38000d2ae0 2023-11-13T10:56:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.047+0000 7f37fcff9640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37e006ae40 0x7f37e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.047+0000 7f37fcff9640 1 -- 172.21.15.16:0/2150334943 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f37e8088e20 con 0x7f38000d2ae0 2023-11-13T10:56:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.048+0000 7f37ff7fe640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37e006ae40 0x7f37e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.049+0000 7f37ff7fe640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37e006ae40 0x7f37e006d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37f0006fd0 tx=0x7f37f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.051+0000 7f37fcff9640 1 -- 172.21.15.16:0/2150334943 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37e8053c30 con 0x7f38000d2ae0 2023-11-13T10:56:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.249+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f380004e4d0 con 0x7f38000d2ae0 2023-11-13T10:56:53.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.250+0000 7f37fcff9640 1 -- 172.21.15.16:0/2150334943 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f37e8017020 con 0x7f38000d2ae0 2023-11-13T10:56:53.256 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:53.257 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37e006ae40 msgr2=0x7f37e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37e006ae40 0x7f37e006d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37f0006fd0 tx=0x7f37f0008040 comp rx=0 tx=0).stop 2023-11-13T10:56:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000d2ae0 msgr2=0x7f380015e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000d2ae0 0x7f380015e5b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c480 tx=0x7f37e800c940 comp rx=0 tx=0).stop 2023-11-13T10:56:53.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 shutdown_connections 2023-11-13T10:56:53.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37e006ae40 0x7f37e006d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f380015cfc0 0x7f38000c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000d2ae0 0x7f380015e5b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 --2- 172.21.15.16:0/2150334943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38000b30e0 0x7f38000ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 >> 172.21.15.16:0/2150334943 conn(0x7f380008c570 msgr2=0x7f38000c86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 shutdown_connections 2023-11-13T10:56:53.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.253+0000 7f3804967640 1 -- 172.21.15.16:0/2150334943 wait complete. 2023-11-13T10:56:53.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:53.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:53.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.506+0000 7fdc4d400640 1 Processor -- start 2023-11-13T10:56:53.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.506+0000 7fdc4d400640 1 -- start start 2023-11-13T10:56:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.507+0000 7fdc4d400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc4815d280 0x7fdc480cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.507+0000 7fdc4d400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480d37d0 0x7fdc480cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.507+0000 7fdc4d400640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc480d0bb0 0x7fdc480d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.507+0000 7fdc4d400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc480cf510 con 0x7fdc4815d280 2023-11-13T10:56:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.507+0000 7fdc4d400640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc480ce990 con 0x7fdc480d0bb0 2023-11-13T10:56:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.507+0000 7fdc4d400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480cec70 con 0x7fdc480d37d0 2023-11-13T10:56:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc46ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc4815d280 0x7fdc480cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc46ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc4815d280 0x7fdc480cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44502/0 (socket says 172.21.15.16:44502) 2023-11-13T10:56:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc46ffd640 1 -- 172.21.15.16:0/3877846557 learned_addr learned my addr 172.21.15.16:0/3877846557 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc477fe640 1 --2- 172.21.15.16:0/3877846557 >> v2:172.21.15.184:3300/0 conn(0x7fdc480d0bb0 0x7fdc480d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc467fc640 1 --2- 172.21.15.16:0/3877846557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480d37d0 0x7fdc480cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc46ffd640 1 -- 172.21.15.16:0/3877846557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480d37d0 msgr2=0x7fdc480cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:53.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc46ffd640 1 --2- 172.21.15.16:0/3877846557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480d37d0 0x7fdc480cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc46ffd640 1 -- 172.21.15.16:0/3877846557 >> v2:172.21.15.184:3300/0 conn(0x7fdc480d0bb0 msgr2=0x7fdc480d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:53.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc46ffd640 1 --2- 172.21.15.16:0/3877846557 >> v2:172.21.15.184:3300/0 conn(0x7fdc480d0bb0 0x7fdc480d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.508+0000 7fdc46ffd640 1 -- 172.21.15.16:0/3877846557 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480cef50 con 0x7fdc4815d280 2023-11-13T10:56:53.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.510+0000 7fdc46ffd640 1 --2- 172.21.15.16:0/3877846557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc4815d280 0x7fdc480cf6c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdc48072690 tx=0x7fdc3802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d183fded1b2f98d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.510+0000 7fdc45ffb640 1 -- 172.21.15.16:0/3877846557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3802ca70 con 0x7fdc4815d280 2023-11-13T10:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.510+0000 7fdc45ffb640 1 -- 172.21.15.16:0/3877846557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3802cbd0 con 0x7fdc4815d280 2023-11-13T10:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.510+0000 7fdc45ffb640 1 -- 172.21.15.16:0/3877846557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380343f0 con 0x7fdc4815d280 2023-11-13T10:56:53.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.511+0000 7fdc4d400640 1 -- 172.21.15.16:0/3877846557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc4815d280 msgr2=0x7fdc480cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.511+0000 7fdc4d400640 1 --2- 172.21.15.16:0/3877846557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc4815d280 0x7fdc480cf6c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdc48072690 tx=0x7fdc3802bed0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.511+0000 7fdc4d400640 1 -- 172.21.15.16:0/3877846557 shutdown_connections 2023-11-13T10:56:53.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.511+0000 7fdc4d400640 1 --2- 172.21.15.16:0/3877846557 >> v2:172.21.15.184:3300/0 conn(0x7fdc480d0bb0 0x7fdc480d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.511+0000 7fdc4d400640 1 --2- 172.21.15.16:0/3877846557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc480d37d0 0x7fdc480cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.511+0000 7fdc4d400640 1 --2- 172.21.15.16:0/3877846557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc4815d280 0x7fdc480cf6c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.511+0000 7fdc4d400640 1 -- 172.21.15.16:0/3877846557 >> 172.21.15.16:0/3877846557 conn(0x7fdc480d2cb0 msgr2=0x7fdc480d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:53.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.511+0000 7fdc4d400640 1 -- 172.21.15.16:0/3877846557 shutdown_connections 2023-11-13T10:56:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.511+0000 7fdc4d400640 1 -- 172.21.15.16:0/3877846557 wait complete. 2023-11-13T10:56:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.513+0000 7fdc4d400640 1 Processor -- start 2023-11-13T10:56:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.513+0000 7fdc4d400640 1 -- start start 2023-11-13T10:56:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc4d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480d0bb0 0x7fdc481f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc4d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc480d37d0 0x7fdc4809ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc4d400640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc4815d280 0x7fdc480b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc4d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4809fa20 con 0x7fdc480d37d0 2023-11-13T10:56:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc4d400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4809fd00 con 0x7fdc480d0bb0 2023-11-13T10:56:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc4d400640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4809ffe0 con 0x7fdc4815d280 2023-11-13T10:56:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc467fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc480d37d0 0x7fdc4809ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc467fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc480d37d0 0x7fdc4809ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44504/0 (socket says 172.21.15.16:44504) 2023-11-13T10:56:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc467fc640 1 -- 172.21.15.16:0/2319374061 learned_addr learned my addr 172.21.15.16:0/2319374061 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc46ffd640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480d0bb0 0x7fdc481f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.515+0000 7fdc477fe640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc4815d280 0x7fdc480b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.516+0000 7fdc467fc640 1 -- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc4815d280 msgr2=0x7fdc480b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.516+0000 7fdc467fc640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc4815d280 0x7fdc480b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.516+0000 7fdc467fc640 1 -- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480d0bb0 msgr2=0x7fdc481f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.516+0000 7fdc467fc640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480d0bb0 0x7fdc481f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.516+0000 7fdc467fc640 1 -- 172.21.15.16:0/2319374061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38009b30 con 0x7fdc480d37d0 2023-11-13T10:56:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.516+0000 7fdc467fc640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc480d37d0 0x7fdc4809ef90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000d800 tx=0x7fdc3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.517+0000 7fdc3f7fe640 1 -- 172.21.15.16:0/2319374061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000b6c0 con 0x7fdc480d37d0 2023-11-13T10:56:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.517+0000 7fdc3f7fe640 1 -- 172.21.15.16:0/2319374061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3000b820 con 0x7fdc480d37d0 2023-11-13T10:56:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.517+0000 7fdc3f7fe640 1 -- 172.21.15.16:0/2319374061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000bbb0 con 0x7fdc480d37d0 2023-11-13T10:56:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.517+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc480a02c0 con 0x7fdc480d37d0 2023-11-13T10:56:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.517+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4815ba80 con 0x7fdc480d37d0 2023-11-13T10:56:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.518+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc480a0c60 con 0x7fdc480d37d0 2023-11-13T10:56:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.520+0000 7fdc3f7fe640 1 -- 172.21.15.16:0/2319374061 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc3000bd10 con 0x7fdc480d37d0 2023-11-13T10:56:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.520+0000 7fdc3f7fe640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc2806ae40 0x7fdc2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.520+0000 7fdc3f7fe640 1 -- 172.21.15.16:0/2319374061 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fdc30089d30 con 0x7fdc480d37d0 2023-11-13T10:56:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.521+0000 7fdc46ffd640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc2806ae40 0x7fdc2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.522+0000 7fdc46ffd640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc2806ae40 0x7fdc2806d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc38035000 tx=0x7fdc38030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.525+0000 7fdc3f7fe640 1 -- 172.21.15.16:0/2319374061 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc30054b40 con 0x7fdc480d37d0 2023-11-13T10:56:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.718+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc4804e5b0 con 0x7fdc480d37d0 2023-11-13T10:56:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.719+0000 7fdc3f7fe640 1 -- 172.21.15.16:0/2319374061 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc30058160 con 0x7fdc480d37d0 2023-11-13T10:56:53.724 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:53.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc2806ae40 msgr2=0x7fdc2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc2806ae40 0x7fdc2806d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc38035000 tx=0x7fdc38030000 comp rx=0 tx=0).stop 2023-11-13T10:56:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc480d37d0 msgr2=0x7fdc4809ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc480d37d0 0x7fdc4809ef90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000d800 tx=0x7fdc3000dcc0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 shutdown_connections 2023-11-13T10:56:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc2806ae40 0x7fdc2806d300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc4815d280 0x7fdc480b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc480d37d0 0x7fdc4809ef90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 --2- 172.21.15.16:0/2319374061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc480d0bb0 0x7fdc481f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 >> 172.21.15.16:0/2319374061 conn(0x7fdc480d2cb0 msgr2=0x7fdc480af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.722+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 shutdown_connections 2023-11-13T10:56:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.723+0000 7fdc4d400640 1 -- 172.21.15.16:0/2319374061 wait complete. 2023-11-13T10:56:53.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.968+0000 7f425b6a2640 1 Processor -- start 2023-11-13T10:56:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.969+0000 7f425b6a2640 1 -- start start 2023-11-13T10:56:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.970+0000 7f425b6a2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 0x7f42540b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.970+0000 7f425b6a2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42540b3b90 0x7f4254070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.970+0000 7f425b6a2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42540b2e00 con 0x7f425406c3b0 2023-11-13T10:56:53.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.970+0000 7f425b6a2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42540b21f0 con 0x7f42540b3b90 2023-11-13T10:56:53.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.970+0000 7f425b6a2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42540b24d0 con 0x7f425415cf90 2023-11-13T10:56:53.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.971+0000 7f4259c3f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42540b3b90 0x7f4254070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.971+0000 7f4259c3f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42540b3b90 0x7f4254070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44512/0 (socket says 172.21.15.16:44512) 2023-11-13T10:56:53.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.971+0000 7f4259c3f640 1 -- 172.21.15.16:0/1376677248 learned_addr learned my addr 172.21.15.16:0/1376677248 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:53.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.971+0000 7f425943e640 1 --2- 172.21.15.16:0/1376677248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 0x7f42540b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.971+0000 7f4259c3f640 1 -- 172.21.15.16:0/1376677248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 msgr2=0x7f42540b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:53.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.971+0000 7f4259c3f640 1 --2- 172.21.15.16:0/1376677248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 0x7f42540b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.971+0000 7f4259c3f640 1 -- 172.21.15.16:0/1376677248 >> v1:172.21.15.184:6789/0 conn(0x7f425406c3b0 legacy=0x7f42540b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:53.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.971+0000 7f4259c3f640 1 -- 172.21.15.16:0/1376677248 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42540b27b0 con 0x7f42540b3b90 2023-11-13T10:56:53.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.971+0000 7f425943e640 1 --2- 172.21.15.16:0/1376677248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 0x7f42540b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:53.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.972+0000 7f4259c3f640 1 --2- 172.21.15.16:0/1376677248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 0x7f4254070440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f424c009870 tx=0x7f424c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdd45ad529eae62 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:53.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.973+0000 7f4253fff640 1 -- 172.21.15.16:0/1376677248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c031640 con 0x7f42540b3b90 2023-11-13T10:56:53.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.973+0000 7f4253fff640 1 -- 172.21.15.16:0/1376677248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c0317a0 con 0x7f42540b3b90 2023-11-13T10:56:53.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.973+0000 7f4253fff640 1 -- 172.21.15.16:0/1376677248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c031b30 con 0x7f42540b3b90 2023-11-13T10:56:53.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.973+0000 7f425b6a2640 1 -- 172.21.15.16:0/1376677248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 msgr2=0x7f4254070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.973+0000 7f425b6a2640 1 --2- 172.21.15.16:0/1376677248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 0x7f4254070440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f424c009870 tx=0x7f424c02b8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.974+0000 7f425b6a2640 1 -- 172.21.15.16:0/1376677248 shutdown_connections 2023-11-13T10:56:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.974+0000 7f425b6a2640 1 --2- 172.21.15.16:0/1376677248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 0x7f4254070440 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.974+0000 7f425b6a2640 1 --2- 172.21.15.16:0/1376677248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 0x7f42540b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.974+0000 7f425b6a2640 1 -- 172.21.15.16:0/1376677248 >> 172.21.15.16:0/1376677248 conn(0x7f425406a190 msgr2=0x7f425406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.974+0000 7f425b6a2640 1 -- 172.21.15.16:0/1376677248 shutdown_connections 2023-11-13T10:56:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.974+0000 7f425b6a2640 1 -- 172.21.15.16:0/1376677248 wait complete. 2023-11-13T10:56:53.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.975+0000 7f425b6a2640 1 Processor -- start 2023-11-13T10:56:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.976+0000 7f425b6a2640 1 -- start start 2023-11-13T10:56:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f425b6a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425406c3b0 0x7f425414f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f425b6a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 0x7f42540af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f425b6a2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 0x7f42540afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f425b6a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425414f630 con 0x7f42540b3b90 2023-11-13T10:56:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f425b6a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42541f6070 con 0x7f425406c3b0 2023-11-13T10:56:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f425b6a2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42541f6350 con 0x7f425415cf90 2023-11-13T10:56:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f4258c3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 0x7f42540af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f4258c3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 0x7f42540af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44516/0 (socket says 172.21.15.16:44516) 2023-11-13T10:56:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f4258c3d640 1 -- 172.21.15.16:0/3071422660 learned_addr learned my addr 172.21.15.16:0/3071422660 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f425943e640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425406c3b0 0x7f425414f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f4259c3f640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 0x7f42540afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f4258c3d640 1 -- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 msgr2=0x7f42540afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f4258c3d640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 0x7f42540afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f4258c3d640 1 -- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425406c3b0 msgr2=0x7f425414f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f4258c3d640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425406c3b0 0x7f425414f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f4258c3d640 1 -- 172.21.15.16:0/3071422660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c0094d0 con 0x7f42540b3b90 2023-11-13T10:56:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.977+0000 7f425943e640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425406c3b0 0x7f425414f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.978+0000 7f4258c3d640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 0x7f42540af7c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f424800c430 tx=0x7f424800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.978+0000 7f4251ffb640 1 -- 172.21.15.16:0/3071422660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248011790 con 0x7f42540b3b90 2023-11-13T10:56:54.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.978+0000 7f4251ffb640 1 -- 172.21.15.16:0/3071422660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42480118f0 con 0x7f42540b3b90 2023-11-13T10:56:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.978+0000 7f4251ffb640 1 -- 172.21.15.16:0/3071422660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248011ce0 con 0x7f42540b3b90 2023-11-13T10:56:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.978+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42541f6630 con 0x7f42540b3b90 2023-11-13T10:56:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.978+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42541f6bf0 con 0x7f42540b3b90 2023-11-13T10:56:54.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.980+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4254072520 con 0x7f42540b3b90 2023-11-13T10:56:54.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.980+0000 7f4251ffb640 1 -- 172.21.15.16:0/3071422660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f424801c460 con 0x7f42540b3b90 2023-11-13T10:56:54.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.981+0000 7f4251ffb640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424006ae40 0x7f424006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.982+0000 7f4251ffb640 1 -- 172.21.15.16:0/3071422660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f42480898c0 con 0x7f42540b3b90 2023-11-13T10:56:54.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.982+0000 7f425943e640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424006ae40 0x7f424006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.983+0000 7f425943e640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424006ae40 0x7f424006d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4244007f40 tx=0x7f4244007490 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:53.987+0000 7f4251ffb640 1 -- 172.21.15.16:0/3071422660 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4248054610 con 0x7f42540b3b90 2023-11-13T10:56:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.186+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42540adea0 con 0x7f42540b3b90 2023-11-13T10:56:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.187+0000 7f4251ffb640 1 -- 172.21.15.16:0/3071422660 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4248057c30 con 0x7f42540b3b90 2023-11-13T10:56:54.192 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.189+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424006ae40 msgr2=0x7f424006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.189+0000 7f425b6a2640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424006ae40 0x7f424006d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4244007f40 tx=0x7f4244007490 comp rx=0 tx=0).stop 2023-11-13T10:56:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.189+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 msgr2=0x7f42540af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.189+0000 7f425b6a2640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 0x7f42540af7c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f424800c430 tx=0x7f424800c8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.190+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 shutdown_connections 2023-11-13T10:56:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.190+0000 7f425b6a2640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424006ae40 0x7f424006d300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.190+0000 7f425b6a2640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f425415cf90 0x7f42540afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.190+0000 7f425b6a2640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42540b3b90 0x7f42540af7c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.190+0000 7f425b6a2640 1 --2- 172.21.15.16:0/3071422660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f425406c3b0 0x7f425414f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.190+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 >> 172.21.15.16:0/3071422660 conn(0x7f425406a190 msgr2=0x7f4254156eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.190+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 shutdown_connections 2023-11-13T10:56:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.190+0000 7f425b6a2640 1 -- 172.21.15.16:0/3071422660 wait complete. 2023-11-13T10:56:54.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.435+0000 7fdfbf59e640 1 Processor -- start 2023-11-13T10:56:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.435+0000 7fdfbf59e640 1 -- start start 2023-11-13T10:56:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.437+0000 7fdfbf59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdfc015ce70 0x7fdfc00cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.437+0000 7fdfbf59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdfc00d1850 0x7fdfc00c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.437+0000 7fdfbf59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc00ce0e0 con 0x7fdfc00cdbb0 2023-11-13T10:56:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.437+0000 7fdfbf59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc00ce3c0 con 0x7fdfc015ce70 2023-11-13T10:56:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.437+0000 7fdfbf59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc00ce6a0 con 0x7fdfc00d1850 2023-11-13T10:56:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.437+0000 7fdfbe59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdfc015ce70 0x7fdfc00cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.437+0000 7fdfbe59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdfc015ce70 0x7fdfc00cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44522/0 (socket says 172.21.15.16:44522) 2023-11-13T10:56:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.437+0000 7fdfbe59c640 1 -- 172.21.15.16:0/3104698357 learned_addr learned my addr 172.21.15.16:0/3104698357 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.438+0000 7fdfbe59c640 1 -- 172.21.15.16:0/3104698357 >> v2:172.21.15.184:3300/0 conn(0x7fdfc00d1850 msgr2=0x7fdfc00c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.438+0000 7fdfbed9d640 1 --2- 172.21.15.16:0/3104698357 >> v2:172.21.15.184:3300/0 conn(0x7fdfc00d1850 0x7fdfc00c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.438+0000 7fdfbe59c640 1 --2- 172.21.15.16:0/3104698357 >> v2:172.21.15.184:3300/0 conn(0x7fdfc00d1850 0x7fdfc00c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.438+0000 7fdfbe59c640 1 -- 172.21.15.16:0/3104698357 >> v1:172.21.15.184:6789/0 conn(0x7fdfc00cdbb0 legacy=0x7fdfc00d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.438+0000 7fdfbe59c640 1 -- 172.21.15.16:0/3104698357 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc00ce980 con 0x7fdfc015ce70 2023-11-13T10:56:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.438+0000 7fdfbed9d640 1 --2- 172.21.15.16:0/3104698357 >> v2:172.21.15.184:3300/0 conn(0x7fdfc00d1850 0x7fdfc00c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.439+0000 7fdfbe59c640 1 --2- 172.21.15.16:0/3104698357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 0x7fdfc00cd670 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdfc008d170 tx=0x7fdfb002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dfacce8016e95c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.439+0000 7fdfbd59a640 1 -- 172.21.15.16:0/3104698357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb002ca80 con 0x7fdfc015ce70 2023-11-13T10:56:54.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.439+0000 7fdfbd59a640 1 -- 172.21.15.16:0/3104698357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb002cbe0 con 0x7fdfc015ce70 2023-11-13T10:56:54.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.439+0000 7fdfbd59a640 1 -- 172.21.15.16:0/3104698357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb00343f0 con 0x7fdfc015ce70 2023-11-13T10:56:54.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.440+0000 7fdfbf59e640 1 -- 172.21.15.16:0/3104698357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 msgr2=0x7fdfc00cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.440+0000 7fdfbf59e640 1 --2- 172.21.15.16:0/3104698357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 0x7fdfc00cd670 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdfc008d170 tx=0x7fdfb002bee0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.440+0000 7fdfbf59e640 1 -- 172.21.15.16:0/3104698357 shutdown_connections 2023-11-13T10:56:54.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.440+0000 7fdfbf59e640 1 --2- 172.21.15.16:0/3104698357 >> v2:172.21.15.184:3300/0 conn(0x7fdfc00d1850 0x7fdfc00c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.440+0000 7fdfbf59e640 1 --2- 172.21.15.16:0/3104698357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 0x7fdfc00cd670 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.440+0000 7fdfbf59e640 1 -- 172.21.15.16:0/3104698357 >> 172.21.15.16:0/3104698357 conn(0x7fdfc008c580 msgr2=0x7fdfc009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:54.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.440+0000 7fdfbf59e640 1 -- 172.21.15.16:0/3104698357 shutdown_connections 2023-11-13T10:56:54.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.440+0000 7fdfbf59e640 1 -- 172.21.15.16:0/3104698357 wait complete. 2023-11-13T10:56:54.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.442+0000 7fdfbf59e640 1 Processor -- start 2023-11-13T10:56:54.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.442+0000 7fdfbf59e640 1 -- start start 2023-11-13T10:56:54.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.443+0000 7fdfbf59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfc00cdbb0 0x7fdfc00b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.443+0000 7fdfbf59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfc00d1850 0x7fdfc00c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.443+0000 7fdfbf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 0x7fdfc00b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.443+0000 7fdfbf59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc00b5cd0 con 0x7fdfc015ce70 2023-11-13T10:56:54.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.443+0000 7fdfbf59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc00b5fb0 con 0x7fdfc00d1850 2023-11-13T10:56:54.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.443+0000 7fdfbf59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc00b6290 con 0x7fdfc00cdbb0 2023-11-13T10:56:54.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.443+0000 7fdfbed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 0x7fdfc00b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.443+0000 7fdfbed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 0x7fdfc00b7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44528/0 (socket says 172.21.15.16:44528) 2023-11-13T10:56:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.443+0000 7fdfbed9d640 1 -- 172.21.15.16:0/198769651 learned_addr learned my addr 172.21.15.16:0/198769651 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.444+0000 7fdfbdd9b640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfc00d1850 0x7fdfc00c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.444+0000 7fdfbed9d640 1 -- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfc00cdbb0 msgr2=0x7fdfc00b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.444+0000 7fdfbed9d640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfc00cdbb0 0x7fdfc00b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.444+0000 7fdfbed9d640 1 -- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfc00d1850 msgr2=0x7fdfc00c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.444+0000 7fdfbed9d640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfc00d1850 0x7fdfc00c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.444+0000 7fdfbed9d640 1 -- 172.21.15.16:0/198769651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0009b30 con 0x7fdfc015ce70 2023-11-13T10:56:54.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.444+0000 7fdfbed9d640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 0x7fdfc00b7fc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800c7f0 tx=0x7fdfb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:54.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.444+0000 7fdfaeffd640 1 -- 172.21.15.16:0/198769651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8004220 con 0x7fdfc015ce70 2023-11-13T10:56:54.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.445+0000 7fdfaeffd640 1 -- 172.21.15.16:0/198769651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb80043a0 con 0x7fdfc015ce70 2023-11-13T10:56:54.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.445+0000 7fdfaeffd640 1 -- 172.21.15.16:0/198769651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8007860 con 0x7fdfc015ce70 2023-11-13T10:56:54.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.445+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc00b6570 con 0x7fdfc015ce70 2023-11-13T10:56:54.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.445+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc00c7e40 con 0x7fdfc015ce70 2023-11-13T10:56:54.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.446+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc00b6a90 con 0x7fdfc015ce70 2023-11-13T10:56:54.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.447+0000 7fdfaeffd640 1 -- 172.21.15.16:0/198769651 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdfb8007bd0 con 0x7fdfc015ce70 2023-11-13T10:56:54.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.448+0000 7fdfaeffd640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfa006ae40 0x7fdfa006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.448+0000 7fdfaeffd640 1 -- 172.21.15.16:0/198769651 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fdfb8089a50 con 0x7fdfc015ce70 2023-11-13T10:56:54.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.448+0000 7fdfbe59c640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfa006ae40 0x7fdfa006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.450+0000 7fdfbe59c640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfa006ae40 0x7fdfa006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0035000 tx=0x7fdfb002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:54.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.452+0000 7fdfaeffd640 1 -- 172.21.15.16:0/198769651 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdfc00b6a90 con 0x7fdfc015ce70 2023-11-13T10:56:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.647+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdfc00b6a90 con 0x7fdfc015ce70 2023-11-13T10:56:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.648+0000 7fdfaeffd640 1 -- 172.21.15.16:0/198769651 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdfc00b6a90 con 0x7fdfc015ce70 2023-11-13T10:56:54.653 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.650+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfa006ae40 msgr2=0x7fdfa006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.650+0000 7fdfbf59e640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfa006ae40 0x7fdfa006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0035000 tx=0x7fdfb002e000 comp rx=0 tx=0).stop 2023-11-13T10:56:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.650+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 msgr2=0x7fdfc00b7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.650+0000 7fdfbf59e640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 0x7fdfc00b7fc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800c7f0 tx=0x7fdfb800ccb0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.651+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 shutdown_connections 2023-11-13T10:56:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.651+0000 7fdfbf59e640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdfa006ae40 0x7fdfa006d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.651+0000 7fdfbf59e640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfc015ce70 0x7fdfc00b7fc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.651+0000 7fdfbf59e640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfc00d1850 0x7fdfc00c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.651+0000 7fdfbf59e640 1 --2- 172.21.15.16:0/198769651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfc00cdbb0 0x7fdfc00b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.651+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 >> 172.21.15.16:0/198769651 conn(0x7fdfc008c580 msgr2=0x7fdfc015f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.651+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 shutdown_connections 2023-11-13T10:56:54.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.651+0000 7fdfbf59e640 1 -- 172.21.15.16:0/198769651 wait complete. 2023-11-13T10:56:54.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.908+0000 7f9bf5327640 1 Processor -- start 2023-11-13T10:56:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.909+0000 7f9bf5327640 1 -- start start 2023-11-13T10:56:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.910+0000 7f9bf5327640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9bf0070cc0 0x7f9bf006bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.910+0000 7f9bf5327640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9bf00710a0 0x7f9bf015baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.910+0000 7f9bf5327640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 0x7f9bf0158550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.910+0000 7f9bf5327640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf015c030 con 0x7f9bf00710a0 2023-11-13T10:56:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.910+0000 7f9bf5327640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf0158a90 con 0x7f9bf0070cc0 2023-11-13T10:56:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.910+0000 7f9bf5327640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf006c280 con 0x7f9bf01580e0 2023-11-13T10:56:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9bef7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9bf00710a0 0x7f9bf015baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9bef7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9bf00710a0 0x7f9bf015baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44536/0 (socket says 172.21.15.16:44536) 2023-11-13T10:56:54.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2957554572 learned_addr learned my addr 172.21.15.16:0/2957554572 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:54.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9bf4b26640 1 --2- 172.21.15.16:0/2957554572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 0x7f9bf0158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9beffff640 1 --2- 172.21.15.16:0/2957554572 >> v2:172.21.15.184:3300/0 conn(0x7f9bf0070cc0 0x7f9bf006bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2957554572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 msgr2=0x7f9bf0158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:54.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9bef7fe640 1 --2- 172.21.15.16:0/2957554572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 0x7f9bf0158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2957554572 >> v2:172.21.15.184:3300/0 conn(0x7f9bf0070cc0 msgr2=0x7f9bf006bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9bef7fe640 1 --2- 172.21.15.16:0/2957554572 >> v2:172.21.15.184:3300/0 conn(0x7f9bf0070cc0 0x7f9bf006bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.911+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2957554572 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00b29a0 con 0x7f9bf00710a0 2023-11-13T10:56:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.913+0000 7f9bef7fe640 1 --2- 172.21.15.16:0/2957554572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 0x7f9bf015baf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b2690 tx=0x7f9be402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899222458bc94e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:54.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.913+0000 7f9beeffd640 1 -- 172.21.15.16:0/2957554572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4031860 con 0x7f9bf00710a0 2023-11-13T10:56:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.913+0000 7f9beeffd640 1 -- 172.21.15.16:0/2957554572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be40319c0 con 0x7f9bf00710a0 2023-11-13T10:56:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.913+0000 7f9beeffd640 1 -- 172.21.15.16:0/2957554572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4031d50 con 0x7f9bf00710a0 2023-11-13T10:56:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.914+0000 7f9bf5327640 1 -- 172.21.15.16:0/2957554572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 msgr2=0x7f9bf015baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.914+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2957554572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 0x7f9bf015baf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b2690 tx=0x7f9be402b900 comp rx=0 tx=0).stop 2023-11-13T10:56:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.914+0000 7f9bf5327640 1 -- 172.21.15.16:0/2957554572 shutdown_connections 2023-11-13T10:56:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.914+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2957554572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 0x7f9bf0158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.914+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2957554572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 0x7f9bf015baf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.914+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2957554572 >> v2:172.21.15.184:3300/0 conn(0x7f9bf0070cc0 0x7f9bf006bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.914+0000 7f9bf5327640 1 -- 172.21.15.16:0/2957554572 >> 172.21.15.16:0/2957554572 conn(0x7f9bf00b1600 msgr2=0x7f9bf00b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.914+0000 7f9bf5327640 1 -- 172.21.15.16:0/2957554572 shutdown_connections 2023-11-13T10:56:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.914+0000 7f9bf5327640 1 -- 172.21.15.16:0/2957554572 wait complete. 2023-11-13T10:56:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.916+0000 7f9bf5327640 1 Processor -- start 2023-11-13T10:56:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.916+0000 7f9bf5327640 1 -- start start 2023-11-13T10:56:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.917+0000 7f9bf5327640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf0070cc0 0x7f9bf0155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.917+0000 7f9bf5327640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 0x7f9bf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.917+0000 7f9bf5327640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 0x7f9bf00c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.917+0000 7f9bf5327640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0148c30 con 0x7f9bf00710a0 2023-11-13T10:56:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.917+0000 7f9bf5327640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0154280 con 0x7f9bf0070cc0 2023-11-13T10:56:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.917+0000 7f9bf5327640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0154560 con 0x7f9bf01580e0 2023-11-13T10:56:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.917+0000 7f9bef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 0x7f9bf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.917+0000 7f9bef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 0x7f9bf006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44538/0 (socket says 172.21.15.16:44538) 2023-11-13T10:56:54.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.917+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2187507663 learned_addr learned my addr 172.21.15.16:0/2187507663 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:54.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9bf4b26640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 0x7f9bf00c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9beffff640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf0070cc0 0x7f9bf0155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 msgr2=0x7f9bf00c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9bef7fe640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 0x7f9bf00c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf0070cc0 msgr2=0x7f9bf0155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9bef7fe640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf0070cc0 0x7f9bf0155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2187507663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40094d0 con 0x7f9bf00710a0 2023-11-13T10:56:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9bf4b26640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 0x7f9bf00c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9bef7fe640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 0x7f9bf006d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9be402b940 tx=0x7f9be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:54.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9becff9640 1 -- 172.21.15.16:0/2187507663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be403ad30 con 0x7f9bf00710a0 2023-11-13T10:56:54.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9becff9640 1 -- 172.21.15.16:0/2187507663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be40443f0 con 0x7f9bf00710a0 2023-11-13T10:56:54.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9becff9640 1 -- 172.21.15.16:0/2187507663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be40447a0 con 0x7f9bf00710a0 2023-11-13T10:56:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.918+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf0154840 con 0x7f9bf00710a0 2023-11-13T10:56:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.919+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf0154e00 con 0x7f9bf00710a0 2023-11-13T10:56:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.920+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf0154ff0 con 0x7f9bf00710a0 2023-11-13T10:56:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.921+0000 7f9becff9640 1 -- 172.21.15.16:0/2187507663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9be404d450 con 0x7f9bf00710a0 2023-11-13T10:56:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.922+0000 7f9becff9640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bdc06ae40 0x7f9bdc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.922+0000 7f9becff9640 1 -- 172.21.15.16:0/2187507663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f9be40ac020 con 0x7f9bf00710a0 2023-11-13T10:56:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.922+0000 7f9beffff640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bdc06ae40 0x7f9bdc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.924+0000 7f9beffff640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bdc06ae40 0x7f9bdc06d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009f30 tx=0x7f9be000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:54.926+0000 7f9becff9640 1 -- 172.21.15.16:0/2187507663 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9be4076c60 con 0x7f9bf00710a0 2023-11-13T10:56:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.120+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9bf004e4d0 con 0x7f9bf00710a0 2023-11-13T10:56:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.121+0000 7f9becff9640 1 -- 172.21.15.16:0/2187507663 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9be4035030 con 0x7f9bf00710a0 2023-11-13T10:56:55.126 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bdc06ae40 msgr2=0x7f9bdc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bdc06ae40 0x7f9bdc06d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009f30 tx=0x7f9be000b040 comp rx=0 tx=0).stop 2023-11-13T10:56:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 msgr2=0x7f9bf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 0x7f9bf006d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9be402b940 tx=0x7f9be40046c0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 shutdown_connections 2023-11-13T10:56:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9bdc06ae40 0x7f9bdc06d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9bf01580e0 0x7f9bf00c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf00710a0 0x7f9bf006d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 --2- 172.21.15.16:0/2187507663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf0070cc0 0x7f9bf0155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 >> 172.21.15.16:0/2187507663 conn(0x7f9bf00b1600 msgr2=0x7f9bf00cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 shutdown_connections 2023-11-13T10:56:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.124+0000 7f9bf5327640 1 -- 172.21.15.16:0/2187507663 wait complete. 2023-11-13T10:56:55.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:56:55.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.368+0000 7f6d5359e640 1 Processor -- start 2023-11-13T10:56:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.369+0000 7f6d5359e640 1 -- start start 2023-11-13T10:56:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.370+0000 7f6d5359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d540d2ae0 0x7f6d540d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.370+0000 7f6d5359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d540b29d0 0x7f6d540b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.370+0000 7f6d5359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d540b3cf0 con 0x7f6d5415cfc0 2023-11-13T10:56:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.370+0000 7f6d5359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d540d3480 con 0x7f6d540d2ae0 2023-11-13T10:56:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.370+0000 7f6d5359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d540cde80 con 0x7f6d540b29d0 2023-11-13T10:56:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d5259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6d5415cfc0 0x7f6d540cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36442/0 (socket says 172.21.15.16:36442) 2023-11-13T10:56:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d5259c640 1 -- 172.21.15.16:0/1950932331 learned_addr learned my addr 172.21.15.16:0/1950932331 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d51d9b640 1 --2- 172.21.15.16:0/1950932331 >> v2:172.21.15.16:3300/0 conn(0x7f6d540d2ae0 0x7f6d540d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:55.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d52d9d640 1 --2- 172.21.15.16:0/1950932331 >> v2:172.21.15.184:3300/0 conn(0x7f6d540b29d0 0x7f6d540b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:55.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d5159a640 1 -- 172.21.15.16:0/1950932331 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516665000 0 0) 0x7f6d540b3cf0 con 0x7f6d5415cfc0 2023-11-13T10:56:55.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d5159a640 1 -- 172.21.15.16:0/1950932331 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d40003520 con 0x7f6d5415cfc0 2023-11-13T10:56:55.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d51d9b640 1 -- 172.21.15.16:0/1950932331 >> v2:172.21.15.184:3300/0 conn(0x7f6d540b29d0 msgr2=0x7f6d540b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:55.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d51d9b640 1 --2- 172.21.15.16:0/1950932331 >> v2:172.21.15.184:3300/0 conn(0x7f6d540b29d0 0x7f6d540b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d51d9b640 1 -- 172.21.15.16:0/1950932331 >> v1:172.21.15.16:6789/0 conn(0x7f6d5415cfc0 legacy=0x7f6d540cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:55.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.371+0000 7f6d51d9b640 1 -- 172.21.15.16:0/1950932331 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d540b3730 con 0x7f6d540d2ae0 2023-11-13T10:56:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.373+0000 7f6d51d9b640 1 --2- 172.21.15.16:0/1950932331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d540d2ae0 0x7f6d540d2f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6d540738c0 tx=0x7f6d3c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784a0e382ab0c977 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.373+0000 7f6d5159a640 1 -- 172.21.15.16:0/1950932331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c02e070 con 0x7f6d540d2ae0 2023-11-13T10:56:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.373+0000 7f6d5159a640 1 -- 172.21.15.16:0/1950932331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c0315a0 con 0x7f6d540d2ae0 2023-11-13T10:56:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.373+0000 7f6d5159a640 1 -- 172.21.15.16:0/1950932331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c031930 con 0x7f6d540d2ae0 2023-11-13T10:56:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.374+0000 7f6d5359e640 1 -- 172.21.15.16:0/1950932331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d540d2ae0 msgr2=0x7f6d540d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.374+0000 7f6d5359e640 1 --2- 172.21.15.16:0/1950932331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d540d2ae0 0x7f6d540d2f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6d540738c0 tx=0x7f6d3c02b840 comp rx=0 tx=0).stop 2023-11-13T10:56:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.374+0000 7f6d5359e640 1 -- 172.21.15.16:0/1950932331 shutdown_connections 2023-11-13T10:56:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.374+0000 7f6d5359e640 1 --2- 172.21.15.16:0/1950932331 >> v2:172.21.15.184:3300/0 conn(0x7f6d540b29d0 0x7f6d540b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.374+0000 7f6d5359e640 1 --2- 172.21.15.16:0/1950932331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d540d2ae0 0x7f6d540d2f40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.374+0000 7f6d5359e640 1 -- 172.21.15.16:0/1950932331 >> 172.21.15.16:0/1950932331 conn(0x7f6d5408c570 msgr2=0x7f6d5409be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.374+0000 7f6d5359e640 1 -- 172.21.15.16:0/1950932331 shutdown_connections 2023-11-13T10:56:55.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.374+0000 7f6d5359e640 1 -- 172.21.15.16:0/1950932331 wait complete. 2023-11-13T10:56:55.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.376+0000 7f6d5359e640 1 Processor -- start 2023-11-13T10:56:55.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.376+0000 7f6d5359e640 1 -- start start 2023-11-13T10:56:55.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.377+0000 7f6d5359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b29d0 0x7f6d5415e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.377+0000 7f6d5359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d540d2ae0 0x7f6d54160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.377+0000 7f6d5359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5415cfc0 0x7f6d540b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.377+0000 7f6d5359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540b7ab0 con 0x7f6d5415cfc0 2023-11-13T10:56:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.377+0000 7f6d5359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d541f59c0 con 0x7f6d540d2ae0 2023-11-13T10:56:55.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.377+0000 7f6d5359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d541f5ca0 con 0x7f6d540b29d0 2023-11-13T10:56:55.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.377+0000 7f6d52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5415cfc0 0x7f6d540b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:55.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.377+0000 7f6d52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5415cfc0 0x7f6d540b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44550/0 (socket says 172.21.15.16:44550) 2023-11-13T10:56:55.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.377+0000 7f6d52d9d640 1 -- 172.21.15.16:0/551312645 learned_addr learned my addr 172.21.15.16:0/551312645 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:55.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.378+0000 7f6d51d9b640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d540d2ae0 0x7f6d54160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:55.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.378+0000 7f6d5259c640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b29d0 0x7f6d5415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:55.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.378+0000 7f6d52d9d640 1 -- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b29d0 msgr2=0x7f6d5415e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:55.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.378+0000 7f6d52d9d640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b29d0 0x7f6d5415e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.378+0000 7f6d52d9d640 1 -- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d540d2ae0 msgr2=0x7f6d54160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:55.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.378+0000 7f6d52d9d640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d540d2ae0 0x7f6d54160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.378+0000 7f6d52d9d640 1 -- 172.21.15.16:0/551312645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c009400 con 0x7f6d5415cfc0 2023-11-13T10:56:55.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.378+0000 7f6d52d9d640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5415cfc0 0x7f6d540b7570 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00d5d0 tx=0x7f6d4c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:55.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.379+0000 7f6d4affd640 1 -- 172.21.15.16:0/551312645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0125f0 con 0x7f6d5415cfc0 2023-11-13T10:56:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.379+0000 7f6d4affd640 1 -- 172.21.15.16:0/551312645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c012770 con 0x7f6d5415cfc0 2023-11-13T10:56:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.379+0000 7f6d4affd640 1 -- 172.21.15.16:0/551312645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c012b20 con 0x7f6d5415cfc0 2023-11-13T10:56:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.379+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d541f5f80 con 0x7f6d5415cfc0 2023-11-13T10:56:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.379+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d541f6540 con 0x7f6d5415cfc0 2023-11-13T10:56:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.380+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5404e4d0 con 0x7f6d5415cfc0 2023-11-13T10:56:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.381+0000 7f6d4affd640 1 -- 172.21.15.16:0/551312645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d4c012c80 con 0x7f6d5415cfc0 2023-11-13T10:56:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.382+0000 7f6d4affd640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d3406ae40 0x7f6d3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.382+0000 7f6d4affd640 1 -- 172.21.15.16:0/551312645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f6d4c089ef0 con 0x7f6d5415cfc0 2023-11-13T10:56:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.383+0000 7f6d5259c640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d3406ae40 0x7f6d3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.384+0000 7f6d5259c640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d3406ae40 0x7f6d3406d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6d44001560 tx=0x7f6d44009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.386+0000 7f6d4affd640 1 -- 172.21.15.16:0/551312645 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d4c054bd0 con 0x7f6d5415cfc0 2023-11-13T10:56:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.596+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6d541f23d0 con 0x7f6d5415cfc0 2023-11-13T10:56:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.597+0000 7f6d4affd640 1 -- 172.21.15.16:0/551312645 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6d4c0581f0 con 0x7f6d5415cfc0 2023-11-13T10:56:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:56:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.599+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d3406ae40 msgr2=0x7f6d3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.599+0000 7f6d5359e640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d3406ae40 0x7f6d3406d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6d44001560 tx=0x7f6d44009040 comp rx=0 tx=0).stop 2023-11-13T10:56:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.599+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5415cfc0 msgr2=0x7f6d540b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.599+0000 7f6d5359e640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5415cfc0 0x7f6d540b7570 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00d5d0 tx=0x7f6d4c00da90 comp rx=0 tx=0).stop 2023-11-13T10:56:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.600+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 shutdown_connections 2023-11-13T10:56:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.600+0000 7f6d5359e640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d3406ae40 0x7f6d3406d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.600+0000 7f6d5359e640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5415cfc0 0x7f6d540b7570 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.600+0000 7f6d5359e640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d540d2ae0 0x7f6d54160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.600+0000 7f6d5359e640 1 --2- 172.21.15.16:0/551312645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d540b29d0 0x7f6d5415e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.600+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 >> 172.21.15.16:0/551312645 conn(0x7f6d5408c570 msgr2=0x7f6d540b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.600+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 shutdown_connections 2023-11-13T10:56:55.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:55.600+0000 7f6d5359e640 1 -- 172.21.15.16:0/551312645 wait complete. 2023-11-13T10:56:55.625 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-13T10:56:55.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:55.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-13T10:56:55.727 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T10:56:55.728 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T10:56:55.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:56:55.729 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-13T10:56:55.787 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:56:55.788 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:55.850 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:56:55.850 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:55.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:56.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.138+0000 7f8ea359e640 1 Processor -- start 2023-11-13T10:56:56.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.138+0000 7f8ea359e640 1 -- start start 2023-11-13T10:56:56.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.138+0000 7f8ea359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8ea415cfc0 0x7f8ea40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.138+0000 7f8ea359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ea40b30e0 0x7f8ea40d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.138+0000 7f8ea359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea40b3cf0 con 0x7f8ea40d2ae0 2023-11-13T10:56:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.139+0000 7f8ea359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea40b3550 con 0x7f8ea40b30e0 2023-11-13T10:56:56.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.139+0000 7f8ea359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea40d3480 con 0x7f8ea415cfc0 2023-11-13T10:56:56.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.139+0000 7f8ea2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ea40b30e0 0x7f8ea40d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.139+0000 7f8ea2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ea40b30e0 0x7f8ea40d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44556/0 (socket says 172.21.15.16:44556) 2023-11-13T10:56:56.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.139+0000 7f8ea2d9d640 1 -- 172.21.15.16:0/1551712559 learned_addr learned my addr 172.21.15.16:0/1551712559 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:56.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.139+0000 7f8ea2d9d640 1 -- 172.21.15.16:0/1551712559 >> v2:172.21.15.184:3300/0 conn(0x7f8ea415cfc0 msgr2=0x7f8ea40cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:56.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.139+0000 7f8ea2d9d640 1 --2- 172.21.15.16:0/1551712559 >> v2:172.21.15.184:3300/0 conn(0x7f8ea415cfc0 0x7f8ea40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.139+0000 7f8ea2d9d640 1 -- 172.21.15.16:0/1551712559 >> v1:172.21.15.184:6789/0 conn(0x7f8ea40d2ae0 legacy=0x7f8ea40b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.139+0000 7f8ea2d9d640 1 -- 172.21.15.16:0/1551712559 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea40b3730 con 0x7f8ea40b30e0 2023-11-13T10:56:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.140+0000 7f8ea2d9d640 1 --2- 172.21.15.16:0/1551712559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea40b30e0 0x7f8ea40d2f40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40d0680 tx=0x7f8e9c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484e1c965f3697c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.141+0000 7f8ea159a640 1 -- 172.21.15.16:0/1551712559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0317e0 con 0x7f8ea40b30e0 2023-11-13T10:56:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.141+0000 7f8ea159a640 1 -- 172.21.15.16:0/1551712559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c031940 con 0x7f8ea40b30e0 2023-11-13T10:56:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.141+0000 7f8ea159a640 1 -- 172.21.15.16:0/1551712559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c031cd0 con 0x7f8ea40b30e0 2023-11-13T10:56:56.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.141+0000 7f8ea359e640 1 -- 172.21.15.16:0/1551712559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea40b30e0 msgr2=0x7f8ea40d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.141+0000 7f8ea359e640 1 --2- 172.21.15.16:0/1551712559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea40b30e0 0x7f8ea40d2f40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40d0680 tx=0x7f8e9c02b880 comp rx=0 tx=0).stop 2023-11-13T10:56:56.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.142+0000 7f8ea359e640 1 -- 172.21.15.16:0/1551712559 shutdown_connections 2023-11-13T10:56:56.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.142+0000 7f8ea359e640 1 --2- 172.21.15.16:0/1551712559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea40b30e0 0x7f8ea40d2f40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.142+0000 7f8ea359e640 1 --2- 172.21.15.16:0/1551712559 >> v2:172.21.15.184:3300/0 conn(0x7f8ea415cfc0 0x7f8ea40cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.142+0000 7f8ea359e640 1 -- 172.21.15.16:0/1551712559 >> 172.21.15.16:0/1551712559 conn(0x7f8ea408c570 msgr2=0x7f8ea409be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:56.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.142+0000 7f8ea359e640 1 -- 172.21.15.16:0/1551712559 shutdown_connections 2023-11-13T10:56:56.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.142+0000 7f8ea359e640 1 -- 172.21.15.16:0/1551712559 wait complete. 2023-11-13T10:56:56.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.144+0000 7f8ea359e640 1 Processor -- start 2023-11-13T10:56:56.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.144+0000 7f8ea359e640 1 -- start start 2023-11-13T10:56:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ea40b30e0 0x7f8ea4153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40d2ae0 0x7f8ea40b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415cfc0 0x7f8ea40b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40c66b0 con 0x7f8ea415cfc0 2023-11-13T10:56:56.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4154120 con 0x7f8ea40d2ae0 2023-11-13T10:56:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41f66f0 con 0x7f8ea40b30e0 2023-11-13T10:56:56.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415cfc0 0x7f8ea40b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415cfc0 0x7f8ea40b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44570/0 (socket says 172.21.15.16:44570) 2023-11-13T10:56:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea2d9d640 1 -- 172.21.15.16:0/1114114520 learned_addr learned my addr 172.21.15.16:0/1114114520 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea259c640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ea40b30e0 0x7f8ea4153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.145+0000 7f8ea1d9b640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40d2ae0 0x7f8ea40b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8ea2d9d640 1 -- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ea40b30e0 msgr2=0x7f8ea4153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8ea2d9d640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ea40b30e0 0x7f8ea4153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8ea2d9d640 1 -- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40d2ae0 msgr2=0x7f8ea40b7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8ea2d9d640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40d2ae0 0x7f8ea40b7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8ea2d9d640 1 -- 172.21.15.16:0/1114114520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c0094d0 con 0x7f8ea415cfc0 2023-11-13T10:56:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8ea1d9b640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40d2ae0 0x7f8ea40b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8ea2d9d640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415cfc0 0x7f8ea40b0910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0090b0 tx=0x7f8e9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8e9affd640 1 -- 172.21.15.16:0/1114114520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c032940 con 0x7f8ea415cfc0 2023-11-13T10:56:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8e9affd640 1 -- 172.21.15.16:0/1114114520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c032ac0 con 0x7f8ea415cfc0 2023-11-13T10:56:56.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea41f69d0 con 0x7f8ea415cfc0 2023-11-13T10:56:56.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8e9affd640 1 -- 172.21.15.16:0/1114114520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c032e70 con 0x7f8ea415cfc0 2023-11-13T10:56:56.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.146+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea41f6f90 con 0x7f8ea415cfc0 2023-11-13T10:56:56.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.147+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea41f7180 con 0x7f8ea415cfc0 2023-11-13T10:56:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.149+0000 7f8e9affd640 1 -- 172.21.15.16:0/1114114520 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e9c04d3f0 con 0x7f8ea415cfc0 2023-11-13T10:56:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.150+0000 7f8e9affd640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8e8406ae40 0x7f8e8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.150+0000 7f8e9affd640 1 -- 172.21.15.16:0/1114114520 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f8e9c0ab840 con 0x7f8ea415cfc0 2023-11-13T10:56:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.151+0000 7f8ea259c640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8e8406ae40 0x7f8e8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.152+0000 7f8ea259c640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8e8406ae40 0x7f8e8406d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400b810 tx=0x7f8e94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.154+0000 7f8e9affd640 1 -- 172.21.15.16:0/1114114520 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ea41f7180 con 0x7f8ea415cfc0 2023-11-13T10:56:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.353+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ea41f7180 con 0x7f8ea415cfc0 2023-11-13T10:56:56.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.356+0000 7f8e9affd640 1 -- 172.21.15.16:0/1114114520 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f8ea41f7180 con 0x7f8ea415cfc0 2023-11-13T10:56:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:56.362 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8e8406ae40 msgr2=0x7f8e8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8e8406ae40 0x7f8e8406d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400b810 tx=0x7f8e94008040 comp rx=0 tx=0).stop 2023-11-13T10:56:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415cfc0 msgr2=0x7f8ea40b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415cfc0 0x7f8ea40b0910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0090b0 tx=0x7f8e9c0046c0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 shutdown_connections 2023-11-13T10:56:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8e8406ae40 0x7f8e8406d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea415cfc0 0x7f8ea40b0910 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ea40d2ae0 0x7f8ea40b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 --2- 172.21.15.16:0/1114114520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ea40b30e0 0x7f8ea4153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.359+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 >> 172.21.15.16:0/1114114520 conn(0x7f8ea408c570 msgr2=0x7f8ea40d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.360+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 shutdown_connections 2023-11-13T10:56:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.360+0000 7f8ea359e640 1 -- 172.21.15.16:0/1114114520 wait complete. 2023-11-13T10:56:56.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:56.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.611+0000 7ff0fe11d640 1 Processor -- start 2023-11-13T10:56:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.611+0000 7ff0fe11d640 1 -- start start 2023-11-13T10:56:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.613+0000 7ff0fe11d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff0f80ce9b0 0x7ff0f80b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.613+0000 7ff0fe11d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f8144950 con 0x7ff0f8144570 2023-11-13T10:56:56.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.613+0000 7ff0fe11d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f80ce300 con 0x7ff0f80ce550 2023-11-13T10:56:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.613+0000 7ff0fe11d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f80cd260 con 0x7ff0f80ce9b0 2023-11-13T10:56:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.613+0000 7ff0f77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff0f8144570 0x7ff0f815f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36452/0 (socket says 172.21.15.16:36452) 2023-11-13T10:56:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.613+0000 7ff0f77fe640 1 -- 172.21.15.16:0/2520095697 learned_addr learned my addr 172.21.15.16:0/2520095697 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.614+0000 7ff0f7fff640 1 --2- 172.21.15.16:0/2520095697 >> v2:172.21.15.184:3300/0 conn(0x7ff0f80ce9b0 0x7ff0f80b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.614+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694037623 0 0) 0x7ff0f8144950 con 0x7ff0f8144570 2023-11-13T10:56:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.614+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e0003520 con 0x7ff0f8144570 2023-11-13T10:56:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.614+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 460404990 0 0) 0x7ff0e0003520 con 0x7ff0f8144570 2023-11-13T10:56:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.614+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 >> v2:172.21.15.184:3300/0 conn(0x7ff0f80ce9b0 msgr2=0x7ff0f80b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.614+0000 7ff0f67fc640 1 --2- 172.21.15.16:0/2520095697 >> v2:172.21.15.184:3300/0 conn(0x7ff0f80ce9b0 0x7ff0f80b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.614+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 >> v1:172.21.15.184:6789/0 conn(0x7ff0f80ce550 legacy=0x7ff0f80b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.614+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f80cd540 con 0x7ff0f8144570 2023-11-13T10:56:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.614+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7ff0e8001670 con 0x7ff0f8144570 2023-11-13T10:56:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 >> v1:172.21.15.16:6789/0 conn(0x7ff0f8144570 legacy=0x7ff0f815f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f67fc640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e0003970 0x7ff0e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f67fc640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e00047c0 0x7ff0e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f67fc640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e0005180 0x7ff0e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f80cd260 con 0x7ff0e0003970 2023-11-13T10:56:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e000c020 con 0x7ff0e00047c0 2023-11-13T10:56:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e000c1e0 con 0x7ff0e0005180 2023-11-13T10:56:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f77fe640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e0003970 0x7ff0e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f77fe640 1 -- 172.21.15.16:0/2520095697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e0005180 msgr2=0x7ff0e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:56.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f77fe640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e0005180 0x7ff0e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f77fe640 1 -- 172.21.15.16:0/2520095697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e00047c0 msgr2=0x7ff0e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f6ffd640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e00047c0 0x7ff0e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.615+0000 7ff0f77fe640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e00047c0 0x7ff0e0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.616+0000 7ff0f77fe640 1 -- 172.21.15.16:0/2520095697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0003520 con 0x7ff0e0003970 2023-11-13T10:56:56.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.616+0000 7ff0f7fff640 1 -- 172.21.15.16:0/2520095697 reap_dead start 2023-11-13T10:56:56.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.617+0000 7ff0f77fe640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e0003970 0x7ff0e0003d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800c190 tx=0x7ff0e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b267014ec7749206 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:56.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.617+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8005ad0 con 0x7ff0e0003970 2023-11-13T10:56:56.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.617+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e8005c30 con 0x7ff0e0003970 2023-11-13T10:56:56.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.617+0000 7ff0f67fc640 1 -- 172.21.15.16:0/2520095697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e802cab0 con 0x7ff0e0003970 2023-11-13T10:56:56.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.617+0000 7ff0fe11d640 1 -- 172.21.15.16:0/2520095697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e0003970 msgr2=0x7ff0e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.617+0000 7ff0fe11d640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e0003970 0x7ff0e0003d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800c190 tx=0x7ff0e00057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.618+0000 7ff0fe11d640 1 -- 172.21.15.16:0/2520095697 shutdown_connections 2023-11-13T10:56:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.618+0000 7ff0fe11d640 1 --2- 172.21.15.16:0/2520095697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e0003970 0x7ff0e0003d90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.618+0000 7ff0fe11d640 1 -- 172.21.15.16:0/2520095697 >> 172.21.15.16:0/2520095697 conn(0x7ff0f806a190 msgr2=0x7ff0f806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.618+0000 7ff0fe11d640 1 -- 172.21.15.16:0/2520095697 shutdown_connections 2023-11-13T10:56:56.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.618+0000 7ff0fe11d640 1 -- 172.21.15.16:0/2520095697 wait complete. 2023-11-13T10:56:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.620+0000 7ff0fe11d640 1 Processor -- start 2023-11-13T10:56:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.620+0000 7ff0fe11d640 1 -- start start 2023-11-13T10:56:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0fe11d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e0003970 0x7ff0f8142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0fe11d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f8156a00 0x7ff0f81f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0fe11d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f8154690 0x7ff0f8154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0fe11d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f8155050 con 0x7ff0f8156a00 2023-11-13T10:56:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0fe11d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f8143200 con 0x7ff0e0003970 2023-11-13T10:56:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0fe11d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f8142960 con 0x7ff0f8154690 2023-11-13T10:56:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f8156a00 0x7ff0f81f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f8156a00 0x7ff0f81f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44580/0 (socket says 172.21.15.16:44580) 2023-11-13T10:56:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3350868846 learned_addr learned my addr 172.21.15.16:0/3350868846 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0f77fe640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e0003970 0x7ff0f8142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f8154690 msgr2=0x7ff0f8154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0f6ffd640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f8154690 0x7ff0f8154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e0003970 msgr2=0x7ff0f8142380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0f6ffd640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e0003970 0x7ff0f8142380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.621+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3350868846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8142c40 con 0x7ff0f8156a00 2023-11-13T10:56:56.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.622+0000 7ff0f6ffd640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f8156a00 0x7ff0f81f5a50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00d400 tx=0x7ff0ec00d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:56.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.622+0000 7ff0dffff640 1 -- 172.21.15.16:0/3350868846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00b070 con 0x7ff0f8156a00 2023-11-13T10:56:56.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.622+0000 7ff0dffff640 1 -- 172.21.15.16:0/3350868846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec0054e0 con 0x7ff0f8156a00 2023-11-13T10:56:56.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.622+0000 7ff0dffff640 1 -- 172.21.15.16:0/3350868846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec003830 con 0x7ff0f8156a00 2023-11-13T10:56:56.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.622+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f8142f20 con 0x7ff0f8156a00 2023-11-13T10:56:56.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.622+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f81f5f90 con 0x7ff0f8156a00 2023-11-13T10:56:56.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.625+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f8072520 con 0x7ff0f8156a00 2023-11-13T10:56:56.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.625+0000 7ff0dffff640 1 -- 172.21.15.16:0/3350868846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff0ec003990 con 0x7ff0f8156a00 2023-11-13T10:56:56.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.625+0000 7ff0dffff640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff0e406ae70 0x7ff0e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:56.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.625+0000 7ff0dffff640 1 -- 172.21.15.16:0/3350868846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7ff0ec0890d0 con 0x7ff0f8156a00 2023-11-13T10:56:56.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.626+0000 7ff0f77fe640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff0e406ae70 0x7ff0e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:56.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.627+0000 7ff0f77fe640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff0e406ae70 0x7ff0e406d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8034000 tx=0x7ff0e8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:56.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.630+0000 7ff0dffff640 1 -- 172.21.15.16:0/3350868846 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0ec053ee0 con 0x7ff0f8156a00 2023-11-13T10:56:56.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.823+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0f804e5b0 con 0x7ff0f8156a00 2023-11-13T10:56:56.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.824+0000 7ff0dffff640 1 -- 172.21.15.16:0/3350868846 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0ec0180a0 con 0x7ff0f8156a00 2023-11-13T10:56:56.829 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:56.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff0e406ae70 msgr2=0x7ff0e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff0e406ae70 0x7ff0e406d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8034000 tx=0x7ff0e8009030 comp rx=0 tx=0).stop 2023-11-13T10:56:56.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f8156a00 msgr2=0x7ff0f81f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:56.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f8156a00 0x7ff0f81f5a50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00d400 tx=0x7ff0ec00d8c0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 shutdown_connections 2023-11-13T10:56:56.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff0e406ae70 0x7ff0e406d330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0f8154690 0x7ff0f8154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f8156a00 0x7ff0f81f5a50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 --2- 172.21.15.16:0/3350868846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e0003970 0x7ff0f8142380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:56.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 >> 172.21.15.16:0/3350868846 conn(0x7ff0f806a190 msgr2=0x7ff0f80b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:56.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 shutdown_connections 2023-11-13T10:56:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:56.827+0000 7ff0fe11d640 1 -- 172.21.15.16:0/3350868846 wait complete. 2023-11-13T10:56:56.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.076+0000 7f8455025640 1 Processor -- start 2023-11-13T10:56:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.076+0000 7f8455025640 1 -- start start 2023-11-13T10:56:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.077+0000 7f8455025640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f845015bcc0 0x7f84500cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.077+0000 7f8455025640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845006d010 con 0x7f845015b8e0 2023-11-13T10:56:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.077+0000 7f8455025640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845006b860 con 0x7f84500ce090 2023-11-13T10:56:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.077+0000 7f8455025640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84500ce540 con 0x7f845015bcc0 2023-11-13T10:56:57.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.077+0000 7f844ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f845015b8e0 0x7f845006c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36462/0 (socket says 172.21.15.16:36462) 2023-11-13T10:56:57.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.077+0000 7f844ffff640 1 -- 172.21.15.16:0/3269198521 learned_addr learned my addr 172.21.15.16:0/3269198521 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.078+0000 7f844f7fe640 1 --2- 172.21.15.16:0/3269198521 >> v2:172.21.15.16:3300/0 conn(0x7f845015bcc0 0x7f84500cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.078+0000 7f844effd640 1 -- 172.21.15.16:0/3269198521 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998558078 0 0) 0x7f845006d010 con 0x7f845015b8e0 2023-11-13T10:56:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.078+0000 7f844effd640 1 -- 172.21.15.16:0/3269198521 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8438003520 con 0x7f845015b8e0 2023-11-13T10:56:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.078+0000 7f844f7fe640 1 -- 172.21.15.16:0/3269198521 >> v1:172.21.15.184:6789/0 conn(0x7f84500ce090 legacy=0x7f84500cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.078+0000 7f844f7fe640 1 -- 172.21.15.16:0/3269198521 >> v1:172.21.15.16:6789/0 conn(0x7f845015b8e0 legacy=0x7f845006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.078+0000 7f844f7fe640 1 -- 172.21.15.16:0/3269198521 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845006bb40 con 0x7f845015bcc0 2023-11-13T10:56:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.082+0000 7f844f7fe640 1 --2- 172.21.15.16:0/3269198521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015bcc0 0x7f84500cdb50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84440097a0 tx=0x7f844402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beb532ddd98f05d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:57.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.082+0000 7f844effd640 1 -- 172.21.15.16:0/3269198521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444031830 con 0x7f845015bcc0 2023-11-13T10:56:57.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.082+0000 7f844effd640 1 -- 172.21.15.16:0/3269198521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8444031990 con 0x7f845015bcc0 2023-11-13T10:56:57.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.082+0000 7f844effd640 1 -- 172.21.15.16:0/3269198521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444031d20 con 0x7f845015bcc0 2023-11-13T10:56:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.082+0000 7f8455025640 1 -- 172.21.15.16:0/3269198521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015bcc0 msgr2=0x7f84500cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.082+0000 7f8455025640 1 --2- 172.21.15.16:0/3269198521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015bcc0 0x7f84500cdb50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84440097a0 tx=0x7f844402b8d0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.082+0000 7f8455025640 1 -- 172.21.15.16:0/3269198521 shutdown_connections 2023-11-13T10:56:57.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.082+0000 7f8455025640 1 --2- 172.21.15.16:0/3269198521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015bcc0 0x7f84500cdb50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.082+0000 7f8455025640 1 -- 172.21.15.16:0/3269198521 >> 172.21.15.16:0/3269198521 conn(0x7f84500b0970 msgr2=0x7f84500b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:57.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.083+0000 7f8455025640 1 -- 172.21.15.16:0/3269198521 shutdown_connections 2023-11-13T10:56:57.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.083+0000 7f8455025640 1 -- 172.21.15.16:0/3269198521 wait complete. 2023-11-13T10:56:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.084+0000 7f8455025640 1 Processor -- start 2023-11-13T10:56:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.085+0000 7f8455025640 1 -- start start 2023-11-13T10:56:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f8455025640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84500ce090 0x7f84500af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f8455025640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015b8e0 0x7f84501f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f8455025640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84501f3590 0x7f84500a8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f8455025640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84500cb300 con 0x7f845015b8e0 2023-11-13T10:56:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f8455025640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84501f3a10 con 0x7f84501f3590 2023-11-13T10:56:57.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f8455025640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84501f0830 con 0x7f84500ce090 2023-11-13T10:56:57.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f844f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015b8e0 0x7f84501f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f844f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015b8e0 0x7f84501f3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44598/0 (socket says 172.21.15.16:44598) 2023-11-13T10:56:57.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f844f7fe640 1 -- 172.21.15.16:0/3200811163 learned_addr learned my addr 172.21.15.16:0/3200811163 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:57.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f844f7fe640 1 -- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84500ce090 msgr2=0x7f84500af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f844f7fe640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84500ce090 0x7f84500af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f844f7fe640 1 -- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84501f3590 msgr2=0x7f84500a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f844f7fe640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84501f3590 0x7f84500a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.086+0000 7f844f7fe640 1 -- 172.21.15.16:0/3200811163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8444009400 con 0x7f845015b8e0 2023-11-13T10:56:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.087+0000 7f844f7fe640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015b8e0 0x7f84501f3050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8444034040 tx=0x7f84440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.087+0000 7f844cff9640 1 -- 172.21.15.16:0/3200811163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844403a630 con 0x7f845015b8e0 2023-11-13T10:56:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.087+0000 7f844cff9640 1 -- 172.21.15.16:0/3200811163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844403a7d0 con 0x7f845015b8e0 2023-11-13T10:56:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.087+0000 7f844cff9640 1 -- 172.21.15.16:0/3200811163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844403ab80 con 0x7f845015b8e0 2023-11-13T10:56:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.087+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84501f0b10 con 0x7f845015b8e0 2023-11-13T10:56:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.087+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84501f10d0 con 0x7f845015b8e0 2023-11-13T10:56:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.089+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845004e4d0 con 0x7f845015b8e0 2023-11-13T10:56:57.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.089+0000 7f844cff9640 1 -- 172.21.15.16:0/3200811163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f844403ace0 con 0x7f845015b8e0 2023-11-13T10:56:57.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.090+0000 7f844cff9640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.091+0000 7f844cff9640 1 -- 172.21.15.16:0/3200811163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f84440ab2b0 con 0x7f845015b8e0 2023-11-13T10:56:57.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.091+0000 7f844ffff640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.092+0000 7f844ffff640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8440001610 tx=0x7f8440007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.095+0000 7f844cff9640 1 -- 172.21.15.16:0/3200811163 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84440761d0 con 0x7f845015b8e0 2023-11-13T10:56:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.288+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84500b1810 con 0x7f845015b8e0 2023-11-13T10:56:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.289+0000 7f844cff9640 1 -- 172.21.15.16:0/3200811163 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84440797f0 con 0x7f845015b8e0 2023-11-13T10:56:57.294 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 msgr2=0x7f843c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8440001610 tx=0x7f8440007930 comp rx=0 tx=0).stop 2023-11-13T10:56:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015b8e0 msgr2=0x7f84501f3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015b8e0 0x7f84501f3050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8444034040 tx=0x7f84440046c0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 shutdown_connections 2023-11-13T10:56:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84501f3590 0x7f84500a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845015b8e0 0x7f84501f3050 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 --2- 172.21.15.16:0/3200811163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84500ce090 0x7f84500af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 >> 172.21.15.16:0/3200811163 conn(0x7f84500b0970 msgr2=0x7f845015c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 shutdown_connections 2023-11-13T10:56:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.292+0000 7f8455025640 1 -- 172.21.15.16:0/3200811163 wait complete. 2023-11-13T10:56:57.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:56:57.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.550+0000 7fe92759e640 1 Processor -- start 2023-11-13T10:56:57.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.550+0000 7fe92759e640 1 -- start start 2023-11-13T10:56:57.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.552+0000 7fe92759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815ce70 0x7fe9280cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.552+0000 7fe92759e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe9280d1850 0x7fe9280c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.552+0000 7fe92759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9280ce0e0 con 0x7fe9280cdbb0 2023-11-13T10:56:57.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.552+0000 7fe92759e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9280ce3c0 con 0x7fe9280d1850 2023-11-13T10:56:57.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.552+0000 7fe92759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280ce6a0 con 0x7fe92815ce70 2023-11-13T10:56:57.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.552+0000 7fe925d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe9280cdbb0 0x7fe9280d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36478/0 (socket says 172.21.15.16:36478) 2023-11-13T10:56:57.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.552+0000 7fe925d9b640 1 -- 172.21.15.16:0/1861273825 learned_addr learned my addr 172.21.15.16:0/1861273825 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:57.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92659c640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815ce70 0x7fe9280cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe926d9d640 1 --2- 172.21.15.16:0/1861273825 >> v2:172.21.15.184:3300/0 conn(0x7fe9280d1850 0x7fe9280c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831430551 0 0) 0x7fe9280ce0e0 con 0x7fe9280cdbb0 2023-11-13T10:56:57.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe910003520 con 0x7fe9280cdbb0 2023-11-13T10:56:57.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656057199 0 0) 0x7fe910003520 con 0x7fe9280cdbb0 2023-11-13T10:56:57.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815ce70 msgr2=0x7fe9280cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:57.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815ce70 0x7fe9280cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 >> v2:172.21.15.184:3300/0 conn(0x7fe9280d1850 msgr2=0x7fe9280c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:57.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 --2- 172.21.15.16:0/1861273825 >> v2:172.21.15.184:3300/0 conn(0x7fe9280d1850 0x7fe9280c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280ce980 con 0x7fe9280cdbb0 2023-11-13T10:56:57.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fe91c001670 con 0x7fe9280cdbb0 2023-11-13T10:56:57.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 >> v1:172.21.15.16:6789/0 conn(0x7fe9280cdbb0 legacy=0x7fe9280d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe910003970 0x7fe910003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815ce70 0x7fe9100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe910005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280ce3c0 con 0x7fe910004d00 2023-11-13T10:56:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280ce6a0 con 0x7fe910003970 2023-11-13T10:56:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.553+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91000b980 con 0x7fe92815ce70 2023-11-13T10:56:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.554+0000 7fe926d9d640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe910005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.554+0000 7fe925d9b640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815ce70 0x7fe9100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.554+0000 7fe926d9d640 1 -- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815ce70 msgr2=0x7fe9100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:57.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.554+0000 7fe926d9d640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815ce70 0x7fe9100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.554+0000 7fe92659c640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe910003970 0x7fe910003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.554+0000 7fe926d9d640 1 -- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe910003970 msgr2=0x7fe910003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.554+0000 7fe926d9d640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe910003970 0x7fe910003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.554+0000 7fe926d9d640 1 -- 172.21.15.16:0/1861273825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe910003520 con 0x7fe910004d00 2023-11-13T10:56:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.554+0000 7fe925d9b640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815ce70 0x7fe9100047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.555+0000 7fe926d9d640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe910005180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe920009f90 tx=0x7fe9100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712676e4e3f74981 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.556+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92002cc20 con 0x7fe910004d00 2023-11-13T10:56:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.556+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92000f040 con 0x7fe910004d00 2023-11-13T10:56:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.556+0000 7fe92559a640 1 -- 172.21.15.16:0/1861273825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920034440 con 0x7fe910004d00 2023-11-13T10:56:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.556+0000 7fe92759e640 1 -- 172.21.15.16:0/1861273825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 msgr2=0x7fe910005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.556+0000 7fe92759e640 1 --2- 172.21.15.16:0/1861273825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe910005180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe920009f90 tx=0x7fe9100057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.556+0000 7fe926d9d640 1 -- 172.21.15.16:0/1861273825 reap_dead start 2023-11-13T10:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.556+0000 7fe92759e640 1 -- 172.21.15.16:0/1861273825 shutdown_connections 2023-11-13T10:56:57.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.556+0000 7fe92759e640 1 -- 172.21.15.16:0/1861273825 >> 172.21.15.16:0/1861273825 conn(0x7fe92808c580 msgr2=0x7fe92809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:57.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.557+0000 7fe92759e640 1 -- 172.21.15.16:0/1861273825 shutdown_connections 2023-11-13T10:56:57.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.557+0000 7fe92759e640 1 -- 172.21.15.16:0/1861273825 wait complete. 2023-11-13T10:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.559+0000 7fe92759e640 1 Processor -- start 2023-11-13T10:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.559+0000 7fe92759e640 1 -- start start 2023-11-13T10:56:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.560+0000 7fe92759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe928159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.560+0000 7fe92759e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9280b81e0 0x7fe9280b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.560+0000 7fe92759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815bab0 0x7fe92815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.560+0000 7fe92759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280b89c0 con 0x7fe910004d00 2023-11-13T10:56:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.560+0000 7fe92759e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280b7450 con 0x7fe9280b81e0 2023-11-13T10:56:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.560+0000 7fe92759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92815c480 con 0x7fe92815bab0 2023-11-13T10:56:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.560+0000 7fe92659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe928159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.560+0000 7fe92659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe928159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44624/0 (socket says 172.21.15.16:44624) 2023-11-13T10:56:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.560+0000 7fe92659c640 1 -- 172.21.15.16:0/1452243701 learned_addr learned my addr 172.21.15.16:0/1452243701 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe926d9d640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815bab0 0x7fe92815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe925d9b640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9280b81e0 0x7fe9280b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe92659c640 1 -- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815bab0 msgr2=0x7fe92815bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe92659c640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815bab0 0x7fe92815bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe92659c640 1 -- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9280b81e0 msgr2=0x7fe9280b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe92659c640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9280b81e0 0x7fe9280b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe92659c640 1 -- 172.21.15.16:0/1452243701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280b86e0 con 0x7fe910004d00 2023-11-13T10:56:57.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe925d9b640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9280b81e0 0x7fe9280b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:57.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe92659c640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe928159e20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe918010890 tx=0x7fe918010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:57.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe916ffd640 1 -- 172.21.15.16:0/1452243701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9180095b0 con 0x7fe910004d00 2023-11-13T10:56:57.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe916ffd640 1 -- 172.21.15.16:0/1452243701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91800eba0 con 0x7fe910004d00 2023-11-13T10:56:57.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe916ffd640 1 -- 172.21.15.16:0/1452243701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91801c400 con 0x7fe910004d00 2023-11-13T10:56:57.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.561+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9280b6d90 con 0x7fe910004d00 2023-11-13T10:56:57.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.562+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9280b5db0 con 0x7fe910004d00 2023-11-13T10:56:57.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.563+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9280b6bf0 con 0x7fe910004d00 2023-11-13T10:56:57.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.564+0000 7fe916ffd640 1 -- 172.21.15.16:0/1452243701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9180143f0 con 0x7fe910004d00 2023-11-13T10:56:57.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.565+0000 7fe916ffd640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe90806ae40 0x7fe90806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:57.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.565+0000 7fe916ffd640 1 -- 172.21.15.16:0/1452243701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fe918089bb0 con 0x7fe910004d00 2023-11-13T10:56:57.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.565+0000 7fe925d9b640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe90806ae40 0x7fe90806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:57.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.567+0000 7fe925d9b640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe90806ae40 0x7fe90806d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0017a0 tx=0x7fe91c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:57.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.569+0000 7fe916ffd640 1 -- 172.21.15.16:0/1452243701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9280b6bf0 con 0x7fe910004d00 2023-11-13T10:56:57.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.760+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9280b6bf0 con 0x7fe910004d00 2023-11-13T10:56:57.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.761+0000 7fe916ffd640 1 -- 172.21.15.16:0/1452243701 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9280b6bf0 con 0x7fe910004d00 2023-11-13T10:56:57.765 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:56:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe90806ae40 msgr2=0x7fe90806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe90806ae40 0x7fe90806d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0017a0 tx=0x7fe91c009040 comp rx=0 tx=0).stop 2023-11-13T10:56:57.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 msgr2=0x7fe928159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe928159e20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe918010890 tx=0x7fe918010d50 comp rx=0 tx=0).stop 2023-11-13T10:56:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 shutdown_connections 2023-11-13T10:56:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe90806ae40 0x7fe90806d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe92815bab0 0x7fe92815bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9280b81e0 0x7fe9280b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 --2- 172.21.15.16:0/1452243701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe910004d00 0x7fe928159e20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 >> 172.21.15.16:0/1452243701 conn(0x7fe92808c580 msgr2=0x7fe92815adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 shutdown_connections 2023-11-13T10:56:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:57.764+0000 7fe92759e640 1 -- 172.21.15.16:0/1452243701 wait complete. 2023-11-13T10:56:57.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:56:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.018+0000 7ff450de2640 1 Processor -- start 2023-11-13T10:56:58.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.018+0000 7ff450de2640 1 -- start start 2023-11-13T10:56:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.020+0000 7ff450de2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff44c15bcc0 0x7ff44c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.020+0000 7ff450de2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff44c0ce090 0x7ff44c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.020+0000 7ff450de2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c06d010 con 0x7ff44c15b8e0 2023-11-13T10:56:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.020+0000 7ff450de2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0ce510 con 0x7ff44c15bcc0 2023-11-13T10:56:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.020+0000 7ff450de2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c06b860 con 0x7ff44c0ce090 2023-11-13T10:56:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff44c15b8e0 0x7ff44c06c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36492/0 (socket says 172.21.15.16:36492) 2023-11-13T10:56:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44b7fe640 1 -- 172.21.15.16:0/2935239550 learned_addr learned my addr 172.21.15.16:0/2935239550 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff442ffd640 1 --2- 172.21.15.16:0/2935239550 >> v2:172.21.15.184:3300/0 conn(0x7ff44c15bcc0 0x7ff44c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265117162 0 0) 0x7ff44c06d010 con 0x7ff44c15b8e0 2023-11-13T10:56:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44bfff640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff44c0ce090 0x7ff44c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff438003520 con 0x7ff44c15b8e0 2023-11-13T10:56:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574387073 0 0) 0x7ff438003520 con 0x7ff44c15b8e0 2023-11-13T10:56:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff44c0ce090 msgr2=0x7ff44c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44affd640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff44c0ce090 0x7ff44c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 >> v2:172.21.15.184:3300/0 conn(0x7ff44c15bcc0 msgr2=0x7ff44c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44affd640 1 --2- 172.21.15.16:0/2935239550 >> v2:172.21.15.184:3300/0 conn(0x7ff44c15bcc0 0x7ff44c0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c06bb40 con 0x7ff44c15b8e0 2023-11-13T10:56:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.021+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7ff43c0019f0 con 0x7ff44c15b8e0 2023-11-13T10:56:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 >> v1:172.21.15.16:6789/0 conn(0x7ff44c15b8e0 legacy=0x7ff44c06c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44affd640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff438003970 0x7ff438003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44affd640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4380047c0 0x7ff438004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44affd640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff43800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0ce510 con 0x7ff438005180 2023-11-13T10:56:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c06b860 con 0x7ff4380047c0 2023-11-13T10:56:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43800bfc0 con 0x7ff438003970 2023-11-13T10:56:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44bfff640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff43800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44b7fe640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff438003970 0x7ff438003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff442ffd640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4380047c0 0x7ff438004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44bfff640 1 -- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff438003970 msgr2=0x7ff438003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.022+0000 7ff44bfff640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff438003970 0x7ff438003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.023+0000 7ff44bfff640 1 -- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4380047c0 msgr2=0x7ff438004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.023+0000 7ff44bfff640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4380047c0 0x7ff438004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.023+0000 7ff44bfff640 1 -- 172.21.15.16:0/2935239550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff438003520 con 0x7ff438005180 2023-11-13T10:56:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.023+0000 7ff44b7fe640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff438003970 0x7ff438003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:56:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.024+0000 7ff44bfff640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff43800b980 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff44400b900 tx=0x7ff4380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc27c2175bdb765 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.024+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444034610 con 0x7ff438005180 2023-11-13T10:56:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.024+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44402e070 con 0x7ff438005180 2023-11-13T10:56:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.024+0000 7ff44affd640 1 -- 172.21.15.16:0/2935239550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444034990 con 0x7ff438005180 2023-11-13T10:56:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.024+0000 7ff450de2640 1 -- 172.21.15.16:0/2935239550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 msgr2=0x7ff43800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.025+0000 7ff450de2640 1 --2- 172.21.15.16:0/2935239550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff43800b980 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff44400b900 tx=0x7ff4380057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.025+0000 7ff44bfff640 1 -- 172.21.15.16:0/2935239550 reap_dead start 2023-11-13T10:56:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.025+0000 7ff450de2640 1 -- 172.21.15.16:0/2935239550 shutdown_connections 2023-11-13T10:56:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.025+0000 7ff450de2640 1 -- 172.21.15.16:0/2935239550 >> 172.21.15.16:0/2935239550 conn(0x7ff44c0b0970 msgr2=0x7ff44c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.025+0000 7ff450de2640 1 -- 172.21.15.16:0/2935239550 shutdown_connections 2023-11-13T10:56:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.025+0000 7ff450de2640 1 -- 172.21.15.16:0/2935239550 wait complete. 2023-11-13T10:56:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.027+0000 7ff450de2640 1 Processor -- start 2023-11-13T10:56:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.027+0000 7ff450de2640 1 -- start start 2023-11-13T10:56:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff450de2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff44c15e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff450de2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff44c1530e0 0x7ff44c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff450de2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff44c070d50 0x7ff44c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff450de2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c153a80 con 0x7ff438005180 2023-11-13T10:56:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff450de2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c06fae0 con 0x7ff44c1530e0 2023-11-13T10:56:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff450de2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c06fdc0 con 0x7ff44c070d50 2023-11-13T10:56:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff44b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff44c15e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff44b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff44c15e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44634/0 (socket says 172.21.15.16:44634) 2023-11-13T10:56:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff44b7fe640 1 -- 172.21.15.16:0/3501554501 learned_addr learned my addr 172.21.15.16:0/3501554501 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff44bfff640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff44c070d50 0x7ff44c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.028+0000 7ff442ffd640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff44c1530e0 0x7ff44c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff44b7fe640 1 -- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff44c070d50 msgr2=0x7ff44c0711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff44b7fe640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff44c070d50 0x7ff44c0711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff44b7fe640 1 -- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff44c1530e0 msgr2=0x7ff44c153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff44b7fe640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff44c1530e0 0x7ff44c153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff44b7fe640 1 -- 172.21.15.16:0/3501554501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c0700a0 con 0x7ff438005180 2023-11-13T10:56:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff44b7fe640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff44c15e550 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff43c005fd0 tx=0x7ff43c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff448ff9640 1 -- 172.21.15.16:0/3501554501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c00ee50 con 0x7ff438005180 2023-11-13T10:56:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff448ff9640 1 -- 172.21.15.16:0/3501554501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c013420 con 0x7ff438005180 2023-11-13T10:56:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff448ff9640 1 -- 172.21.15.16:0/3501554501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c013790 con 0x7ff438005180 2023-11-13T10:56:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c070380 con 0x7ff438005180 2023-11-13T10:56:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.029+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c071710 con 0x7ff438005180 2023-11-13T10:56:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.031+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c04e4d0 con 0x7ff438005180 2023-11-13T10:56:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.032+0000 7ff448ff9640 1 -- 172.21.15.16:0/3501554501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff43c0138f0 con 0x7ff438005180 2023-11-13T10:56:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.033+0000 7ff448ff9640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff42c06ae70 0x7ff42c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.033+0000 7ff448ff9640 1 -- 172.21.15.16:0/3501554501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7ff43c08a270 con 0x7ff438005180 2023-11-13T10:56:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.034+0000 7ff442ffd640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff42c06ae70 0x7ff42c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.035+0000 7ff442ffd640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff42c06ae70 0x7ff42c06d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff44c15ce20 tx=0x7ff434008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.038+0000 7ff448ff9640 1 -- 172.21.15.16:0/3501554501 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff43c054eb0 con 0x7ff438005180 2023-11-13T10:56:58.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.230+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff44c0b17a0 con 0x7ff438005180 2023-11-13T10:56:58.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.231+0000 7ff448ff9640 1 -- 172.21.15.16:0/3501554501 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff43c0584d0 con 0x7ff438005180 2023-11-13T10:56:58.235 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:56:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.233+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff42c06ae70 msgr2=0x7ff42c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.233+0000 7ff450de2640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff42c06ae70 0x7ff42c06d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff44c15ce20 tx=0x7ff434008040 comp rx=0 tx=0).stop 2023-11-13T10:56:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.233+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 msgr2=0x7ff44c15e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.233+0000 7ff450de2640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff44c15e550 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff43c005fd0 tx=0x7ff43c00e530 comp rx=0 tx=0).stop 2023-11-13T10:56:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.233+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 shutdown_connections 2023-11-13T10:56:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.233+0000 7ff450de2640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff42c06ae70 0x7ff42c06d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.233+0000 7ff450de2640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff44c070d50 0x7ff44c0711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.233+0000 7ff450de2640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff44c1530e0 0x7ff44c153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.233+0000 7ff450de2640 1 --2- 172.21.15.16:0/3501554501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff438005180 0x7ff44c15e550 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.234+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 >> 172.21.15.16:0/3501554501 conn(0x7ff44c0b0970 msgr2=0x7ff44c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.234+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 shutdown_connections 2023-11-13T10:56:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.234+0000 7ff450de2640 1 -- 172.21.15.16:0/3501554501 wait complete. 2023-11-13T10:56:58.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:56:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.463+0000 7f5d96f11640 1 Processor -- start 2023-11-13T10:56:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.463+0000 7f5d96f11640 1 -- start start 2023-11-13T10:56:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.464+0000 7f5d96f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d9015aa90 0x7f5d9006a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.464+0000 7f5d96f11640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9015d630 con 0x7f5d90070a70 2023-11-13T10:56:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.464+0000 7f5d96f11640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9015d910 con 0x7f5d9006a6d0 2023-11-13T10:56:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.464+0000 7f5d96f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9006ab50 con 0x7f5d9015aa90 2023-11-13T10:56:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.464+0000 7f5d96710640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d90070a70 0x7f5d9006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36504/0 (socket says 172.21.15.16:36504) 2023-11-13T10:56:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d96710640 1 -- 172.21.15.16:0/1898492414 learned_addr learned my addr 172.21.15.16:0/1898492414 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d95f0f640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d9015aa90 0x7f5d9006a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77806908 0 0) 0x7f5d9015d630 con 0x7f5d90070a70 2023-11-13T10:56:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7c003520 con 0x7f5d90070a70 2023-11-13T10:56:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1932062576 0 0) 0x7f5d7c003520 con 0x7f5d90070a70 2023-11-13T10:56:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d9015aa90 msgr2=0x7f5d9006a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d9015aa90 0x7f5d9006a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 >> v1:172.21.15.184:6789/0 conn(0x7f5d9006a6d0 legacy=0x7f5d90070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9015dbf0 con 0x7f5d90070a70 2023-11-13T10:56:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f5d880019f0 con 0x7f5d90070a70 2023-11-13T10:56:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 >> v1:172.21.15.16:6789/0 conn(0x7f5d90070a70 legacy=0x7f5d9006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.465+0000 7f5d94f0d640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7c0039a0 0x7f5d7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d94f0d640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d9015aa90 0x7f5d7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d94f0d640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d7c004d00 0x7f5d7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9006ab50 con 0x7f5d7c0039a0 2023-11-13T10:56:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c00bab0 con 0x7f5d7c004d00 2023-11-13T10:56:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c00bc70 con 0x7f5d9015aa90 2023-11-13T10:56:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d95f0f640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7c0039a0 0x7f5d7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d9570e640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d9015aa90 0x7f5d7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d95f0f640 1 -- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d9015aa90 msgr2=0x7f5d7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d96710640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d7c004d00 0x7f5d7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d95f0f640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d9015aa90 0x7f5d7c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d95f0f640 1 -- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d7c004d00 msgr2=0x7f5d7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:58.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d95f0f640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d7c004d00 0x7f5d7c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.466+0000 7f5d95f0f640 1 -- 172.21.15.16:0/1898492414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c003520 con 0x7f5d7c0039a0 2023-11-13T10:56:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.467+0000 7f5d95f0f640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7c0039a0 0x7f5d7c003dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009ac0 tx=0x7f5d7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427db9ea8e7c0816 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c02c600 con 0x7f5d7c0039a0 2023-11-13T10:56:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c02c760 con 0x7f5d7c0039a0 2023-11-13T10:56:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d94f0d640 1 -- 172.21.15.16:0/1898492414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c02caf0 con 0x7f5d7c0039a0 2023-11-13T10:56:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d96f11640 1 -- 172.21.15.16:0/1898492414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7c0039a0 msgr2=0x7f5d7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d96f11640 1 --2- 172.21.15.16:0/1898492414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7c0039a0 0x7f5d7c003dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009ac0 tx=0x7f5d7c0057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d96710640 1 -- 172.21.15.16:0/1898492414 reap_dead start 2023-11-13T10:56:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d96f11640 1 -- 172.21.15.16:0/1898492414 shutdown_connections 2023-11-13T10:56:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d96f11640 1 -- 172.21.15.16:0/1898492414 >> 172.21.15.16:0/1898492414 conn(0x7f5d900b04a0 msgr2=0x7f5d900b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d96f11640 1 -- 172.21.15.16:0/1898492414 shutdown_connections 2023-11-13T10:56:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.468+0000 7f5d96f11640 1 -- 172.21.15.16:0/1898492414 wait complete. 2023-11-13T10:56:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.470+0000 7f5d96f11640 1 Processor -- start 2023-11-13T10:56:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.470+0000 7f5d96f11640 1 -- start start 2023-11-13T10:56:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.471+0000 7f5d96f11640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d7c0039a0 0x7f5d900cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.471+0000 7f5d96f11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90159020 0x7f5d90169330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.471+0000 7f5d96f11640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d90147130 0x7f5d9006c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.471+0000 7f5d96f11640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d900ce4e0 con 0x7f5d90159020 2023-11-13T10:56:58.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.471+0000 7f5d96f11640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d90168920 con 0x7f5d7c0039a0 2023-11-13T10:56:58.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.471+0000 7f5d96f11640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d90168360 con 0x7f5d90147130 2023-11-13T10:56:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d9570e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90159020 0x7f5d90169330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d9570e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90159020 0x7f5d90169330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44658/0 (socket says 172.21.15.16:44658) 2023-11-13T10:56:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d9570e640 1 -- 172.21.15.16:0/3932507666 learned_addr learned my addr 172.21.15.16:0/3932507666 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:58.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d95f0f640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d7c0039a0 0x7f5d900cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d96710640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d90147130 0x7f5d9006c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d9570e640 1 -- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d90147130 msgr2=0x7f5d9006c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d9570e640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d90147130 0x7f5d9006c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d9570e640 1 -- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d7c0039a0 msgr2=0x7f5d900cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d9570e640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d7c0039a0 0x7f5d900cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d9570e640 1 -- 172.21.15.16:0/3932507666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d90159780 con 0x7f5d90159020 2023-11-13T10:56:58.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.472+0000 7f5d9570e640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90159020 0x7f5d90169330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000e580 tx=0x7f5d8000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:58.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.473+0000 7f5d867fc640 1 -- 172.21.15.16:0/3932507666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d800052e0 con 0x7f5d90159020 2023-11-13T10:56:58.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.473+0000 7f5d867fc640 1 -- 172.21.15.16:0/3932507666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d80005440 con 0x7f5d90159020 2023-11-13T10:56:58.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.473+0000 7f5d867fc640 1 -- 172.21.15.16:0/3932507666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80003810 con 0x7f5d90159020 2023-11-13T10:56:58.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.473+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d90159a60 con 0x7f5d90159020 2023-11-13T10:56:58.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.473+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d90169870 con 0x7f5d90159020 2023-11-13T10:56:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.474+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9004e4d0 con 0x7f5d90159020 2023-11-13T10:56:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.475+0000 7f5d867fc640 1 -- 172.21.15.16:0/3932507666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d80003970 con 0x7f5d90159020 2023-11-13T10:56:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.476+0000 7f5d867fc640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5d7406ae40 0x7f5d7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:58.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.476+0000 7f5d867fc640 1 -- 172.21.15.16:0/3932507666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f5d800896e0 con 0x7f5d90159020 2023-11-13T10:56:58.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.476+0000 7f5d95f0f640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5d7406ae40 0x7f5d7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:58.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.477+0000 7f5d95f0f640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5d7406ae40 0x7f5d7406d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c036000 tx=0x7f5d8c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.480+0000 7f5d867fc640 1 -- 172.21.15.16:0/3932507666 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d80054440 con 0x7f5d90159020 2023-11-13T10:56:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.689+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5d9006c220 con 0x7f5d90159020 2023-11-13T10:56:58.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.689+0000 7f5d867fc640 1 -- 172.21.15.16:0/3932507666 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5d80057a60 con 0x7f5d90159020 2023-11-13T10:56:58.693 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:56:58.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5d7406ae40 msgr2=0x7f5d7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5d7406ae40 0x7f5d7406d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c036000 tx=0x7f5d8c030000 comp rx=0 tx=0).stop 2023-11-13T10:56:58.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90159020 msgr2=0x7f5d90169330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:58.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90159020 0x7f5d90169330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000e580 tx=0x7f5d8000ea40 comp rx=0 tx=0).stop 2023-11-13T10:56:58.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 shutdown_connections 2023-11-13T10:56:58.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5d7406ae40 0x7f5d7406d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d90147130 0x7f5d9006c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90159020 0x7f5d90169330 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 --2- 172.21.15.16:0/3932507666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d7c0039a0 0x7f5d900cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:58.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 >> 172.21.15.16:0/3932507666 conn(0x7f5d900b04a0 msgr2=0x7f5d9015bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:58.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 shutdown_connections 2023-11-13T10:56:58.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:58.692+0000 7f5d96f11640 1 -- 172.21.15.16:0/3932507666 wait complete. 2023-11-13T10:56:58.716 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T10:56:58.716 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:56:58.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:56:59.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.023+0000 7f365b1c9640 1 Processor -- start 2023-11-13T10:56:59.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.023+0000 7f365b1c9640 1 -- start start 2023-11-13T10:56:59.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.025+0000 7f365b1c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36540d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.025+0000 7f365b1c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36540cfe20 0x7f36540d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.025+0000 7f365b1c9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36540d3c70 con 0x7f36540d0ac0 2023-11-13T10:56:59.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.025+0000 7f365b1c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36540c69d0 con 0x7f36540cfe20 2023-11-13T10:56:59.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.025+0000 7f365b1c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36540c6cb0 con 0x7f365415d0f0 2023-11-13T10:56:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.025+0000 7f3653fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36540cfe20 0x7f36540d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.025+0000 7f3653fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36540cfe20 0x7f36540d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44664/0 (socket says 172.21.15.16:44664) 2023-11-13T10:56:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.025+0000 7f3653fff640 1 -- 172.21.15.16:0/2875371405 learned_addr learned my addr 172.21.15.16:0/2875371405 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.026+0000 7f3653fff640 1 -- 172.21.15.16:0/2875371405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 msgr2=0x7f36540d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.026+0000 7f3658f65640 1 --2- 172.21.15.16:0/2875371405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36540d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.026+0000 7f3653fff640 1 --2- 172.21.15.16:0/2875371405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36540d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.026+0000 7f3653fff640 1 -- 172.21.15.16:0/2875371405 >> v1:172.21.15.184:6789/0 conn(0x7f36540d0ac0 legacy=0x7f36540b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:59.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.026+0000 7f3653fff640 1 -- 172.21.15.16:0/2875371405 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36540c6f90 con 0x7f36540cfe20 2023-11-13T10:56:59.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.026+0000 7f3658f65640 1 --2- 172.21.15.16:0/2875371405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36540d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:59.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.027+0000 7f3653fff640 1 --2- 172.21.15.16:0/2875371405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 0x7f36540d1800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f363c009870 tx=0x7f363c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e58ca1d1519a0f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:59.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.027+0000 7f36537fe640 1 -- 172.21.15.16:0/2875371405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c030840 con 0x7f36540cfe20 2023-11-13T10:56:59.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.027+0000 7f36537fe640 1 -- 172.21.15.16:0/2875371405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c0309a0 con 0x7f36540cfe20 2023-11-13T10:56:59.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.027+0000 7f36537fe640 1 -- 172.21.15.16:0/2875371405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c030d30 con 0x7f36540cfe20 2023-11-13T10:56:59.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.028+0000 7f365b1c9640 1 -- 172.21.15.16:0/2875371405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 msgr2=0x7f36540d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.028+0000 7f365b1c9640 1 --2- 172.21.15.16:0/2875371405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 0x7f36540d1800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f363c009870 tx=0x7f363c02b980 comp rx=0 tx=0).stop 2023-11-13T10:56:59.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.028+0000 7f365b1c9640 1 -- 172.21.15.16:0/2875371405 shutdown_connections 2023-11-13T10:56:59.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.028+0000 7f365b1c9640 1 --2- 172.21.15.16:0/2875371405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 0x7f36540d1800 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.028+0000 7f365b1c9640 1 --2- 172.21.15.16:0/2875371405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36540d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.028+0000 7f365b1c9640 1 -- 172.21.15.16:0/2875371405 >> 172.21.15.16:0/2875371405 conn(0x7f36540d2ce0 msgr2=0x7f36540d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:59.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.028+0000 7f365b1c9640 1 -- 172.21.15.16:0/2875371405 shutdown_connections 2023-11-13T10:56:59.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.028+0000 7f365b1c9640 1 -- 172.21.15.16:0/2875371405 wait complete. 2023-11-13T10:56:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.029+0000 7f365b1c9640 1 Processor -- start 2023-11-13T10:56:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.030+0000 7f365b1c9640 1 -- start start 2023-11-13T10:56:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f365b1c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 0x7f365415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f365b1c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36540d0ac0 0x7f36541e7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f365b1c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36541e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f365b1c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36541e6150 con 0x7f36540cfe20 2023-11-13T10:56:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f365b1c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365415c400 con 0x7f36540d0ac0 2023-11-13T10:56:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f365b1c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36541e7ff0 con 0x7f365415d0f0 2023-11-13T10:56:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3658f65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 0x7f365415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3658f65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 0x7f365415b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44674/0 (socket says 172.21.15.16:44674) 2023-11-13T10:56:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3658f65640 1 -- 172.21.15.16:0/846537484 learned_addr learned my addr 172.21.15.16:0/846537484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3658f65640 1 -- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 msgr2=0x7f36541e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3653fff640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36540d0ac0 0x7f36541e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3659766640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36541e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3658f65640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36541e79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3658f65640 1 -- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36540d0ac0 msgr2=0x7f36541e7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3658f65640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36540d0ac0 0x7f36541e7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3658f65640 1 -- 172.21.15.16:0/846537484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0094d0 con 0x7f36540cfe20 2023-11-13T10:56:59.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3653fff640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36540d0ac0 0x7f36541e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.031+0000 7f3659766640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36541e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:56:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.032+0000 7f3658f65640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 0x7f365415b850 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f364400ce00 tx=0x7f364400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:59.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.032+0000 7f36517fa640 1 -- 172.21.15.16:0/846537484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364400f890 con 0x7f36540cfe20 2023-11-13T10:56:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.032+0000 7f36517fa640 1 -- 172.21.15.16:0/846537484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364400f9f0 con 0x7f36540cfe20 2023-11-13T10:56:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.032+0000 7f36517fa640 1 -- 172.21.15.16:0/846537484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364400fdc0 con 0x7f36540cfe20 2023-11-13T10:56:59.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.032+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36541e5820 con 0x7f36540cfe20 2023-11-13T10:56:59.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.032+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36541e5de0 con 0x7f36540cfe20 2023-11-13T10:56:59.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.033+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3654067090 con 0x7f36540cfe20 2023-11-13T10:56:59.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.034+0000 7f36517fa640 1 -- 172.21.15.16:0/846537484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f364400b040 con 0x7f36540cfe20 2023-11-13T10:56:59.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.035+0000 7f36517fa640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f363406ae40 0x7f363406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.035+0000 7f36517fa640 1 -- 172.21.15.16:0/846537484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f364408a2f0 con 0x7f36540cfe20 2023-11-13T10:56:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.036+0000 7f3653fff640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f363406ae40 0x7f363406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.037+0000 7f3653fff640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f363406ae40 0x7f363406d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f363c036000 tx=0x7f363c03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:59.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.039+0000 7f36517fa640 1 -- 172.21.15.16:0/846537484 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3644055050 con 0x7f36540cfe20 2023-11-13T10:56:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.235+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36540d10a0 con 0x7f36540cfe20 2023-11-13T10:56:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.238+0000 7f36517fa640 1 -- 172.21.15.16:0/846537484 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f3644058670 con 0x7f36540cfe20 2023-11-13T10:56:59.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:56:59.249 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:56:59.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.240+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f363406ae40 msgr2=0x7f363406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.240+0000 7f365b1c9640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f363406ae40 0x7f363406d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f363c036000 tx=0x7f363c03b040 comp rx=0 tx=0).stop 2023-11-13T10:56:59.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.240+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 msgr2=0x7f365415b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.240+0000 7f365b1c9640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 0x7f365415b850 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f364400ce00 tx=0x7f364400a650 comp rx=0 tx=0).stop 2023-11-13T10:56:59.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.241+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 shutdown_connections 2023-11-13T10:56:59.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.241+0000 7f365b1c9640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f363406ae40 0x7f363406d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.241+0000 7f365b1c9640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f365415d0f0 0x7f36541e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.241+0000 7f365b1c9640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36540d0ac0 0x7f36541e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.241+0000 7f365b1c9640 1 --2- 172.21.15.16:0/846537484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36540cfe20 0x7f365415b850 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.241+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 >> 172.21.15.16:0/846537484 conn(0x7f36540d2ce0 msgr2=0x7f365406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:59.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.241+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 shutdown_connections 2023-11-13T10:56:59.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.241+0000 7f365b1c9640 1 -- 172.21.15.16:0/846537484 wait complete. 2023-11-13T10:56:59.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:56:59.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:56:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.492+0000 7f01d2eec640 1 Processor -- start 2023-11-13T10:56:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.492+0000 7f01d2eec640 1 -- start start 2023-11-13T10:56:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.494+0000 7f01d2eec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f01cc0d2ed0 0x7f01cc167a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.494+0000 7f01d2eec640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01cc0cac40 con 0x7f01cc0d1850 2023-11-13T10:56:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.494+0000 7f01d2eec640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01cc0ce0e0 con 0x7f01cc15ce70 2023-11-13T10:56:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.494+0000 7f01d2eec640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01cc0ce3c0 con 0x7f01cc0d2ed0 2023-11-13T10:56:59.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.494+0000 7f01d16e9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f01cc0d1850 0x7f01cc0d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36506/0 (socket says 172.21.15.16:36506) 2023-11-13T10:56:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.494+0000 7f01d16e9640 1 -- 172.21.15.16:0/3302001022 learned_addr learned my addr 172.21.15.16:0/3302001022 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.494+0000 7f01d26eb640 1 --2- 172.21.15.16:0/3302001022 >> v2:172.21.15.184:3300/0 conn(0x7f01cc0d2ed0 0x7f01cc167a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404324420 0 0) 0x7f01cc0cac40 con 0x7f01cc0d1850 2023-11-13T10:56:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b4003520 con 0x7f01cc0d1850 2023-11-13T10:56:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4256078046 0 0) 0x7f01b4003520 con 0x7f01cc0d1850 2023-11-13T10:56:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 >> v2:172.21.15.184:3300/0 conn(0x7f01cc0d2ed0 msgr2=0x7f01cc167a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 --2- 172.21.15.16:0/3302001022 >> v2:172.21.15.184:3300/0 conn(0x7f01cc0d2ed0 0x7f01cc167a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 >> v1:172.21.15.184:6789/0 conn(0x7f01cc15ce70 legacy=0x7f01cc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0ce6a0 con 0x7f01cc0d1850 2023-11-13T10:56:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f01c0001670 con 0x7f01cc0d1850 2023-11-13T10:56:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 >> v1:172.21.15.16:6789/0 conn(0x7f01cc0d1850 legacy=0x7f01cc0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01b40039c0 0x7f01b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b4005180 0x7f01b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0ce3c0 con 0x7f01b40047c0 2023-11-13T10:56:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b400c080 con 0x7f01b4005180 2023-11-13T10:56:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.495+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b400c240 con 0x7f01b40039c0 2023-11-13T10:56:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.496+0000 7f01d16e9640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.496+0000 7f01d26eb640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b4005180 0x7f01b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.496+0000 7f01d16e9640 1 -- 172.21.15.16:0/3302001022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01b40039c0 msgr2=0x7f01b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.496+0000 7f01d16e9640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01b40039c0 0x7f01b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.496+0000 7f01d16e9640 1 -- 172.21.15.16:0/3302001022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b4005180 msgr2=0x7f01b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:56:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.496+0000 7f01d16e9640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b4005180 0x7f01b400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.496+0000 7f01d16e9640 1 -- 172.21.15.16:0/3302001022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4003520 con 0x7f01b40047c0 2023-11-13T10:56:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.496+0000 7f01d26eb640 1 -- 172.21.15.16:0/3302001022 reap_dead start 2023-11-13T10:56:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.497+0000 7f01d16e9640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01b4004c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f01c000c0e0 tx=0x7f01b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d881928ae38a764 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.497+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0005ad0 con 0x7f01b40047c0 2023-11-13T10:56:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.497+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c0005c30 con 0x7f01b40047c0 2023-11-13T10:56:59.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.497+0000 7f01d0ee8640 1 -- 172.21.15.16:0/3302001022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c002ca50 con 0x7f01b40047c0 2023-11-13T10:56:59.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.498+0000 7f01d2eec640 1 -- 172.21.15.16:0/3302001022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 msgr2=0x7f01b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.498+0000 7f01d2eec640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01b4004c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f01c000c0e0 tx=0x7f01b40057f0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.498+0000 7f01d2eec640 1 -- 172.21.15.16:0/3302001022 shutdown_connections 2023-11-13T10:56:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.498+0000 7f01d2eec640 1 --2- 172.21.15.16:0/3302001022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01b4004c40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.498+0000 7f01d2eec640 1 -- 172.21.15.16:0/3302001022 >> 172.21.15.16:0/3302001022 conn(0x7f01cc08c580 msgr2=0x7f01cc09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.498+0000 7f01d2eec640 1 -- 172.21.15.16:0/3302001022 shutdown_connections 2023-11-13T10:56:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.498+0000 7f01d2eec640 1 -- 172.21.15.16:0/3302001022 wait complete. 2023-11-13T10:56:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.500+0000 7f01d2eec640 1 Processor -- start 2023-11-13T10:56:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.500+0000 7f01d2eec640 1 -- start start 2023-11-13T10:56:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.501+0000 7f01d2eec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01cc1f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.501+0000 7f01d2eec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01cc1f3350 0x7f01cc15e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.501+0000 7f01d2eec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01cc0b3da0 0x7f01cc0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.501+0000 7f01d2eec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc15f510 con 0x7f01b40047c0 2023-11-13T10:56:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.501+0000 7f01d2eec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0b4760 con 0x7f01cc0b3da0 2023-11-13T10:56:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.501+0000 7f01d2eec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc1f24f0 con 0x7f01cc1f3350 2023-11-13T10:56:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d1eea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01cc1f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d1eea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01cc1f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44690/0 (socket says 172.21.15.16:44690) 2023-11-13T10:56:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d1eea640 1 -- 172.21.15.16:0/3399481803 learned_addr learned my addr 172.21.15.16:0/3399481803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:56:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d16e9640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01cc1f3350 0x7f01cc15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d26eb640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01cc0b3da0 0x7f01cc0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d1eea640 1 -- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01cc1f3350 msgr2=0x7f01cc15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d1eea640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01cc1f3350 0x7f01cc15e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d1eea640 1 -- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01cc0b3da0 msgr2=0x7f01cc0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d1eea640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01cc0b3da0 0x7f01cc0b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d1eea640 1 -- 172.21.15.16:0/3399481803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc15ec70 con 0x7f01b40047c0 2023-11-13T10:56:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.502+0000 7f01d1eea640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01cc1f1fb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00c430 tx=0x7f01bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.503+0000 7f01ca7fc640 1 -- 172.21.15.16:0/3399481803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0052d0 con 0x7f01b40047c0 2023-11-13T10:56:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.503+0000 7f01ca7fc640 1 -- 172.21.15.16:0/3399481803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc005450 con 0x7f01b40047c0 2023-11-13T10:56:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.503+0000 7f01ca7fc640 1 -- 172.21.15.16:0/3399481803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc003830 con 0x7f01b40047c0 2023-11-13T10:56:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.503+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc15ef50 con 0x7f01b40047c0 2023-11-13T10:56:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.503+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc1f3a00 con 0x7f01b40047c0 2023-11-13T10:56:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.504+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc1f3bc0 con 0x7f01b40047c0 2023-11-13T10:56:59.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.505+0000 7f01ca7fc640 1 -- 172.21.15.16:0/3399481803 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01bc003990 con 0x7f01b40047c0 2023-11-13T10:56:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.506+0000 7f01ca7fc640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f01b806ae70 0x7f01b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.506+0000 7f01ca7fc640 1 -- 172.21.15.16:0/3399481803 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f01bc088b20 con 0x7f01b40047c0 2023-11-13T10:56:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.507+0000 7f01d16e9640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f01b806ae70 0x7f01b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.508+0000 7f01d16e9640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f01b806ae70 0x7f01b806d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f01c0034000 tx=0x7f01c0009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:59.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.510+0000 7f01ca7fc640 1 -- 172.21.15.16:0/3399481803 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01bc057270 con 0x7f01b40047c0 2023-11-13T10:56:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.708+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01cc1f3860 con 0x7f01b40047c0 2023-11-13T10:56:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.709+0000 7f01ca7fc640 1 -- 172.21.15.16:0/3399481803 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01cc1f3860 con 0x7f01b40047c0 2023-11-13T10:56:59.712 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:56:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f01b806ae70 msgr2=0x7f01b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f01b806ae70 0x7f01b806d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f01c0034000 tx=0x7f01c0009030 comp rx=0 tx=0).stop 2023-11-13T10:56:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 msgr2=0x7f01cc1f1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01cc1f1fb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00c430 tx=0x7f01bc00c8f0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 shutdown_connections 2023-11-13T10:56:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f01b806ae70 0x7f01b806d330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01cc0b3da0 0x7f01cc0b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01cc1f3350 0x7f01cc15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 --2- 172.21.15.16:0/3399481803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b40047c0 0x7f01cc1f1fb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.711+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 >> 172.21.15.16:0/3399481803 conn(0x7f01cc08c580 msgr2=0x7f01cc15c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.712+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 shutdown_connections 2023-11-13T10:56:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.712+0000 7f01d2eec640 1 -- 172.21.15.16:0/3399481803 wait complete. 2023-11-13T10:56:59.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:56:59.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.974+0000 7f426a747640 1 Processor -- start 2023-11-13T10:56:59.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.974+0000 7f426a747640 1 -- start start 2023-11-13T10:56:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.975+0000 7f426a747640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f426415ce70 0x7f42640cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:56:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.976+0000 7f426a747640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42640ce0e0 con 0x7f42640cdbb0 2023-11-13T10:56:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.976+0000 7f426a747640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42640ce3c0 con 0x7f42640d1850 2023-11-13T10:56:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.976+0000 7f426a747640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42640ce6a0 con 0x7f426415ce70 2023-11-13T10:56:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.976+0000 7f4268f44640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f42640cdbb0 0x7f42640d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36520/0 (socket says 172.21.15.16:36520) 2023-11-13T10:56:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.976+0000 7f4268f44640 1 -- 172.21.15.16:0/3372012489 learned_addr learned my addr 172.21.15.16:0/3372012489 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:56:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.976+0000 7f4269745640 1 --2- 172.21.15.16:0/3372012489 >> v2:172.21.15.16:3300/0 conn(0x7f426415ce70 0x7f42640cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:56:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.977+0000 7f425bfff640 1 -- 172.21.15.16:0/3372012489 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783812947 0 0) 0x7f42640ce0e0 con 0x7f42640cdbb0 2023-11-13T10:56:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.977+0000 7f425bfff640 1 -- 172.21.15.16:0/3372012489 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4250003520 con 0x7f42640cdbb0 2023-11-13T10:56:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.977+0000 7f4269745640 1 -- 172.21.15.16:0/3372012489 >> v1:172.21.15.184:6789/0 conn(0x7f42640d1850 legacy=0x7f42640c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.977+0000 7f4269745640 1 -- 172.21.15.16:0/3372012489 >> v1:172.21.15.16:6789/0 conn(0x7f42640cdbb0 legacy=0x7f42640d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.977+0000 7f4269745640 1 -- 172.21.15.16:0/3372012489 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42640ce980 con 0x7f426415ce70 2023-11-13T10:56:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.978+0000 7f4269745640 1 --2- 172.21.15.16:0/3372012489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426415ce70 0x7f42640cd670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4260009d90 tx=0x7f426002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2935d0f8f37e808c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:56:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.978+0000 7f425bfff640 1 -- 172.21.15.16:0/3372012489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426002cd00 con 0x7f426415ce70 2023-11-13T10:56:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.978+0000 7f425bfff640 1 -- 172.21.15.16:0/3372012489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426002ce60 con 0x7f426415ce70 2023-11-13T10:56:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.978+0000 7f425bfff640 1 -- 172.21.15.16:0/3372012489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260034660 con 0x7f426415ce70 2023-11-13T10:56:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.979+0000 7f426a747640 1 -- 172.21.15.16:0/3372012489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426415ce70 msgr2=0x7f42640cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:56:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.979+0000 7f426a747640 1 --2- 172.21.15.16:0/3372012489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426415ce70 0x7f42640cd670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4260009d90 tx=0x7f426002bf50 comp rx=0 tx=0).stop 2023-11-13T10:56:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.979+0000 7f426a747640 1 -- 172.21.15.16:0/3372012489 shutdown_connections 2023-11-13T10:56:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.979+0000 7f426a747640 1 --2- 172.21.15.16:0/3372012489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426415ce70 0x7f42640cd670 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:56:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.979+0000 7f426a747640 1 -- 172.21.15.16:0/3372012489 >> 172.21.15.16:0/3372012489 conn(0x7f426408c580 msgr2=0x7f426409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:56:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.979+0000 7f426a747640 1 -- 172.21.15.16:0/3372012489 shutdown_connections 2023-11-13T10:56:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.979+0000 7f426a747640 1 -- 172.21.15.16:0/3372012489 wait complete. 2023-11-13T10:56:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.981+0000 7f426a747640 1 Processor -- start 2023-11-13T10:56:59.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.981+0000 7f426a747640 1 -- start start 2023-11-13T10:57:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.982+0000 7f426a747640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42640cdbb0 0x7f426415c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.982+0000 7f426a747640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42640d1850 0x7f42640c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.982+0000 7f426a747640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f426415ce70 0x7f42640b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.982+0000 7f426a747640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c7a90 con 0x7f42640d1850 2023-11-13T10:57:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.982+0000 7f426a747640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c7d70 con 0x7f426415ce70 2023-11-13T10:57:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.982+0000 7f426a747640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42640c8050 con 0x7f42640cdbb0 2023-11-13T10:57:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.982+0000 7f4268f44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42640d1850 0x7f42640c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.982+0000 7f4268f44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42640d1850 0x7f42640c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44702/0 (socket says 172.21.15.16:44702) 2023-11-13T10:57:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.982+0000 7f4268f44640 1 -- 172.21.15.16:0/3999561823 learned_addr learned my addr 172.21.15.16:0/3999561823 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4269745640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42640cdbb0 0x7f426415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4268f44640 1 -- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42640cdbb0 msgr2=0x7f426415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4268f44640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42640cdbb0 0x7f426415c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4268f44640 1 -- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f426415ce70 msgr2=0x7f42640b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4268f44640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f426415ce70 0x7f42640b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4268f44640 1 -- 172.21.15.16:0/3999561823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260009a70 con 0x7f42640d1850 2023-11-13T10:57:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4269745640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42640cdbb0 0x7f426415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4268f44640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42640d1850 0x7f42640c7040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f425400c4d0 tx=0x7f425400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4259ffb640 1 -- 172.21.15.16:0/3999561823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254005ce0 con 0x7f42640d1850 2023-11-13T10:57:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4259ffb640 1 -- 172.21.15.16:0/3999561823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4254005e40 con 0x7f42640d1850 2023-11-13T10:57:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.983+0000 7f4259ffb640 1 -- 172.21.15.16:0/3999561823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254007270 con 0x7f42640d1850 2023-11-13T10:57:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.984+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42640c8330 con 0x7f42640d1850 2023-11-13T10:57:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.984+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426409d9a0 con 0x7f42640d1850 2023-11-13T10:57:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.985+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426409dbf0 con 0x7f42640d1850 2023-11-13T10:57:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.986+0000 7f4259ffb640 1 -- 172.21.15.16:0/3999561823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4254007490 con 0x7f42640d1850 2023-11-13T10:57:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.987+0000 7f4259ffb640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424806ae40 0x7f424806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.987+0000 7f4259ffb640 1 -- 172.21.15.16:0/3999561823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f4254088ee0 con 0x7f42640d1850 2023-11-13T10:57:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.987+0000 7f4269745640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424806ae40 0x7f424806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.989+0000 7f4269745640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424806ae40 0x7f424806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4260036000 tx=0x7f4260030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:00.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:56:59.991+0000 7f4259ffb640 1 -- 172.21.15.16:0/3999561823 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4254053c60 con 0x7f42640d1850 2023-11-13T10:57:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.184+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f426404e4d0 con 0x7f42640d1850 2023-11-13T10:57:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.185+0000 7f4259ffb640 1 -- 172.21.15.16:0/3999561823 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4254057280 con 0x7f42640d1850 2023-11-13T10:57:00.189 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:57:00.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.187+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424806ae40 msgr2=0x7f424806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.187+0000 7f426a747640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424806ae40 0x7f424806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4260036000 tx=0x7f4260030000 comp rx=0 tx=0).stop 2023-11-13T10:57:00.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.187+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42640d1850 msgr2=0x7f42640c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.187+0000 7f426a747640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42640d1850 0x7f42640c7040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f425400c4d0 tx=0x7f425400c990 comp rx=0 tx=0).stop 2023-11-13T10:57:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.188+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 shutdown_connections 2023-11-13T10:57:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.188+0000 7f426a747640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f424806ae40 0x7f424806d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.188+0000 7f426a747640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f426415ce70 0x7f42640b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.188+0000 7f426a747640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42640d1850 0x7f42640c7040 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.188+0000 7f426a747640 1 --2- 172.21.15.16:0/3999561823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42640cdbb0 0x7f426415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.188+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 >> 172.21.15.16:0/3999561823 conn(0x7f426408c580 msgr2=0x7f426409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:00.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.188+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 shutdown_connections 2023-11-13T10:57:00.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.188+0000 7f426a747640 1 -- 172.21.15.16:0/3999561823 wait complete. 2023-11-13T10:57:00.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.434+0000 7fc2624d1640 1 Processor -- start 2023-11-13T10:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.434+0000 7fc2624d1640 1 -- start start 2023-11-13T10:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.436+0000 7fc2624d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc25c15b8e0 0x7fc25c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.436+0000 7fc2624d1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c06b860 con 0x7fc25c0ce260 2023-11-13T10:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.436+0000 7fc2624d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c06bb40 con 0x7fc25c15bcc0 2023-11-13T10:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.436+0000 7fc2624d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c06be20 con 0x7fc25c15b8e0 2023-11-13T10:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.437+0000 7fc261cd0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc25c0ce260 0x7fc25c0cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36530/0 (socket says 172.21.15.16:36530) 2023-11-13T10:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.437+0000 7fc261cd0640 1 -- 172.21.15.16:0/3588901861 learned_addr learned my addr 172.21.15.16:0/3588901861 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.437+0000 7fc2614cf640 1 --2- 172.21.15.16:0/3588901861 >> v2:172.21.15.184:3300/0 conn(0x7fc25c15b8e0 0x7fc25c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.437+0000 7fc253fff640 1 -- 172.21.15.16:0/3588901861 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2555291991 0 0) 0x7fc25c06b860 con 0x7fc25c0ce260 2023-11-13T10:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.437+0000 7fc253fff640 1 -- 172.21.15.16:0/3588901861 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc248003520 con 0x7fc25c0ce260 2023-11-13T10:57:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.438+0000 7fc253fff640 1 -- 172.21.15.16:0/3588901861 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1372023952 0 0) 0x7fc248003520 con 0x7fc25c0ce260 2023-11-13T10:57:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.438+0000 7fc2614cf640 1 -- 172.21.15.16:0/3588901861 >> v1:172.21.15.184:6789/0 conn(0x7fc25c15bcc0 legacy=0x7fc25c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.438+0000 7fc2614cf640 1 -- 172.21.15.16:0/3588901861 >> v1:172.21.15.16:6789/0 conn(0x7fc25c0ce260 legacy=0x7fc25c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.438+0000 7fc2614cf640 1 -- 172.21.15.16:0/3588901861 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c06c100 con 0x7fc25c15b8e0 2023-11-13T10:57:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.439+0000 7fc2614cf640 1 --2- 172.21.15.16:0/3588901861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc25c15b8e0 0x7fc25c06c900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc2580097a0 tx=0x7fc25802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc5429f641d0b73f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.439+0000 7fc253fff640 1 -- 172.21.15.16:0/3588901861 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25802f050 con 0x7fc25c15b8e0 2023-11-13T10:57:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.440+0000 7fc253fff640 1 -- 172.21.15.16:0/3588901861 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc258031500 con 0x7fc25c15b8e0 2023-11-13T10:57:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.440+0000 7fc253fff640 1 -- 172.21.15.16:0/3588901861 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2540019f0 con 0x7fc25c15b8e0 2023-11-13T10:57:00.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.440+0000 7fc2624d1640 1 -- 172.21.15.16:0/3588901861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc25c15b8e0 msgr2=0x7fc25c06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.440+0000 7fc2624d1640 1 --2- 172.21.15.16:0/3588901861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc25c15b8e0 0x7fc25c06c900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc2580097a0 tx=0x7fc25802b820 comp rx=0 tx=0).stop 2023-11-13T10:57:00.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.440+0000 7fc2624d1640 1 -- 172.21.15.16:0/3588901861 shutdown_connections 2023-11-13T10:57:00.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.440+0000 7fc2624d1640 1 --2- 172.21.15.16:0/3588901861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc25c15b8e0 0x7fc25c06c900 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.440+0000 7fc2624d1640 1 -- 172.21.15.16:0/3588901861 >> 172.21.15.16:0/3588901861 conn(0x7fc25c0b0970 msgr2=0x7fc25c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.441+0000 7fc2624d1640 1 -- 172.21.15.16:0/3588901861 shutdown_connections 2023-11-13T10:57:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.441+0000 7fc2624d1640 1 -- 172.21.15.16:0/3588901861 wait complete. 2023-11-13T10:57:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.443+0000 7fc2624d1640 1 Processor -- start 2023-11-13T10:57:00.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.443+0000 7fc2624d1640 1 -- start start 2023-11-13T10:57:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.444+0000 7fc2624d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc25c0ce260 0x7fc25c146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.444+0000 7fc2624d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c15b8e0 0x7fc25c1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc2624d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc25c153050 0x7fc25c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc2624d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c146650 con 0x7fc25c15b8e0 2023-11-13T10:57:00.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc2624d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c154550 con 0x7fc25c0ce260 2023-11-13T10:57:00.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc2624d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c154830 con 0x7fc25c153050 2023-11-13T10:57:00.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc260cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c15b8e0 0x7fc25c1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc260cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c15b8e0 0x7fc25c1555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44714/0 (socket says 172.21.15.16:44714) 2023-11-13T10:57:00.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc260cce640 1 -- 172.21.15.16:0/2641536382 learned_addr learned my addr 172.21.15.16:0/2641536382 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:00.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc2614cf640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc25c0ce260 0x7fc25c146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc260cce640 1 -- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc25c153050 msgr2=0x7fc25c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc260cce640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc25c153050 0x7fc25c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc260cce640 1 -- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc25c0ce260 msgr2=0x7fc25c146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc260cce640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc25c0ce260 0x7fc25c146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.445+0000 7fc260cce640 1 -- 172.21.15.16:0/2641536382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc258009400 con 0x7fc25c15b8e0 2023-11-13T10:57:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.446+0000 7fc260cce640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c15b8e0 0x7fc25c1555f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00c400 tx=0x7fc24c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.446+0000 7fc251ffb640 1 -- 172.21.15.16:0/2641536382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0052d0 con 0x7fc25c15b8e0 2023-11-13T10:57:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.446+0000 7fc251ffb640 1 -- 172.21.15.16:0/2641536382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c005430 con 0x7fc25c15b8e0 2023-11-13T10:57:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.446+0000 7fc251ffb640 1 -- 172.21.15.16:0/2641536382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c003830 con 0x7fc25c15b8e0 2023-11-13T10:57:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.446+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c154b10 con 0x7fc25c15b8e0 2023-11-13T10:57:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.446+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c1550d0 con 0x7fc25c15b8e0 2023-11-13T10:57:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.448+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c04e4d0 con 0x7fc25c15b8e0 2023-11-13T10:57:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.448+0000 7fc251ffb640 1 -- 172.21.15.16:0/2641536382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc24c003990 con 0x7fc25c15b8e0 2023-11-13T10:57:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.449+0000 7fc251ffb640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc24006ae40 0x7fc24006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.449+0000 7fc251ffb640 1 -- 172.21.15.16:0/2641536382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fc24c088f70 con 0x7fc25c15b8e0 2023-11-13T10:57:00.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.450+0000 7fc2614cf640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc24006ae40 0x7fc24006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.451+0000 7fc2614cf640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc24006ae40 0x7fc24006d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc258035000 tx=0x7fc258008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:00.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.456+0000 7fc251ffb640 1 -- 172.21.15.16:0/2641536382 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc24c0bbe90 con 0x7fc25c15b8e0 2023-11-13T10:57:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.647+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc25c0b18c0 con 0x7fc25c15b8e0 2023-11-13T10:57:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.648+0000 7fc251ffb640 1 -- 172.21.15.16:0/2641536382 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc24c074e60 con 0x7fc25c15b8e0 2023-11-13T10:57:00.652 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:57:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc24006ae40 msgr2=0x7fc24006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc24006ae40 0x7fc24006d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc258035000 tx=0x7fc258008030 comp rx=0 tx=0).stop 2023-11-13T10:57:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c15b8e0 msgr2=0x7fc25c1555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c15b8e0 0x7fc25c1555f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00c400 tx=0x7fc24c00c8c0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 shutdown_connections 2023-11-13T10:57:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc24006ae40 0x7fc24006d300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc25c153050 0x7fc25c0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c15b8e0 0x7fc25c1555f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 --2- 172.21.15.16:0/2641536382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc25c0ce260 0x7fc25c146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 >> 172.21.15.16:0/2641536382 conn(0x7fc25c0b0970 msgr2=0x7fc25c15a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.651+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 shutdown_connections 2023-11-13T10:57:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.652+0000 7fc2624d1640 1 -- 172.21.15.16:0/2641536382 wait complete. 2023-11-13T10:57:00.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:57:00.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.898+0000 7febb2fa7640 1 Processor -- start 2023-11-13T10:57:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.899+0000 7febb2fa7640 1 -- start start 2023-11-13T10:57:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.900+0000 7febb2fa7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac0ce260 0x7febac0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.900+0000 7febb2fa7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac06d010 con 0x7febac15b8e0 2023-11-13T10:57:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.900+0000 7febb2fa7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac06b860 con 0x7febac15bcc0 2023-11-13T10:57:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.900+0000 7febb2fa7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac06bb40 con 0x7febac0ce260 2023-11-13T10:57:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.900+0000 7febb1fa5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7febac15b8e0 0x7febac06c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36546/0 (socket says 172.21.15.16:36546) 2023-11-13T10:57:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.900+0000 7febb1fa5640 1 -- 172.21.15.16:0/4263141154 learned_addr learned my addr 172.21.15.16:0/4263141154 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:00.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb27a6640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac0ce260 0x7febac0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197670643 0 0) 0x7febac06d010 con 0x7febac15b8e0 2023-11-13T10:57:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb98003520 con 0x7febac15b8e0 2023-11-13T10:57:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2843108261 0 0) 0x7feb98003520 con 0x7febac15b8e0 2023-11-13T10:57:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac0ce260 msgr2=0x7febac0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:00.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac0ce260 0x7febac0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 >> v1:172.21.15.184:6789/0 conn(0x7febac15bcc0 legacy=0x7febac0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac06be20 con 0x7febac15b8e0 2023-11-13T10:57:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7feba8001670 con 0x7febac15b8e0 2023-11-13T10:57:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 >> v1:172.21.15.16:6789/0 conn(0x7febac15b8e0 legacy=0x7febac06c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac0ce260 0x7feb980039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98003f70 0x7feb980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.901+0000 7febb0fa3640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb98004d00 0x7feb98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac06bb40 con 0x7feb98004d00 2023-11-13T10:57:00.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9800bac0 con 0x7feb98003f70 2023-11-13T10:57:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9800bc80 con 0x7febac0ce260 2023-11-13T10:57:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb27a6640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb98004d00 0x7feb98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb1fa5640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac0ce260 0x7feb980039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb17a4640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98003f70 0x7feb980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb27a6640 1 -- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac0ce260 msgr2=0x7feb980039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb27a6640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac0ce260 0x7feb980039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb27a6640 1 -- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98003f70 msgr2=0x7feb980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb27a6640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98003f70 0x7feb980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb27a6640 1 -- 172.21.15.16:0/4263141154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb98003520 con 0x7feb98004d00 2023-11-13T10:57:00.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.902+0000 7febb17a4640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98003f70 0x7feb980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.903+0000 7febb27a6640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb98004d00 0x7feb98005180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feba4009fc0 tx=0x7feb980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb4cc3ca6c7b5b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba402cc20 con 0x7feb98004d00 2023-11-13T10:57:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba400f040 con 0x7feb98004d00 2023-11-13T10:57:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb0fa3640 1 -- 172.21.15.16:0/4263141154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4034440 con 0x7feb98004d00 2023-11-13T10:57:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb2fa7640 1 -- 172.21.15.16:0/4263141154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb98004d00 msgr2=0x7feb98005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb2fa7640 1 --2- 172.21.15.16:0/4263141154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb98004d00 0x7feb98005180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feba4009fc0 tx=0x7feb980057f0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb27a6640 1 -- 172.21.15.16:0/4263141154 reap_dead start 2023-11-13T10:57:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb2fa7640 1 -- 172.21.15.16:0/4263141154 shutdown_connections 2023-11-13T10:57:00.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb2fa7640 1 -- 172.21.15.16:0/4263141154 >> 172.21.15.16:0/4263141154 conn(0x7febac0b0970 msgr2=0x7febac0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb2fa7640 1 -- 172.21.15.16:0/4263141154 shutdown_connections 2023-11-13T10:57:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.904+0000 7febb2fa7640 1 -- 172.21.15.16:0/4263141154 wait complete. 2023-11-13T10:57:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.906+0000 7febb2fa7640 1 Processor -- start 2023-11-13T10:57:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.906+0000 7febb2fa7640 1 -- start start 2023-11-13T10:57:00.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.907+0000 7febb2fa7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98004d00 0x7febac150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.907+0000 7febb2fa7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac151470 0x7febac1e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.907+0000 7febb2fa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febac1e56f0 0x7febac15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.907+0000 7febb2fa7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac1e4430 con 0x7febac1e56f0 2023-11-13T10:57:00.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.907+0000 7febb2fa7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac1e46d0 con 0x7feb98004d00 2023-11-13T10:57:00.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.907+0000 7febb2fa7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac1e49b0 con 0x7febac151470 2023-11-13T10:57:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb27a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febac1e56f0 0x7febac15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb27a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febac1e56f0 0x7febac15e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44746/0 (socket says 172.21.15.16:44746) 2023-11-13T10:57:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb27a6640 1 -- 172.21.15.16:0/1974433450 learned_addr learned my addr 172.21.15.16:0/1974433450 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb1fa5640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98004d00 0x7febac150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb27a6640 1 -- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac151470 msgr2=0x7febac1e51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb27a6640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac151470 0x7febac1e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb27a6640 1 -- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98004d00 msgr2=0x7febac150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb27a6640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98004d00 0x7febac150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb27a6640 1 -- 172.21.15.16:0/1974433450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac1e4c90 con 0x7febac1e56f0 2023-11-13T10:57:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.908+0000 7febb1fa5640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98004d00 0x7febac150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.909+0000 7febb27a6640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febac1e56f0 0x7febac15e2e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feba4002230 tx=0x7feba4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.909+0000 7feba27fc640 1 -- 172.21.15.16:0/1974433450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4034cd0 con 0x7febac1e56f0 2023-11-13T10:57:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.909+0000 7feba27fc640 1 -- 172.21.15.16:0/1974433450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba4034e30 con 0x7febac1e56f0 2023-11-13T10:57:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.909+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac15ddc0 con 0x7febac1e56f0 2023-11-13T10:57:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.909+0000 7feba27fc640 1 -- 172.21.15.16:0/1974433450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4003bd0 con 0x7febac1e56f0 2023-11-13T10:57:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.909+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac15d8e0 con 0x7febac1e56f0 2023-11-13T10:57:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.910+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac15db90 con 0x7febac1e56f0 2023-11-13T10:57:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.911+0000 7feba27fc640 1 -- 172.21.15.16:0/1974433450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feba4003d30 con 0x7febac1e56f0 2023-11-13T10:57:00.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.912+0000 7feba27fc640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb9006ae40 0x7feb9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:00.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.912+0000 7feba27fc640 1 -- 172.21.15.16:0/1974433450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7feba40ab200 con 0x7febac1e56f0 2023-11-13T10:57:00.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.913+0000 7febb1fa5640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb9006ae40 0x7feb9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:00.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.914+0000 7febb1fa5640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb9006ae40 0x7feb9006d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feba8001eb0 tx=0x7feba8000f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:00.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:00.917+0000 7feba27fc640 1 -- 172.21.15.16:0/1974433450 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feba4075f70 con 0x7febac1e56f0 2023-11-13T10:57:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.112+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7febac04e4d0 con 0x7febac1e56f0 2023-11-13T10:57:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.113+0000 7feba27fc640 1 -- 172.21.15.16:0/1974433450 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7feba4079590 con 0x7febac1e56f0 2023-11-13T10:57:01.116 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:57:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.115+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb9006ae40 msgr2=0x7feb9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.115+0000 7febb2fa7640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb9006ae40 0x7feb9006d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feba8001eb0 tx=0x7feba8000f90 comp rx=0 tx=0).stop 2023-11-13T10:57:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.115+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febac1e56f0 msgr2=0x7febac15e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.115+0000 7febb2fa7640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febac1e56f0 0x7febac15e2e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feba4002230 tx=0x7feba4009750 comp rx=0 tx=0).stop 2023-11-13T10:57:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.116+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 shutdown_connections 2023-11-13T10:57:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.116+0000 7febb2fa7640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb9006ae40 0x7feb9006d300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.116+0000 7febb2fa7640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febac1e56f0 0x7febac15e2e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.116+0000 7febb2fa7640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7febac151470 0x7febac1e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.116+0000 7febb2fa7640 1 --2- 172.21.15.16:0/1974433450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb98004d00 0x7febac150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.116+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 >> 172.21.15.16:0/1974433450 conn(0x7febac0b0970 msgr2=0x7febac0cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.116+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 shutdown_connections 2023-11-13T10:57:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.116+0000 7febb2fa7640 1 -- 172.21.15.16:0/1974433450 wait complete. 2023-11-13T10:57:01.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T10:57:01.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.364+0000 7ff7d69ff640 1 Processor -- start 2023-11-13T10:57:01.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.365+0000 7ff7d69ff640 1 -- start start 2023-11-13T10:57:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.366+0000 7ff7d69ff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7d00cb190 0x7ff7d015b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.366+0000 7ff7d69ff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d0158c20 con 0x7ff7d00716b0 2023-11-13T10:57:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.367+0000 7ff7d69ff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d00b1830 con 0x7ff7d006a190 2023-11-13T10:57:01.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.367+0000 7ff7d69ff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d015b780 con 0x7ff7d00cb190 2023-11-13T10:57:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.367+0000 7ff7d59fd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7d00cb190 0x7ff7d015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.367+0000 7ff7d51fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff7d00716b0 0x7ff7d006f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36558/0 (socket says 172.21.15.16:36558) 2023-11-13T10:57:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.367+0000 7ff7d51fc640 1 -- 172.21.15.16:0/2967512673 learned_addr learned my addr 172.21.15.16:0/2967512673 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.367+0000 7ff7d59fd640 1 -- 172.21.15.16:0/2967512673 >> v1:172.21.15.184:6789/0 conn(0x7ff7d006a190 legacy=0x7ff7d00b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.367+0000 7ff7d59fd640 1 -- 172.21.15.16:0/2967512673 >> v1:172.21.15.16:6789/0 conn(0x7ff7d00716b0 legacy=0x7ff7d006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.367+0000 7ff7d59fd640 1 -- 172.21.15.16:0/2967512673 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d015ba60 con 0x7ff7d00cb190 2023-11-13T10:57:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.369+0000 7ff7d59fd640 1 --2- 172.21.15.16:0/2967512673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00cb190 0x7ff7d015b190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0097a0 tx=0x7ff7cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebde6c48ffb86c2e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:01.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.369+0000 7ff7d49fb640 1 -- 172.21.15.16:0/2967512673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc031830 con 0x7ff7d00cb190 2023-11-13T10:57:01.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.369+0000 7ff7d49fb640 1 -- 172.21.15.16:0/2967512673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc031990 con 0x7ff7d00cb190 2023-11-13T10:57:01.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.369+0000 7ff7d49fb640 1 -- 172.21.15.16:0/2967512673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc031d20 con 0x7ff7d00cb190 2023-11-13T10:57:01.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.369+0000 7ff7d69ff640 1 -- 172.21.15.16:0/2967512673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00cb190 msgr2=0x7ff7d015b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:01.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.369+0000 7ff7d69ff640 1 --2- 172.21.15.16:0/2967512673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00cb190 0x7ff7d015b190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0097a0 tx=0x7ff7cc02b8d0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.370+0000 7ff7d69ff640 1 -- 172.21.15.16:0/2967512673 shutdown_connections 2023-11-13T10:57:01.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.370+0000 7ff7d69ff640 1 --2- 172.21.15.16:0/2967512673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00cb190 0x7ff7d015b190 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.370+0000 7ff7d69ff640 1 -- 172.21.15.16:0/2967512673 >> 172.21.15.16:0/2967512673 conn(0x7ff7d00b04a0 msgr2=0x7ff7d00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:01.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.370+0000 7ff7d69ff640 1 -- 172.21.15.16:0/2967512673 shutdown_connections 2023-11-13T10:57:01.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.370+0000 7ff7d69ff640 1 -- 172.21.15.16:0/2967512673 wait complete. 2023-11-13T10:57:01.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.372+0000 7ff7d69ff640 1 Processor -- start 2023-11-13T10:57:01.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.372+0000 7ff7d69ff640 1 -- start start 2023-11-13T10:57:01.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d69ff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7d006a190 0x7ff7d014ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:01.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d69ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00716b0 0x7ff7d01f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:01.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d69ff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d00cb190 0x7ff7d00ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:01.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d69ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d006cda0 con 0x7ff7d00716b0 2023-11-13T10:57:01.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d69ff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d014d3b0 con 0x7ff7d00cb190 2023-11-13T10:57:01.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d69ff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d006bd00 con 0x7ff7d006a190 2023-11-13T10:57:01.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d51fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00716b0 0x7ff7d01f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d51fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00716b0 0x7ff7d01f6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44764/0 (socket says 172.21.15.16:44764) 2023-11-13T10:57:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d51fc640 1 -- 172.21.15.16:0/1860531193 learned_addr learned my addr 172.21.15.16:0/1860531193 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.374+0000 7ff7d59fd640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7d006a190 0x7ff7d014ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d51fc640 1 -- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7d006a190 msgr2=0x7ff7d014ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d51fc640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7d006a190 0x7ff7d014ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d51fc640 1 -- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d00cb190 msgr2=0x7ff7d00ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:01.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d61fe640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d00cb190 0x7ff7d00ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:01.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d51fc640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d00cb190 0x7ff7d00ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d51fc640 1 -- 172.21.15.16:0/1860531193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc009400 con 0x7ff7d00716b0 2023-11-13T10:57:01.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d61fe640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d00cb190 0x7ff7d00ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:01.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d51fc640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00716b0 0x7ff7d01f6d00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c4a0 tx=0x7ff7c000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:01.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7c67fc640 1 -- 172.21.15.16:0/1860531193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0005ce0 con 0x7ff7d00716b0 2023-11-13T10:57:01.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7c67fc640 1 -- 172.21.15.16:0/1860531193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c0005e40 con 0x7ff7d00716b0 2023-11-13T10:57:01.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7c67fc640 1 -- 172.21.15.16:0/1860531193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0007270 con 0x7ff7d00716b0 2023-11-13T10:57:01.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d006bfe0 con 0x7ff7d00716b0 2023-11-13T10:57:01.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.375+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d006c5a0 con 0x7ff7d00716b0 2023-11-13T10:57:01.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.377+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d004e4d0 con 0x7ff7d00716b0 2023-11-13T10:57:01.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.377+0000 7ff7c67fc640 1 -- 172.21.15.16:0/1860531193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7c0007490 con 0x7ff7d00716b0 2023-11-13T10:57:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.379+0000 7ff7c67fc640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff7b406ae40 0x7ff7b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.379+0000 7ff7c67fc640 1 -- 172.21.15.16:0/1860531193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7ff7c0088c90 con 0x7ff7d00716b0 2023-11-13T10:57:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.379+0000 7ff7d59fd640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff7b406ae40 0x7ff7b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.381+0000 7ff7d59fd640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff7b406ae40 0x7ff7b406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc036000 tx=0x7ff7cc03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.383+0000 7ff7c67fc640 1 -- 172.21.15.16:0/1860531193 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff7c0053b20 con 0x7ff7d00716b0 2023-11-13T10:57:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.588+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff7d015abd0 con 0x7ff7d00716b0 2023-11-13T10:57:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.589+0000 7ff7c67fc640 1 -- 172.21.15.16:0/1860531193 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff7c0017020 con 0x7ff7d00716b0 2023-11-13T10:57:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T10:57:01.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff7b406ae40 msgr2=0x7ff7b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:01.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff7b406ae40 0x7ff7b406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc036000 tx=0x7ff7cc03c040 comp rx=0 tx=0).stop 2023-11-13T10:57:01.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00716b0 msgr2=0x7ff7d01f6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:01.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00716b0 0x7ff7d01f6d00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000c4a0 tx=0x7ff7c000c960 comp rx=0 tx=0).stop 2023-11-13T10:57:01.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 shutdown_connections 2023-11-13T10:57:01.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff7b406ae40 0x7ff7b406d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d00cb190 0x7ff7d00ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d00716b0 0x7ff7d01f6d00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 --2- 172.21.15.16:0/1860531193 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7d006a190 0x7ff7d014ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:01.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.592+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 >> 172.21.15.16:0/1860531193 conn(0x7ff7d00b04a0 msgr2=0x7ff7d00cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:01.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.593+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 shutdown_connections 2023-11-13T10:57:01.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:01.593+0000 7ff7d69ff640 1 -- 172.21.15.16:0/1860531193 wait complete. 2023-11-13T10:57:01.627 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi184.front.sepia.ceph.com... 2023-11-13T10:57:01.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:01.628 DEBUG:teuthology.orchestra.run.smithi184:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-13T10:57:01.733 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T10:57:01.734 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T10:57:01.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:01.735 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-13T10:57:01.787 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-13T10:57:01.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:57:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.028+0000 7efc41083640 1 Processor -- start 2023-11-13T10:57:02.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.029+0000 7efc41083640 1 -- start start 2023-11-13T10:57:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.030+0000 7efc41083640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc3c15b0e0 0x7efc3c15c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.030+0000 7efc41083640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc3c0d16a0 0x7efc3c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.030+0000 7efc41083640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc3c0ced30 0x7efc3c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.030+0000 7efc41083640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c15bd10 con 0x7efc3c15b0e0 2023-11-13T10:57:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.030+0000 7efc41083640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c0cf6c0 con 0x7efc3c0d16a0 2023-11-13T10:57:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.030+0000 7efc41083640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15b670 con 0x7efc3c0ced30 2023-11-13T10:57:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.030+0000 7efc3ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc3c15b0e0 0x7efc3c15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.031+0000 7efc3ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc3c15b0e0 0x7efc3c15c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44780/0 (socket says 172.21.15.16:44780) 2023-11-13T10:57:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.031+0000 7efc3ad9d640 1 -- 172.21.15.16:0/3875670047 learned_addr learned my addr 172.21.15.16:0/3875670047 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.031+0000 7efc3ad9d640 1 -- 172.21.15.16:0/3875670047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc3c0ced30 msgr2=0x7efc3c0cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.031+0000 7efc3ad9d640 1 --2- 172.21.15.16:0/3875670047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc3c0ced30 0x7efc3c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.031+0000 7efc3ad9d640 1 -- 172.21.15.16:0/3875670047 >> v2:172.21.15.184:3300/0 conn(0x7efc3c0d16a0 msgr2=0x7efc3c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.031+0000 7efc3ad9d640 1 --2- 172.21.15.16:0/3875670047 >> v2:172.21.15.184:3300/0 conn(0x7efc3c0d16a0 0x7efc3c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.031+0000 7efc3ad9d640 1 -- 172.21.15.16:0/3875670047 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c15f680 con 0x7efc3c15b0e0 2023-11-13T10:57:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.032+0000 7efc3ad9d640 1 --2- 172.21.15.16:0/3875670047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c15b0e0 0x7efc3c15c130 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0ce5a0 tx=0x7efc2c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b909351532e9e2b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.032+0000 7efc39d9b640 1 -- 172.21.15.16:0/3875670047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c02ca80 con 0x7efc3c15b0e0 2023-11-13T10:57:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc39d9b640 1 -- 172.21.15.16:0/3875670047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c02cbe0 con 0x7efc3c15b0e0 2023-11-13T10:57:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc39d9b640 1 -- 172.21.15.16:0/3875670047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0343f0 con 0x7efc3c15b0e0 2023-11-13T10:57:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc41083640 1 -- 172.21.15.16:0/3875670047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c15b0e0 msgr2=0x7efc3c15c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc41083640 1 --2- 172.21.15.16:0/3875670047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c15b0e0 0x7efc3c15c130 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0ce5a0 tx=0x7efc2c02bee0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc41083640 1 -- 172.21.15.16:0/3875670047 shutdown_connections 2023-11-13T10:57:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc41083640 1 --2- 172.21.15.16:0/3875670047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc3c0ced30 0x7efc3c0cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc41083640 1 --2- 172.21.15.16:0/3875670047 >> v2:172.21.15.184:3300/0 conn(0x7efc3c0d16a0 0x7efc3c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc41083640 1 --2- 172.21.15.16:0/3875670047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c15b0e0 0x7efc3c15c130 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc41083640 1 -- 172.21.15.16:0/3875670047 >> 172.21.15.16:0/3875670047 conn(0x7efc3c0d0aa0 msgr2=0x7efc3c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc41083640 1 -- 172.21.15.16:0/3875670047 shutdown_connections 2023-11-13T10:57:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.033+0000 7efc41083640 1 -- 172.21.15.16:0/3875670047 wait complete. 2023-11-13T10:57:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.036+0000 7efc41083640 1 Processor -- start 2023-11-13T10:57:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.036+0000 7efc41083640 1 -- start start 2023-11-13T10:57:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.037+0000 7efc41083640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc3c0ced30 0x7efc3c0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.037+0000 7efc41083640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c0d16a0 0x7efc3c0b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.037+0000 7efc41083640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc3c15b0e0 0x7efc3c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.037+0000 7efc41083640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c159560 con 0x7efc3c0d16a0 2023-11-13T10:57:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.037+0000 7efc41083640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0c7b70 con 0x7efc3c15b0e0 2023-11-13T10:57:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.037+0000 7efc41083640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0c8130 con 0x7efc3c0ced30 2023-11-13T10:57:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.037+0000 7efc3a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c0d16a0 0x7efc3c0b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.037+0000 7efc3a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c0d16a0 0x7efc3c0b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44784/0 (socket says 172.21.15.16:44784) 2023-11-13T10:57:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.037+0000 7efc3a59c640 1 -- 172.21.15.16:0/1063840684 learned_addr learned my addr 172.21.15.16:0/1063840684 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.038+0000 7efc3ad9d640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc3c0ced30 0x7efc3c0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.038+0000 7efc3b59e640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc3c15b0e0 0x7efc3c15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.038+0000 7efc3a59c640 1 -- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc3c0ced30 msgr2=0x7efc3c0c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.038+0000 7efc3a59c640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc3c0ced30 0x7efc3c0c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.038+0000 7efc3a59c640 1 -- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc3c15b0e0 msgr2=0x7efc3c15e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.038+0000 7efc3a59c640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc3c15b0e0 0x7efc3c15e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.038+0000 7efc3a59c640 1 -- 172.21.15.16:0/1063840684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c009b30 con 0x7efc3c0d16a0 2023-11-13T10:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.038+0000 7efc3a59c640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c0d16a0 0x7efc3c0b16b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efc3000b4a0 tx=0x7efc3000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.039+0000 7efc237fe640 1 -- 172.21.15.16:0/1063840684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30010610 con 0x7efc3c0d16a0 2023-11-13T10:57:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.039+0000 7efc237fe640 1 -- 172.21.15.16:0/1063840684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc30010770 con 0x7efc3c0d16a0 2023-11-13T10:57:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.039+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c15ef60 con 0x7efc3c0d16a0 2023-11-13T10:57:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.039+0000 7efc237fe640 1 -- 172.21.15.16:0/1063840684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30010b20 con 0x7efc3c0d16a0 2023-11-13T10:57:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.039+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c15f2f0 con 0x7efc3c0d16a0 2023-11-13T10:57:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.040+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c0c8bc0 con 0x7efc3c0d16a0 2023-11-13T10:57:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.041+0000 7efc237fe640 1 -- 172.21.15.16:0/1063840684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc30010c80 con 0x7efc3c0d16a0 2023-11-13T10:57:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.042+0000 7efc237fe640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc2806ae40 0x7efc2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.042+0000 7efc237fe640 1 -- 172.21.15.16:0/1063840684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7efc300894e0 con 0x7efc3c0d16a0 2023-11-13T10:57:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.043+0000 7efc3ad9d640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc2806ae40 0x7efc2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.044+0000 7efc3ad9d640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc2806ae40 0x7efc2806d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc2c035000 tx=0x7efc2c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.047+0000 7efc237fe640 1 -- 172.21.15.16:0/1063840684 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc3c0c8bc0 con 0x7efc3c0d16a0 2023-11-13T10:57:02.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.284+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc3c0c8bc0 con 0x7efc3c0d16a0 2023-11-13T10:57:02.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.285+0000 7efc237fe640 1 -- 172.21.15.16:0/1063840684 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7efc3c0c8bc0 con 0x7efc3c0d16a0 2023-11-13T10:57:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:02.290 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:53.560913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-13T10:57:02.291 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-11-13T10:57:02.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc2806ae40 msgr2=0x7efc2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc2806ae40 0x7efc2806d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc2c035000 tx=0x7efc2c030000 comp rx=0 tx=0).stop 2023-11-13T10:57:02.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c0d16a0 msgr2=0x7efc3c0b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c0d16a0 0x7efc3c0b16b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efc3000b4a0 tx=0x7efc3000b960 comp rx=0 tx=0).stop 2023-11-13T10:57:02.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 shutdown_connections 2023-11-13T10:57:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc2806ae40 0x7efc2806d300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc3c15b0e0 0x7efc3c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3c0d16a0 0x7efc3c0b16b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 --2- 172.21.15.16:0/1063840684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc3c0ced30 0x7efc3c0c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 >> 172.21.15.16:0/1063840684 conn(0x7efc3c0d0aa0 msgr2=0x7efc3c0cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 shutdown_connections 2023-11-13T10:57:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.288+0000 7efc41083640 1 -- 172.21.15.16:0/1063840684 wait complete. 2023-11-13T10:57:02.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:57:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.536+0000 7f2e34864640 1 Processor -- start 2023-11-13T10:57:02.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.536+0000 7f2e34864640 1 -- start start 2023-11-13T10:57:02.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.538+0000 7f2e34864640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300ccc60 0x7f2e30071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.538+0000 7f2e34864640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e30071980 0x7f2e3006fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.538+0000 7f2e34864640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e300cd0e0 con 0x7f2e30155410 2023-11-13T10:57:02.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.538+0000 7f2e34864640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e30071e10 con 0x7f2e30071980 2023-11-13T10:57:02.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.538+0000 7f2e34864640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3015b500 con 0x7f2e300ccc60 2023-11-13T10:57:02.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.538+0000 7f2e2ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e30071980 0x7f2e3006fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.538+0000 7f2e2e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2e30155410 0x7f2e300ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36564/0 (socket says 172.21.15.16:36564) 2023-11-13T10:57:02.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.538+0000 7f2e2e59c640 1 -- 172.21.15.16:0/4222759729 learned_addr learned my addr 172.21.15.16:0/4222759729 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:02.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.539+0000 7f2e27fff640 1 --2- 172.21.15.16:0/4222759729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300ccc60 0x7f2e30071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.539+0000 7f2e2ed9d640 1 -- 172.21.15.16:0/4222759729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300ccc60 msgr2=0x7f2e30071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:02.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.539+0000 7f2e2ed9d640 1 --2- 172.21.15.16:0/4222759729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300ccc60 0x7f2e30071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.539+0000 7f2e2ed9d640 1 -- 172.21.15.16:0/4222759729 >> v1:172.21.15.16:6789/0 conn(0x7f2e30155410 legacy=0x7f2e300ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.539+0000 7f2e2ed9d640 1 -- 172.21.15.16:0/4222759729 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3015b7e0 con 0x7f2e30071980 2023-11-13T10:57:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.539+0000 7f2e27fff640 1 --2- 172.21.15.16:0/4222759729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300ccc60 0x7f2e30071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:57:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.540+0000 7f2e2ed9d640 1 --2- 172.21.15.16:0/4222759729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e30071980 0x7f2e3006fe40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009d90 tx=0x7f2e2802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132e6fa7ce8f93e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.540+0000 7f2e2dd9b640 1 -- 172.21.15.16:0/4222759729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2802cc60 con 0x7f2e30071980 2023-11-13T10:57:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.540+0000 7f2e2dd9b640 1 -- 172.21.15.16:0/4222759729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2802cdc0 con 0x7f2e30071980 2023-11-13T10:57:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.540+0000 7f2e2dd9b640 1 -- 172.21.15.16:0/4222759729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28034660 con 0x7f2e30071980 2023-11-13T10:57:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.541+0000 7f2e34864640 1 -- 172.21.15.16:0/4222759729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e30071980 msgr2=0x7f2e3006fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.541+0000 7f2e34864640 1 --2- 172.21.15.16:0/4222759729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e30071980 0x7f2e3006fe40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009d90 tx=0x7f2e2802beb0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.541+0000 7f2e34864640 1 -- 172.21.15.16:0/4222759729 shutdown_connections 2023-11-13T10:57:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.541+0000 7f2e34864640 1 --2- 172.21.15.16:0/4222759729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e30071980 0x7f2e3006fe40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.541+0000 7f2e34864640 1 --2- 172.21.15.16:0/4222759729 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300ccc60 0x7f2e30071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.541+0000 7f2e34864640 1 -- 172.21.15.16:0/4222759729 >> 172.21.15.16:0/4222759729 conn(0x7f2e3006a190 msgr2=0x7f2e3006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.541+0000 7f2e34864640 1 -- 172.21.15.16:0/4222759729 shutdown_connections 2023-11-13T10:57:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.541+0000 7f2e34864640 1 -- 172.21.15.16:0/4222759729 wait complete. 2023-11-13T10:57:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.543+0000 7f2e34864640 1 Processor -- start 2023-11-13T10:57:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.543+0000 7f2e34864640 1 -- start start 2023-11-13T10:57:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.544+0000 7f2e34864640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30071980 0x7f2e300b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.544+0000 7f2e34864640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e300ccc60 0x7f2e30157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.544+0000 7f2e34864640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30155410 0x7f2e30139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.544+0000 7f2e34864640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e30152760 con 0x7f2e300ccc60 2023-11-13T10:57:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.544+0000 7f2e34864640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e30158740 con 0x7f2e30071980 2023-11-13T10:57:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.544+0000 7f2e34864640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e30152150 con 0x7f2e30155410 2023-11-13T10:57:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e300ccc60 0x7f2e30157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e300ccc60 0x7f2e30157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44806/0 (socket says 172.21.15.16:44806) 2023-11-13T10:57:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e27fff640 1 -- 172.21.15.16:0/2730735172 learned_addr learned my addr 172.21.15.16:0/2730735172 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e2e59c640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30071980 0x7f2e300b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e2ed9d640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30155410 0x7f2e30139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e27fff640 1 -- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30155410 msgr2=0x7f2e30139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e27fff640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30155410 0x7f2e30139e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e27fff640 1 -- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30071980 msgr2=0x7f2e300b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e27fff640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30071980 0x7f2e300b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e27fff640 1 -- 172.21.15.16:0/2730735172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28009a70 con 0x7f2e300ccc60 2023-11-13T10:57:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e2e59c640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30071980 0x7f2e300b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.545+0000 7f2e27fff640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e300ccc60 0x7f2e30157980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800e7e0 tx=0x7f2e1800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.546+0000 7f2e26ffd640 1 -- 172.21.15.16:0/2730735172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18004030 con 0x7f2e300ccc60 2023-11-13T10:57:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.546+0000 7f2e26ffd640 1 -- 172.21.15.16:0/2730735172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e18004190 con 0x7f2e300ccc60 2023-11-13T10:57:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.546+0000 7f2e26ffd640 1 -- 172.21.15.16:0/2730735172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18004520 con 0x7f2e300ccc60 2023-11-13T10:57:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.546+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e30158460 con 0x7f2e300ccc60 2023-11-13T10:57:02.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.546+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e300b2300 con 0x7f2e300ccc60 2023-11-13T10:57:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.547+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e301528d0 con 0x7f2e300ccc60 2023-11-13T10:57:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.548+0000 7f2e26ffd640 1 -- 172.21.15.16:0/2730735172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e1800c6d0 con 0x7f2e300ccc60 2023-11-13T10:57:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.549+0000 7f2e26ffd640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2e1006ae70 0x7f2e1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:02.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.549+0000 7f2e26ffd640 1 -- 172.21.15.16:0/2730735172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f2e180896d0 con 0x7f2e300ccc60 2023-11-13T10:57:02.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.550+0000 7f2e2e59c640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2e1006ae70 0x7f2e1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.551+0000 7f2e2e59c640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2e1006ae70 0x7f2e1006d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005880 tx=0x7f2e20009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.553+0000 7f2e26ffd640 1 -- 172.21.15.16:0/2730735172 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e180543a0 con 0x7f2e300ccc60 2023-11-13T10:57:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.755+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e30067090 con 0x7f2e300ccc60 2023-11-13T10:57:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.757+0000 7f2e26ffd640 1 -- 172.21.15.16:0/2730735172 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f2e180579c0 con 0x7f2e300ccc60 2023-11-13T10:57:02.761 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:02.762 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:57:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2e1006ae70 msgr2=0x7f2e1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2e1006ae70 0x7f2e1006d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005880 tx=0x7f2e20009040 comp rx=0 tx=0).stop 2023-11-13T10:57:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e300ccc60 msgr2=0x7f2e30157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e300ccc60 0x7f2e30157980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800e7e0 tx=0x7f2e1800eca0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 shutdown_connections 2023-11-13T10:57:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2e1006ae70 0x7f2e1006d330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30155410 0x7f2e30139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e300ccc60 0x7f2e30157980 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 --2- 172.21.15.16:0/2730735172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30071980 0x7f2e300b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.760+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 >> 172.21.15.16:0/2730735172 conn(0x7f2e3006a190 msgr2=0x7f2e3006fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.761+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 shutdown_connections 2023-11-13T10:57:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:02.761+0000 7f2e34864640 1 -- 172.21.15.16:0/2730735172 wait complete. 2023-11-13T10:57:02.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:55:24.262070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:55:25.326765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-13T10:55:28.924045+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-13T10:57:02.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:57:03.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.015+0000 7f987ed90640 1 Processor -- start 2023-11-13T10:57:03.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.016+0000 7f987ed90640 1 -- start start 2023-11-13T10:57:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.017+0000 7f987ed90640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f987815e0e0 0x7f987806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.017+0000 7f987ed90640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987806d910 con 0x7f987806c930 2023-11-13T10:57:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.017+0000 7f987ed90640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987806db80 con 0x7f98780719b0 2023-11-13T10:57:03.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.017+0000 7f987ed90640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98780cf0c0 con 0x7f987815e0e0 2023-11-13T10:57:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.018+0000 7f9877fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f987815e0e0 0x7f987806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.018+0000 7f987d32d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f987806c930 0x7f9878161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60572/0 (socket says 172.21.15.16:60572) 2023-11-13T10:57:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.018+0000 7f987d32d640 1 -- 172.21.15.16:0/2742916264 learned_addr learned my addr 172.21.15.16:0/2742916264 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.018+0000 7f98777fe640 1 -- 172.21.15.16:0/2742916264 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752536991 0 0) 0x7f987806d910 con 0x7f987806c930 2023-11-13T10:57:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.018+0000 7f98777fe640 1 -- 172.21.15.16:0/2742916264 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9868003520 con 0x7f987806c930 2023-11-13T10:57:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.018+0000 7f9877fff640 1 -- 172.21.15.16:0/2742916264 >> v1:172.21.15.184:6789/0 conn(0x7f98780719b0 legacy=0x7f98780b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.018+0000 7f9877fff640 1 -- 172.21.15.16:0/2742916264 >> v1:172.21.15.16:6789/0 conn(0x7f987806c930 legacy=0x7f9878161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.019+0000 7f9877fff640 1 -- 172.21.15.16:0/2742916264 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987806de60 con 0x7f987815e0e0 2023-11-13T10:57:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.020+0000 7f9877fff640 1 --2- 172.21.15.16:0/2742916264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987815e0e0 0x7f987806c3f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f98640097a0 tx=0x7f986402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3403746acdcd987 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.020+0000 7f98777fe640 1 -- 172.21.15.16:0/2742916264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864030610 con 0x7f987815e0e0 2023-11-13T10:57:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.020+0000 7f98777fe640 1 -- 172.21.15.16:0/2742916264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864030770 con 0x7f987815e0e0 2023-11-13T10:57:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.020+0000 7f98777fe640 1 -- 172.21.15.16:0/2742916264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864030b00 con 0x7f987815e0e0 2023-11-13T10:57:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.021+0000 7f987ed90640 1 -- 172.21.15.16:0/2742916264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987815e0e0 msgr2=0x7f987806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.021+0000 7f987ed90640 1 --2- 172.21.15.16:0/2742916264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987815e0e0 0x7f987806c3f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f98640097a0 tx=0x7f986402b960 comp rx=0 tx=0).stop 2023-11-13T10:57:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.021+0000 7f987ed90640 1 -- 172.21.15.16:0/2742916264 shutdown_connections 2023-11-13T10:57:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.021+0000 7f987ed90640 1 --2- 172.21.15.16:0/2742916264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987815e0e0 0x7f987806c3f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.021+0000 7f987ed90640 1 -- 172.21.15.16:0/2742916264 >> 172.21.15.16:0/2742916264 conn(0x7f9878070890 msgr2=0x7f98780b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.021+0000 7f987ed90640 1 -- 172.21.15.16:0/2742916264 shutdown_connections 2023-11-13T10:57:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.021+0000 7f987ed90640 1 -- 172.21.15.16:0/2742916264 wait complete. 2023-11-13T10:57:03.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.023+0000 7f987ed90640 1 Processor -- start 2023-11-13T10:57:03.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.023+0000 7f987ed90640 1 -- start start 2023-11-13T10:57:03.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.024+0000 7f987ed90640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987806c930 0x7f9878158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.024+0000 7f987ed90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98780719b0 0x7f98781f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.024+0000 7f987ed90640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987815e0e0 0x7f987815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.024+0000 7f987ed90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9878157620 con 0x7f98780719b0 2023-11-13T10:57:03.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.024+0000 7f987ed90640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98781f2a70 con 0x7f987815e0e0 2023-11-13T10:57:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.024+0000 7f987ed90640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9878154590 con 0x7f987806c930 2023-11-13T10:57:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.024+0000 7f9877fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98780719b0 0x7f98781f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.024+0000 7f9877fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98780719b0 0x7f98781f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56330/0 (socket says 172.21.15.16:56330) 2023-11-13T10:57:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.024+0000 7f9877fff640 1 -- 172.21.15.16:0/219168482 learned_addr learned my addr 172.21.15.16:0/219168482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f987cb2c640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987806c930 0x7f9878158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f987d32d640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987815e0e0 0x7f987815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f9877fff640 1 -- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987806c930 msgr2=0x7f9878158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f9877fff640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987806c930 0x7f9878158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f9877fff640 1 -- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987815e0e0 msgr2=0x7f987815f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f9877fff640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987815e0e0 0x7f987815f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f9877fff640 1 -- 172.21.15.16:0/219168482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864009400 con 0x7f98780719b0 2023-11-13T10:57:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f987d32d640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987815e0e0 0x7f987815f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f987cb2c640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987806c930 0x7f9878158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.025+0000 7f9877fff640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98780719b0 0x7f98781f2530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98640098c0 tx=0x7f98640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.026+0000 7f98757fa640 1 -- 172.21.15.16:0/219168482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986403f070 con 0x7f98780719b0 2023-11-13T10:57:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.026+0000 7f98757fa640 1 -- 172.21.15.16:0/219168482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864030e10 con 0x7f98780719b0 2023-11-13T10:57:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.026+0000 7f98757fa640 1 -- 172.21.15.16:0/219168482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986403ac60 con 0x7f98780719b0 2023-11-13T10:57:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.026+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9878154870 con 0x7f98780719b0 2023-11-13T10:57:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.026+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98781451d0 con 0x7f98780719b0 2023-11-13T10:57:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.027+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9878070610 con 0x7f98780719b0 2023-11-13T10:57:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.028+0000 7f98757fa640 1 -- 172.21.15.16:0/219168482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f986404c400 con 0x7f98780719b0 2023-11-13T10:57:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.030+0000 7f98757fa640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f985c06ae40 0x7f985c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.030+0000 7f98757fa640 1 -- 172.21.15.16:0/219168482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f98640abd30 con 0x7f98780719b0 2023-11-13T10:57:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.030+0000 7f987cb2c640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f985c06ae40 0x7f985c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.032+0000 7f987cb2c640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f985c06ae40 0x7f985c06d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f986c000f60 tx=0x7f986c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.034+0000 7f98757fa640 1 -- 172.21.15.16:0/219168482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9864076a90 con 0x7f98780719b0 2023-11-13T10:57:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.232+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9878157760 con 0x7f98780719b0 2023-11-13T10:57:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.234+0000 7f98757fa640 1 -- 172.21.15.16:0/219168482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9878157760 con 0x7f98780719b0 2023-11-13T10:57:03.238 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:57:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f985c06ae40 msgr2=0x7f985c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f985c06ae40 0x7f985c06d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f986c000f60 tx=0x7f986c009040 comp rx=0 tx=0).stop 2023-11-13T10:57:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98780719b0 msgr2=0x7f98781f2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98780719b0 0x7f98781f2530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98640098c0 tx=0x7f98640046c0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 shutdown_connections 2023-11-13T10:57:03.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f985c06ae40 0x7f985c06d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f987815e0e0 0x7f987815f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98780719b0 0x7f98781f2530 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 --2- 172.21.15.16:0/219168482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f987806c930 0x7f9878158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 >> 172.21.15.16:0/219168482 conn(0x7f9878070890 msgr2=0x7f98780aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 shutdown_connections 2023-11-13T10:57:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.237+0000 7f987ed90640 1 -- 172.21.15.16:0/219168482 wait complete. 2023-11-13T10:57:03.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T10:57:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.498+0000 7fcfbd01a640 1 Processor -- start 2023-11-13T10:57:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.498+0000 7fcfbd01a640 1 -- start start 2023-11-13T10:57:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.499+0000 7fcfbd01a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcfb815ce70 0x7fcfb80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.499+0000 7fcfbd01a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80cdbb0 0x7fcfb80d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.499+0000 7fcfbd01a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb80cac40 con 0x7fcfb80d1850 2023-11-13T10:57:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.499+0000 7fcfbd01a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb80ce0e0 con 0x7fcfb815ce70 2023-11-13T10:57:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.499+0000 7fcfbd01a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb80ce3c0 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfbc819640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcfb80d1850 0x7fcfb80cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60588/0 (socket says 172.21.15.16:60588) 2023-11-13T10:57:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfbc819640 1 -- 172.21.15.16:0/2920640499 learned_addr learned my addr 172.21.15.16:0/2920640499 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfb7fff640 1 --2- 172.21.15.16:0/2920640499 >> v2:172.21.15.16:3300/0 conn(0x7fcfb815ce70 0x7fcfb80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfaf7fe640 1 --2- 172.21.15.16:0/2920640499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80cdbb0 0x7fcfb80d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfb77fe640 1 -- 172.21.15.16:0/2920640499 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319155494 0 0) 0x7fcfb80cac40 con 0x7fcfb80d1850 2023-11-13T10:57:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfb77fe640 1 -- 172.21.15.16:0/2920640499 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa4003520 con 0x7fcfb80d1850 2023-11-13T10:57:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfb7fff640 1 -- 172.21.15.16:0/2920640499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80cdbb0 msgr2=0x7fcfb80d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfb7fff640 1 --2- 172.21.15.16:0/2920640499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80cdbb0 0x7fcfb80d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfb7fff640 1 -- 172.21.15.16:0/2920640499 >> v1:172.21.15.16:6789/0 conn(0x7fcfb80d1850 legacy=0x7fcfb80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.500+0000 7fcfb7fff640 1 -- 172.21.15.16:0/2920640499 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80ce6a0 con 0x7fcfb815ce70 2023-11-13T10:57:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.502+0000 7fcfb7fff640 1 --2- 172.21.15.16:0/2920640499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb815ce70 0x7fcfb80cd670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80097d0 tx=0x7fcfa802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ee691bb4e75c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.502+0000 7fcfb77fe640 1 -- 172.21.15.16:0/2920640499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8031800 con 0x7fcfb815ce70 2023-11-13T10:57:03.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.502+0000 7fcfb77fe640 1 -- 172.21.15.16:0/2920640499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8031960 con 0x7fcfb815ce70 2023-11-13T10:57:03.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.502+0000 7fcfb77fe640 1 -- 172.21.15.16:0/2920640499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8031cf0 con 0x7fcfb815ce70 2023-11-13T10:57:03.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.503+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2920640499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb815ce70 msgr2=0x7fcfb80cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.503+0000 7fcfbd01a640 1 --2- 172.21.15.16:0/2920640499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb815ce70 0x7fcfb80cd670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80097d0 tx=0x7fcfa802b8a0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.503+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2920640499 shutdown_connections 2023-11-13T10:57:03.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.503+0000 7fcfbd01a640 1 --2- 172.21.15.16:0/2920640499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80cdbb0 0x7fcfb80d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.503+0000 7fcfbd01a640 1 --2- 172.21.15.16:0/2920640499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb815ce70 0x7fcfb80cd670 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.503+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2920640499 >> 172.21.15.16:0/2920640499 conn(0x7fcfb808c580 msgr2=0x7fcfb809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:03.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.503+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2920640499 shutdown_connections 2023-11-13T10:57:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.503+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2920640499 wait complete. 2023-11-13T10:57:03.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.505+0000 7fcfbd01a640 1 Processor -- start 2023-11-13T10:57:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.505+0000 7fcfbd01a640 1 -- start start 2023-11-13T10:57:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.506+0000 7fcfbd01a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb80cdbb0 0x7fcfb8152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.506+0000 7fcfbd01a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80d1850 0x7fcfb815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.506+0000 7fcfbd01a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb815ce70 0x7fcfb815f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.506+0000 7fcfbd01a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb815eef0 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.506+0000 7fcfbd01a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb815de50 con 0x7fcfb815ce70 2023-11-13T10:57:03.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.506+0000 7fcfbd01a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb815e130 con 0x7fcfb80d1850 2023-11-13T10:57:03.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.506+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb80cdbb0 0x7fcfb8152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.506+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb80cdbb0 0x7fcfb8152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56358/0 (socket says 172.21.15.16:56358) 2023-11-13T10:57:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.506+0000 7fcfb7fff640 1 -- 172.21.15.16:0/2090619078 learned_addr learned my addr 172.21.15.16:0/2090619078 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.507+0000 7fcfaf7fe640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80d1850 0x7fcfb815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.507+0000 7fcfbc819640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb815ce70 0x7fcfb815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.507+0000 7fcfb7fff640 1 -- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80d1850 msgr2=0x7fcfb815d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.507+0000 7fcfb7fff640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80d1850 0x7fcfb815d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.507+0000 7fcfb7fff640 1 -- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb815ce70 msgr2=0x7fcfb815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.507+0000 7fcfb7fff640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb815ce70 0x7fcfb815f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.507+0000 7fcfb7fff640 1 -- 172.21.15.16:0/2090619078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8009400 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.507+0000 7fcfb7fff640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb80cdbb0 0x7fcfb8152060 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8034040 tx=0x7fcfa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:03.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.508+0000 7fcfb57fa640 1 -- 172.21.15.16:0/2090619078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa803ac60 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.508+0000 7fcfb57fa640 1 -- 172.21.15.16:0/2090619078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa803ade0 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.508+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb815e410 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.508+0000 7fcfb57fa640 1 -- 172.21.15.16:0/2090619078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8044640 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.508+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb815e9d0 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.509+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb80675b0 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.510+0000 7fcfb57fa640 1 -- 172.21.15.16:0/2090619078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcfa804d440 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.511+0000 7fcfb57fa640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcf9806ae40 0x7fcf9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.511+0000 7fcfb57fa640 1 -- 172.21.15.16:0/2090619078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fcfa802e070 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.512+0000 7fcfaf7fe640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcf9806ae40 0x7fcf9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.513+0000 7fcfaf7fe640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcf9806ae40 0x7fcf9806d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80c95e0 tx=0x7fcfa0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.515+0000 7fcfb57fa640 1 -- 172.21.15.16:0/2090619078 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfa8076a50 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.715+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfb804e4d0 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.716+0000 7fcfb57fa640 1 -- 172.21.15.16:0/2090619078 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfa8035020 con 0x7fcfb80cdbb0 2023-11-13T10:57:03.721 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:57:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcf9806ae40 msgr2=0x7fcf9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcf9806ae40 0x7fcf9806d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80c95e0 tx=0x7fcfa0008040 comp rx=0 tx=0).stop 2023-11-13T10:57:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb80cdbb0 msgr2=0x7fcfb8152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb80cdbb0 0x7fcfb8152060 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8034040 tx=0x7fcfa80046c0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 shutdown_connections 2023-11-13T10:57:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcf9806ae40 0x7fcf9806d300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb815ce70 0x7fcfb815f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb80d1850 0x7fcfb815d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 --2- 172.21.15.16:0/2090619078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfb80cdbb0 0x7fcfb8152060 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 >> 172.21.15.16:0/2090619078 conn(0x7fcfb808c580 msgr2=0x7fcfb809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:03.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 shutdown_connections 2023-11-13T10:57:03.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.719+0000 7fcfbd01a640 1 -- 172.21.15.16:0/2090619078 wait complete. 2023-11-13T10:57:03.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T10:57:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.980+0000 7f29e198e640 1 Processor -- start 2023-11-13T10:57:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.980+0000 7f29e198e640 1 -- start start 2023-11-13T10:57:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.982+0000 7f29e198e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29dc15ce70 0x7f29dc0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.982+0000 7f29e198e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc0ce0e0 con 0x7f29dc0d1850 2023-11-13T10:57:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.982+0000 7f29e198e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc0ce3c0 con 0x7f29dc0cdbb0 2023-11-13T10:57:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.982+0000 7f29e198e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29dc0ce6a0 con 0x7f29dc15ce70 2023-11-13T10:57:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.982+0000 7f29e118d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f29dc0d1850 0x7f29dc0c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60598/0 (socket says 172.21.15.16:60598) 2023-11-13T10:57:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.982+0000 7f29e118d640 1 -- 172.21.15.16:0/3909033032 learned_addr learned my addr 172.21.15.16:0/3909033032 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.982+0000 7f29e098c640 1 --2- 172.21.15.16:0/3909033032 >> v2:172.21.15.16:3300/0 conn(0x7f29dc15ce70 0x7f29dc0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.983+0000 7f29e098c640 1 -- 172.21.15.16:0/3909033032 >> v1:172.21.15.184:6789/0 conn(0x7f29dc0cdbb0 legacy=0x7f29dc0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.983+0000 7f29db7fe640 1 -- 172.21.15.16:0/3909033032 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657262575 0 0) 0x7f29dc0ce0e0 con 0x7f29dc0d1850 2023-11-13T10:57:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.983+0000 7f29e098c640 1 -- 172.21.15.16:0/3909033032 >> v1:172.21.15.16:6789/0 conn(0x7f29dc0d1850 legacy=0x7f29dc0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.983+0000 7f29e098c640 1 -- 172.21.15.16:0/3909033032 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc0ce980 con 0x7f29dc15ce70 2023-11-13T10:57:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.984+0000 7f29e098c640 1 --2- 172.21.15.16:0/3909033032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc15ce70 0x7f29dc0cd670 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f29d00097a0 tx=0x7f29d002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92feb150d287e37 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.985+0000 7f29db7fe640 1 -- 172.21.15.16:0/3909033032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0030610 con 0x7f29dc15ce70 2023-11-13T10:57:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.985+0000 7f29db7fe640 1 -- 172.21.15.16:0/3909033032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d0030770 con 0x7f29dc15ce70 2023-11-13T10:57:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.985+0000 7f29db7fe640 1 -- 172.21.15.16:0/3909033032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0030b00 con 0x7f29dc15ce70 2023-11-13T10:57:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.985+0000 7f29e198e640 1 -- 172.21.15.16:0/3909033032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc15ce70 msgr2=0x7f29dc0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.985+0000 7f29e198e640 1 --2- 172.21.15.16:0/3909033032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc15ce70 0x7f29dc0cd670 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f29d00097a0 tx=0x7f29d002b960 comp rx=0 tx=0).stop 2023-11-13T10:57:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.986+0000 7f29e198e640 1 -- 172.21.15.16:0/3909033032 shutdown_connections 2023-11-13T10:57:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.986+0000 7f29e198e640 1 --2- 172.21.15.16:0/3909033032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc15ce70 0x7f29dc0cd670 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.986+0000 7f29e198e640 1 -- 172.21.15.16:0/3909033032 >> 172.21.15.16:0/3909033032 conn(0x7f29dc08c580 msgr2=0x7f29dc09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.986+0000 7f29e198e640 1 -- 172.21.15.16:0/3909033032 shutdown_connections 2023-11-13T10:57:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.986+0000 7f29e198e640 1 -- 172.21.15.16:0/3909033032 wait complete. 2023-11-13T10:57:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.987+0000 7f29e198e640 1 Processor -- start 2023-11-13T10:57:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.988+0000 7f29e198e640 1 -- start start 2023-11-13T10:57:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.988+0000 7f29e198e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cdbb0 0x7f29dc09d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.988+0000 7f29e198e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29dc0d1850 0x7f29dc09d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e198e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc15ce70 0x7f29dc0c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e198e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0a07e0 con 0x7f29dc0cdbb0 2023-11-13T10:57:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e198e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc09fc60 con 0x7f29dc15ce70 2023-11-13T10:57:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e198e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc09ff40 con 0x7f29dc0d1850 2023-11-13T10:57:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e098c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cdbb0 0x7f29dc09d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e098c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cdbb0 0x7f29dc09d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56374/0 (socket says 172.21.15.16:56374) 2023-11-13T10:57:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e098c640 1 -- 172.21.15.16:0/3420869186 learned_addr learned my addr 172.21.15.16:0/3420869186 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e098c640 1 -- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29dc0d1850 msgr2=0x7f29dc09d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e098c640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29dc0d1850 0x7f29dc09d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e098c640 1 -- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc15ce70 msgr2=0x7f29dc0c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e098c640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc15ce70 0x7f29dc0c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e098c640 1 -- 172.21.15.16:0/3420869186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d0009400 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.989+0000 7f29e098c640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cdbb0 0x7f29dc09d3b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f29d0000c00 tx=0x7f29d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.990+0000 7f29d97fa640 1 -- 172.21.15.16:0/3420869186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d003f070 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.990+0000 7f29d97fa640 1 -- 172.21.15.16:0/3420869186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d0004030 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.990+0000 7f29d97fa640 1 -- 172.21.15.16:0/3420869186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00043e0 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.990+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc0a0220 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.990+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc152310 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.991+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc0c65e0 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.992+0000 7f29d97fa640 1 -- 172.21.15.16:0/3420869186 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f29d0043070 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.993+0000 7f29d97fa640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f29c006ae40 0x7f29c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.993+0000 7f29d97fa640 1 -- 172.21.15.16:0/3420869186 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f29d00ab750 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.994+0000 7f29dbfff640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f29c006ae40 0x7f29c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.995+0000 7f29dbfff640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f29c006ae40 0x7f29c006d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f29c8000f60 tx=0x7f29c80079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:03.998+0000 7f29d97fa640 1 -- 172.21.15.16:0/3420869186 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29dc0c65e0 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.194+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29dc0c65e0 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.195+0000 7f29d97fa640 1 -- 172.21.15.16:0/3420869186 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29dc0c65e0 con 0x7f29dc0cdbb0 2023-11-13T10:57:04.200 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T10:57:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f29c006ae40 msgr2=0x7f29c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f29c006ae40 0x7f29c006d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f29c8000f60 tx=0x7f29c80079b0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cdbb0 msgr2=0x7f29dc09d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cdbb0 0x7f29dc09d3b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f29d0000c00 tx=0x7f29d00046c0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 shutdown_connections 2023-11-13T10:57:04.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f29c006ae40 0x7f29c006d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc15ce70 0x7f29dc0c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29dc0d1850 0x7f29dc09d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 --2- 172.21.15.16:0/3420869186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cdbb0 0x7f29dc09d3b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 >> 172.21.15.16:0/3420869186 conn(0x7f29dc08c580 msgr2=0x7f29dc0d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:04.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 shutdown_connections 2023-11-13T10:57:04.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.198+0000 7f29e198e640 1 -- 172.21.15.16:0/3420869186 wait complete. 2023-11-13T10:57:04.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-13T10:57:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.460+0000 7fc018b33640 1 Processor -- start 2023-11-13T10:57:04.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.460+0000 7fc018b33640 1 -- start start 2023-11-13T10:57:04.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.462+0000 7fc018b33640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0140ce5b0 0x7fc01415a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.462+0000 7fc018b33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 0x7fc01415a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.462+0000 7fc018b33640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01415d8d0 con 0x7fc0140ce1d0 2023-11-13T10:57:04.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.462+0000 7fc018b33640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01415af00 con 0x7fc0140ce5b0 2023-11-13T10:57:04.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.462+0000 7fc018b33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01415db90 con 0x7fc0140cf0e0 2023-11-13T10:57:04.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.462+0000 7fc011d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0140ce5b0 0x7fc01415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.462+0000 7fc011d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0140ce5b0 0x7fc01415a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56390/0 (socket says 172.21.15.16:56390) 2023-11-13T10:57:04.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.462+0000 7fc011d9b640 1 -- 172.21.15.16:0/4182220293 learned_addr learned my addr 172.21.15.16:0/4182220293 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:04.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.462+0000 7fc012d9d640 1 --2- 172.21.15.16:0/4182220293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 0x7fc01415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.463+0000 7fc011d9b640 1 -- 172.21.15.16:0/4182220293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 msgr2=0x7fc01415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:04.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.463+0000 7fc011d9b640 1 --2- 172.21.15.16:0/4182220293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 0x7fc01415a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.463+0000 7fc011d9b640 1 -- 172.21.15.16:0/4182220293 >> v1:172.21.15.184:6789/0 conn(0x7fc0140ce1d0 legacy=0x7fc0140ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.463+0000 7fc011d9b640 1 -- 172.21.15.16:0/4182220293 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01415de70 con 0x7fc0140ce5b0 2023-11-13T10:57:04.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.464+0000 7fc011d9b640 1 --2- 172.21.15.16:0/4182220293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce5b0 0x7fc01415a130 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009870 tx=0x7fbffc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3370f239aa9cfb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:04.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.464+0000 7fc01159a640 1 -- 172.21.15.16:0/4182220293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc030840 con 0x7fc0140ce5b0 2023-11-13T10:57:04.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.464+0000 7fc01159a640 1 -- 172.21.15.16:0/4182220293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc0309a0 con 0x7fc0140ce5b0 2023-11-13T10:57:04.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.464+0000 7fc01159a640 1 -- 172.21.15.16:0/4182220293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc030d30 con 0x7fc0140ce5b0 2023-11-13T10:57:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.465+0000 7fc018b33640 1 -- 172.21.15.16:0/4182220293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce5b0 msgr2=0x7fc01415a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.465+0000 7fc018b33640 1 --2- 172.21.15.16:0/4182220293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce5b0 0x7fc01415a130 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009870 tx=0x7fbffc02b980 comp rx=0 tx=0).stop 2023-11-13T10:57:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.465+0000 7fc018b33640 1 -- 172.21.15.16:0/4182220293 shutdown_connections 2023-11-13T10:57:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.465+0000 7fc018b33640 1 --2- 172.21.15.16:0/4182220293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 0x7fc01415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.465+0000 7fc018b33640 1 --2- 172.21.15.16:0/4182220293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce5b0 0x7fc01415a130 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.465+0000 7fc018b33640 1 -- 172.21.15.16:0/4182220293 >> 172.21.15.16:0/4182220293 conn(0x7fc01406a190 msgr2=0x7fc01406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:04.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.465+0000 7fc018b33640 1 -- 172.21.15.16:0/4182220293 shutdown_connections 2023-11-13T10:57:04.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.465+0000 7fc018b33640 1 -- 172.21.15.16:0/4182220293 wait complete. 2023-11-13T10:57:04.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.467+0000 7fc018b33640 1 Processor -- start 2023-11-13T10:57:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.467+0000 7fc018b33640 1 -- start start 2023-11-13T10:57:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.468+0000 7fc018b33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce1d0 0x7fc0141412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.468+0000 7fc018b33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 0x7fc0141547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.468+0000 7fc018b33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc014154d20 0x7fc014158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.468+0000 7fc018b33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc014142c90 con 0x7fc0140ce1d0 2023-11-13T10:57:04.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.468+0000 7fc018b33640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc014141bf0 con 0x7fc014154d20 2023-11-13T10:57:04.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.468+0000 7fc018b33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc014141ed0 con 0x7fc0140cf0e0 2023-11-13T10:57:04.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.468+0000 7fc01259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce1d0 0x7fc0141412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.468+0000 7fc01259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce1d0 0x7fc0141412a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56394/0 (socket says 172.21.15.16:56394) 2023-11-13T10:57:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.468+0000 7fc01259c640 1 -- 172.21.15.16:0/3588902008 learned_addr learned my addr 172.21.15.16:0/3588902008 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc011d9b640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 0x7fc0141547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc012d9d640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc014154d20 0x7fc014158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc01259c640 1 -- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 msgr2=0x7fc0141547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc01259c640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 0x7fc0141547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc01259c640 1 -- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc014154d20 msgr2=0x7fc014158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc01259c640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc014154d20 0x7fc014158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc01259c640 1 -- 172.21.15.16:0/3588902008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc0094d0 con 0x7fc0140ce1d0 2023-11-13T10:57:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc012d9d640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc014154d20 0x7fc014158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc01259c640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce1d0 0x7fc0141412a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc00400c730 tx=0x7fc00400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc00affd640 1 -- 172.21.15.16:0/3588902008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004003b70 con 0x7fc0140ce1d0 2023-11-13T10:57:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc00affd640 1 -- 172.21.15.16:0/3588902008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc004003cf0 con 0x7fc0140ce1d0 2023-11-13T10:57:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc00affd640 1 -- 172.21.15.16:0/3588902008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004003ea0 con 0x7fc0140ce1d0 2023-11-13T10:57:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0141421b0 con 0x7fc0140ce1d0 2023-11-13T10:57:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.469+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc014142770 con 0x7fc0140ce1d0 2023-11-13T10:57:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.471+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc014142960 con 0x7fc0140ce1d0 2023-11-13T10:57:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.471+0000 7fc00affd640 1 -- 172.21.15.16:0/3588902008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc00400aa90 con 0x7fc0140ce1d0 2023-11-13T10:57:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.473+0000 7fc00affd640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff406ae70 0x7fbff406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.473+0000 7fc00affd640 1 -- 172.21.15.16:0/3588902008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fc0040897d0 con 0x7fc0140ce1d0 2023-11-13T10:57:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.474+0000 7fc011d9b640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff406ae70 0x7fbff406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.475+0000 7fc011d9b640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff406ae70 0x7fbff406d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbffc036000 tx=0x7fbffc03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.477+0000 7fc00affd640 1 -- 172.21.15.16:0/3588902008 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0040544a0 con 0x7fc0140ce1d0 2023-11-13T10:57:04.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.671+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc014072520 con 0x7fc0140ce1d0 2023-11-13T10:57:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.672+0000 7fc00affd640 1 -- 172.21.15.16:0/3588902008 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc004057ac0 con 0x7fc0140ce1d0 2023-11-13T10:57:04.676 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T10:57:04.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff406ae70 msgr2=0x7fbff406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff406ae70 0x7fbff406d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbffc036000 tx=0x7fbffc03b040 comp rx=0 tx=0).stop 2023-11-13T10:57:04.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce1d0 msgr2=0x7fc0141412a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce1d0 0x7fc0141412a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc00400c730 tx=0x7fc00400cbf0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 shutdown_connections 2023-11-13T10:57:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbff406ae70 0x7fbff406d330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc014154d20 0x7fc014158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0140cf0e0 0x7fc0141547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 --2- 172.21.15.16:0/3588902008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140ce1d0 0x7fc0141412a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 >> 172.21.15.16:0/3588902008 conn(0x7fc01406a190 msgr2=0x7fc0140cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 shutdown_connections 2023-11-13T10:57:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.675+0000 7fc018b33640 1 -- 172.21.15.16:0/3588902008 wait complete. 2023-11-13T10:57:04.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:57:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.925+0000 7f20625dc640 1 Processor -- start 2023-11-13T10:57:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.925+0000 7f20625dc640 1 -- start start 2023-11-13T10:57:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.927+0000 7f20625dc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f205c15e240 0x7f205c0b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.927+0000 7f20625dc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c06db90 con 0x7f205c15d9f0 2023-11-13T10:57:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.927+0000 7f20625dc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c15a7e0 con 0x7f205c15ddd0 2023-11-13T10:57:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.927+0000 7f20625dc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c15b070 con 0x7f205c15e240 2023-11-13T10:57:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.927+0000 7f205bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f205c15d9f0 0x7f205c06d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60614/0 (socket says 172.21.15.16:60614) 2023-11-13T10:57:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.927+0000 7f205bfff640 1 -- 172.21.15.16:0/2960812015 learned_addr learned my addr 172.21.15.16:0/2960812015 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.927+0000 7f2060b79640 1 --2- 172.21.15.16:0/2960812015 >> v2:172.21.15.16:3300/0 conn(0x7f205c15e240 0x7f205c0b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.928+0000 7f205affd640 1 -- 172.21.15.16:0/2960812015 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716102143 0 0) 0x7f205c06db90 con 0x7f205c15d9f0 2023-11-13T10:57:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.928+0000 7f205affd640 1 -- 172.21.15.16:0/2960812015 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2048003520 con 0x7f205c15d9f0 2023-11-13T10:57:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.928+0000 7f2060b79640 1 -- 172.21.15.16:0/2960812015 >> v1:172.21.15.184:6789/0 conn(0x7f205c15ddd0 legacy=0x7f205c15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.928+0000 7f2060b79640 1 -- 172.21.15.16:0/2960812015 >> v1:172.21.15.16:6789/0 conn(0x7f205c15d9f0 legacy=0x7f205c06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.928+0000 7f2060b79640 1 -- 172.21.15.16:0/2960812015 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c15aab0 con 0x7f205c15e240 2023-11-13T10:57:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.929+0000 7f2060b79640 1 --2- 172.21.15.16:0/2960812015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15e240 0x7f205c0b2630 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f20540097a0 tx=0x7f205402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07a79be2fb62d22 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.929+0000 7f205affd640 1 -- 172.21.15.16:0/2960812015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054031830 con 0x7f205c15e240 2023-11-13T10:57:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.929+0000 7f205affd640 1 -- 172.21.15.16:0/2960812015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2054031990 con 0x7f205c15e240 2023-11-13T10:57:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.929+0000 7f205affd640 1 -- 172.21.15.16:0/2960812015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054031d20 con 0x7f205c15e240 2023-11-13T10:57:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.930+0000 7f20625dc640 1 -- 172.21.15.16:0/2960812015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15e240 msgr2=0x7f205c0b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.930+0000 7f20625dc640 1 --2- 172.21.15.16:0/2960812015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15e240 0x7f205c0b2630 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f20540097a0 tx=0x7f205402b8d0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.930+0000 7f20625dc640 1 -- 172.21.15.16:0/2960812015 shutdown_connections 2023-11-13T10:57:04.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.930+0000 7f20625dc640 1 --2- 172.21.15.16:0/2960812015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15e240 0x7f205c0b2630 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.930+0000 7f20625dc640 1 -- 172.21.15.16:0/2960812015 >> 172.21.15.16:0/2960812015 conn(0x7f205c06a190 msgr2=0x7f205c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.930+0000 7f20625dc640 1 -- 172.21.15.16:0/2960812015 shutdown_connections 2023-11-13T10:57:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.930+0000 7f20625dc640 1 -- 172.21.15.16:0/2960812015 wait complete. 2023-11-13T10:57:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.932+0000 7f20625dc640 1 Processor -- start 2023-11-13T10:57:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.932+0000 7f20625dc640 1 -- start start 2023-11-13T10:57:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f20625dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15d9f0 0x7f205c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f20625dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f205c15ddd0 0x7f205c1445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f20625dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f205c15e240 0x7f205c144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f20625dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c143780 con 0x7f205c15d9f0 2023-11-13T10:57:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f20625dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c143a60 con 0x7f205c15e240 2023-11-13T10:57:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f20625dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c143d40 con 0x7f205c15ddd0 2023-11-13T10:57:04.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f205bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15d9f0 0x7f205c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f205bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15d9f0 0x7f205c151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56420/0 (socket says 172.21.15.16:56420) 2023-11-13T10:57:04.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f205bfff640 1 -- 172.21.15.16:0/2665761665 learned_addr learned my addr 172.21.15.16:0/2665761665 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:04.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f205bfff640 1 -- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f205c15ddd0 msgr2=0x7f205c1445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f205bfff640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f205c15ddd0 0x7f205c1445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f2060b79640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f205c15e240 0x7f205c144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f205bfff640 1 -- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f205c15e240 msgr2=0x7f205c144b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f205bfff640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f205c15e240 0x7f205c144b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.933+0000 7f205bfff640 1 -- 172.21.15.16:0/2665761665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2054009400 con 0x7f205c15d9f0 2023-11-13T10:57:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.934+0000 7f205bfff640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15d9f0 0x7f205c151ba0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f204400c410 tx=0x7f204400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.934+0000 7f2058ff9640 1 -- 172.21.15.16:0/2665761665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044005ce0 con 0x7f205c15d9f0 2023-11-13T10:57:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.934+0000 7f2058ff9640 1 -- 172.21.15.16:0/2665761665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2044005e40 con 0x7f205c15d9f0 2023-11-13T10:57:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.934+0000 7f2058ff9640 1 -- 172.21.15.16:0/2665761665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20440072f0 con 0x7f205c15d9f0 2023-11-13T10:57:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.934+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c144020 con 0x7f205c15d9f0 2023-11-13T10:57:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.934+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c143410 con 0x7f205c15d9f0 2023-11-13T10:57:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.935+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c067090 con 0x7f205c15d9f0 2023-11-13T10:57:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.936+0000 7f2058ff9640 1 -- 172.21.15.16:0/2665761665 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2044007450 con 0x7f205c15d9f0 2023-11-13T10:57:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.937+0000 7f2058ff9640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f203c06ae70 0x7f203c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.937+0000 7f2058ff9640 1 -- 172.21.15.16:0/2665761665 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f2044088fe0 con 0x7f205c15d9f0 2023-11-13T10:57:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.938+0000 7f205b7fe640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f203c06ae70 0x7f203c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:04.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.939+0000 7f205b7fe640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f203c06ae70 0x7f203c06d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f204c0012e0 tx=0x7f204c007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:04.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:04.942+0000 7f2058ff9640 1 -- 172.21.15.16:0/2665761665 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2044057890 con 0x7f205c15d9f0 2023-11-13T10:57:05.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.172+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f205c1438c0 con 0x7f205c15d9f0 2023-11-13T10:57:05.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.174+0000 7f2058ff9640 1 -- 172.21.15.16:0/2665761665 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f205c1438c0 con 0x7f205c15d9f0 2023-11-13T10:57:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:05.178 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:53.560913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-13T10:57:05.179 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-11-13T10:57:05.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.176+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f203c06ae70 msgr2=0x7f203c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.176+0000 7f20625dc640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f203c06ae70 0x7f203c06d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f204c0012e0 tx=0x7f204c007d30 comp rx=0 tx=0).stop 2023-11-13T10:57:05.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.176+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15d9f0 msgr2=0x7f205c151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.176+0000 7f20625dc640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15d9f0 0x7f205c151ba0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f204400c410 tx=0x7f204400c8d0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.177+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 shutdown_connections 2023-11-13T10:57:05.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.177+0000 7f20625dc640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f203c06ae70 0x7f203c06d330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.177+0000 7f20625dc640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f205c15e240 0x7f205c144b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.177+0000 7f20625dc640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f205c15ddd0 0x7f205c1445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.177+0000 7f20625dc640 1 --2- 172.21.15.16:0/2665761665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15d9f0 0x7f205c151ba0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.177+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 >> 172.21.15.16:0/2665761665 conn(0x7f205c06a190 msgr2=0x7f205c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:05.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.177+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 shutdown_connections 2023-11-13T10:57:05.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.177+0000 7f20625dc640 1 -- 172.21.15.16:0/2665761665 wait complete. 2023-11-13T10:57:05.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:57:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.430+0000 7f8ec8911640 1 Processor -- start 2023-11-13T10:57:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.430+0000 7f8ec8911640 1 -- start start 2023-11-13T10:57:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.432+0000 7f8ec8911640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ec415b190 0x7f8ec406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.432+0000 7f8ec8911640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8ec406a190 0x7f8ec406a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.432+0000 7f8ec8911640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec4158c20 con 0x7f8ec40cb190 2023-11-13T10:57:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.432+0000 7f8ec8911640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec406ab40 con 0x7f8ec415b190 2023-11-13T10:57:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.432+0000 7f8ec8911640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec40b1830 con 0x7f8ec406a190 2023-11-13T10:57:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.432+0000 7f8ec37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8ec40cb190 0x7f8ec40716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60630/0 (socket says 172.21.15.16:60630) 2023-11-13T10:57:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.432+0000 7f8ec37fe640 1 -- 172.21.15.16:0/997143188 learned_addr learned my addr 172.21.15.16:0/997143188 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.432+0000 7f8ebaffd640 1 --2- 172.21.15.16:0/997143188 >> v2:172.21.15.16:3300/0 conn(0x7f8ec415b190 0x7f8ec406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.433+0000 7f8ec3fff640 1 --2- 172.21.15.16:0/997143188 >> v2:172.21.15.184:3300/0 conn(0x7f8ec406a190 0x7f8ec406a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.433+0000 7f8ec2ffd640 1 -- 172.21.15.16:0/997143188 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402637948 0 0) 0x7f8ec4158c20 con 0x7f8ec40cb190 2023-11-13T10:57:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.433+0000 7f8ec2ffd640 1 -- 172.21.15.16:0/997143188 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eb0003520 con 0x7f8ec40cb190 2023-11-13T10:57:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.433+0000 7f8ebaffd640 1 -- 172.21.15.16:0/997143188 >> v2:172.21.15.184:3300/0 conn(0x7f8ec406a190 msgr2=0x7f8ec406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.433+0000 7f8ebaffd640 1 --2- 172.21.15.16:0/997143188 >> v2:172.21.15.184:3300/0 conn(0x7f8ec406a190 0x7f8ec406a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.433+0000 7f8ebaffd640 1 -- 172.21.15.16:0/997143188 >> v1:172.21.15.16:6789/0 conn(0x7f8ec40cb190 legacy=0x7f8ec40716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.433+0000 7f8ebaffd640 1 -- 172.21.15.16:0/997143188 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec415b780 con 0x7f8ec415b190 2023-11-13T10:57:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.434+0000 7f8ebaffd640 1 --2- 172.21.15.16:0/997143188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec415b190 0x7f8ec406f970 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40b2ba0 tx=0x7f8eac02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746b2eab7fdeffbc server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.434+0000 7f8ec2ffd640 1 -- 172.21.15.16:0/997143188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac02cbe0 con 0x7f8ec415b190 2023-11-13T10:57:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.434+0000 7f8ec2ffd640 1 -- 172.21.15.16:0/997143188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac02cd40 con 0x7f8ec415b190 2023-11-13T10:57:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.434+0000 7f8ec2ffd640 1 -- 172.21.15.16:0/997143188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac034660 con 0x7f8ec415b190 2023-11-13T10:57:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.435+0000 7f8ec8911640 1 -- 172.21.15.16:0/997143188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec415b190 msgr2=0x7f8ec406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.435+0000 7f8ec8911640 1 --2- 172.21.15.16:0/997143188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec415b190 0x7f8ec406f970 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40b2ba0 tx=0x7f8eac02be30 comp rx=0 tx=0).stop 2023-11-13T10:57:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.435+0000 7f8ec8911640 1 -- 172.21.15.16:0/997143188 shutdown_connections 2023-11-13T10:57:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.435+0000 7f8ec8911640 1 --2- 172.21.15.16:0/997143188 >> v2:172.21.15.184:3300/0 conn(0x7f8ec406a190 0x7f8ec406a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.435+0000 7f8ec8911640 1 --2- 172.21.15.16:0/997143188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec415b190 0x7f8ec406f970 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.435+0000 7f8ec8911640 1 -- 172.21.15.16:0/997143188 >> 172.21.15.16:0/997143188 conn(0x7f8ec40b04a0 msgr2=0x7f8ec40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.435+0000 7f8ec8911640 1 -- 172.21.15.16:0/997143188 shutdown_connections 2023-11-13T10:57:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.435+0000 7f8ec8911640 1 -- 172.21.15.16:0/997143188 wait complete. 2023-11-13T10:57:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.437+0000 7f8ec8911640 1 Processor -- start 2023-11-13T10:57:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.437+0000 7f8ec8911640 1 -- start start 2023-11-13T10:57:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.438+0000 7f8ec8911640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec406a190 0x7f8ec4140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.438+0000 7f8ec8911640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40cb190 0x7f8ec4153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.438+0000 7f8ec8911640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ec415b190 0x7f8ec406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.438+0000 7f8ec8911640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4143770 con 0x7f8ec40cb190 2023-11-13T10:57:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.438+0000 7f8ec8911640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4143a50 con 0x7f8ec406a190 2023-11-13T10:57:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.438+0000 7f8ec8911640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4143d30 con 0x7f8ec415b190 2023-11-13T10:57:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.438+0000 7f8ebaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40cb190 0x7f8ec4153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ebaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40cb190 0x7f8ec4153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56430/0 (socket says 172.21.15.16:56430) 2023-11-13T10:57:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ebaffd640 1 -- 172.21.15.16:0/2764389919 learned_addr learned my addr 172.21.15.16:0/2764389919 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ec37fe640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec406a190 0x7f8ec4140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ebaffd640 1 -- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ec415b190 msgr2=0x7f8ec406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ebaffd640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ec415b190 0x7f8ec406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ebaffd640 1 -- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec406a190 msgr2=0x7f8ec4140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ebaffd640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec406a190 0x7f8ec4140b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ebaffd640 1 -- 172.21.15.16:0/2764389919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac009a70 con 0x7f8ec40cb190 2023-11-13T10:57:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ec37fe640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec406a190 0x7f8ec4140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.439+0000 7f8ebaffd640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40cb190 0x7f8ec4153cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8eac037040 tx=0x7f8eac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.440+0000 7f8ec0ff9640 1 -- 172.21.15.16:0/2764389919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac040650 con 0x7f8ec40cb190 2023-11-13T10:57:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.440+0000 7f8ec0ff9640 1 -- 172.21.15.16:0/2764389919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac0407b0 con 0x7f8ec40cb190 2023-11-13T10:57:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.440+0000 7f8ec0ff9640 1 -- 172.21.15.16:0/2764389919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac040b60 con 0x7f8ec40cb190 2023-11-13T10:57:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.440+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec4144010 con 0x7f8ec40cb190 2023-11-13T10:57:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.440+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec41410c0 con 0x7f8ec40cb190 2023-11-13T10:57:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.441+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec404e4d0 con 0x7f8ec40cb190 2023-11-13T10:57:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.442+0000 7f8ec0ff9640 1 -- 172.21.15.16:0/2764389919 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8eac040cc0 con 0x7f8ec40cb190 2023-11-13T10:57:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.443+0000 7f8ec0ff9640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ea406ae40 0x7f8ea406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.443+0000 7f8ec0ff9640 1 -- 172.21.15.16:0/2764389919 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f8eac0ac7f0 con 0x7f8ec40cb190 2023-11-13T10:57:05.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.444+0000 7f8ec37fe640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ea406ae40 0x7f8ea406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.445+0000 7f8ec37fe640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ea406ae40 0x7f8ea406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40011e0 tx=0x7f8eb4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:05.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.447+0000 7f8ec0ff9640 1 -- 172.21.15.16:0/2764389919 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8eac077550 con 0x7f8ec40cb190 2023-11-13T10:57:05.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.644+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ec40b3100 con 0x7f8ec40cb190 2023-11-13T10:57:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.647+0000 7f8ec0ff9640 1 -- 172.21.15.16:0/2764389919 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f8eac07ab70 con 0x7f8ec40cb190 2023-11-13T10:57:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:05.653 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:57:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ea406ae40 msgr2=0x7f8ea406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ea406ae40 0x7f8ea406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40011e0 tx=0x7f8eb4009040 comp rx=0 tx=0).stop 2023-11-13T10:57:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40cb190 msgr2=0x7f8ec4153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40cb190 0x7f8ec4153cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8eac037040 tx=0x7f8eac004a40 comp rx=0 tx=0).stop 2023-11-13T10:57:05.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 shutdown_connections 2023-11-13T10:57:05.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ea406ae40 0x7f8ea406d300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ec415b190 0x7f8ec406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40cb190 0x7f8ec4153cb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 --2- 172.21.15.16:0/2764389919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ec406a190 0x7f8ec4140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 >> 172.21.15.16:0/2764389919 conn(0x7f8ec40b04a0 msgr2=0x7f8ec415d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.650+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 shutdown_connections 2023-11-13T10:57:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.651+0000 7f8ec8911640 1 -- 172.21.15.16:0/2764389919 wait complete. 2023-11-13T10:57:05.673 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-13T10:57:05.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:57:05.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.910+0000 7ff3e9741640 1 Processor -- start 2023-11-13T10:57:05.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.910+0000 7ff3e9741640 1 -- start start 2023-11-13T10:57:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.911+0000 7ff3e9741640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e4159170 0x7ff3e406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.911+0000 7ff3e9741640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e406d700 con 0x7ff3e415d500 2023-11-13T10:57:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.912+0000 7ff3e9741640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e406d950 con 0x7ff3e40adf90 2023-11-13T10:57:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.912+0000 7ff3e9741640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e406df10 con 0x7ff3e4159170 2023-11-13T10:57:05.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.912+0000 7ff3e37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff3e415d500 0x7ff3e4070220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60636/0 (socket says 172.21.15.16:60636) 2023-11-13T10:57:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.912+0000 7ff3e37fe640 1 -- 172.21.15.16:0/3378266486 learned_addr learned my addr 172.21.15.16:0/3378266486 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.912+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564154897 0 0) 0x7ff3e406d700 con 0x7ff3e415d500 2023-11-13T10:57:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.912+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d0003520 con 0x7ff3e415d500 2023-11-13T10:57:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.912+0000 7ff3e2ffd640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e4159170 0x7ff3e406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1047477196 0 0) 0x7ff3d0003520 con 0x7ff3e415d500 2023-11-13T10:57:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e4159170 msgr2=0x7ff3e406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e4159170 0x7ff3e406cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 >> v1:172.21.15.184:6789/0 conn(0x7ff3e40adf90 legacy=0x7ff3e415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e406dc30 con 0x7ff3e415d500 2023-11-13T10:57:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7ff3dc0019f0 con 0x7ff3e415d500 2023-11-13T10:57:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 >> v1:172.21.15.16:6789/0 conn(0x7ff3e415d500 legacy=0x7ff3e4070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d00039a0 0x7ff3d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e4159170 0x7ff3d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e406df10 con 0x7ff3d0004d50 2023-11-13T10:57:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d000bc00 con 0x7ff3d00039a0 2023-11-13T10:57:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d000bdc0 con 0x7ff3e4159170 2023-11-13T10:57:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3e37fe640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.913+0000 7ff3dbfff640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e4159170 0x7ff3d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.914+0000 7ff3e37fe640 1 -- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e4159170 msgr2=0x7ff3d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.914+0000 7ff3e37fe640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e4159170 0x7ff3d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.914+0000 7ff3e37fe640 1 -- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d00039a0 msgr2=0x7ff3d0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.914+0000 7ff3e2ffd640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d00039a0 0x7ff3d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.914+0000 7ff3e37fe640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3d00039a0 0x7ff3d0003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.914+0000 7ff3e37fe640 1 -- 172.21.15.16:0/3378266486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0003520 con 0x7ff3d0004d50 2023-11-13T10:57:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.914+0000 7ff3dbfff640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e4159170 0x7ff3d00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.915+0000 7ff3e37fe640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3d00051d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff3e415cd50 tx=0x7ff3d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3ca219ee7d8e94 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.915+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0072d0 con 0x7ff3d0004d50 2023-11-13T10:57:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.915+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc007430 con 0x7ff3d0004d50 2023-11-13T10:57:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.915+0000 7ff3e27fc640 1 -- 172.21.15.16:0/3378266486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc007820 con 0x7ff3d0004d50 2023-11-13T10:57:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.916+0000 7ff3e9741640 1 -- 172.21.15.16:0/3378266486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 msgr2=0x7ff3d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.916+0000 7ff3e9741640 1 --2- 172.21.15.16:0/3378266486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3d00051d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff3e415cd50 tx=0x7ff3d00057f0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.916+0000 7ff3e37fe640 1 -- 172.21.15.16:0/3378266486 reap_dead start 2023-11-13T10:57:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.916+0000 7ff3e9741640 1 -- 172.21.15.16:0/3378266486 shutdown_connections 2023-11-13T10:57:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.916+0000 7ff3e9741640 1 -- 172.21.15.16:0/3378266486 >> 172.21.15.16:0/3378266486 conn(0x7ff3e406a190 msgr2=0x7ff3e406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.916+0000 7ff3e9741640 1 -- 172.21.15.16:0/3378266486 shutdown_connections 2023-11-13T10:57:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.916+0000 7ff3e9741640 1 -- 172.21.15.16:0/3378266486 wait complete. 2023-11-13T10:57:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.918+0000 7ff3e9741640 1 Processor -- start 2023-11-13T10:57:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.918+0000 7ff3e9741640 1 -- start start 2023-11-13T10:57:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e9741640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3e4159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e9741640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e415b960 0x7ff3e40acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e9741640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3e40cd9d0 0x7ff3e40cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e9741640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e415ab20 con 0x7ff3d0004d50 2023-11-13T10:57:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e9741640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40ce390 con 0x7ff3e40cd9d0 2023-11-13T10:57:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e9741640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e4159ea0 con 0x7ff3e415b960 2023-11-13T10:57:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3e4159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3e4159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56460/0 (socket says 172.21.15.16:56460) 2023-11-13T10:57:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e2ffd640 1 -- 172.21.15.16:0/3988452023 learned_addr learned my addr 172.21.15.16:0/3988452023 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3dbfff640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e415b960 0x7ff3e40acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.919+0000 7ff3e2ffd640 1 -- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e415b960 msgr2=0x7ff3e40acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3e2ffd640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e415b960 0x7ff3e40acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3e2ffd640 1 -- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3e40cd9d0 msgr2=0x7ff3e40cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3e2ffd640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3e40cd9d0 0x7ff3e40cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3e2ffd640 1 -- 172.21.15.16:0/3988452023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e415a180 con 0x7ff3d0004d50 2023-11-13T10:57:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3dbfff640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e415b960 0x7ff3e40acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3e2ffd640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3e4159850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400ea50 tx=0x7ff3d400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3db7fe640 1 -- 172.21.15.16:0/3988452023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400ccc0 con 0x7ff3d0004d50 2023-11-13T10:57:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3db7fe640 1 -- 172.21.15.16:0/3988452023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d400ce20 con 0x7ff3d0004d50 2023-11-13T10:57:05.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3db7fe640 1 -- 172.21.15.16:0/3988452023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40115f0 con 0x7ff3d0004d50 2023-11-13T10:57:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e415a460 con 0x7ff3d0004d50 2023-11-13T10:57:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.920+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e415bde0 con 0x7ff3d0004d50 2023-11-13T10:57:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.922+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e41f33c0 con 0x7ff3d0004d50 2023-11-13T10:57:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.923+0000 7ff3db7fe640 1 -- 172.21.15.16:0/3988452023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3d4011750 con 0x7ff3d0004d50 2023-11-13T10:57:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.924+0000 7ff3db7fe640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff3c406ae40 0x7ff3c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.924+0000 7ff3db7fe640 1 -- 172.21.15.16:0/3988452023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7ff3d4089b80 con 0x7ff3d0004d50 2023-11-13T10:57:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.924+0000 7ff3dbfff640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff3c406ae40 0x7ff3c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.925+0000 7ff3dbfff640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff3c406ae40 0x7ff3c406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40cacd0 tx=0x7ff3cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:05.928+0000 7ff3db7fe640 1 -- 172.21.15.16:0/3988452023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3d4054840 con 0x7ff3d0004d50 2023-11-13T10:57:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.164+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3e415bf70 con 0x7ff3d0004d50 2023-11-13T10:57:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.165+0000 7ff3db7fe640 1 -- 172.21.15.16:0/3988452023 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7ff3d4021090 con 0x7ff3d0004d50 2023-11-13T10:57:06.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:06.170 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:53.560913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-13T10:57:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-11-13T10:57:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.168+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff3c406ae40 msgr2=0x7ff3c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.168+0000 7ff3e9741640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff3c406ae40 0x7ff3c406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40cacd0 tx=0x7ff3cc009040 comp rx=0 tx=0).stop 2023-11-13T10:57:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.168+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 msgr2=0x7ff3e4159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.168+0000 7ff3e9741640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3e4159850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400ea50 tx=0x7ff3d400ef10 comp rx=0 tx=0).stop 2023-11-13T10:57:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.169+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 shutdown_connections 2023-11-13T10:57:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.169+0000 7ff3e9741640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff3c406ae40 0x7ff3c406d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.169+0000 7ff3e9741640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3e40cd9d0 0x7ff3e40cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.169+0000 7ff3e9741640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3e415b960 0x7ff3e40acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.169+0000 7ff3e9741640 1 --2- 172.21.15.16:0/3988452023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0004d50 0x7ff3e4159850 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.169+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 >> 172.21.15.16:0/3988452023 conn(0x7ff3e406a190 msgr2=0x7ff3e4070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.169+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 shutdown_connections 2023-11-13T10:57:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.169+0000 7ff3e9741640 1 -- 172.21.15.16:0/3988452023 wait complete. 2023-11-13T10:57:06.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.419+0000 7f1c48f0a640 1 Processor -- start 2023-11-13T10:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.420+0000 7f1c48f0a640 1 -- start start 2023-11-13T10:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.421+0000 7f1c48f0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c4415cfc0 0x7f1c440cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.421+0000 7f1c48f0a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c440b3cf0 con 0x7f1c440b30e0 2023-11-13T10:57:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.421+0000 7f1c48f0a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c440b3730 con 0x7f1c440d2ae0 2023-11-13T10:57:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.421+0000 7f1c48f0a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c440b3550 con 0x7f1c4415cfc0 2023-11-13T10:57:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.421+0000 7f1c437fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c4415cfc0 0x7f1c440cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:06.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.421+0000 7f1c43fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1c440b30e0 0x7f1c440d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60652/0 (socket says 172.21.15.16:60652) 2023-11-13T10:57:06.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.421+0000 7f1c43fff640 1 -- 172.21.15.16:0/2115433676 learned_addr learned my addr 172.21.15.16:0/2115433676 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:06.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.422+0000 7f1c437fe640 1 -- 172.21.15.16:0/2115433676 >> v1:172.21.15.184:6789/0 conn(0x7f1c440d2ae0 legacy=0x7f1c440b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.422+0000 7f1c437fe640 1 -- 172.21.15.16:0/2115433676 >> v1:172.21.15.16:6789/0 conn(0x7f1c440b30e0 legacy=0x7f1c440d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.422+0000 7f1c437fe640 1 -- 172.21.15.16:0/2115433676 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c440b3a10 con 0x7f1c4415cfc0 2023-11-13T10:57:06.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.423+0000 7f1c437fe640 1 --2- 172.21.15.16:0/2115433676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c4415cfc0 0x7f1c440cd770 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009d90 tx=0x7f1c3402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71d32e1bf7dd8e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:06.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.423+0000 7f1c427fc640 1 -- 172.21.15.16:0/2115433676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3402cbd0 con 0x7f1c4415cfc0 2023-11-13T10:57:06.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.423+0000 7f1c427fc640 1 -- 172.21.15.16:0/2115433676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3402cd30 con 0x7f1c4415cfc0 2023-11-13T10:57:06.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.423+0000 7f1c427fc640 1 -- 172.21.15.16:0/2115433676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3402cea0 con 0x7f1c4415cfc0 2023-11-13T10:57:06.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.424+0000 7f1c48f0a640 1 -- 172.21.15.16:0/2115433676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c4415cfc0 msgr2=0x7f1c440cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.424+0000 7f1c48f0a640 1 --2- 172.21.15.16:0/2115433676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c4415cfc0 0x7f1c440cd770 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009d90 tx=0x7f1c3402be20 comp rx=0 tx=0).stop 2023-11-13T10:57:06.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.424+0000 7f1c48f0a640 1 -- 172.21.15.16:0/2115433676 shutdown_connections 2023-11-13T10:57:06.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.424+0000 7f1c48f0a640 1 --2- 172.21.15.16:0/2115433676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c4415cfc0 0x7f1c440cd770 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.424+0000 7f1c48f0a640 1 -- 172.21.15.16:0/2115433676 >> 172.21.15.16:0/2115433676 conn(0x7f1c4408c570 msgr2=0x7f1c4409be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:06.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.424+0000 7f1c48f0a640 1 -- 172.21.15.16:0/2115433676 shutdown_connections 2023-11-13T10:57:06.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.424+0000 7f1c48f0a640 1 -- 172.21.15.16:0/2115433676 wait complete. 2023-11-13T10:57:06.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.426+0000 7f1c48f0a640 1 Processor -- start 2023-11-13T10:57:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.426+0000 7f1c48f0a640 1 -- start start 2023-11-13T10:57:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.427+0000 7f1c48f0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c440b30e0 0x7f1c44150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.427+0000 7f1c48f0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c440d2ae0 0x7f1c440c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.427+0000 7f1c48f0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c4415cfc0 0x7f1c440b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.427+0000 7f1c48f0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4415d430 con 0x7f1c440d2ae0 2023-11-13T10:57:06.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.427+0000 7f1c48f0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4415e290 con 0x7f1c440b30e0 2023-11-13T10:57:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.427+0000 7f1c48f0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4415d680 con 0x7f1c4415cfc0 2023-11-13T10:57:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.427+0000 7f1c42ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c440d2ae0 0x7f1c440c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.427+0000 7f1c42ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c440d2ae0 0x7f1c440c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56480/0 (socket says 172.21.15.16:56480) 2023-11-13T10:57:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.427+0000 7f1c42ffd640 1 -- 172.21.15.16:0/1073057424 learned_addr learned my addr 172.21.15.16:0/1073057424 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.428+0000 7f1c437fe640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c440b30e0 0x7f1c44150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.428+0000 7f1c43fff640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c4415cfc0 0x7f1c440b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.428+0000 7f1c42ffd640 1 -- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c4415cfc0 msgr2=0x7f1c440b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.428+0000 7f1c42ffd640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c4415cfc0 0x7f1c440b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.428+0000 7f1c42ffd640 1 -- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c440b30e0 msgr2=0x7f1c44150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.428+0000 7f1c42ffd640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c440b30e0 0x7f1c44150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.428+0000 7f1c42ffd640 1 -- 172.21.15.16:0/1073057424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c34009a70 con 0x7f1c440d2ae0 2023-11-13T10:57:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.428+0000 7f1c42ffd640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c440d2ae0 0x7f1c440c9de0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800c460 tx=0x7f1c3800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.429+0000 7f1c2bfff640 1 -- 172.21.15.16:0/1073057424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38012830 con 0x7f1c440d2ae0 2023-11-13T10:57:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.429+0000 7f1c2bfff640 1 -- 172.21.15.16:0/1073057424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c380129d0 con 0x7f1c440d2ae0 2023-11-13T10:57:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.429+0000 7f1c2bfff640 1 -- 172.21.15.16:0/1073057424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38012d80 con 0x7f1c440d2ae0 2023-11-13T10:57:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.429+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4415d960 con 0x7f1c440d2ae0 2023-11-13T10:57:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.429+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4415df20 con 0x7f1c440d2ae0 2023-11-13T10:57:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.430+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4404e4d0 con 0x7f1c440d2ae0 2023-11-13T10:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.431+0000 7f1c2bfff640 1 -- 172.21.15.16:0/1073057424 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c3801a3b0 con 0x7f1c440d2ae0 2023-11-13T10:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.432+0000 7f1c2bfff640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c3006ae40 0x7f1c3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.432+0000 7f1c2bfff640 1 -- 172.21.15.16:0/1073057424 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f1c38089900 con 0x7f1c440d2ae0 2023-11-13T10:57:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.432+0000 7f1c437fe640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c3006ae40 0x7f1c3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.434+0000 7f1c437fe640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c3006ae40 0x7f1c3006d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1c34036000 tx=0x7f1c34030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.436+0000 7f1c2bfff640 1 -- 172.21.15.16:0/1073057424 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c380545d0 con 0x7f1c440d2ae0 2023-11-13T10:57:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.669+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c4415e3d0 con 0x7f1c440d2ae0 2023-11-13T10:57:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.670+0000 7f1c2bfff640 1 -- 172.21.15.16:0/1073057424 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f1c38057bf0 con 0x7f1c440d2ae0 2023-11-13T10:57:06.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:06.674 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3474457294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3474457294},{"type":"v1","addr":"172.21.15.184:6835","nonce":3474457294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/4195730200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":4195730200},{"type":"v1","addr":"172.21.15.184:6837","nonce":4195730200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3446345519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3446345519},{"type":"v1","addr":"172.21.15.16:6833","nonce":3446345519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:55:26.312366+0000","modified":"2023-11-13T10:55:53.560913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3826676517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3826676517},{"type":"v1","addr":"172.21.15.16:6835","nonce":3826676517}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-13T10:57:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-11-13T10:57:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c3006ae40 msgr2=0x7f1c3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c3006ae40 0x7f1c3006d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1c34036000 tx=0x7f1c34030000 comp rx=0 tx=0).stop 2023-11-13T10:57:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c440d2ae0 msgr2=0x7f1c440c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c440d2ae0 0x7f1c440c9de0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800c460 tx=0x7f1c3800c920 comp rx=0 tx=0).stop 2023-11-13T10:57:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 shutdown_connections 2023-11-13T10:57:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c3006ae40 0x7f1c3006d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c4415cfc0 0x7f1c440b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c440d2ae0 0x7f1c440c9de0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 --2- 172.21.15.16:0/1073057424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c440b30e0 0x7f1c44150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 >> 172.21.15.16:0/1073057424 conn(0x7f1c4408c570 msgr2=0x7f1c4406a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:06.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 shutdown_connections 2023-11-13T10:57:06.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.673+0000 7f1c48f0a640 1 -- 172.21.15.16:0/1073057424 wait complete. 2023-11-13T10:57:06.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:57:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.919+0000 7fa7d46af640 1 Processor -- start 2023-11-13T10:57:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.920+0000 7fa7d46af640 1 -- start start 2023-11-13T10:57:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.921+0000 7fa7d46af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc155410 0x7fa7cc0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.921+0000 7fa7d46af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa7cc0ce990 0x7fa7cc0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.921+0000 7fa7d46af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa7cc071440 0x7fa7cc0718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.921+0000 7fa7d46af640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7cc071e10 con 0x7fa7cc071440 2023-11-13T10:57:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.921+0000 7fa7d46af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7cc15b500 con 0x7fa7cc0ce990 2023-11-13T10:57:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.921+0000 7fa7d46af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc15b7e0 con 0x7fa7cc155410 2023-11-13T10:57:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.922+0000 7fa7d2c4c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa7cc071440 0x7fa7cc0718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.922+0000 7fa7d2c4c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa7cc071440 0x7fa7cc0718d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56494/0 (socket says 172.21.15.16:56494) 2023-11-13T10:57:06.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.922+0000 7fa7d2c4c640 1 -- 172.21.15.16:0/4074905128 learned_addr learned my addr 172.21.15.16:0/4074905128 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:06.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.922+0000 7fa7d2c4c640 1 -- 172.21.15.16:0/4074905128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc155410 msgr2=0x7fa7cc0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:06.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.922+0000 7fa7d2c4c640 1 --2- 172.21.15.16:0/4074905128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc155410 0x7fa7cc0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.922+0000 7fa7d2c4c640 1 -- 172.21.15.16:0/4074905128 >> v2:172.21.15.184:3300/0 conn(0x7fa7cc0ce990 msgr2=0x7fa7cc0cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.922+0000 7fa7d2c4c640 1 --2- 172.21.15.16:0/4074905128 >> v2:172.21.15.184:3300/0 conn(0x7fa7cc0ce990 0x7fa7cc0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.922+0000 7fa7d2c4c640 1 -- 172.21.15.16:0/4074905128 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc15bac0 con 0x7fa7cc071440 2023-11-13T10:57:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.924+0000 7fa7d2c4c640 1 --2- 172.21.15.16:0/4074905128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc071440 0x7fa7cc0718d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc06df10 tx=0x7fa7c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d338e2752a56bdf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.924+0000 7fa7d1449640 1 -- 172.21.15.16:0/4074905128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c402ca80 con 0x7fa7cc071440 2023-11-13T10:57:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.924+0000 7fa7d1449640 1 -- 172.21.15.16:0/4074905128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c402cbe0 con 0x7fa7cc071440 2023-11-13T10:57:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.924+0000 7fa7d1449640 1 -- 172.21.15.16:0/4074905128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c40343f0 con 0x7fa7cc071440 2023-11-13T10:57:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.925+0000 7fa7d46af640 1 -- 172.21.15.16:0/4074905128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc071440 msgr2=0x7fa7cc0718d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.925+0000 7fa7d46af640 1 --2- 172.21.15.16:0/4074905128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc071440 0x7fa7cc0718d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc06df10 tx=0x7fa7c402bee0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.925+0000 7fa7d46af640 1 -- 172.21.15.16:0/4074905128 shutdown_connections 2023-11-13T10:57:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.925+0000 7fa7d46af640 1 --2- 172.21.15.16:0/4074905128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc071440 0x7fa7cc0718d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.925+0000 7fa7d46af640 1 --2- 172.21.15.16:0/4074905128 >> v2:172.21.15.184:3300/0 conn(0x7fa7cc0ce990 0x7fa7cc0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.925+0000 7fa7d46af640 1 --2- 172.21.15.16:0/4074905128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc155410 0x7fa7cc0ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.925+0000 7fa7d46af640 1 -- 172.21.15.16:0/4074905128 >> 172.21.15.16:0/4074905128 conn(0x7fa7cc06a190 msgr2=0x7fa7cc06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.925+0000 7fa7d46af640 1 -- 172.21.15.16:0/4074905128 shutdown_connections 2023-11-13T10:57:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.925+0000 7fa7d46af640 1 -- 172.21.15.16:0/4074905128 wait complete. 2023-11-13T10:57:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.927+0000 7fa7d46af640 1 Processor -- start 2023-11-13T10:57:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.927+0000 7fa7d46af640 1 -- start start 2023-11-13T10:57:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.928+0000 7fa7d46af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc071440 0x7fa7cc14df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.928+0000 7fa7d46af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc0ce990 0x7fa7cc153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d46af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7cc155410 0x7fa7cc153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d46af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc14eaf0 con 0x7fa7cc0ce990 2023-11-13T10:57:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d46af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc15d840 con 0x7fa7cc155410 2023-11-13T10:57:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d46af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1559a0 con 0x7fa7cc071440 2023-11-13T10:57:06.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d1c4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc0ce990 0x7fa7cc153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d1c4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc0ce990 0x7fa7cc153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56508/0 (socket says 172.21.15.16:56508) 2023-11-13T10:57:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d1c4a640 1 -- 172.21.15.16:0/2339402646 learned_addr learned my addr 172.21.15.16:0/2339402646 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d1c4a640 1 -- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc071440 msgr2=0x7fa7cc14df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d1c4a640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc071440 0x7fa7cc14df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d1c4a640 1 -- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7cc155410 msgr2=0x7fa7cc153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d1c4a640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7cc155410 0x7fa7cc153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.929+0000 7fa7d1c4a640 1 -- 172.21.15.16:0/2339402646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c4009b30 con 0x7fa7cc0ce990 2023-11-13T10:57:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.930+0000 7fa7d1c4a640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc0ce990 0x7fa7cc153220 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000b4a0 tx=0x7fa7c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.930+0000 7fa7beffd640 1 -- 172.21.15.16:0/2339402646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0010610 con 0x7fa7cc0ce990 2023-11-13T10:57:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.930+0000 7fa7beffd640 1 -- 172.21.15.16:0/2339402646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c0010770 con 0x7fa7cc0ce990 2023-11-13T10:57:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.930+0000 7fa7beffd640 1 -- 172.21.15.16:0/2339402646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0010b20 con 0x7fa7cc0ce990 2023-11-13T10:57:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.930+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc155c80 con 0x7fa7cc0ce990 2023-11-13T10:57:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.930+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc156240 con 0x7fa7cc0ce990 2023-11-13T10:57:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.932+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7cc156460 con 0x7fa7cc0ce990 2023-11-13T10:57:06.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.933+0000 7fa7beffd640 1 -- 172.21.15.16:0/2339402646 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7c0010c80 con 0x7fa7cc0ce990 2023-11-13T10:57:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.934+0000 7fa7beffd640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b006ae40 0x7fa7b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:06.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.934+0000 7fa7beffd640 1 -- 172.21.15.16:0/2339402646 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fa7c0089560 con 0x7fa7cc0ce990 2023-11-13T10:57:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.935+0000 7fa7d244b640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b006ae40 0x7fa7b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.936+0000 7fa7d244b640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b006ae40 0x7fa7b006d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009670 tx=0x7fa7c8006bf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:06.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:06.938+0000 7fa7beffd640 1 -- 172.21.15.16:0/2339402646 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7c00542c0 con 0x7fa7cc0ce990 2023-11-13T10:57:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.133+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7cc067090 con 0x7fa7cc0ce990 2023-11-13T10:57:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.136+0000 7fa7beffd640 1 -- 172.21.15.16:0/2339402646 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fa7c00578e0 con 0x7fa7cc0ce990 2023-11-13T10:57:07.141 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:07.141 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:56:07.890744+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:55:24.262070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-13T10:55:25.326765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-13T10:55:28.924045+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.2","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":4}]},{"pgid":"4.16","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:57:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b006ae40 msgr2=0x7fa7b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b006ae40 0x7fa7b006d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009670 tx=0x7fa7c8006bf0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc0ce990 msgr2=0x7fa7cc153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc0ce990 0x7fa7cc153220 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000b4a0 tx=0x7fa7c000b960 comp rx=0 tx=0).stop 2023-11-13T10:57:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 shutdown_connections 2023-11-13T10:57:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b006ae40 0x7fa7b006d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7cc155410 0x7fa7cc153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7cc0ce990 0x7fa7cc153220 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 --2- 172.21.15.16:0/2339402646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc071440 0x7fa7cc14df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 >> 172.21.15.16:0/2339402646 conn(0x7fa7cc06a190 msgr2=0x7fa7cc070220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 shutdown_connections 2023-11-13T10:57:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.139+0000 7fa7d46af640 1 -- 172.21.15.16:0/2339402646 wait complete. 2023-11-13T10:57:07.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-13T10:57:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.378+0000 7fc35d7d2640 1 Processor -- start 2023-11-13T10:57:07.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.378+0000 7fc35d7d2640 1 -- start start 2023-11-13T10:57:07.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.379+0000 7fc35d7d2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 0x7fc35806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.380+0000 7fc35d7d2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc35806a190 0x7fc35806a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.380+0000 7fc35d7d2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc358158c20 con 0x7fc3580cb190 2023-11-13T10:57:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.380+0000 7fc35d7d2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3580b1830 con 0x7fc35806a190 2023-11-13T10:57:07.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.380+0000 7fc35d7d2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35806ab10 con 0x7fc35815b190 2023-11-13T10:57:07.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.380+0000 7fc35cfd1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc35806a190 0x7fc35806a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.380+0000 7fc35cfd1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc35806a190 0x7fc35806a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56518/0 (socket says 172.21.15.16:56518) 2023-11-13T10:57:07.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.380+0000 7fc35cfd1640 1 -- 172.21.15.16:0/4234313614 learned_addr learned my addr 172.21.15.16:0/4234313614 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:07.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.380+0000 7fc34ffff640 1 --2- 172.21.15.16:0/4234313614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 0x7fc35806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.381+0000 7fc35cfd1640 1 -- 172.21.15.16:0/4234313614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 msgr2=0x7fc35806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:07.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.381+0000 7fc35cfd1640 1 --2- 172.21.15.16:0/4234313614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 0x7fc35806f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.381+0000 7fc35cfd1640 1 -- 172.21.15.16:0/4234313614 >> v1:172.21.15.184:6789/0 conn(0x7fc3580cb190 legacy=0x7fc3580716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:07.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.381+0000 7fc35cfd1640 1 -- 172.21.15.16:0/4234313614 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35815b780 con 0x7fc35806a190 2023-11-13T10:57:07.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.381+0000 7fc34ffff640 1 --2- 172.21.15.16:0/4234313614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 0x7fc35806f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:07.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.382+0000 7fc35cfd1640 1 --2- 172.21.15.16:0/4234313614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35806a190 0x7fc35806a5d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc350009e50 tx=0x7fc35002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79454238feea141d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:07.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.382+0000 7fc3577fe640 1 -- 172.21.15.16:0/4234313614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35002c9f0 con 0x7fc35806a190 2023-11-13T10:57:07.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.382+0000 7fc3577fe640 1 -- 172.21.15.16:0/4234313614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35002cb50 con 0x7fc35806a190 2023-11-13T10:57:07.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.382+0000 7fc3577fe640 1 -- 172.21.15.16:0/4234313614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3500343f0 con 0x7fc35806a190 2023-11-13T10:57:07.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.383+0000 7fc35d7d2640 1 -- 172.21.15.16:0/4234313614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35806a190 msgr2=0x7fc35806a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.383+0000 7fc35d7d2640 1 --2- 172.21.15.16:0/4234313614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35806a190 0x7fc35806a5d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc350009e50 tx=0x7fc35002be50 comp rx=0 tx=0).stop 2023-11-13T10:57:07.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.383+0000 7fc35d7d2640 1 -- 172.21.15.16:0/4234313614 shutdown_connections 2023-11-13T10:57:07.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.383+0000 7fc35d7d2640 1 --2- 172.21.15.16:0/4234313614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35806a190 0x7fc35806a5d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.383+0000 7fc35d7d2640 1 --2- 172.21.15.16:0/4234313614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 0x7fc35806f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.383+0000 7fc35d7d2640 1 -- 172.21.15.16:0/4234313614 >> 172.21.15.16:0/4234313614 conn(0x7fc3580b04a0 msgr2=0x7fc3580b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:07.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.383+0000 7fc35d7d2640 1 -- 172.21.15.16:0/4234313614 shutdown_connections 2023-11-13T10:57:07.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.383+0000 7fc35d7d2640 1 -- 172.21.15.16:0/4234313614 wait complete. 2023-11-13T10:57:07.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.385+0000 7fc35d7d2640 1 Processor -- start 2023-11-13T10:57:07.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.385+0000 7fc35d7d2640 1 -- start start 2023-11-13T10:57:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.386+0000 7fc35d7d2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc35806a190 0x7fc3581ee410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.386+0000 7fc35d7d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580cb190 0x7fc3581f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.386+0000 7fc35d7d2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 0x7fc358154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.386+0000 7fc35d7d2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35806ba70 con 0x7fc3580cb190 2023-11-13T10:57:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.386+0000 7fc35d7d2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3581f1720 con 0x7fc35806a190 2023-11-13T10:57:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.386+0000 7fc35d7d2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc358154060 con 0x7fc35815b190 2023-11-13T10:57:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc34ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580cb190 0x7fc3581f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc34ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580cb190 0x7fc3581f11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56534/0 (socket says 172.21.15.16:56534) 2023-11-13T10:57:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc34ffff640 1 -- 172.21.15.16:0/3299486188 learned_addr learned my addr 172.21.15.16:0/3299486188 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc357fff640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc35806a190 0x7fc3581ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc35cfd1640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 0x7fc358154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc34ffff640 1 -- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 msgr2=0x7fc358154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc34ffff640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 0x7fc358154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc34ffff640 1 -- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc35806a190 msgr2=0x7fc3581ee410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc34ffff640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc35806a190 0x7fc3581ee410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.387+0000 7fc34ffff640 1 -- 172.21.15.16:0/3299486188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc350009b30 con 0x7fc3580cb190 2023-11-13T10:57:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.388+0000 7fc34ffff640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580cb190 0x7fc3581f11e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c7f0 tx=0x7fc34000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.388+0000 7fc3557fa640 1 -- 172.21.15.16:0/3299486188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340007820 con 0x7fc3580cb190 2023-11-13T10:57:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.388+0000 7fc3557fa640 1 -- 172.21.15.16:0/3299486188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc340007980 con 0x7fc3580cb190 2023-11-13T10:57:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.388+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc358154340 con 0x7fc3580cb190 2023-11-13T10:57:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.388+0000 7fc3557fa640 1 -- 172.21.15.16:0/3299486188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340007cf0 con 0x7fc3580cb190 2023-11-13T10:57:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.388+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35806da30 con 0x7fc3580cb190 2023-11-13T10:57:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.389+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35804e4d0 con 0x7fc3580cb190 2023-11-13T10:57:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.390+0000 7fc3557fa640 1 -- 172.21.15.16:0/3299486188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3400113f0 con 0x7fc3580cb190 2023-11-13T10:57:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.391+0000 7fc3557fa640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc33806ae40 0x7fc33806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.391+0000 7fc3557fa640 1 -- 172.21.15.16:0/3299486188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fc340089d30 con 0x7fc3580cb190 2023-11-13T10:57:07.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.392+0000 7fc357fff640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc33806ae40 0x7fc33806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.393+0000 7fc357fff640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc33806ae40 0x7fc33806d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc348004ea0 tx=0x7fc348009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.395+0000 7fc3557fa640 1 -- 172.21.15.16:0/3299486188 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc340054a00 con 0x7fc3580cb190 2023-11-13T10:57:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.626+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc35806bbb0 con 0x7fc3580cb190 2023-11-13T10:57:07.630 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-13T10:57:07.627+0000 7fe1550d2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-13T10:57:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.630+0000 7fc3557fa640 1 -- 172.21.15.16:0/3299486188 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc34001f090 con 0x7fc3580cb190 2023-11-13T10:57:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-13T10:57:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.634+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc33806ae40 msgr2=0x7fc33806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.634+0000 7fc35d7d2640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc33806ae40 0x7fc33806d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc348004ea0 tx=0x7fc348009040 comp rx=0 tx=0).stop 2023-11-13T10:57:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.634+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580cb190 msgr2=0x7fc3581f11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.634+0000 7fc35d7d2640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580cb190 0x7fc3581f11e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c7f0 tx=0x7fc34000ccb0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.634+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 shutdown_connections 2023-11-13T10:57:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.634+0000 7fc35d7d2640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc33806ae40 0x7fc33806d300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.634+0000 7fc35d7d2640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc35815b190 0x7fc358154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.634+0000 7fc35d7d2640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3580cb190 0x7fc3581f11e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.634+0000 7fc35d7d2640 1 --2- 172.21.15.16:0/3299486188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc35806a190 0x7fc3581ee410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.635+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 >> 172.21.15.16:0/3299486188 conn(0x7fc3580b04a0 msgr2=0x7fc3580ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.635+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 shutdown_connections 2023-11-13T10:57:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.635+0000 7fc35d7d2640 1 -- 172.21.15.16:0/3299486188 wait complete. 2023-11-13T10:57:07.658 INFO:tasks.ceph.mds.c.smithi016.stderr: -5381> 2023-11-13T10:55:32.092+0000 7f16742f5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-13T10:57:07.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-13T10:57:07.728 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-13T10:57:07.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.854+0000 7f44713e9640 1 Processor -- start 2023-11-13T10:57:07.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.855+0000 7f44713e9640 1 -- start start 2023-11-13T10:57:07.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.856+0000 7f44713e9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f446c15ce70 0x7f446c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.856+0000 7f44713e9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c0ce0e0 con 0x7f446c0d1850 2023-11-13T10:57:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.856+0000 7f44713e9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c0ce3c0 con 0x7f446c0cdbb0 2023-11-13T10:57:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.856+0000 7f44713e9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446c0ce6a0 con 0x7f446c15ce70 2023-11-13T10:57:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.856+0000 7f4470be8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f446c0d1850 0x7f446c0c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60672/0 (socket says 172.21.15.16:60672) 2023-11-13T10:57:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.856+0000 7f4470be8640 1 -- 172.21.15.16:0/1095035344 learned_addr learned my addr 172.21.15.16:0/1095035344 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446bfff640 1 --2- 172.21.15.16:0/1095035344 >> v2:172.21.15.184:3300/0 conn(0x7f446c15ce70 0x7f446c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315239618 0 0) 0x7f446c0ce0e0 con 0x7f446c0d1850 2023-11-13T10:57:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4458003520 con 0x7f446c0d1850 2023-11-13T10:57:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397886048 0 0) 0x7f4458003520 con 0x7f446c0d1850 2023-11-13T10:57:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 >> v2:172.21.15.184:3300/0 conn(0x7f446c15ce70 msgr2=0x7f446c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 --2- 172.21.15.16:0/1095035344 >> v2:172.21.15.184:3300/0 conn(0x7f446c15ce70 0x7f446c0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 >> v1:172.21.15.184:6789/0 conn(0x7f446c0cdbb0 legacy=0x7f446c0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0ce980 con 0x7f446c0d1850 2023-11-13T10:57:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f4464001670 con 0x7f446c0d1850 2023-11-13T10:57:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 >> v1:172.21.15.16:6789/0 conn(0x7f446c0d1850 legacy=0x7f446c0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 --2- 172.21.15.16:0/1095035344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4458003970 0x7f4458003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.857+0000 7f446affd640 1 --2- 172.21.15.16:0/1095035344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44580047c0 0x7f4458004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446affd640 1 --2- 172.21.15.16:0/1095035344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4458005180 0x7f445800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0ce6a0 con 0x7f4458003970 2023-11-13T10:57:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445800c020 con 0x7f4458005180 2023-11-13T10:57:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445800c1e0 con 0x7f44580047c0 2023-11-13T10:57:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446bfff640 1 --2- 172.21.15.16:0/1095035344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4458003970 0x7f4458003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446bfff640 1 -- 172.21.15.16:0/1095035344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44580047c0 msgr2=0x7f4458004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446bfff640 1 --2- 172.21.15.16:0/1095035344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44580047c0 0x7f4458004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446bfff640 1 -- 172.21.15.16:0/1095035344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4458005180 msgr2=0x7f445800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446bfff640 1 --2- 172.21.15.16:0/1095035344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4458005180 0x7f445800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f446bfff640 1 -- 172.21.15.16:0/1095035344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4458003520 con 0x7f4458003970 2023-11-13T10:57:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.858+0000 7f4470be8640 1 -- 172.21.15.16:0/1095035344 reap_dead start 2023-11-13T10:57:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.859+0000 7f446bfff640 1 --2- 172.21.15.16:0/1095035344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4458003970 0x7f4458003d90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f445c009fc0 tx=0x7f44580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4880784b32b277a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c02cc60 con 0x7f4458003970 2023-11-13T10:57:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c00f040 con 0x7f4458003970 2023-11-13T10:57:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f446affd640 1 -- 172.21.15.16:0/1095035344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c034440 con 0x7f4458003970 2023-11-13T10:57:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f44713e9640 1 -- 172.21.15.16:0/1095035344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4458003970 msgr2=0x7f4458003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f44713e9640 1 --2- 172.21.15.16:0/1095035344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4458003970 0x7f4458003d90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f445c009fc0 tx=0x7f44580057f0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f44713e9640 1 -- 172.21.15.16:0/1095035344 shutdown_connections 2023-11-13T10:57:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f44713e9640 1 --2- 172.21.15.16:0/1095035344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4458003970 0x7f4458003d90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f44713e9640 1 -- 172.21.15.16:0/1095035344 >> 172.21.15.16:0/1095035344 conn(0x7f446c08c580 msgr2=0x7f446c09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f44713e9640 1 -- 172.21.15.16:0/1095035344 shutdown_connections 2023-11-13T10:57:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.860+0000 7f44713e9640 1 -- 172.21.15.16:0/1095035344 wait complete. 2023-11-13T10:57:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.862+0000 7f44713e9640 1 Processor -- start 2023-11-13T10:57:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.862+0000 7f44713e9640 1 -- start start 2023-11-13T10:57:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.863+0000 7f44713e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4458003970 0x7f446c152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.863+0000 7f44713e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f446c155400 0x7f446c1f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.863+0000 7f44713e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c155880 0x7f446c06b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.863+0000 7f44713e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c1f6970 con 0x7f446c155400 2023-11-13T10:57:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.863+0000 7f44713e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c1f6c50 con 0x7f446c155880 2023-11-13T10:57:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.863+0000 7f44713e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c154c00 con 0x7f4458003970 2023-11-13T10:57:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.863+0000 7f446b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f446c155400 0x7f446c1f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.863+0000 7f446b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f446c155400 0x7f446c1f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56576/0 (socket says 172.21.15.16:56576) 2023-11-13T10:57:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.863+0000 7f446b7fe640 1 -- 172.21.15.16:0/3021205416 learned_addr learned my addr 172.21.15.16:0/3021205416 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f446bfff640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4458003970 0x7f446c152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f446b7fe640 1 -- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4458003970 msgr2=0x7f446c152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f446b7fe640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4458003970 0x7f446c152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f446b7fe640 1 -- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c155880 msgr2=0x7f446c06b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f4470be8640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c155880 0x7f446c06b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f446b7fe640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c155880 0x7f446c06b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:07.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f446b7fe640 1 -- 172.21.15.16:0/3021205416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c154ee0 con 0x7f446c155400 2023-11-13T10:57:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f446b7fe640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f446c155400 0x7f446c1f5ef0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f445400c3c0 tx=0x7f445400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f4468ff9640 1 -- 172.21.15.16:0/3021205416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44540052d0 con 0x7f446c155400 2023-11-13T10:57:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f4468ff9640 1 -- 172.21.15.16:0/3021205416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4454005450 con 0x7f446c155400 2023-11-13T10:57:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f4468ff9640 1 -- 172.21.15.16:0/3021205416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454007260 con 0x7f446c155400 2023-11-13T10:57:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.864+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c1525a0 con 0x7f446c155400 2023-11-13T10:57:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.865+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c1600d0 con 0x7f446c155400 2023-11-13T10:57:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.865+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c04e4d0 con 0x7f446c155400 2023-11-13T10:57:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.867+0000 7f4468ff9640 1 -- 172.21.15.16:0/3021205416 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44540073c0 con 0x7f446c155400 2023-11-13T10:57:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.868+0000 7f4468ff9640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f444c06ae40 0x7f444c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.868+0000 7f4468ff9640 1 -- 172.21.15.16:0/3021205416 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6662+0+0 (secure 0 0 0) 0x7f4454088dc0 con 0x7f446c155400 2023-11-13T10:57:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.868+0000 7f446bfff640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f444c06ae40 0x7f444c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.869+0000 7f446bfff640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f444c06ae40 0x7f444c06d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f445c036000 tx=0x7f445c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:07.872+0000 7f4468ff9640 1 -- 172.21.15.16:0/3021205416 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4454053ad0 con 0x7f446c155400 2023-11-13T10:57:08.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.094+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f446c06a7c0 con 0x7f446c155400 2023-11-13T10:57:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.634+0000 7f4468ff9640 1 -- 172.21.15.16:0/3021205416 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f44540570f0 con 0x7f446c155400 2023-11-13T10:57:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f444c06ae40 msgr2=0x7f444c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f444c06ae40 0x7f444c06d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f445c036000 tx=0x7f445c02e000 comp rx=0 tx=0).stop 2023-11-13T10:57:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f446c155400 msgr2=0x7f446c1f5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f446c155400 0x7f446c1f5ef0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f445400c3c0 tx=0x7f445400c880 comp rx=0 tx=0).stop 2023-11-13T10:57:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 shutdown_connections 2023-11-13T10:57:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f444c06ae40 0x7f444c06d300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f446c155880 0x7f446c06b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f446c155400 0x7f446c1f5ef0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 --2- 172.21.15.16:0/3021205416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4458003970 0x7f446c152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 >> 172.21.15.16:0/3021205416 conn(0x7f446c08c580 msgr2=0x7f446c0cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.638+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 shutdown_connections 2023-11-13T10:57:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.639+0000 7f44713e9640 1 -- 172.21.15.16:0/3021205416 wait complete. 2023-11-13T10:57:08.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-13T10:57:08.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.893+0000 7fafa0820640 1 Processor -- start 2023-11-13T10:57:08.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.894+0000 7fafa0820640 1 -- start start 2023-11-13T10:57:08.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.895+0000 7fafa0820640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf9c1608c0 0x7faf9c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:08.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.895+0000 7fafa0820640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faf9c160ca0 0x7faf9c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:08.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.895+0000 7fafa0820640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c0d1b80 con 0x7faf9c15c7a0 2023-11-13T10:57:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.895+0000 7fafa0820640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c0cea50 con 0x7faf9c1608c0 2023-11-13T10:57:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.895+0000 7fafa0820640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c0d1000 con 0x7faf9c160ca0 2023-11-13T10:57:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.895+0000 7faf9ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf9c1608c0 0x7faf9c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.895+0000 7faf9ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf9c1608c0 0x7faf9c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56582/0 (socket says 172.21.15.16:56582) 2023-11-13T10:57:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.896+0000 7faf9ad9d640 1 -- 172.21.15.16:0/319281110 learned_addr learned my addr 172.21.15.16:0/319281110 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:08.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.896+0000 7faf9a59c640 1 --2- 172.21.15.16:0/319281110 >> v2:172.21.15.184:3300/0 conn(0x7faf9c160ca0 0x7faf9c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.896+0000 7faf9ad9d640 1 -- 172.21.15.16:0/319281110 >> v2:172.21.15.184:3300/0 conn(0x7faf9c160ca0 msgr2=0x7faf9c0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.896+0000 7faf9ad9d640 1 --2- 172.21.15.16:0/319281110 >> v2:172.21.15.184:3300/0 conn(0x7faf9c160ca0 0x7faf9c0d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.896+0000 7faf9ad9d640 1 -- 172.21.15.16:0/319281110 >> v1:172.21.15.184:6789/0 conn(0x7faf9c15c7a0 legacy=0x7faf9c15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.896+0000 7faf9ad9d640 1 -- 172.21.15.16:0/319281110 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0d12e0 con 0x7faf9c1608c0 2023-11-13T10:57:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.897+0000 7faf9ad9d640 1 --2- 172.21.15.16:0/319281110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c1608c0 0x7faf9c0ce510 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0d2aa0 tx=0x7faf8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ad8f680c922ab server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7faf99d9b640 1 -- 172.21.15.16:0/319281110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c031870 con 0x7faf9c1608c0 2023-11-13T10:57:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7faf99d9b640 1 -- 172.21.15.16:0/319281110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c0319d0 con 0x7faf9c1608c0 2023-11-13T10:57:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7faf99d9b640 1 -- 172.21.15.16:0/319281110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c031d60 con 0x7faf9c1608c0 2023-11-13T10:57:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7fafa0820640 1 -- 172.21.15.16:0/319281110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c1608c0 msgr2=0x7faf9c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7fafa0820640 1 --2- 172.21.15.16:0/319281110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c1608c0 0x7faf9c0ce510 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0d2aa0 tx=0x7faf8c02b910 comp rx=0 tx=0).stop 2023-11-13T10:57:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7fafa0820640 1 -- 172.21.15.16:0/319281110 shutdown_connections 2023-11-13T10:57:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7fafa0820640 1 --2- 172.21.15.16:0/319281110 >> v2:172.21.15.184:3300/0 conn(0x7faf9c160ca0 0x7faf9c0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7fafa0820640 1 --2- 172.21.15.16:0/319281110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c1608c0 0x7faf9c0ce510 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7fafa0820640 1 -- 172.21.15.16:0/319281110 >> 172.21.15.16:0/319281110 conn(0x7faf9c0d2b50 msgr2=0x7faf9c0d2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.898+0000 7fafa0820640 1 -- 172.21.15.16:0/319281110 shutdown_connections 2023-11-13T10:57:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.899+0000 7fafa0820640 1 -- 172.21.15.16:0/319281110 wait complete. 2023-11-13T10:57:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.901+0000 7fafa0820640 1 Processor -- start 2023-11-13T10:57:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.901+0000 7fafa0820640 1 -- start start 2023-11-13T10:57:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.902+0000 7fafa0820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c15c7a0 0x7faf9c0c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.902+0000 7fafa0820640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf9c1608c0 0x7faf9c0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.902+0000 7fafa0820640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf9c160ca0 0x7faf9c0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.902+0000 7fafa0820640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c149270 con 0x7faf9c15c7a0 2023-11-13T10:57:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.902+0000 7fafa0820640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c15c060 con 0x7faf9c160ca0 2023-11-13T10:57:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.902+0000 7fafa0820640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0cb4a0 con 0x7faf9c1608c0 2023-11-13T10:57:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c15c7a0 0x7faf9c0c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c15c7a0 0x7faf9c0c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56588/0 (socket says 172.21.15.16:56588) 2023-11-13T10:57:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9ad9d640 1 -- 172.21.15.16:0/460229885 learned_addr learned my addr 172.21.15.16:0/460229885 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9b59e640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf9c160ca0 0x7faf9c0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9a59c640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf9c1608c0 0x7faf9c0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9ad9d640 1 -- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf9c1608c0 msgr2=0x7faf9c0ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9ad9d640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf9c1608c0 0x7faf9c0ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9ad9d640 1 -- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf9c160ca0 msgr2=0x7faf9c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9ad9d640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf9c160ca0 0x7faf9c0cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9ad9d640 1 -- 172.21.15.16:0/460229885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0094d0 con 0x7faf9c15c7a0 2023-11-13T10:57:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9b59e640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf9c160ca0 0x7faf9c0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.903+0000 7faf9ad9d640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c15c7a0 0x7faf9c0c8d00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faf8c000c00 tx=0x7faf8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.904+0000 7faf837fe640 1 -- 172.21.15.16:0/460229885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c031840 con 0x7faf9c15c7a0 2023-11-13T10:57:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.904+0000 7faf837fe640 1 -- 172.21.15.16:0/460229885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c0443f0 con 0x7faf9c15c7a0 2023-11-13T10:57:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.904+0000 7faf837fe640 1 -- 172.21.15.16:0/460229885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c0447a0 con 0x7faf9c15c7a0 2023-11-13T10:57:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.904+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c0cb780 con 0x7faf9c15c7a0 2023-11-13T10:57:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.904+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c148e90 con 0x7faf9c15c7a0 2023-11-13T10:57:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.905+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c04e4d0 con 0x7faf9c15c7a0 2023-11-13T10:57:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.906+0000 7faf837fe640 1 -- 172.21.15.16:0/460229885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf8c04d450 con 0x7faf9c15c7a0 2023-11-13T10:57:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.907+0000 7faf837fe640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf8806ae70 0x7faf8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.907+0000 7faf837fe640 1 -- 172.21.15.16:0/460229885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6662+0+0 (secure 0 0 0) 0x7faf8c02d040 con 0x7faf9c15c7a0 2023-11-13T10:57:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.907+0000 7faf9a59c640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf8806ae70 0x7faf8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:08.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.909+0000 7faf9a59c640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf8806ae70 0x7faf8806d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faf90006fd0 tx=0x7faf90008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:08.911+0000 7faf837fe640 1 -- 172.21.15.16:0/460229885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf8c076ce0 con 0x7faf9c15c7a0 2023-11-13T10:57:09.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.104+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf9c0d1b80 con 0x7faf9c15c7a0 2023-11-13T10:57:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.644+0000 7faf837fe640 1 -- 172.21.15.16:0/460229885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7faf9c0d1b80 con 0x7faf9c15c7a0 2023-11-13T10:57:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-13T10:57:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf8806ae70 msgr2=0x7faf8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf8806ae70 0x7faf8806d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faf90006fd0 tx=0x7faf90008040 comp rx=0 tx=0).stop 2023-11-13T10:57:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c15c7a0 msgr2=0x7faf9c0c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c15c7a0 0x7faf9c0c8d00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faf8c000c00 tx=0x7faf8c0046c0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 shutdown_connections 2023-11-13T10:57:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faf8806ae70 0x7faf8806d330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf9c160ca0 0x7faf9c0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf9c1608c0 0x7faf9c0ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 --2- 172.21.15.16:0/460229885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c15c7a0 0x7faf9c0c8d00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.647+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 >> 172.21.15.16:0/460229885 conn(0x7faf9c0d2b50 msgr2=0x7faf9c06bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:09.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.648+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 shutdown_connections 2023-11-13T10:57:09.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.648+0000 7fafa0820640 1 -- 172.21.15.16:0/460229885 wait complete. 2023-11-13T10:57:09.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-13T10:57:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.922+0000 7fb43aa34640 1 Processor -- start 2023-11-13T10:57:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.922+0000 7fb43aa34640 1 -- start start 2023-11-13T10:57:09.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.923+0000 7fb43aa34640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340cdae0 0x7fb43406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:09.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.923+0000 7fb43aa34640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb43415e340 0x7fb43406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:09.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.923+0000 7fb43aa34640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb43406e2a0 0x7fb43406ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:09.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.924+0000 7fb43aa34640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4340cdec0 con 0x7fb43415e340 2023-11-13T10:57:09.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.924+0000 7fb43aa34640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4340cf1c0 con 0x7fb43406e2a0 2023-11-13T10:57:09.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.924+0000 7fb43aa34640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340cf4a0 con 0x7fb4340cdae0 2023-11-13T10:57:09.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.924+0000 7fb4337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb43415e340 0x7fb43406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:09.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.924+0000 7fb4337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb43415e340 0x7fb43406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56596/0 (socket says 172.21.15.16:56596) 2023-11-13T10:57:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.924+0000 7fb4337fe640 1 -- 172.21.15.16:0/1990652594 learned_addr learned my addr 172.21.15.16:0/1990652594 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.924+0000 7fb433fff640 1 --2- 172.21.15.16:0/1990652594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340cdae0 0x7fb43406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.924+0000 7fb438fd1640 1 --2- 172.21.15.16:0/1990652594 >> v2:172.21.15.184:3300/0 conn(0x7fb43406e2a0 0x7fb43406ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.925+0000 7fb4337fe640 1 -- 172.21.15.16:0/1990652594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340cdae0 msgr2=0x7fb43406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.925+0000 7fb4337fe640 1 --2- 172.21.15.16:0/1990652594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340cdae0 0x7fb43406dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.925+0000 7fb4337fe640 1 -- 172.21.15.16:0/1990652594 >> v2:172.21.15.184:3300/0 conn(0x7fb43406e2a0 msgr2=0x7fb43406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.925+0000 7fb4337fe640 1 --2- 172.21.15.16:0/1990652594 >> v2:172.21.15.184:3300/0 conn(0x7fb43406e2a0 0x7fb43406ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.925+0000 7fb4337fe640 1 -- 172.21.15.16:0/1990652594 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340cf780 con 0x7fb43415e340 2023-11-13T10:57:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.925+0000 7fb438fd1640 1 --2- 172.21.15.16:0/1990652594 >> v2:172.21.15.184:3300/0 conn(0x7fb43406e2a0 0x7fb43406ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:57:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.926+0000 7fb4337fe640 1 --2- 172.21.15.16:0/1990652594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43415e340 0x7fb43406e720 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb4340b36d0 tx=0x7fb41c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac3099e0a4cfec6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.926+0000 7fb432ffd640 1 -- 172.21.15.16:0/1990652594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c031880 con 0x7fb43415e340 2023-11-13T10:57:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.926+0000 7fb432ffd640 1 -- 172.21.15.16:0/1990652594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c0319e0 con 0x7fb43415e340 2023-11-13T10:57:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.926+0000 7fb432ffd640 1 -- 172.21.15.16:0/1990652594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c031d70 con 0x7fb43415e340 2023-11-13T10:57:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.927+0000 7fb43aa34640 1 -- 172.21.15.16:0/1990652594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43415e340 msgr2=0x7fb43406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.927+0000 7fb43aa34640 1 --2- 172.21.15.16:0/1990652594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43415e340 0x7fb43406e720 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb4340b36d0 tx=0x7fb41c02b920 comp rx=0 tx=0).stop 2023-11-13T10:57:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.927+0000 7fb43aa34640 1 -- 172.21.15.16:0/1990652594 shutdown_connections 2023-11-13T10:57:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.927+0000 7fb43aa34640 1 --2- 172.21.15.16:0/1990652594 >> v2:172.21.15.184:3300/0 conn(0x7fb43406e2a0 0x7fb43406ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.927+0000 7fb43aa34640 1 --2- 172.21.15.16:0/1990652594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43415e340 0x7fb43406e720 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.927+0000 7fb43aa34640 1 --2- 172.21.15.16:0/1990652594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4340cdae0 0x7fb43406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.927+0000 7fb43aa34640 1 -- 172.21.15.16:0/1990652594 >> 172.21.15.16:0/1990652594 conn(0x7fb4340d0b10 msgr2=0x7fb4340d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.927+0000 7fb43aa34640 1 -- 172.21.15.16:0/1990652594 shutdown_connections 2023-11-13T10:57:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.927+0000 7fb43aa34640 1 -- 172.21.15.16:0/1990652594 wait complete. 2023-11-13T10:57:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.929+0000 7fb43aa34640 1 Processor -- start 2023-11-13T10:57:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.929+0000 7fb43aa34640 1 -- start start 2023-11-13T10:57:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.930+0000 7fb43aa34640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb43406e2a0 0x7fb43415e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.930+0000 7fb43aa34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cdae0 0x7fb4341f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.930+0000 7fb43aa34640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43415e340 0x7fb4340c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.930+0000 7fb43aa34640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4341457c0 con 0x7fb4340cdae0 2023-11-13T10:57:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.930+0000 7fb43aa34640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434147af0 con 0x7fb43406e2a0 2023-11-13T10:57:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.930+0000 7fb43aa34640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4341f7020 con 0x7fb43415e340 2023-11-13T10:57:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.930+0000 7fb4337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cdae0 0x7fb4341f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.930+0000 7fb4337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cdae0 0x7fb4341f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56608/0 (socket says 172.21.15.16:56608) 2023-11-13T10:57:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.930+0000 7fb4337fe640 1 -- 172.21.15.16:0/2483646503 learned_addr learned my addr 172.21.15.16:0/2483646503 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.931+0000 7fb438fd1640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43415e340 0x7fb4340c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.931+0000 7fb4337fe640 1 -- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43415e340 msgr2=0x7fb4340c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.931+0000 7fb4337fe640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43415e340 0x7fb4340c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.931+0000 7fb433fff640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb43406e2a0 0x7fb43415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.931+0000 7fb4337fe640 1 -- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb43406e2a0 msgr2=0x7fb43415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.931+0000 7fb4337fe640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb43406e2a0 0x7fb43415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.931+0000 7fb4337fe640 1 -- 172.21.15.16:0/2483646503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0094d0 con 0x7fb4340cdae0 2023-11-13T10:57:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.931+0000 7fb4337fe640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cdae0 0x7fb4341f6ae0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb41c02b960 tx=0x7fb41c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.932+0000 7fb430ff9640 1 -- 172.21.15.16:0/2483646503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c03ad30 con 0x7fb4340cdae0 2023-11-13T10:57:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.932+0000 7fb430ff9640 1 -- 172.21.15.16:0/2483646503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c0443f0 con 0x7fb4340cdae0 2023-11-13T10:57:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.932+0000 7fb430ff9640 1 -- 172.21.15.16:0/2483646503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0447a0 con 0x7fb4340cdae0 2023-11-13T10:57:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.932+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43415ed20 con 0x7fb4340cdae0 2023-11-13T10:57:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.932+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb434148020 con 0x7fb4340cdae0 2023-11-13T10:57:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.933+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb434067090 con 0x7fb4340cdae0 2023-11-13T10:57:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.934+0000 7fb430ff9640 1 -- 172.21.15.16:0/2483646503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb41c04d450 con 0x7fb4340cdae0 2023-11-13T10:57:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.935+0000 7fb430ff9640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb41406ae40 0x7fb41406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.935+0000 7fb430ff9640 1 -- 172.21.15.16:0/2483646503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6224+0+0 (secure 0 0 0) 0x7fb41c0abd30 con 0x7fb4340cdae0 2023-11-13T10:57:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.935+0000 7fb433fff640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb41406ae40 0x7fb41406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.937+0000 7fb433fff640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb41406ae40 0x7fb41406d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb424009f30 tx=0x7fb42400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:09.939+0000 7fb430ff9640 1 -- 172.21.15.16:0/2483646503 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb41c076c70 con 0x7fb4340cdae0 2023-11-13T10:57:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.146+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb4340b6a90 con 0x7fb4340cdae0 2023-11-13T10:57:10.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.647+0000 7fb430ff9640 1 -- 172.21.15.16:0/2483646503 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb41c035070 con 0x7fb4340cdae0 2023-11-13T10:57:10.653 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-13T10:57:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.651+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb41406ae40 msgr2=0x7fb41406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.651+0000 7fb43aa34640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb41406ae40 0x7fb41406d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb424009f30 tx=0x7fb42400b040 comp rx=0 tx=0).stop 2023-11-13T10:57:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.651+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cdae0 msgr2=0x7fb4341f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.651+0000 7fb43aa34640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cdae0 0x7fb4341f6ae0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb41c02b960 tx=0x7fb41c0046c0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.651+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 shutdown_connections 2023-11-13T10:57:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.651+0000 7fb43aa34640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb41406ae40 0x7fb41406d300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.651+0000 7fb43aa34640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb43415e340 0x7fb4340c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.651+0000 7fb43aa34640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4340cdae0 0x7fb4341f6ae0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.651+0000 7fb43aa34640 1 --2- 172.21.15.16:0/2483646503 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb43406e2a0 0x7fb43415e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.652+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 >> 172.21.15.16:0/2483646503 conn(0x7fb4340d0b10 msgr2=0x7fb4340ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.652+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 shutdown_connections 2023-11-13T10:57:10.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.652+0000 7fb43aa34640 1 -- 172.21.15.16:0/2483646503 wait complete. 2023-11-13T10:57:10.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-13T10:57:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.900+0000 7f9dd238a640 1 Processor -- start 2023-11-13T10:57:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.901+0000 7f9dd238a640 1 -- start start 2023-11-13T10:57:10.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.902+0000 7f9dd238a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9dcc071ae0 0x7f9dcc15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:10.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.902+0000 7f9dd238a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dcc1575b0 0x7f9dcc0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:10.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.902+0000 7f9dd238a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dcc06cb60 con 0x7f9dcc1571d0 2023-11-13T10:57:10.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.902+0000 7f9dd238a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc15e330 con 0x7f9dcc1575b0 2023-11-13T10:57:10.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.902+0000 7f9dd238a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc06bf50 con 0x7f9dcc071ae0 2023-11-13T10:57:10.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.902+0000 7f9dd0927640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dcc1575b0 0x7f9dcc0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:10.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.902+0000 7f9dd0927640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dcc1575b0 0x7f9dcc0acb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56618/0 (socket says 172.21.15.16:56618) 2023-11-13T10:57:10.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.902+0000 7f9dd0927640 1 -- 172.21.15.16:0/2819664184 learned_addr learned my addr 172.21.15.16:0/2819664184 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:10.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.903+0000 7f9dc3fff640 1 --2- 172.21.15.16:0/2819664184 >> v2:172.21.15.184:3300/0 conn(0x7f9dcc071ae0 0x7f9dcc15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:10.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.903+0000 7f9dd0927640 1 -- 172.21.15.16:0/2819664184 >> v2:172.21.15.184:3300/0 conn(0x7f9dcc071ae0 msgr2=0x7f9dcc15ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:10.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.903+0000 7f9dd0927640 1 --2- 172.21.15.16:0/2819664184 >> v2:172.21.15.184:3300/0 conn(0x7f9dcc071ae0 0x7f9dcc15ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.903+0000 7f9dd0927640 1 -- 172.21.15.16:0/2819664184 >> v1:172.21.15.184:6789/0 conn(0x7f9dcc1571d0 legacy=0x7f9dcc0713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:10.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.903+0000 7f9dd0927640 1 -- 172.21.15.16:0/2819664184 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc06c230 con 0x7f9dcc1575b0 2023-11-13T10:57:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.904+0000 7f9dd0927640 1 --2- 172.21.15.16:0/2819664184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 0x7f9dcc0acb20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0cf9e0 tx=0x7f9dc402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f56f26a0d2ef416 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.904+0000 7f9dcb7fe640 1 -- 172.21.15.16:0/2819664184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc402cb20 con 0x7f9dcc1575b0 2023-11-13T10:57:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.904+0000 7f9dcb7fe640 1 -- 172.21.15.16:0/2819664184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc402cc80 con 0x7f9dcc1575b0 2023-11-13T10:57:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.904+0000 7f9dcb7fe640 1 -- 172.21.15.16:0/2819664184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40343f0 con 0x7f9dcc1575b0 2023-11-13T10:57:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.905+0000 7f9dd238a640 1 -- 172.21.15.16:0/2819664184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 msgr2=0x7f9dcc0acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.905+0000 7f9dd238a640 1 --2- 172.21.15.16:0/2819664184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 0x7f9dcc0acb20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0cf9e0 tx=0x7f9dc402bf80 comp rx=0 tx=0).stop 2023-11-13T10:57:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.905+0000 7f9dd238a640 1 -- 172.21.15.16:0/2819664184 shutdown_connections 2023-11-13T10:57:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.905+0000 7f9dd238a640 1 --2- 172.21.15.16:0/2819664184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 0x7f9dcc0acb20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.905+0000 7f9dd238a640 1 --2- 172.21.15.16:0/2819664184 >> v2:172.21.15.184:3300/0 conn(0x7f9dcc071ae0 0x7f9dcc15ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.905+0000 7f9dd238a640 1 -- 172.21.15.16:0/2819664184 >> 172.21.15.16:0/2819664184 conn(0x7f9dcc06a190 msgr2=0x7f9dcc06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.905+0000 7f9dd238a640 1 -- 172.21.15.16:0/2819664184 shutdown_connections 2023-11-13T10:57:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.905+0000 7f9dd238a640 1 -- 172.21.15.16:0/2819664184 wait complete. 2023-11-13T10:57:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.907+0000 7f9dd238a640 1 Processor -- start 2023-11-13T10:57:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.907+0000 7f9dd238a640 1 -- start start 2023-11-13T10:57:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.908+0000 7f9dd238a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dcc071ae0 0x7f9dcc1535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:10.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.908+0000 7f9dd238a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dcc1571d0 0x7f9dcc0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.908+0000 7f9dd238a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 0x7f9dcc1430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:10.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.908+0000 7f9dd238a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc1531a0 con 0x7f9dcc1575b0 2023-11-13T10:57:10.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.908+0000 7f9dd238a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0b12b0 con 0x7f9dcc071ae0 2023-11-13T10:57:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.908+0000 7f9dd238a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc152ae0 con 0x7f9dcc1571d0 2023-11-13T10:57:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.908+0000 7f9dd0927640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 0x7f9dcc1430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.908+0000 7f9dd0927640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 0x7f9dcc1430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56632/0 (socket says 172.21.15.16:56632) 2023-11-13T10:57:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.908+0000 7f9dd0927640 1 -- 172.21.15.16:0/4126297303 learned_addr learned my addr 172.21.15.16:0/4126297303 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dc3fff640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dcc1571d0 0x7f9dcc0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dcbfff640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dcc071ae0 0x7f9dcc1535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dd0927640 1 -- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dcc1571d0 msgr2=0x7f9dcc0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dd0927640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dcc1571d0 0x7f9dcc0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dd0927640 1 -- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dcc071ae0 msgr2=0x7f9dcc1535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dd0927640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dcc071ae0 0x7f9dcc1535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dd0927640 1 -- 172.21.15.16:0/4126297303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4009b30 con 0x7f9dcc1575b0 2023-11-13T10:57:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dcbfff640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dcc071ae0 0x7f9dcc1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dc3fff640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dcc1571d0 0x7f9dcc0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.909+0000 7f9dd0927640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 0x7f9dcc1430e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4036040 tx=0x7f9dc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.910+0000 7f9dc97fa640 1 -- 172.21.15.16:0/4126297303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4009510 con 0x7f9dcc1575b0 2023-11-13T10:57:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.910+0000 7f9dc97fa640 1 -- 172.21.15.16:0/4126297303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc4007e50 con 0x7f9dcc1575b0 2023-11-13T10:57:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.910+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc152dc0 con 0x7f9dcc1575b0 2023-11-13T10:57:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.910+0000 7f9dc97fa640 1 -- 172.21.15.16:0/4126297303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc403e670 con 0x7f9dcc1575b0 2023-11-13T10:57:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.910+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc1f23d0 con 0x7f9dcc1575b0 2023-11-13T10:57:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.911+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dcc152940 con 0x7f9dcc1575b0 2023-11-13T10:57:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.912+0000 7f9dc97fa640 1 -- 172.21.15.16:0/4126297303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9dc403e7d0 con 0x7f9dcc1575b0 2023-11-13T10:57:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.913+0000 7f9dc97fa640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9dac06ae40 0x7f9dac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.914+0000 7f9dc97fa640 1 -- 172.21.15.16:0/4126297303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f9dc40ac2e0 con 0x7f9dcc1575b0 2023-11-13T10:57:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.914+0000 7f9dcbfff640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9dac06ae40 0x7f9dac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.915+0000 7f9dcbfff640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9dac06ae40 0x7f9dac06d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc003eb0 tx=0x7f9dbc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:10.918+0000 7f9dc97fa640 1 -- 172.21.15.16:0/4126297303 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9dcc152940 con 0x7f9dcc1575b0 2023-11-13T10:57:11.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.113+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9dcc1532e0 con 0x7f9dcc1575b0 2023-11-13T10:57:11.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.820+0000 7f9dc97fa640 1 -- 172.21.15.16:0/4126297303 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7f9dcc1532e0 con 0x7f9dcc1575b0 2023-11-13T10:57:11.824 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2023-11-13T10:57:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9dac06ae40 msgr2=0x7f9dac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9dac06ae40 0x7f9dac06d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc003eb0 tx=0x7f9dbc009040 comp rx=0 tx=0).stop 2023-11-13T10:57:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 msgr2=0x7f9dcc1430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 0x7f9dcc1430e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4036040 tx=0x7f9dc4004a40 comp rx=0 tx=0).stop 2023-11-13T10:57:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 shutdown_connections 2023-11-13T10:57:11.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9dac06ae40 0x7f9dac06d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:11.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc1575b0 0x7f9dcc1430e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:11.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9dcc1571d0 0x7f9dcc0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:11.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 --2- 172.21.15.16:0/4126297303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9dcc071ae0 0x7f9dcc1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:11.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 >> 172.21.15.16:0/4126297303 conn(0x7f9dcc06a190 msgr2=0x7f9dcc06fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:11.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 shutdown_connections 2023-11-13T10:57:11.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:11.824+0000 7f9dd238a640 1 -- 172.21.15.16:0/4126297303 wait complete. 2023-11-13T10:57:11.852 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-13T10:57:11.853 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-13T10:57:11.853 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-13T10:57:11.853 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T10:57:11.854 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-13T10:57:11.854 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-13T10:57:11.854 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-13T10:57:11.854 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T10:57:11.855 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-13T10:57:11.855 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-13T10:57:11.855 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-13T10:57:11.856 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T10:57:11.856 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-13T10:57:11.856 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-13T10:57:11.857 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-13T10:57:11.857 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T10:57:11.878 INFO:tasks.ceph.mds.c:Stopped 2023-11-13T10:57:11.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-13T10:57:11.880 INFO:tasks.ceph.mds.a:Stopped 2023-11-13T10:57:11.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-13T10:57:11.881 INFO:tasks.ceph.mds.b:Stopped 2023-11-13T10:57:11.882 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-13T10:57:11.883 INFO:tasks.ceph.mds.d:Stopped 2023-11-13T10:57:11.883 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-13T10:57:11.932 INFO:tasks.ceph.mds.b:Started 2023-11-13T10:57:11.932 DEBUG:teuthology.parallel:result is None 2023-11-13T10:57:11.933 INFO:tasks.ceph.mds.c:Started 2023-11-13T10:57:11.934 DEBUG:teuthology.parallel:result is None 2023-11-13T10:57:11.939 INFO:tasks.ceph.mds.d:Started 2023-11-13T10:57:11.939 DEBUG:teuthology.parallel:result is None 2023-11-13T10:57:11.940 INFO:tasks.ceph.mds.a:Started 2023-11-13T10:57:11.940 DEBUG:teuthology.parallel:result is None 2023-11-13T10:57:11.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-13T10:57:12.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-13T10:57:12.090 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-13T10:57:12.093 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-11-13T10:57:12.093 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-11-13T10:57:12.095 INFO:tasks.ceph.mds.d.smithi184.stderr:2023-11-13T10:57:12.091+0000 7fc60679a640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-13T10:57:12.103 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-13T10:57:12.107 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-13T10:57:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.139+0000 7fe907087640 1 Processor -- start 2023-11-13T10:57:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.140+0000 7fe907087640 1 -- start start 2023-11-13T10:57:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.141+0000 7fe907087640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900159c30 0x7fe9000b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.141+0000 7fe907087640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe90006e3b0 0x7fe900070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.141+0000 7fe907087640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90015e120 con 0x7fe900071120 2023-11-13T10:57:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.141+0000 7fe907087640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000715b0 con 0x7fe90006e3b0 2023-11-13T10:57:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.141+0000 7fe907087640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90015a010 con 0x7fe900159c30 2023-11-13T10:57:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.142+0000 7fe904e23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900159c30 0x7fe9000b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.142+0000 7fe904e23640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900159c30 0x7fe9000b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:49330/0 (socket says 172.21.15.16:49330) 2023-11-13T10:57:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.142+0000 7fe904e23640 1 -- 172.21.15.16:0/499566848 learned_addr learned my addr 172.21.15.16:0/499566848 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.142+0000 7fe8fffff640 1 --2- 172.21.15.16:0/499566848 >> v2:172.21.15.16:3300/0 conn(0x7fe90006e3b0 0x7fe900070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.142+0000 7fe904e23640 1 -- 172.21.15.16:0/499566848 >> v2:172.21.15.16:3300/0 conn(0x7fe90006e3b0 msgr2=0x7fe900070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.142+0000 7fe904e23640 1 --2- 172.21.15.16:0/499566848 >> v2:172.21.15.16:3300/0 conn(0x7fe90006e3b0 0x7fe900070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.142+0000 7fe904e23640 1 -- 172.21.15.16:0/499566848 >> v1:172.21.15.184:6789/0 conn(0x7fe900071120 legacy=0x7fe90006c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.142+0000 7fe904e23640 1 -- 172.21.15.16:0/499566848 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90006de40 con 0x7fe900159c30 2023-11-13T10:57:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.142+0000 7fe8fffff640 1 --2- 172.21.15.16:0/499566848 >> v2:172.21.15.16:3300/0 conn(0x7fe90006e3b0 0x7fe900070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:57:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.144+0000 7fe904e23640 1 --2- 172.21.15.16:0/499566848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900159c30 0x7fe9000b41f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009870 tx=0x7fe8f402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683862167a7f9127 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.144+0000 7fe8ff7fe640 1 -- 172.21.15.16:0/499566848 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4030840 con 0x7fe900159c30 2023-11-13T10:57:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.145+0000 7fe8ff7fe640 1 -- 172.21.15.16:0/499566848 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f40309a0 con 0x7fe900159c30 2023-11-13T10:57:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.145+0000 7fe8ff7fe640 1 -- 172.21.15.16:0/499566848 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4030d30 con 0x7fe900159c30 2023-11-13T10:57:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.145+0000 7fe907087640 1 -- 172.21.15.16:0/499566848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900159c30 msgr2=0x7fe9000b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.145+0000 7fe907087640 1 --2- 172.21.15.16:0/499566848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900159c30 0x7fe9000b41f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009870 tx=0x7fe8f402b980 comp rx=0 tx=0).stop 2023-11-13T10:57:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.146+0000 7fe907087640 1 -- 172.21.15.16:0/499566848 shutdown_connections 2023-11-13T10:57:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.146+0000 7fe907087640 1 --2- 172.21.15.16:0/499566848 >> v2:172.21.15.16:3300/0 conn(0x7fe90006e3b0 0x7fe900070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.146+0000 7fe907087640 1 --2- 172.21.15.16:0/499566848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900159c30 0x7fe9000b41f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.146+0000 7fe907087640 1 -- 172.21.15.16:0/499566848 >> 172.21.15.16:0/499566848 conn(0x7fe9000b17c0 msgr2=0x7fe9000b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.146+0000 7fe907087640 1 -- 172.21.15.16:0/499566848 shutdown_connections 2023-11-13T10:57:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.146+0000 7fe907087640 1 -- 172.21.15.16:0/499566848 wait complete. 2023-11-13T10:57:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.149+0000 7fe907087640 1 Processor -- start 2023-11-13T10:57:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.149+0000 7fe907087640 1 -- start start 2023-11-13T10:57:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe907087640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe90006e3b0 0x7fe900142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe907087640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900071120 0x7fe9000cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe907087640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe900159c30 0x7fe9000cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe907087640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000cd3c0 con 0x7fe900159c30 2023-11-13T10:57:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe907087640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe900142a90 con 0x7fe90006e3b0 2023-11-13T10:57:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe907087640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000d0490 con 0x7fe900071120 2023-11-13T10:57:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe904e23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe90006e3b0 0x7fe900142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe8fffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900071120 0x7fe9000cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe904e23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe90006e3b0 0x7fe900142550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:39356/0 (socket says 172.21.15.16:39356) 2023-11-13T10:57:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe904e23640 1 -- 172.21.15.16:0/1208008271 learned_addr learned my addr 172.21.15.16:0/1208008271 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.151+0000 7fe905624640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe900159c30 0x7fe9000cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.152+0000 7fe8fffff640 1 -- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe90006e3b0 msgr2=0x7fe900142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.152+0000 7fe8fffff640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe90006e3b0 0x7fe900142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.152+0000 7fe8fffff640 1 -- 172.21.15.16:0/1208008271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe900159c30 msgr2=0x7fe9000cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.152+0000 7fe8fffff640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe900159c30 0x7fe9000cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.152+0000 7fe8fffff640 1 -- 172.21.15.16:0/1208008271 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec009400 con 0x7fe900071120 2023-11-13T10:57:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.152+0000 7fe905624640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe900159c30 0x7fe9000cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:57:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.152+0000 7fe904e23640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe90006e3b0 0x7fe900142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:57:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.153+0000 7fe8fffff640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900071120 0x7fe9000cfa60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00e900 tx=0x7fe8ec00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.153+0000 7fe8fd7fa640 1 -- 172.21.15.16:0/1208008271 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00ca00 con 0x7fe900071120 2023-11-13T10:57:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.154+0000 7fe8fd7fa640 1 -- 172.21.15.16:0/1208008271 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec00cb60 con 0x7fe900071120 2023-11-13T10:57:12.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.154+0000 7fe8fd7fa640 1 -- 172.21.15.16:0/1208008271 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec013470 con 0x7fe900071120 2023-11-13T10:57:12.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.154+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f40094d0 con 0x7fe900071120 2023-11-13T10:57:12.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.154+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9000ccd70 con 0x7fe900071120 2023-11-13T10:57:12.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.156+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe900067090 con 0x7fe900071120 2023-11-13T10:57:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.158+0000 7fe8fd7fa640 1 -- 172.21.15.16:0/1208008271 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe8ec0135d0 con 0x7fe900071120 2023-11-13T10:57:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.159+0000 7fe8fd7fa640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8e406ae90 0x7fe8e406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.159+0000 7fe8fd7fa640 1 -- 172.21.15.16:0/1208008271 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe8ec0895f0 con 0x7fe900071120 2023-11-13T10:57:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.159+0000 7fe904e23640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8e406ae90 0x7fe8e406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.160+0000 7fe904e23640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8e406ae90 0x7fe8e406d350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402b9c0 tx=0x7fe8f4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.166+0000 7fe8fd7fa640 1 -- 172.21.15.16:0/1208008271 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8ec054970 con 0x7fe900071120 2023-11-13T10:57:12.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.380+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe90014ded0 con 0x7fe900071120 2023-11-13T10:57:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.855+0000 7fe8fd7fa640 1 -- 172.21.15.16:0/1208008271 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe8ec057f90 con 0x7fe900071120 2023-11-13T10:57:12.859 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-11-13T10:57:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8e406ae90 msgr2=0x7fe8e406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8e406ae90 0x7fe8e406d350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402b9c0 tx=0x7fe8f4008030 comp rx=0 tx=0).stop 2023-11-13T10:57:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900071120 msgr2=0x7fe9000cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900071120 0x7fe9000cfa60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00e900 tx=0x7fe8ec00edc0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 shutdown_connections 2023-11-13T10:57:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8e406ae90 0x7fe8e406d350 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe900159c30 0x7fe9000cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe900071120 0x7fe9000cfa60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 --2- 172.21.15.16:0/1208008271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe90006e3b0 0x7fe900142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 >> 172.21.15.16:0/1208008271 conn(0x7fe9000b17c0 msgr2=0x7fe9000b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 shutdown_connections 2023-11-13T10:57:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:12.859+0000 7fe907087640 1 -- 172.21.15.16:0/1208008271 wait complete. 2023-11-13T10:57:12.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-13T10:57:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.104+0000 7fa7d3fff640 1 Processor -- start 2023-11-13T10:57:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.104+0000 7fa7d3fff640 1 -- start start 2023-11-13T10:57:13.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.106+0000 7fa7d3fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa7d40ae5b0 0x7fa7d415cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.106+0000 7fa7d3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d40cfbf0 0x7fa7d406c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.106+0000 7fa7d3fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d406c1c0 con 0x7fa7d40b4170 2023-11-13T10:57:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.106+0000 7fa7d3fff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d40ae990 con 0x7fa7d40ae5b0 2023-11-13T10:57:13.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.106+0000 7fa7d3fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d415d240 con 0x7fa7d40cfbf0 2023-11-13T10:57:13.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.106+0000 7fa7d27fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa7d40b4170 0x7fa7d40cf4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60232/0 (socket says 172.21.15.16:60232) 2023-11-13T10:57:13.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.106+0000 7fa7d27fc640 1 -- 172.21.15.16:0/2054745339 learned_addr learned my addr 172.21.15.16:0/2054745339 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:13.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d37fe640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d40cfbf0 0x7fa7d406c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d2ffd640 1 --2- 172.21.15.16:0/2054745339 >> v2:172.21.15.184:3300/0 conn(0x7fa7d40ae5b0 0x7fa7d415cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383218480 0 0) 0x7fa7d406c1c0 con 0x7fa7d40b4170 2023-11-13T10:57:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c0003520 con 0x7fa7d40b4170 2023-11-13T10:57:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4183439848 0 0) 0x7fa7c0003520 con 0x7fa7d40b4170 2023-11-13T10:57:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d40cfbf0 msgr2=0x7fa7d406c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d40cfbf0 0x7fa7d406c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 >> v2:172.21.15.184:3300/0 conn(0x7fa7d40ae5b0 msgr2=0x7fa7d415cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 --2- 172.21.15.16:0/2054745339 >> v2:172.21.15.184:3300/0 conn(0x7fa7d40ae5b0 0x7fa7d415cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d415d4f0 con 0x7fa7d40b4170 2023-11-13T10:57:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fa7bc001670 con 0x7fa7d40b4170 2023-11-13T10:57:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.107+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 >> v1:172.21.15.16:6789/0 conn(0x7fa7d40b4170 legacy=0x7fa7d40cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d1ffb640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7c0003970 0x7fa7c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d1ffb640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c00047c0 0x7fa7c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d1ffb640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d40cfbf0 0x7fa7c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40ae990 con 0x7fa7c0003970 2023-11-13T10:57:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d415d240 con 0x7fa7c00047c0 2023-11-13T10:57:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c000ba00 con 0x7fa7d40cfbf0 2023-11-13T10:57:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d2ffd640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7c0003970 0x7fa7c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d27fc640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c00047c0 0x7fa7c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d2ffd640 1 -- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d40cfbf0 msgr2=0x7fa7c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d37fe640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d40cfbf0 0x7fa7c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d2ffd640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d40cfbf0 0x7fa7c0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d2ffd640 1 -- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c00047c0 msgr2=0x7fa7c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d2ffd640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c00047c0 0x7fa7c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.108+0000 7fa7d2ffd640 1 -- 172.21.15.16:0/2054745339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c0003520 con 0x7fa7c0003970 2023-11-13T10:57:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.109+0000 7fa7d27fc640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c00047c0 0x7fa7c0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.110+0000 7fa7d2ffd640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7c0003970 0x7fa7c0003d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009fc0 tx=0x7fa7c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fd84b6b633711d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.110+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c402cc60 con 0x7fa7c0003970 2023-11-13T10:57:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.110+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c400f040 con 0x7fa7c0003970 2023-11-13T10:57:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.110+0000 7fa7d1ffb640 1 -- 172.21.15.16:0/2054745339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4034440 con 0x7fa7c0003970 2023-11-13T10:57:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.110+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2054745339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7c0003970 msgr2=0x7fa7c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.110+0000 7fa7d3fff640 1 --2- 172.21.15.16:0/2054745339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7c0003970 0x7fa7c0003d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009fc0 tx=0x7fa7c00057f0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.110+0000 7fa7d37fe640 1 -- 172.21.15.16:0/2054745339 reap_dead start 2023-11-13T10:57:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.110+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2054745339 shutdown_connections 2023-11-13T10:57:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.110+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2054745339 >> 172.21.15.16:0/2054745339 conn(0x7fa7d406a5b0 msgr2=0x7fa7d406a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.111+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2054745339 shutdown_connections 2023-11-13T10:57:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.111+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2054745339 wait complete. 2023-11-13T10:57:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.112+0000 7fa7d3fff640 1 Processor -- start 2023-11-13T10:57:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.113+0000 7fa7d3fff640 1 -- start start 2023-11-13T10:57:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.113+0000 7fa7d3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c0003970 0x7fa7d409c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d415af90 0x7fa7d415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d4159430 0x7fa7d41598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d3fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4159df0 con 0x7fa7d415af90 2023-11-13T10:57:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d3fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d414d570 con 0x7fa7c0003970 2023-11-13T10:57:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d3fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d415b930 con 0x7fa7d4159430 2023-11-13T10:57:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d415af90 0x7fa7d415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d415af90 0x7fa7d415b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52020/0 (socket says 172.21.15.16:52020) 2023-11-13T10:57:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d27fc640 1 -- 172.21.15.16:0/2894345289 learned_addr learned my addr 172.21.15.16:0/2894345289 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d2ffd640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c0003970 0x7fa7d409c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d27fc640 1 -- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d4159430 msgr2=0x7fa7d41598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d37fe640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d4159430 0x7fa7d41598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d27fc640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d4159430 0x7fa7d41598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d27fc640 1 -- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c0003970 msgr2=0x7fa7d409c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d27fc640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c0003970 0x7fa7d409c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d27fc640 1 -- 172.21.15.16:0/2894345289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d409caa0 con 0x7fa7d415af90 2023-11-13T10:57:13.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.114+0000 7fa7d2ffd640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c0003970 0x7fa7d409c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:13.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.115+0000 7fa7d27fc640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d415af90 0x7fa7d415b3f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00d870 tx=0x7fa7bc00dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:13.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.115+0000 7fa7cb7fe640 1 -- 172.21.15.16:0/2894345289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0074e0 con 0x7fa7d415af90 2023-11-13T10:57:13.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.115+0000 7fa7cb7fe640 1 -- 172.21.15.16:0/2894345289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc007640 con 0x7fa7d415af90 2023-11-13T10:57:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.115+0000 7fa7cb7fe640 1 -- 172.21.15.16:0/2894345289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc004a20 con 0x7fa7d415af90 2023-11-13T10:57:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.115+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d415a010 con 0x7fa7d415af90 2023-11-13T10:57:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.115+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d415a5d0 con 0x7fa7d415af90 2023-11-13T10:57:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.117+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d404e4d0 con 0x7fa7d415af90 2023-11-13T10:57:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.117+0000 7fa7cb7fe640 1 -- 172.21.15.16:0/2894345289 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7bc004b80 con 0x7fa7d415af90 2023-11-13T10:57:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.118+0000 7fa7cb7fe640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b406ae30 0x7fa7b406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.118+0000 7fa7cb7fe640 1 -- 172.21.15.16:0/2894345289 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa7bc088e70 con 0x7fa7d415af90 2023-11-13T10:57:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.119+0000 7fa7d2ffd640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b406ae30 0x7fa7b406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.120+0000 7fa7d2ffd640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b406ae30 0x7fa7b406d2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa7c402c600 tx=0x7fa7c402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.122+0000 7fa7cb7fe640 1 -- 172.21.15.16:0/2894345289 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7bc0540c0 con 0x7fa7d415af90 2023-11-13T10:57:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.382+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa7d40b0780 con 0x7fa7d415af90 2023-11-13T10:57:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.382+0000 7fa7cb7fe640 1 -- 172.21.15.16:0/2894345289 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fa7bc019070 con 0x7fa7d415af90 2023-11-13T10:57:13.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:13.387 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T10:57:13.387 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-11-13T10:57:13.387 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.388 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-11-13T10:57:13.388 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBcAFJlFf7FKBAAzl2CzFIHt1vexIHicDcMOQ==", 2023-11-13T10:57:13.388 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.388 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T10:57:13.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T10:57:13.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T10:57:13.389 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.389 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.390 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.390 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-11-13T10:57:13.390 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJlxksXERAApz8MD99vmb9+TVqjK4yoHQ==", 2023-11-13T10:57:13.390 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.391 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T10:57:13.391 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T10:57:13.391 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T10:57:13.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.391 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.392 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-11-13T10:57:13.392 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJl3f8tABAAn77Y5rtLX6KXfDt6O64jHw==", 2023-11-13T10:57:13.392 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.393 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T10:57:13.396 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T10:57:13.396 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T10:57:13.396 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.396 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.397 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.397 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-11-13T10:57:13.397 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJliC8UHxAAp4RpLcBhknvkfcD6+u0cWg==", 2023-11-13T10:57:13.399 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.399 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T10:57:13.400 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T10:57:13.400 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T10:57:13.400 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.400 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.401 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.401 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-11-13T10:57:13.401 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBgAFJl6OfsFRAA/pxN6JyHj0g/ZTvQTTBvHg==", 2023-11-13T10:57:13.401 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T10:57:13.402 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T10:57:13.402 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.402 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.403 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.403 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-11-13T10:57:13.403 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBjAFJlCcthIRAAD9rCdKqLeqILvx+RcVu3aA==", 2023-11-13T10:57:13.403 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.403 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T10:57:13.404 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T10:57:13.404 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.404 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.405 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-11-13T10:57:13.405 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBmAFJldeAFLRAA6BDzhw1Sml5AWptU6G7MEA==", 2023-11-13T10:57:13.406 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.406 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T10:57:13.406 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T10:57:13.407 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.407 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.407 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.407 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-11-13T10:57:13.408 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBqAFJlhz82DBAAmRhdLjHKXpAyNNqUkXk5qw==", 2023-11-13T10:57:13.409 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.410 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T10:57:13.410 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T10:57:13.410 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.419 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.419 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.419 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-11-13T10:57:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBxAFJlgXLmHxAAHpM0gY+C/Ionsp0sG9KoKQ==", 2023-11-13T10:57:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T10:57:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T10:57:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-11-13T10:57:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQB0AFJlfsdPKxAA5Ba0gTAcUVQ6PueL9Ul6qQ==", 2023-11-13T10:57:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T10:57:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T10:57:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.424 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-11-13T10:57:13.424 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQB4AFJlD6j5CBAAXuiAkwIbpfUSJ1j1j3QP4w==", 2023-11-13T10:57:13.424 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.424 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T10:57:13.424 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T10:57:13.425 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.425 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.426 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.426 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-11-13T10:57:13.426 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQB7AFJlkX9TIhAAPjuDlChUGXjej7ot5x6vDg==", 2023-11-13T10:57:13.426 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.426 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T10:57:13.426 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T10:57:13.427 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.427 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.428 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.428 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-11-13T10:57:13.428 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJl9p66LRAAYRThccLplmNpsr+W647dOA==", 2023-11-13T10:57:13.428 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.428 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T10:57:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T10:57:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T10:57:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T10:57:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-11-13T10:57:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBeAFJlkymOCRAATnUAZgjg1er9vu82gnrxtw==", 2023-11-13T10:57:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.432 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T10:57:13.432 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T10:57:13.432 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T10:57:13.432 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T10:57:13.432 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.433 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.433 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-11-13T10:57:13.433 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJlrk4zORAA7tCfwi0UlPhDcL1GZ6UIIg==", 2023-11-13T10:57:13.433 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.434 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T10:57:13.434 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T10:57:13.434 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T10:57:13.434 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T10:57:13.434 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.435 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.435 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-11-13T10:57:13.435 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBeAFJlrh6DFBAAtrGSamisl/Jexs9ewdiAkQ==", 2023-11-13T10:57:13.435 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.436 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T10:57:13.436 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T10:57:13.436 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T10:57:13.436 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T10:57:13.436 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.437 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.437 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-11-13T10:57:13.437 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBbAFJlhaZyGxAANZe05VzLnzM3KiFjOH93JA==", 2023-11-13T10:57:13.437 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.438 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-13T10:57:13.438 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T10:57:13.438 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T10:57:13.439 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.439 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.445 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.445 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2023-11-13T10:57:13.445 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFAFJluGVDABAAsnP8V6Vcn8WF/B/TuVWPFw==", 2023-11-13T10:57:13.446 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.446 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2023-11-13T10:57:13.446 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.446 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.447 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2023-11-13T10:57:13.447 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFAFJlIHVDABAALMOFstAP3LtKAFwpl2TwbQ==", 2023-11-13T10:57:13.447 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.447 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-13T10:57:13.448 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.448 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.448 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2023-11-13T10:57:13.448 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFAFJlMoVDABAAoWJ+Ou1Ubi9d62UpUNn9sQ==", 2023-11-13T10:57:13.449 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.449 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2023-11-13T10:57:13.449 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.450 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.450 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2023-11-13T10:57:13.450 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFAFJlyZJDABAAVktFwsXPpKmBSRzh9DTJwQ==", 2023-11-13T10:57:13.450 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.450 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-13T10:57:13.451 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.451 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.451 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-13T10:57:13.451 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFAFJl7p9DABAAiO70aTnHwb6LMoCnISb+cQ==", 2023-11-13T10:57:13.452 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.452 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-13T10:57:13.452 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.453 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.453 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2023-11-13T10:57:13.453 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCFAFJlOa1DABAA45/3bsTpdjqQPejAy55y4g==", 2023-11-13T10:57:13.453 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.454 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-13T10:57:13.454 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.454 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.454 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-11-13T10:57:13.455 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBcAFJlvFoIHBAAIXc5UQJIKTtHmeVznGZPSA==", 2023-11-13T10:57:13.455 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.455 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-13T10:57:13.455 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-13T10:57:13.455 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.455 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:13.457 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:13.457 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-11-13T10:57:13.457 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBcAFJluuvTERAAn6zxosZEaeBkgENpZDAHqg==", 2023-11-13T10:57:13.458 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T10:57:13.458 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-13T10:57:13.458 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-13T10:57:13.458 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T10:57:13.458 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.459 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:13.459 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-11-13T10:57:13.459 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T10:57:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b406ae30 msgr2=0x7fa7b406d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b406ae30 0x7fa7b406d2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa7c402c600 tx=0x7fa7c402e000 comp rx=0 tx=0).stop 2023-11-13T10:57:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d415af90 msgr2=0x7fa7d415b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d415af90 0x7fa7d415b3f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00d870 tx=0x7fa7bc00dd30 comp rx=0 tx=0).stop 2023-11-13T10:57:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 shutdown_connections 2023-11-13T10:57:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa7b406ae30 0x7fa7b406d2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d4159430 0x7fa7d41598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d415af90 0x7fa7d415b3f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 --2- 172.21.15.16:0/2894345289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7c0003970 0x7fa7d409c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.385+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 >> 172.21.15.16:0/2894345289 conn(0x7fa7d406a5b0 msgr2=0x7fa7d40cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.386+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 shutdown_connections 2023-11-13T10:57:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.386+0000 7fa7d3fff640 1 -- 172.21.15.16:0/2894345289 wait complete. 2023-11-13T10:57:13.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-13T10:57:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.701+0000 7f1471ce5640 1 Processor -- start 2023-11-13T10:57:13.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.701+0000 7f1471ce5640 1 -- start start 2023-11-13T10:57:13.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.703+0000 7f1471ce5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c160470 0x7f146c0b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.703+0000 7f1471ce5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f146c0cf440 0x7f146c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.703+0000 7f1471ce5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f146c0b59c0 0x7f146c0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.703+0000 7f1471ce5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c0d1cd0 con 0x7f146c0cf440 2023-11-13T10:57:13.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.703+0000 7f1471ce5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c0cfe00 con 0x7f146c0b59c0 2023-11-13T10:57:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.703+0000 7f1471ce5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0d1840 con 0x7f146c160470 2023-11-13T10:57:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f146bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f146c0cf440 0x7f146c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f146bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f146c0cf440 0x7f146c0cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52036/0 (socket says 172.21.15.16:52036) 2023-11-13T10:57:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f146bfff640 1 -- 172.21.15.16:0/363996343 learned_addr learned my addr 172.21.15.16:0/363996343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f14714e4640 1 --2- 172.21.15.16:0/363996343 >> v2:172.21.15.184:3300/0 conn(0x7f146c0b59c0 0x7f146c0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f1470ce3640 1 --2- 172.21.15.16:0/363996343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c160470 0x7f146c0b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f146bfff640 1 -- 172.21.15.16:0/363996343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c160470 msgr2=0x7f146c0b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f146bfff640 1 --2- 172.21.15.16:0/363996343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c160470 0x7f146c0b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f146bfff640 1 -- 172.21.15.16:0/363996343 >> v2:172.21.15.184:3300/0 conn(0x7f146c0b59c0 msgr2=0x7f146c0b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f146bfff640 1 --2- 172.21.15.16:0/363996343 >> v2:172.21.15.184:3300/0 conn(0x7f146c0b59c0 0x7f146c0b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.704+0000 7f146bfff640 1 -- 172.21.15.16:0/363996343 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0d3730 con 0x7f146c0cf440 2023-11-13T10:57:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.705+0000 7f146bfff640 1 --2- 172.21.15.16:0/363996343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 0x7f146c0cf8c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f146c0cd250 tx=0x7f145802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4cf7dda517a780 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.706+0000 7f146b7fe640 1 -- 172.21.15.16:0/363996343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458031860 con 0x7f146c0cf440 2023-11-13T10:57:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.706+0000 7f146b7fe640 1 -- 172.21.15.16:0/363996343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14580319c0 con 0x7f146c0cf440 2023-11-13T10:57:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.706+0000 7f146b7fe640 1 -- 172.21.15.16:0/363996343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458031d50 con 0x7f146c0cf440 2023-11-13T10:57:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.706+0000 7f1471ce5640 1 -- 172.21.15.16:0/363996343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 msgr2=0x7f146c0cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.706+0000 7f1471ce5640 1 --2- 172.21.15.16:0/363996343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 0x7f146c0cf8c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f146c0cd250 tx=0x7f145802b900 comp rx=0 tx=0).stop 2023-11-13T10:57:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.707+0000 7f1471ce5640 1 -- 172.21.15.16:0/363996343 shutdown_connections 2023-11-13T10:57:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.707+0000 7f1471ce5640 1 --2- 172.21.15.16:0/363996343 >> v2:172.21.15.184:3300/0 conn(0x7f146c0b59c0 0x7f146c0b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.707+0000 7f1471ce5640 1 --2- 172.21.15.16:0/363996343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 0x7f146c0cf8c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.707+0000 7f1471ce5640 1 --2- 172.21.15.16:0/363996343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c160470 0x7f146c0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.707+0000 7f1471ce5640 1 -- 172.21.15.16:0/363996343 >> 172.21.15.16:0/363996343 conn(0x7f146c0d2b70 msgr2=0x7f146c0d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.707+0000 7f1471ce5640 1 -- 172.21.15.16:0/363996343 shutdown_connections 2023-11-13T10:57:13.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.707+0000 7f1471ce5640 1 -- 172.21.15.16:0/363996343 wait complete. 2023-11-13T10:57:13.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.709+0000 7f1471ce5640 1 Processor -- start 2023-11-13T10:57:13.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.709+0000 7f1471ce5640 1 -- start start 2023-11-13T10:57:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.710+0000 7f1471ce5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c0b59c0 0x7f146c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.710+0000 7f1471ce5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 0x7f146c1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.710+0000 7f1471ce5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146c160470 0x7f146c1f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.710+0000 7f1471ce5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0cc0c0 con 0x7f146c0cf440 2023-11-13T10:57:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.710+0000 7f1471ce5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c06c6e0 con 0x7f146c160470 2023-11-13T10:57:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.710+0000 7f1471ce5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c1f3e10 con 0x7f146c0b59c0 2023-11-13T10:57:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.710+0000 7f146bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 0x7f146c1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.710+0000 7f146bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 0x7f146c1f5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52050/0 (socket says 172.21.15.16:52050) 2023-11-13T10:57:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.710+0000 7f146bfff640 1 -- 172.21.15.16:0/1018499110 learned_addr learned my addr 172.21.15.16:0/1018499110 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.711+0000 7f1470ce3640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c0b59c0 0x7f146c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.711+0000 7f14714e4640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146c160470 0x7f146c1f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.711+0000 7f146bfff640 1 -- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c0b59c0 msgr2=0x7f146c0cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.711+0000 7f146bfff640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c0b59c0 0x7f146c0cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.711+0000 7f146bfff640 1 -- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146c160470 msgr2=0x7f146c1f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.711+0000 7f146bfff640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146c160470 0x7f146c1f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.711+0000 7f146bfff640 1 -- 172.21.15.16:0/1018499110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14580094d0 con 0x7f146c0cf440 2023-11-13T10:57:13.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.711+0000 7f146bfff640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 0x7f146c1f5d50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f145802b940 tx=0x7f14580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.711+0000 7f14697fa640 1 -- 172.21.15.16:0/1018499110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145803ac60 con 0x7f146c0cf440 2023-11-13T10:57:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.712+0000 7f14697fa640 1 -- 172.21.15.16:0/1018499110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145803adc0 con 0x7f146c0cf440 2023-11-13T10:57:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.712+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c0cc2b0 con 0x7f146c0cf440 2023-11-13T10:57:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.712+0000 7f14697fa640 1 -- 172.21.15.16:0/1018499110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458044610 con 0x7f146c0cf440 2023-11-13T10:57:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.712+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c1f3aa0 con 0x7f146c0cf440 2023-11-13T10:57:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.713+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c0cc7d0 con 0x7f146c0cf440 2023-11-13T10:57:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.714+0000 7f14697fa640 1 -- 172.21.15.16:0/1018499110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1458044770 con 0x7f146c0cf440 2023-11-13T10:57:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.715+0000 7f14697fa640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f145006ae40 0x7f145006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.715+0000 7f14697fa640 1 -- 172.21.15.16:0/1018499110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f14580abb20 con 0x7f146c0cf440 2023-11-13T10:57:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.716+0000 7f1470ce3640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f145006ae40 0x7f145006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:13.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.717+0000 7f1470ce3640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f145006ae40 0x7f145006d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1460006fd0 tx=0x7f1460008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:13.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.719+0000 7f14697fa640 1 -- 172.21.15.16:0/1018499110 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1458076de0 con 0x7f146c0cf440 2023-11-13T10:57:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.991+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f146c04e4d0 con 0x7f146c0cf440 2023-11-13T10:57:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.996+0000 7f14697fa640 1 -- 172.21.15.16:0/1018499110 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f145807a400 con 0x7f146c0cf440 2023-11-13T10:57:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f145006ae40 msgr2=0x7f145006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f145006ae40 0x7f145006d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1460006fd0 tx=0x7f1460008040 comp rx=0 tx=0).stop 2023-11-13T10:57:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 msgr2=0x7f146c1f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 0x7f146c1f5d50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f145802b940 tx=0x7f14580046c0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 shutdown_connections 2023-11-13T10:57:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f145006ae40 0x7f145006d300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f146c160470 0x7f146c1f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f146c0cf440 0x7f146c1f5d50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 --2- 172.21.15.16:0/1018499110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f146c0b59c0 0x7f146c0cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 >> 172.21.15.16:0/1018499110 conn(0x7f146c0d2b70 msgr2=0x7f146c0cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 shutdown_connections 2023-11-13T10:57:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:13.999+0000 7f1471ce5640 1 -- 172.21.15.16:0/1018499110 wait complete. 2023-11-13T10:57:14.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-13T10:57:14.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.247+0000 7f9c46b59640 1 Processor -- start 2023-11-13T10:57:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.247+0000 7f9c46b59640 1 -- start start 2023-11-13T10:57:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.249+0000 7f9c46b59640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c4015ece0 0x7f9c40071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.249+0000 7f9c46b59640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4006c070 con 0x7f9c40155080 2023-11-13T10:57:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.249+0000 7f9c46b59640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c40071bc0 con 0x7f9c40070330 2023-11-13T10:57:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.249+0000 7f9c46b59640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c400707e0 con 0x7f9c4015ece0 2023-11-13T10:57:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.249+0000 7f9c45b57640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9c40155080 0x7f9c400ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60248/0 (socket says 172.21.15.16:60248) 2023-11-13T10:57:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.249+0000 7f9c45b57640 1 -- 172.21.15.16:0/3680843394 learned_addr learned my addr 172.21.15.16:0/3680843394 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.249+0000 7f9c46358640 1 --2- 172.21.15.16:0/3680843394 >> v2:172.21.15.16:3300/0 conn(0x7f9c4015ece0 0x7f9c40071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.249+0000 7f9c44b55640 1 -- 172.21.15.16:0/3680843394 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150667557 0 0) 0x7f9c4006c070 con 0x7f9c40155080 2023-11-13T10:57:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.249+0000 7f9c44b55640 1 -- 172.21.15.16:0/3680843394 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c2c003520 con 0x7f9c40155080 2023-11-13T10:57:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.250+0000 7f9c46358640 1 -- 172.21.15.16:0/3680843394 >> v1:172.21.15.184:6789/0 conn(0x7f9c40070330 legacy=0x7f9c4015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.250+0000 7f9c46358640 1 -- 172.21.15.16:0/3680843394 >> v1:172.21.15.16:6789/0 conn(0x7f9c40155080 legacy=0x7f9c400ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.250+0000 7f9c46358640 1 -- 172.21.15.16:0/3680843394 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c400709c0 con 0x7f9c4015ece0 2023-11-13T10:57:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.251+0000 7f9c46358640 1 --2- 172.21.15.16:0/3680843394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c4015ece0 0x7f9c40071680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009d90 tx=0x7f9c3802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61e7eddffbfb24b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.251+0000 7f9c44b55640 1 -- 172.21.15.16:0/3680843394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3802cc60 con 0x7f9c4015ece0 2023-11-13T10:57:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.251+0000 7f9c44b55640 1 -- 172.21.15.16:0/3680843394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3802cdc0 con 0x7f9c4015ece0 2023-11-13T10:57:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.251+0000 7f9c44b55640 1 -- 172.21.15.16:0/3680843394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38034660 con 0x7f9c4015ece0 2023-11-13T10:57:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.251+0000 7f9c46b59640 1 -- 172.21.15.16:0/3680843394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c4015ece0 msgr2=0x7f9c40071680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.251+0000 7f9c46b59640 1 --2- 172.21.15.16:0/3680843394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c4015ece0 0x7f9c40071680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009d90 tx=0x7f9c3802beb0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.252+0000 7f9c46b59640 1 -- 172.21.15.16:0/3680843394 shutdown_connections 2023-11-13T10:57:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.252+0000 7f9c46b59640 1 --2- 172.21.15.16:0/3680843394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c4015ece0 0x7f9c40071680 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.252+0000 7f9c46b59640 1 -- 172.21.15.16:0/3680843394 >> 172.21.15.16:0/3680843394 conn(0x7f9c400b0580 msgr2=0x7f9c400b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.252+0000 7f9c46b59640 1 -- 172.21.15.16:0/3680843394 shutdown_connections 2023-11-13T10:57:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.252+0000 7f9c46b59640 1 -- 172.21.15.16:0/3680843394 wait complete. 2023-11-13T10:57:14.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.254+0000 7f9c46b59640 1 Processor -- start 2023-11-13T10:57:14.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.254+0000 7f9c46b59640 1 -- start start 2023-11-13T10:57:14.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c46b59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c40070330 0x7f9c40150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c46b59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c40155080 0x7f9c4014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c46b59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c4015ece0 0x7f9c4015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c46b59640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c40144b50 con 0x7f9c40070330 2023-11-13T10:57:14.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c46b59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401516e0 con 0x7f9c4015ece0 2023-11-13T10:57:14.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c46b59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401519c0 con 0x7f9c40155080 2023-11-13T10:57:14.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c45b57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c40070330 0x7f9c40150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:14.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c45b57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c40070330 0x7f9c40150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52076/0 (socket says 172.21.15.16:52076) 2023-11-13T10:57:14.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c45b57640 1 -- 172.21.15.16:0/3019391368 learned_addr learned my addr 172.21.15.16:0/3019391368 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:14.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c45b57640 1 -- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c40155080 msgr2=0x7f9c4014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c45b57640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c40155080 0x7f9c4014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c45b57640 1 -- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c4015ece0 msgr2=0x7f9c4015d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c45b57640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c4015ece0 0x7f9c4015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.255+0000 7f9c45b57640 1 -- 172.21.15.16:0/3019391368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c38009a70 con 0x7f9c40070330 2023-11-13T10:57:14.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.256+0000 7f9c45b57640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c40070330 0x7f9c40150fe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c500 tx=0x7f9c3c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:14.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.256+0000 7f9c367fc640 1 -- 172.21.15.16:0/3019391368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c005ce0 con 0x7f9c40070330 2023-11-13T10:57:14.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.256+0000 7f9c367fc640 1 -- 172.21.15.16:0/3019391368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c005e40 con 0x7f9c40070330 2023-11-13T10:57:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.256+0000 7f9c367fc640 1 -- 172.21.15.16:0/3019391368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c007330 con 0x7f9c40070330 2023-11-13T10:57:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.256+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c40151ca0 con 0x7f9c40070330 2023-11-13T10:57:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.256+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c40152260 con 0x7f9c40070330 2023-11-13T10:57:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.257+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4004e4d0 con 0x7f9c40070330 2023-11-13T10:57:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.258+0000 7f9c367fc640 1 -- 172.21.15.16:0/3019391368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c3c007550 con 0x7f9c40070330 2023-11-13T10:57:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.260+0000 7f9c367fc640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9c2406ae40 0x7f9c2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.260+0000 7f9c367fc640 1 -- 172.21.15.16:0/3019391368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9c3c088a70 con 0x7f9c40070330 2023-11-13T10:57:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.260+0000 7f9c45356640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9c2406ae40 0x7f9c2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.262+0000 7f9c45356640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9c2406ae40 0x7f9c2406d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c40152970 tx=0x7f9c300079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.265+0000 7f9c367fc640 1 -- 172.21.15.16:0/3019391368 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c3c053de0 con 0x7f9c40070330 2023-11-13T10:57:14.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.532+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9c400b13b0 con 0x7f9c40070330 2023-11-13T10:57:14.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.537+0000 7f9c367fc640 1 -- 172.21.15.16:0/3019391368 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9c3c017020 con 0x7f9c40070330 2023-11-13T10:57:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9c2406ae40 msgr2=0x7f9c2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9c2406ae40 0x7f9c2406d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c40152970 tx=0x7f9c300079b0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c40070330 msgr2=0x7f9c40150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c40070330 0x7f9c40150fe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c500 tx=0x7f9c3c00c9c0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 shutdown_connections 2023-11-13T10:57:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9c2406ae40 0x7f9c2406d300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c4015ece0 0x7f9c4015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9c40155080 0x7f9c4014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 --2- 172.21.15.16:0/3019391368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c40070330 0x7f9c40150fe0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.540+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 >> 172.21.15.16:0/3019391368 conn(0x7f9c400b0580 msgr2=0x7f9c4014f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.541+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 shutdown_connections 2023-11-13T10:57:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.541+0000 7f9c46b59640 1 -- 172.21.15.16:0/3019391368 wait complete. 2023-11-13T10:57:14.564 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-13T10:57:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.790+0000 7f322f26d640 1 Processor -- start 2023-11-13T10:57:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.791+0000 7f322f26d640 1 -- start start 2023-11-13T10:57:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.792+0000 7f322f26d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32280b5970 0x7f32280cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.792+0000 7f322f26d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f32280b3b70 0x7f32280cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.792+0000 7f322f26d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32280d0820 con 0x7f3228160340 2023-11-13T10:57:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.792+0000 7f322f26d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32280ce1a0 con 0x7f32280b5970 2023-11-13T10:57:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.792+0000 7f322f26d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32280ce480 con 0x7f32280b3b70 2023-11-13T10:57:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.793+0000 7f322da6a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32280b5970 0x7f32280cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.793+0000 7f322da6a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32280b5970 0x7f32280cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52084/0 (socket says 172.21.15.16:52084) 2023-11-13T10:57:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.793+0000 7f322da6a640 1 -- 172.21.15.16:0/438785937 learned_addr learned my addr 172.21.15.16:0/438785937 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.793+0000 7f322ea6c640 1 --2- 172.21.15.16:0/438785937 >> v2:172.21.15.184:3300/0 conn(0x7f32280b3b70 0x7f32280cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.793+0000 7f322da6a640 1 -- 172.21.15.16:0/438785937 >> v2:172.21.15.184:3300/0 conn(0x7f32280b3b70 msgr2=0x7f32280cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.793+0000 7f322da6a640 1 --2- 172.21.15.16:0/438785937 >> v2:172.21.15.184:3300/0 conn(0x7f32280b3b70 0x7f32280cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.793+0000 7f322da6a640 1 -- 172.21.15.16:0/438785937 >> v1:172.21.15.184:6789/0 conn(0x7f3228160340 legacy=0x7f32280cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.793+0000 7f322da6a640 1 -- 172.21.15.16:0/438785937 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32280ce760 con 0x7f32280b5970 2023-11-13T10:57:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.793+0000 7f322ea6c640 1 --2- 172.21.15.16:0/438785937 >> v2:172.21.15.184:3300/0 conn(0x7f32280b3b70 0x7f32280cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.795+0000 7f322da6a640 1 --2- 172.21.15.16:0/438785937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32280b5970 0x7f32280cf510 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f322808bc40 tx=0x7f321c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c27b39854634ee4 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.795+0000 7f322d269640 1 -- 172.21.15.16:0/438785937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c030860 con 0x7f32280b5970 2023-11-13T10:57:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.795+0000 7f322d269640 1 -- 172.21.15.16:0/438785937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0309c0 con 0x7f32280b5970 2023-11-13T10:57:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.795+0000 7f322d269640 1 -- 172.21.15.16:0/438785937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c030d50 con 0x7f32280b5970 2023-11-13T10:57:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.796+0000 7f322f26d640 1 -- 172.21.15.16:0/438785937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32280b5970 msgr2=0x7f32280cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.796+0000 7f322f26d640 1 --2- 172.21.15.16:0/438785937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32280b5970 0x7f32280cf510 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f322808bc40 tx=0x7f321c02b9a0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.796+0000 7f322f26d640 1 -- 172.21.15.16:0/438785937 shutdown_connections 2023-11-13T10:57:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.796+0000 7f322f26d640 1 --2- 172.21.15.16:0/438785937 >> v2:172.21.15.184:3300/0 conn(0x7f32280b3b70 0x7f32280cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.796+0000 7f322f26d640 1 --2- 172.21.15.16:0/438785937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32280b5970 0x7f32280cf510 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.796+0000 7f322f26d640 1 -- 172.21.15.16:0/438785937 >> 172.21.15.16:0/438785937 conn(0x7f322809be00 msgr2=0x7f32280d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.796+0000 7f322f26d640 1 -- 172.21.15.16:0/438785937 shutdown_connections 2023-11-13T10:57:14.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.796+0000 7f322f26d640 1 -- 172.21.15.16:0/438785937 wait complete. 2023-11-13T10:57:14.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.798+0000 7f322f26d640 1 Processor -- start 2023-11-13T10:57:14.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.798+0000 7f322f26d640 1 -- start start 2023-11-13T10:57:14.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322f26d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32280b5970 0x7f32281508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322f26d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3228160340 0x7f322815f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322f26d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322814fe60 0x7f32280cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322f26d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3228151ad0 con 0x7f3228160340 2023-11-13T10:57:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322f26d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322815f5d0 con 0x7f322814fe60 2023-11-13T10:57:14.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322f26d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3228150f50 con 0x7f32280b5970 2023-11-13T10:57:14.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322da6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3228160340 0x7f322815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:14.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322da6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3228160340 0x7f322815f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52092/0 (socket says 172.21.15.16:52092) 2023-11-13T10:57:14.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322da6a640 1 -- 172.21.15.16:0/3844691320 learned_addr learned my addr 172.21.15.16:0/3844691320 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:14.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322e26b640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32280b5970 0x7f32281508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.799+0000 7f322ea6c640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322814fe60 0x7f32280cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f322da6a640 1 -- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32280b5970 msgr2=0x7f32281508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f322da6a640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32280b5970 0x7f32281508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f322da6a640 1 -- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322814fe60 msgr2=0x7f32280cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f322da6a640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322814fe60 0x7f32280cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f322da6a640 1 -- 172.21.15.16:0/3844691320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0094d0 con 0x7f3228160340 2023-11-13T10:57:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f322da6a640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3228160340 0x7f322815f090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f321c02b9e0 tx=0x7f321c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f3226ffd640 1 -- 172.21.15.16:0/3844691320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c030850 con 0x7f3228160340 2023-11-13T10:57:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f3226ffd640 1 -- 172.21.15.16:0/3844691320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c043400 con 0x7f3228160340 2023-11-13T10:57:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f3226ffd640 1 -- 172.21.15.16:0/3844691320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0437b0 con 0x7f3228160340 2023-11-13T10:57:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.800+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3228151230 con 0x7f3228160340 2023-11-13T10:57:14.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.801+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32281517f0 con 0x7f3228160340 2023-11-13T10:57:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.802+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322804e4d0 con 0x7f3228160340 2023-11-13T10:57:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.803+0000 7f3226ffd640 1 -- 172.21.15.16:0/3844691320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f321c043910 con 0x7f3228160340 2023-11-13T10:57:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.804+0000 7f3226ffd640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f321406ae40 0x7f321406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.804+0000 7f3226ffd640 1 -- 172.21.15.16:0/3844691320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f321c03e070 con 0x7f3228160340 2023-11-13T10:57:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.804+0000 7f322e26b640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f321406ae40 0x7f321406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.805+0000 7f322e26b640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f321406ae40 0x7f321406d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3218007fd0 tx=0x7f3218009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:14.808+0000 7f3226ffd640 1 -- 172.21.15.16:0/3844691320 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f321c077b00 con 0x7f3228160340 2023-11-13T10:57:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.073+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f32280cfb90 con 0x7f3228160340 2023-11-13T10:57:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.078+0000 7f3226ffd640 1 -- 172.21.15.16:0/3844691320 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f321c07b120 con 0x7f3228160340 2023-11-13T10:57:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f321406ae40 msgr2=0x7f321406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f321406ae40 0x7f321406d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3218007fd0 tx=0x7f3218009040 comp rx=0 tx=0).stop 2023-11-13T10:57:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3228160340 msgr2=0x7f322815f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3228160340 0x7f322815f090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f321c02b9e0 tx=0x7f321c004730 comp rx=0 tx=0).stop 2023-11-13T10:57:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 shutdown_connections 2023-11-13T10:57:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f321406ae40 0x7f321406d300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f322814fe60 0x7f32280cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3228160340 0x7f322815f090 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 --2- 172.21.15.16:0/3844691320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32280b5970 0x7f32281508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 >> 172.21.15.16:0/3844691320 conn(0x7f322809be00 msgr2=0x7f32280cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 shutdown_connections 2023-11-13T10:57:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.081+0000 7f322f26d640 1 -- 172.21.15.16:0/3844691320 wait complete. 2023-11-13T10:57:15.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-13T10:57:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.323+0000 7f9e49a06640 1 Processor -- start 2023-11-13T10:57:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.324+0000 7f9e49a06640 1 -- start start 2023-11-13T10:57:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.325+0000 7f9e49a06640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e44070330 0x7f9e44071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.325+0000 7f9e49a06640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 0x7f9e4415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.325+0000 7f9e49a06640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4406c070 con 0x7f9e44155080 2023-11-13T10:57:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.325+0000 7f9e49a06640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e440707e0 con 0x7f9e44070330 2023-11-13T10:57:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.325+0000 7f9e49a06640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440cd1c0 con 0x7f9e4415e5d0 2023-11-13T10:57:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.325+0000 7f9e43fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e44070330 0x7f9e44071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.325+0000 7f9e48a04640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e44155080 0x7f9e440ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60258/0 (socket says 172.21.15.16:60258) 2023-11-13T10:57:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.325+0000 7f9e48a04640 1 -- 172.21.15.16:0/1381976547 learned_addr learned my addr 172.21.15.16:0/1381976547 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.326+0000 7f9e49205640 1 --2- 172.21.15.16:0/1381976547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 0x7f9e4415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.326+0000 7f9e43fff640 1 -- 172.21.15.16:0/1381976547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 msgr2=0x7f9e4415ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.326+0000 7f9e43fff640 1 --2- 172.21.15.16:0/1381976547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 0x7f9e4415ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.326+0000 7f9e43fff640 1 -- 172.21.15.16:0/1381976547 >> v1:172.21.15.16:6789/0 conn(0x7f9e44155080 legacy=0x7f9e440ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.326+0000 7f9e43fff640 1 -- 172.21.15.16:0/1381976547 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e44071bc0 con 0x7f9e44070330 2023-11-13T10:57:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.326+0000 7f9e437fe640 1 -- 172.21.15.16:0/1381976547 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680838040 0 0) 0x7f9e4406c070 con 0x7f9e44155080 2023-11-13T10:57:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.327+0000 7f9e43fff640 1 --2- 172.21.15.16:0/1381976547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 0x7f9e44071680 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9e300097a0 tx=0x7f9e3002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c70ef63c35e494 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.327+0000 7f9e437fe640 1 -- 172.21.15.16:0/1381976547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3002d040 con 0x7f9e44070330 2023-11-13T10:57:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e437fe640 1 -- 172.21.15.16:0/1381976547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e30031810 con 0x7f9e44070330 2023-11-13T10:57:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e437fe640 1 -- 172.21.15.16:0/1381976547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30031ba0 con 0x7f9e44070330 2023-11-13T10:57:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e49a06640 1 -- 172.21.15.16:0/1381976547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 msgr2=0x7f9e44071680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e49a06640 1 --2- 172.21.15.16:0/1381976547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 0x7f9e44071680 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9e300097a0 tx=0x7f9e3002b8b0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e49a06640 1 -- 172.21.15.16:0/1381976547 shutdown_connections 2023-11-13T10:57:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e49a06640 1 --2- 172.21.15.16:0/1381976547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 0x7f9e4415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e49a06640 1 --2- 172.21.15.16:0/1381976547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 0x7f9e44071680 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e49a06640 1 -- 172.21.15.16:0/1381976547 >> 172.21.15.16:0/1381976547 conn(0x7f9e440b0580 msgr2=0x7f9e440b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e49a06640 1 -- 172.21.15.16:0/1381976547 shutdown_connections 2023-11-13T10:57:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.328+0000 7f9e49a06640 1 -- 172.21.15.16:0/1381976547 wait complete. 2023-11-13T10:57:15.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.330+0000 7f9e49a06640 1 Processor -- start 2023-11-13T10:57:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.330+0000 7f9e49a06640 1 -- start start 2023-11-13T10:57:15.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.331+0000 7f9e49a06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 0x7f9e4414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.331+0000 7f9e49a06640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e44155080 0x7f9e4414a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.331+0000 7f9e49a06640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 0x7f9e441491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.331+0000 7f9e49a06640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e441f0670 con 0x7f9e44070330 2023-11-13T10:57:15.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.331+0000 7f9e49a06640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4414a840 con 0x7f9e44155080 2023-11-13T10:57:15.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.331+0000 7f9e49a06640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4414aac0 con 0x7f9e4415e5d0 2023-11-13T10:57:15.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.331+0000 7f9e48a04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 0x7f9e4414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e48a04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 0x7f9e4414f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52118/0 (socket says 172.21.15.16:52118) 2023-11-13T10:57:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e48a04640 1 -- 172.21.15.16:0/1406295804 learned_addr learned my addr 172.21.15.16:0/1406295804 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e49205640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 0x7f9e441491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e48a04640 1 -- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 msgr2=0x7f9e441491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e48a04640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 0x7f9e441491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e43fff640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e44155080 0x7f9e4414a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e48a04640 1 -- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e44155080 msgr2=0x7f9e4414a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e48a04640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e44155080 0x7f9e4414a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e48a04640 1 -- 172.21.15.16:0/1406295804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30009400 con 0x7f9e44070330 2023-11-13T10:57:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e49205640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 0x7f9e441491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.332+0000 7f9e48a04640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 0x7f9e4414f360 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c4d0 tx=0x7f9e3800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:15.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.333+0000 7f9e417fa640 1 -- 172.21.15.16:0/1406295804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e380074e0 con 0x7f9e44070330 2023-11-13T10:57:15.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.333+0000 7f9e417fa640 1 -- 172.21.15.16:0/1406295804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38007660 con 0x7f9e44070330 2023-11-13T10:57:15.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.333+0000 7f9e417fa640 1 -- 172.21.15.16:0/1406295804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38004a20 con 0x7f9e44070330 2023-11-13T10:57:15.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.333+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4414ada0 con 0x7f9e44070330 2023-11-13T10:57:15.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.333+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4414b360 con 0x7f9e44070330 2023-11-13T10:57:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.334+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4404e4d0 con 0x7f9e44070330 2023-11-13T10:57:15.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.335+0000 7f9e417fa640 1 -- 172.21.15.16:0/1406295804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e38004b80 con 0x7f9e44070330 2023-11-13T10:57:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.336+0000 7f9e417fa640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e2806ae40 0x7f9e2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.336+0000 7f9e417fa640 1 -- 172.21.15.16:0/1406295804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9e38088510 con 0x7f9e44070330 2023-11-13T10:57:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.337+0000 7f9e43fff640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e2806ae40 0x7f9e2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.338+0000 7f9e43fff640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e2806ae40 0x7f9e2806d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e30036000 tx=0x7f9e3003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:15.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.341+0000 7f9e417fa640 1 -- 172.21.15.16:0/1406295804 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e38053860 con 0x7f9e44070330 2023-11-13T10:57:15.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.606+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9e440b13b0 con 0x7f9e44070330 2023-11-13T10:57:15.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.611+0000 7f9e417fa640 1 -- 172.21.15.16:0/1406295804 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9e380180a0 con 0x7f9e44070330 2023-11-13T10:57:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e2806ae40 msgr2=0x7f9e2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e2806ae40 0x7f9e2806d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e30036000 tx=0x7f9e3003c040 comp rx=0 tx=0).stop 2023-11-13T10:57:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 msgr2=0x7f9e4414f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 0x7f9e4414f360 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c4d0 tx=0x7f9e3800c990 comp rx=0 tx=0).stop 2023-11-13T10:57:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 shutdown_connections 2023-11-13T10:57:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e2806ae40 0x7f9e2806d300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e4415e5d0 0x7f9e441491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e44155080 0x7f9e4414a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 --2- 172.21.15.16:0/1406295804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e44070330 0x7f9e4414f360 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 >> 172.21.15.16:0/1406295804 conn(0x7f9e440b0580 msgr2=0x7f9e4414d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 shutdown_connections 2023-11-13T10:57:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.614+0000 7f9e49a06640 1 -- 172.21.15.16:0/1406295804 wait complete. 2023-11-13T10:57:15.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-13T10:57:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.856+0000 7fadc1600640 1 Processor -- start 2023-11-13T10:57:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.856+0000 7fadc1600640 1 -- start start 2023-11-13T10:57:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.858+0000 7fadc1600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.858+0000 7fadc1600640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fadbc15dc70 0x7fadbc15e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.858+0000 7fadc1600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc0b0c70 con 0x7fadbc15e720 2023-11-13T10:57:15.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.858+0000 7fadc1600640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc15b040 con 0x7fadbc15dc70 2023-11-13T10:57:15.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.858+0000 7fadc1600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc15d560 con 0x7fadbc15ac60 2023-11-13T10:57:15.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadbaffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadba7fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fadbc15dc70 0x7fadbc15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadbaffd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:55840/0 (socket says 172.21.15.16:55840) 2023-11-13T10:57:15.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadba7fc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fadbc15dc70 0x7fadbc15e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:50190/0 (socket says 172.21.15.16:50190) 2023-11-13T10:57:15.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadbb7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fadbc15e720 0x7fadbc071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60274/0 (socket says 172.21.15.16:60274) 2023-11-13T10:57:15.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadba7fc640 1 -- 172.21.15.16:0/2617353325 learned_addr learned my addr 172.21.15.16:0/2617353325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:15.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadb9ffb640 1 -- 172.21.15.16:0/2617353325 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600392747 0 0) 0x7fadbc0b0c70 con 0x7fadbc15e720 2023-11-13T10:57:15.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadb9ffb640 1 -- 172.21.15.16:0/2617353325 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada4003520 con 0x7fadbc15e720 2023-11-13T10:57:15.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadbaffd640 1 -- 172.21.15.16:0/2617353325 >> v2:172.21.15.184:3300/0 conn(0x7fadbc15dc70 msgr2=0x7fadbc15e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:15.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadbaffd640 1 --2- 172.21.15.16:0/2617353325 >> v2:172.21.15.184:3300/0 conn(0x7fadbc15dc70 0x7fadbc15e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadbaffd640 1 -- 172.21.15.16:0/2617353325 >> v1:172.21.15.16:6789/0 conn(0x7fadbc15e720 legacy=0x7fadbc071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.859+0000 7fadbaffd640 1 -- 172.21.15.16:0/2617353325 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0b2230 con 0x7fadbc15ac60 2023-11-13T10:57:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.861+0000 7fadbaffd640 1 --2- 172.21.15.16:0/2617353325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc072a50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fadac0097a0 tx=0x7fadac02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da3672e0f0aba20 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.861+0000 7fadb9ffb640 1 -- 172.21.15.16:0/2617353325 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac030800 con 0x7fadbc15ac60 2023-11-13T10:57:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.861+0000 7fadb9ffb640 1 -- 172.21.15.16:0/2617353325 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac030960 con 0x7fadbc15ac60 2023-11-13T10:57:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.861+0000 7fadb9ffb640 1 -- 172.21.15.16:0/2617353325 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac030cf0 con 0x7fadbc15ac60 2023-11-13T10:57:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.862+0000 7fadc1600640 1 -- 172.21.15.16:0/2617353325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 msgr2=0x7fadbc072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.862+0000 7fadc1600640 1 --2- 172.21.15.16:0/2617353325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc072a50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fadac0097a0 tx=0x7fadac02b940 comp rx=0 tx=0).stop 2023-11-13T10:57:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.862+0000 7fadc1600640 1 -- 172.21.15.16:0/2617353325 shutdown_connections 2023-11-13T10:57:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.862+0000 7fadc1600640 1 --2- 172.21.15.16:0/2617353325 >> v2:172.21.15.184:3300/0 conn(0x7fadbc15dc70 0x7fadbc15e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.862+0000 7fadc1600640 1 --2- 172.21.15.16:0/2617353325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc072a50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.862+0000 7fadc1600640 1 -- 172.21.15.16:0/2617353325 >> 172.21.15.16:0/2617353325 conn(0x7fadbc0b48d0 msgr2=0x7fadbc0b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.862+0000 7fadc1600640 1 -- 172.21.15.16:0/2617353325 shutdown_connections 2023-11-13T10:57:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.862+0000 7fadc1600640 1 -- 172.21.15.16:0/2617353325 wait complete. 2023-11-13T10:57:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.864+0000 7fadc1600640 1 Processor -- start 2023-11-13T10:57:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.864+0000 7fadc1600640 1 -- start start 2023-11-13T10:57:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.865+0000 7fadc1600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc15e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.865+0000 7fadc1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadbc15dc70 0x7fadbc155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.865+0000 7fadc1600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadbc15e720 0x7fadbc0992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.865+0000 7fadc1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc09cdc0 con 0x7fadbc15dc70 2023-11-13T10:57:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.865+0000 7fadc1600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc09c800 con 0x7fadbc15e720 2023-11-13T10:57:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.865+0000 7fadc1600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc09cae0 con 0x7fadbc15ac60 2023-11-13T10:57:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.865+0000 7fadba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadbc15dc70 0x7fadbc155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.865+0000 7fadba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadbc15dc70 0x7fadbc155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52126/0 (socket says 172.21.15.16:52126) 2023-11-13T10:57:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.865+0000 7fadba7fc640 1 -- 172.21.15.16:0/2460220104 learned_addr learned my addr 172.21.15.16:0/2460220104 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fadbaffd640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fadbb7fe640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadbc15e720 0x7fadbc0992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fadba7fc640 1 -- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 msgr2=0x7fadbc15e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fadba7fc640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc15e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fadba7fc640 1 -- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadbc15e720 msgr2=0x7fadbc0992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fadba7fc640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadbc15e720 0x7fadbc0992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fadba7fc640 1 -- 172.21.15.16:0/2460220104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac009400 con 0x7fadbc15dc70 2023-11-13T10:57:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fadba7fc640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadbc15dc70 0x7fadbc155390 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009fd0 tx=0x7fadb000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fada37fe640 1 -- 172.21.15.16:0/2460220104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb000c940 con 0x7fadbc15dc70 2023-11-13T10:57:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fada37fe640 1 -- 172.21.15.16:0/2460220104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb000caa0 con 0x7fadbc15dc70 2023-11-13T10:57:15.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fada37fe640 1 -- 172.21.15.16:0/2460220104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb000ce30 con 0x7fadbc15dc70 2023-11-13T10:57:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.866+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc0add50 con 0x7fadbc15dc70 2023-11-13T10:57:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.867+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc1593f0 con 0x7fadbc15dc70 2023-11-13T10:57:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.868+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc0adba0 con 0x7fadbc15dc70 2023-11-13T10:57:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.869+0000 7fada37fe640 1 -- 172.21.15.16:0/2460220104 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadb0013470 con 0x7fadbc15dc70 2023-11-13T10:57:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.869+0000 7fada37fe640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fada806ae40 0x7fada806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.870+0000 7fada37fe640 1 -- 172.21.15.16:0/2460220104 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fadb0089700 con 0x7fadbc15dc70 2023-11-13T10:57:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.870+0000 7fadbaffd640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fada806ae40 0x7fada806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.871+0000 7fadbaffd640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fada806ae40 0x7fada806d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fadac036000 tx=0x7fadac008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:15.874+0000 7fada37fe640 1 -- 172.21.15.16:0/2460220104 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadb0054890 con 0x7fadbc15dc70 2023-11-13T10:57:16.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.139+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fadbc1596d0 con 0x7fadbc15dc70 2023-11-13T10:57:16.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.143+0000 7fada37fe640 1 -- 172.21.15.16:0/2460220104 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fadb00170a0 con 0x7fadbc15dc70 2023-11-13T10:57:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fada806ae40 msgr2=0x7fada806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fada806ae40 0x7fada806d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fadac036000 tx=0x7fadac008030 comp rx=0 tx=0).stop 2023-11-13T10:57:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadbc15dc70 msgr2=0x7fadbc155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadbc15dc70 0x7fadbc155390 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009fd0 tx=0x7fadb000ed40 comp rx=0 tx=0).stop 2023-11-13T10:57:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 shutdown_connections 2023-11-13T10:57:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fada806ae40 0x7fada806d300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadbc15e720 0x7fadbc0992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadbc15dc70 0x7fadbc155390 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 --2- 172.21.15.16:0/2460220104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadbc15ac60 0x7fadbc15e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.146+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 >> 172.21.15.16:0/2460220104 conn(0x7fadbc0b48d0 msgr2=0x7fadbc071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.147+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 shutdown_connections 2023-11-13T10:57:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.147+0000 7fadc1600640 1 -- 172.21.15.16:0/2460220104 wait complete. 2023-11-13T10:57:16.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-13T10:57:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.393+0000 7f860bfff640 1 Processor -- start 2023-11-13T10:57:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.394+0000 7f860bfff640 1 -- start start 2023-11-13T10:57:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.395+0000 7f860bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f860c0adc00 0x7f860c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.395+0000 7f860bfff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860c159e90 con 0x7f860c06f920 2023-11-13T10:57:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.395+0000 7f860bfff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860c0ad9b0 con 0x7f860c06b6a0 2023-11-13T10:57:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.395+0000 7f860bfff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f860c15b070 con 0x7f860c0adc00 2023-11-13T10:57:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.396+0000 7f860affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f860c06f920 0x7f860c06fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60288/0 (socket says 172.21.15.16:60288) 2023-11-13T10:57:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.396+0000 7f860affd640 1 -- 172.21.15.16:0/4042694773 learned_addr learned my addr 172.21.15.16:0/4042694773 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.396+0000 7f860b7fe640 1 --2- 172.21.15.16:0/4042694773 >> v2:172.21.15.184:3300/0 conn(0x7f860c0adc00 0x7f860c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.396+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933157119 0 0) 0x7f860c159e90 con 0x7f860c06f920 2023-11-13T10:57:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.396+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85f8003520 con 0x7f860c06f920 2023-11-13T10:57:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1897647684 0 0) 0x7f85f8003520 con 0x7f860c06f920 2023-11-13T10:57:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 >> v2:172.21.15.184:3300/0 conn(0x7f860c0adc00 msgr2=0x7f860c15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 --2- 172.21.15.16:0/4042694773 >> v2:172.21.15.184:3300/0 conn(0x7f860c0adc00 0x7f860c15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 >> v1:172.21.15.184:6789/0 conn(0x7f860c06b6a0 legacy=0x7f860c15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c06d0e0 con 0x7f860c06f920 2023-11-13T10:57:16.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f85fc001670 con 0x7f860c06f920 2023-11-13T10:57:16.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 >> v1:172.21.15.16:6789/0 conn(0x7f860c06f920 legacy=0x7f860c06fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f8003970 0x7f85f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85f80047c0 0x7f85f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f8005180 0x7f85f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c15b070 con 0x7f85f8003970 2023-11-13T10:57:16.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f800c020 con 0x7f85f80047c0 2023-11-13T10:57:16.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f800c1e0 con 0x7f85f8005180 2023-11-13T10:57:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f860affd640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f8003970 0x7f85f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.397+0000 7f8603fff640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85f80047c0 0x7f85f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.398+0000 7f860affd640 1 -- 172.21.15.16:0/4042694773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f8005180 msgr2=0x7f85f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.398+0000 7f860affd640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85f8005180 0x7f85f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.398+0000 7f860affd640 1 -- 172.21.15.16:0/4042694773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85f80047c0 msgr2=0x7f85f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:16.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.398+0000 7f860affd640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85f80047c0 0x7f85f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.398+0000 7f860affd640 1 -- 172.21.15.16:0/4042694773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f8003520 con 0x7f85f8003970 2023-11-13T10:57:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.398+0000 7f860b7fe640 1 -- 172.21.15.16:0/4042694773 reap_dead start 2023-11-13T10:57:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.399+0000 7f860affd640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f8003970 0x7f85f8003d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00c240 tx=0x7f85f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd8cd3505afcf6a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.399+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc005ad0 con 0x7f85f8003970 2023-11-13T10:57:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.399+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc005c30 con 0x7f85f8003970 2023-11-13T10:57:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.399+0000 7f860a7fc640 1 -- 172.21.15.16:0/4042694773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc02ca90 con 0x7f85f8003970 2023-11-13T10:57:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.400+0000 7f860bfff640 1 -- 172.21.15.16:0/4042694773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f8003970 msgr2=0x7f85f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.400+0000 7f860bfff640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f8003970 0x7f85f8003d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00c240 tx=0x7f85f80057f0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.400+0000 7f860bfff640 1 -- 172.21.15.16:0/4042694773 shutdown_connections 2023-11-13T10:57:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.400+0000 7f860bfff640 1 --2- 172.21.15.16:0/4042694773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f8003970 0x7f85f8003d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.400+0000 7f860bfff640 1 -- 172.21.15.16:0/4042694773 >> 172.21.15.16:0/4042694773 conn(0x7f860c0b0580 msgr2=0x7f860c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.400+0000 7f860bfff640 1 -- 172.21.15.16:0/4042694773 shutdown_connections 2023-11-13T10:57:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.400+0000 7f860bfff640 1 -- 172.21.15.16:0/4042694773 wait complete. 2023-11-13T10:57:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.402+0000 7f860bfff640 1 Processor -- start 2023-11-13T10:57:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.402+0000 7f860bfff640 1 -- start start 2023-11-13T10:57:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.403+0000 7f860bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85f8003970 0x7f860c158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.403+0000 7f860bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c147e00 0x7f860c148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.403+0000 7f860bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860c156c50 0x7f860c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.403+0000 7f860bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c148780 con 0x7f860c156c50 2023-11-13T10:57:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.403+0000 7f860bfff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c1579d0 con 0x7f85f8003970 2023-11-13T10:57:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.403+0000 7f860bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860c1e23e0 con 0x7f860c147e00 2023-11-13T10:57:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.403+0000 7f860b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860c156c50 0x7f860c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.403+0000 7f860b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860c156c50 0x7f860c1570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52146/0 (socket says 172.21.15.16:52146) 2023-11-13T10:57:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.403+0000 7f860b7fe640 1 -- 172.21.15.16:0/332348611 learned_addr learned my addr 172.21.15.16:0/332348611 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:16.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f860affd640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85f8003970 0x7f860c158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f8603fff640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c147e00 0x7f860c148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f860b7fe640 1 -- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c147e00 msgr2=0x7f860c148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f860b7fe640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c147e00 0x7f860c148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f860b7fe640 1 -- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85f8003970 msgr2=0x7f860c158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f860b7fe640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85f8003970 0x7f860c158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f860b7fe640 1 -- 172.21.15.16:0/332348611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c1e26c0 con 0x7f860c156c50 2023-11-13T10:57:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f860b7fe640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860c156c50 0x7f860c1570d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f860c0ad5d0 tx=0x7f860400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f86037fe640 1 -- 172.21.15.16:0/332348611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604010040 con 0x7f860c156c50 2023-11-13T10:57:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f86037fe640 1 -- 172.21.15.16:0/332348611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860400bb70 con 0x7f860c156c50 2023-11-13T10:57:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f86037fe640 1 -- 172.21.15.16:0/332348611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604014400 con 0x7f860c156c50 2023-11-13T10:57:16.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.404+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860c1e29a0 con 0x7f860c156c50 2023-11-13T10:57:16.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.405+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860c1585a0 con 0x7f860c156c50 2023-11-13T10:57:16.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.406+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860c04e4d0 con 0x7f860c156c50 2023-11-13T10:57:16.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.407+0000 7f86037fe640 1 -- 172.21.15.16:0/332348611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8604014560 con 0x7f860c156c50 2023-11-13T10:57:16.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.408+0000 7f86037fe640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f85ec06ae40 0x7f85ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.408+0000 7f86037fe640 1 -- 172.21.15.16:0/332348611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8604089700 con 0x7f860c156c50 2023-11-13T10:57:16.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.409+0000 7f860affd640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f85ec06ae40 0x7f85ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.410+0000 7f860affd640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f85ec06ae40 0x7f85ec06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f85fc034000 tx=0x7f85fc009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:16.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.412+0000 7f86037fe640 1 -- 172.21.15.16:0/332348611 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8604054920 con 0x7f860c156c50 2023-11-13T10:57:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.674+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f860c06bd10 con 0x7f860c156c50 2023-11-13T10:57:16.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.679+0000 7f86037fe640 1 -- 172.21.15.16:0/332348611 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8604057f40 con 0x7f860c156c50 2023-11-13T10:57:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f85ec06ae40 msgr2=0x7f85ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f85ec06ae40 0x7f85ec06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f85fc034000 tx=0x7f85fc009030 comp rx=0 tx=0).stop 2023-11-13T10:57:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860c156c50 msgr2=0x7f860c1570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860c156c50 0x7f860c1570d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f860c0ad5d0 tx=0x7f860400ef30 comp rx=0 tx=0).stop 2023-11-13T10:57:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 shutdown_connections 2023-11-13T10:57:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f85ec06ae40 0x7f85ec06d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860c156c50 0x7f860c1570d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c147e00 0x7f860c148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 --2- 172.21.15.16:0/332348611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85f8003970 0x7f860c158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.681+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 >> 172.21.15.16:0/332348611 conn(0x7f860c0b0580 msgr2=0x7f860c15c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.682+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 shutdown_connections 2023-11-13T10:57:16.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.682+0000 7f860bfff640 1 -- 172.21.15.16:0/332348611 wait complete. 2023-11-13T10:57:16.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:57:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.917+0000 7ff2dfe32640 1 Processor -- start 2023-11-13T10:57:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.917+0000 7ff2dfe32640 1 -- start start 2023-11-13T10:57:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.918+0000 7ff2dfe32640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2d815d0f0 0x7ff2d80d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.919+0000 7ff2dfe32640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d80d3c70 con 0x7ff2d80cfe20 2023-11-13T10:57:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.919+0000 7ff2dfe32640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d80c69d0 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.919+0000 7ff2dfe32640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d80c6cb0 con 0x7ff2d815d0f0 2023-11-13T10:57:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.919+0000 7ff2ddbce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2d815d0f0 0x7ff2d80d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.919+0000 7ff2ddbce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2d815d0f0 0x7ff2d80d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52154/0 (socket says 172.21.15.16:52154) 2023-11-13T10:57:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.919+0000 7ff2ddbce640 1 -- 172.21.15.16:0/3782767785 learned_addr learned my addr 172.21.15.16:0/3782767785 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:16.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.919+0000 7ff2dd3cd640 1 --1- 172.21.15.16:0/3782767785 >> v1:172.21.15.16:6789/0 conn(0x7ff2d80cfe20 0x7ff2d80b3950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60298/0 (socket says 172.21.15.16:60298) 2023-11-13T10:57:16.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.919+0000 7ff2ddbce640 1 -- 172.21.15.16:0/3782767785 >> v1:172.21.15.184:6789/0 conn(0x7ff2d80d0ac0 legacy=0x7ff2d80b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.920+0000 7ff2ddbce640 1 -- 172.21.15.16:0/3782767785 >> v1:172.21.15.16:6789/0 conn(0x7ff2d80cfe20 legacy=0x7ff2d80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.920+0000 7ff2ddbce640 1 -- 172.21.15.16:0/3782767785 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80c6f90 con 0x7ff2d815d0f0 2023-11-13T10:57:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.921+0000 7ff2ddbce640 1 --2- 172.21.15.16:0/3782767785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d815d0f0 0x7ff2d80d12c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009e50 tx=0x7ff2d402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf149feb69efb2e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.921+0000 7ff2dcbcc640 1 -- 172.21.15.16:0/3782767785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d402cbb0 con 0x7ff2d815d0f0 2023-11-13T10:57:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.921+0000 7ff2dcbcc640 1 -- 172.21.15.16:0/3782767785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d402cd10 con 0x7ff2d815d0f0 2023-11-13T10:57:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.921+0000 7ff2dcbcc640 1 -- 172.21.15.16:0/3782767785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4034610 con 0x7ff2d815d0f0 2023-11-13T10:57:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.921+0000 7ff2dfe32640 1 -- 172.21.15.16:0/3782767785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d815d0f0 msgr2=0x7ff2d80d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.921+0000 7ff2dfe32640 1 --2- 172.21.15.16:0/3782767785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d815d0f0 0x7ff2d80d12c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009e50 tx=0x7ff2d402c010 comp rx=0 tx=0).stop 2023-11-13T10:57:16.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.922+0000 7ff2dfe32640 1 -- 172.21.15.16:0/3782767785 shutdown_connections 2023-11-13T10:57:16.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.922+0000 7ff2dfe32640 1 --2- 172.21.15.16:0/3782767785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d815d0f0 0x7ff2d80d12c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.922+0000 7ff2dfe32640 1 -- 172.21.15.16:0/3782767785 >> 172.21.15.16:0/3782767785 conn(0x7ff2d80d2ce0 msgr2=0x7ff2d80d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:16.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.922+0000 7ff2dfe32640 1 -- 172.21.15.16:0/3782767785 shutdown_connections 2023-11-13T10:57:16.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.922+0000 7ff2dfe32640 1 -- 172.21.15.16:0/3782767785 wait complete. 2023-11-13T10:57:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.923+0000 7ff2dfe32640 1 Processor -- start 2023-11-13T10:57:16.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.924+0000 7ff2dfe32640 1 -- start start 2023-11-13T10:57:16.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dfe32640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2d80cfe20 0x7ff2d80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dfe32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d80d0ac0 0x7ff2d815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dfe32640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2d815d0f0 0x7ff2d80b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dfe32640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d80cb790 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dfe32640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d80cba70 con 0x7ff2d80cfe20 2023-11-13T10:57:16.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dfe32640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d80cbd50 con 0x7ff2d815d0f0 2023-11-13T10:57:16.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dd3cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d80d0ac0 0x7ff2d815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dd3cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d80d0ac0 0x7ff2d815b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52156/0 (socket says 172.21.15.16:52156) 2023-11-13T10:57:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dd3cd640 1 -- 172.21.15.16:0/695919130 learned_addr learned my addr 172.21.15.16:0/695919130 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dd3cd640 1 -- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2d815d0f0 msgr2=0x7ff2d80b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2ddbce640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2d80cfe20 0x7ff2d80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2de3cf640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2d815d0f0 0x7ff2d80b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dd3cd640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2d815d0f0 0x7ff2d80b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dd3cd640 1 -- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2d80cfe20 msgr2=0x7ff2d80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dd3cd640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2d80cfe20 0x7ff2d80cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2dd3cd640 1 -- 172.21.15.16:0/695919130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d4009b30 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.925+0000 7ff2de3cf640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2d815d0f0 0x7ff2d80b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.926+0000 7ff2dd3cd640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d80d0ac0 0x7ff2d815b080 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c450 tx=0x7ff2c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:16.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.926+0000 7ff2ce7fc640 1 -- 172.21.15.16:0/695919130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8007480 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.926+0000 7ff2ce7fc640 1 -- 172.21.15.16:0/695919130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c80075e0 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.926+0000 7ff2ce7fc640 1 -- 172.21.15.16:0/695919130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8004a00 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.926+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d80cc030 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.926+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d80cb410 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.928+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d8067090 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.928+0000 7ff2ce7fc640 1 -- 172.21.15.16:0/695919130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2c8004b60 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.929+0000 7ff2ce7fc640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2bc06ae40 0x7ff2bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.930+0000 7ff2ce7fc640 1 -- 172.21.15.16:0/695919130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff2c80884b0 con 0x7ff2d80d0ac0 2023-11-13T10:57:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.930+0000 7ff2ddbce640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2bc06ae40 0x7ff2bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.931+0000 7ff2ddbce640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2bc06ae40 0x7ff2bc06d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4035000 tx=0x7ff2d4030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:16.934+0000 7ff2ce7fc640 1 -- 172.21.15.16:0/695919130 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2c80536d0 con 0x7ff2d80d0ac0 2023-11-13T10:57:17.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:57:17.112 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:57:17.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.123+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2d80d3c70 con 0x7ff2d80d0ac0 2023-11-13T10:57:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.124+0000 7ff2ce7fc640 1 -- 172.21.15.16:0/695919130 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13074 (secure 0 0 0) 0x7ff2d80d3c70 con 0x7ff2d80d0ac0 2023-11-13T10:57:17.136 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:17.137 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":36,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:57:12.863249+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:57:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2bc06ae40 msgr2=0x7ff2bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2bc06ae40 0x7ff2bc06d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4035000 tx=0x7ff2d4030000 comp rx=0 tx=0).stop 2023-11-13T10:57:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d80d0ac0 msgr2=0x7ff2d815b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d80d0ac0 0x7ff2d815b080 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c450 tx=0x7ff2c800c910 comp rx=0 tx=0).stop 2023-11-13T10:57:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 shutdown_connections 2023-11-13T10:57:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff2bc06ae40 0x7ff2bc06d300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2d815d0f0 0x7ff2d80b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2d80d0ac0 0x7ff2d815b080 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 --2- 172.21.15.16:0/695919130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2d80cfe20 0x7ff2d80cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 >> 172.21.15.16:0/695919130 conn(0x7ff2d80d2ce0 msgr2=0x7ff2d80cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 shutdown_connections 2023-11-13T10:57:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.126+0000 7ff2dfe32640 1 -- 172.21.15.16:0/695919130 wait complete. 2023-11-13T10:57:17.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T10:57:17.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T10:57:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.425+0000 7fb5127d7640 1 Processor -- start 2023-11-13T10:57:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.426+0000 7fb5127d7640 1 -- start start 2023-11-13T10:57:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.427+0000 7fb5127d7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.427+0000 7fb5127d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb50c161640 0x7fb50c0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.427+0000 7fb5127d7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50c165b40 con 0x7fb50c0b2970 2023-11-13T10:57:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.427+0000 7fb5127d7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c0b2de0 con 0x7fb50c161640 2023-11-13T10:57:17.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.427+0000 7fb5127d7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c1652a0 con 0x7fb50c15b5c0 2023-11-13T10:57:17.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb510fd4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb50c161640 0x7fb50c0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb510fd4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb50c161640 0x7fb50c0b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52172/0 (socket says 172.21.15.16:52172) 2023-11-13T10:57:17.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb510fd4640 1 -- 172.21.15.16:0/3687454372 learned_addr learned my addr 172.21.15.16:0/3687454372 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:17.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb510fd4640 1 -- 172.21.15.16:0/3687454372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 msgr2=0x7fb50c164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:17.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb5117d5640 1 --2- 172.21.15.16:0/3687454372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb510fd4640 1 --2- 172.21.15.16:0/3687454372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb503fff640 1 -- 172.21.15.16:0/3687454372 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455700360 0 0) 0x7fb50c165b40 con 0x7fb50c0b2970 2023-11-13T10:57:17.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb510fd4640 1 -- 172.21.15.16:0/3687454372 >> v1:172.21.15.16:6789/0 conn(0x7fb50c0b2970 legacy=0x7fb50c0ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb510fd4640 1 -- 172.21.15.16:0/3687454372 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c165580 con 0x7fb50c161640 2023-11-13T10:57:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.428+0000 7fb5117d5640 1 --2- 172.21.15.16:0/3687454372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.429+0000 7fb510fd4640 1 --2- 172.21.15.16:0/3687454372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 0x7fb50c0b2430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009870 tx=0x7fb4fc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb14b1b6eb28e66 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb503fff640 1 -- 172.21.15.16:0/3687454372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc030840 con 0x7fb50c161640 2023-11-13T10:57:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb503fff640 1 -- 172.21.15.16:0/3687454372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc0309a0 con 0x7fb50c161640 2023-11-13T10:57:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb503fff640 1 -- 172.21.15.16:0/3687454372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc030d30 con 0x7fb50c161640 2023-11-13T10:57:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb5127d7640 1 -- 172.21.15.16:0/3687454372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 msgr2=0x7fb50c0b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb5127d7640 1 --2- 172.21.15.16:0/3687454372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 0x7fb50c0b2430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009870 tx=0x7fb4fc02b980 comp rx=0 tx=0).stop 2023-11-13T10:57:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb5127d7640 1 -- 172.21.15.16:0/3687454372 shutdown_connections 2023-11-13T10:57:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb5127d7640 1 --2- 172.21.15.16:0/3687454372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 0x7fb50c0b2430 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb5127d7640 1 --2- 172.21.15.16:0/3687454372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb5127d7640 1 -- 172.21.15.16:0/3687454372 >> 172.21.15.16:0/3687454372 conn(0x7fb50c0b0970 msgr2=0x7fb50c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:17.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb5127d7640 1 -- 172.21.15.16:0/3687454372 shutdown_connections 2023-11-13T10:57:17.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.430+0000 7fb5127d7640 1 -- 172.21.15.16:0/3687454372 wait complete. 2023-11-13T10:57:17.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.432+0000 7fb5127d7640 1 Processor -- start 2023-11-13T10:57:17.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.432+0000 7fb5127d7640 1 -- start start 2023-11-13T10:57:17.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.433+0000 7fb5127d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb50c0b2970 0x7fb50c1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.433+0000 7fb5127d7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.433+0000 7fb5127d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 0x7fb50c15ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.433+0000 7fb5127d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c151810 con 0x7fb50c161640 2023-11-13T10:57:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.433+0000 7fb5127d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c156340 con 0x7fb50c0b2970 2023-11-13T10:57:17.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.433+0000 7fb5127d7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c156620 con 0x7fb50c15b5c0 2023-11-13T10:57:17.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.433+0000 7fb511fd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 0x7fb50c15ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.433+0000 7fb511fd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 0x7fb50c15ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52178/0 (socket says 172.21.15.16:52178) 2023-11-13T10:57:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.433+0000 7fb511fd6640 1 -- 172.21.15.16:0/549904850 learned_addr learned my addr 172.21.15.16:0/549904850 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb510fd4640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb511fd6640 1 -- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 msgr2=0x7fb50c1573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb511fd6640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c1573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb511fd6640 1 -- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb50c0b2970 msgr2=0x7fb50c1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb5117d5640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb50c0b2970 0x7fb50c1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb511fd6640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb50c0b2970 0x7fb50c1512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb511fd6640 1 -- 172.21.15.16:0/549904850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0094d0 con 0x7fb50c161640 2023-11-13T10:57:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb510fd4640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb511fd6640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 0x7fb50c15ffc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb504001560 tx=0x7fb50400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb501ffb640 1 -- 172.21.15.16:0/549904850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504003da0 con 0x7fb50c161640 2023-11-13T10:57:17.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb501ffb640 1 -- 172.21.15.16:0/549904850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb504005070 con 0x7fb50c161640 2023-11-13T10:57:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.434+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c156900 con 0x7fb50c161640 2023-11-13T10:57:17.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.435+0000 7fb501ffb640 1 -- 172.21.15.16:0/549904850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504005420 con 0x7fb50c161640 2023-11-13T10:57:17.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.435+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c156ec0 con 0x7fb50c161640 2023-11-13T10:57:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.436+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50c04e4d0 con 0x7fb50c161640 2023-11-13T10:57:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.437+0000 7fb501ffb640 1 -- 172.21.15.16:0/549904850 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb504005580 con 0x7fb50c161640 2023-11-13T10:57:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.438+0000 7fb501ffb640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb4f006ae70 0x7fb4f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.438+0000 7fb501ffb640 1 -- 172.21.15.16:0/549904850 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb504088d50 con 0x7fb50c161640 2023-11-13T10:57:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.438+0000 7fb5117d5640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb4f006ae70 0x7fb4f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.440+0000 7fb5117d5640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb4f006ae70 0x7fb4f006d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb508008850 tx=0x7fb508008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.442+0000 7fb501ffb640 1 -- 172.21.15.16:0/549904850 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb504053f70 con 0x7fb50c161640 2023-11-13T10:57:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.631+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb50c070200 con 0x7fb50c161640 2023-11-13T10:57:17.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.632+0000 7fb501ffb640 1 -- 172.21.15.16:0/549904850 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb504017070 con 0x7fb50c161640 2023-11-13T10:57:17.636 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T10:57:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb4f006ae70 msgr2=0x7fb4f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb4f006ae70 0x7fb4f006d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb508008850 tx=0x7fb508008040 comp rx=0 tx=0).stop 2023-11-13T10:57:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 msgr2=0x7fb50c15ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 0x7fb50c15ffc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb504001560 tx=0x7fb50400cc70 comp rx=0 tx=0).stop 2023-11-13T10:57:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 shutdown_connections 2023-11-13T10:57:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb4f006ae70 0x7fb4f006d330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c161640 0x7fb50c15ffc0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb50c15b5c0 0x7fb50c1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 --2- 172.21.15.16:0/549904850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb50c0b2970 0x7fb50c1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 >> 172.21.15.16:0/549904850 conn(0x7fb50c0b0970 msgr2=0x7fb50c0afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 shutdown_connections 2023-11-13T10:57:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.634+0000 7fb5127d7640 1 -- 172.21.15.16:0/549904850 wait complete. 2023-11-13T10:57:17.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-13T10:57:17.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.875+0000 7fbb2bfff640 1 Processor -- start 2023-11-13T10:57:17.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.875+0000 7fbb2bfff640 1 -- start start 2023-11-13T10:57:17.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.877+0000 7fbb2bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c070a70 0x7fbb2c164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.877+0000 7fbb2bfff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c06d680 con 0x7fbb2c06a8a0 2023-11-13T10:57:17.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.877+0000 7fbb2bfff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2c15d630 con 0x7fbb2c15aa90 2023-11-13T10:57:17.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.877+0000 7fbb2bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15d910 con 0x7fbb2c070a70 2023-11-13T10:57:17.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.877+0000 7fbb2a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb2c06a8a0 0x7fbb2c070360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60316/0 (socket says 172.21.15.16:60316) 2023-11-13T10:57:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.877+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/1329662862 learned_addr learned my addr 172.21.15.16:0/1329662862 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.878+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c070a70 0x7fbb2c164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.878+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466281099 0 0) 0x7fbb2c06d680 con 0x7fbb2c06a8a0 2023-11-13T10:57:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.878+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb14003520 con 0x7fbb2c06a8a0 2023-11-13T10:57:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.878+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644745614 0 0) 0x7fbb14003520 con 0x7fbb2c06a8a0 2023-11-13T10:57:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.878+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c070a70 msgr2=0x7fbb2c164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.878+0000 7fbb29ffb640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c070a70 0x7fbb2c164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.878+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 >> v1:172.21.15.184:6789/0 conn(0x7fbb2c15aa90 legacy=0x7fbb2c06a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.878+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c15dbf0 con 0x7fbb2c06a8a0 2023-11-13T10:57:17.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.878+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fbb20001670 con 0x7fbb2c06a8a0 2023-11-13T10:57:17.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 >> v1:172.21.15.16:6789/0 conn(0x7fbb2c06a8a0 legacy=0x7fbb2c070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb29ffb640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb140039a0 0x7fbb14003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb29ffb640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140047c0 0x7fbb14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb29ffb640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb140052a0 0x7fbb1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15d910 con 0x7fbb140052a0 2023-11-13T10:57:17.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1400c120 con 0x7fbb140039a0 2023-11-13T10:57:17.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1400c2e0 con 0x7fbb140047c0 2023-11-13T10:57:17.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c070a70 0x7fbb2c164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:57:17.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb140052a0 0x7fbb1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb2a7fc640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140047c0 0x7fbb14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb2affd640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb140039a0 0x7fbb14003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb2b7fe640 1 -- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140047c0 msgr2=0x7fbb14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:17.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb140047c0 0x7fbb14004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb2b7fe640 1 -- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb140039a0 msgr2=0x7fbb14003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:17.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb140039a0 0x7fbb14003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.879+0000 7fbb2b7fe640 1 -- 172.21.15.16:0/1329662862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14003520 con 0x7fbb140052a0 2023-11-13T10:57:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.881+0000 7fbb2b7fe640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb140052a0 0x7fbb1400b980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c070e50 tx=0x7fbb140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d40c6c7da728ff1 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:17.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.881+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2402ee70 con 0x7fbb140052a0 2023-11-13T10:57:17.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.881+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb24003990 con 0x7fbb140052a0 2023-11-13T10:57:17.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.881+0000 7fbb29ffb640 1 -- 172.21.15.16:0/1329662862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24003d20 con 0x7fbb140052a0 2023-11-13T10:57:17.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.881+0000 7fbb2bfff640 1 -- 172.21.15.16:0/1329662862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb140052a0 msgr2=0x7fbb1400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.881+0000 7fbb2bfff640 1 --2- 172.21.15.16:0/1329662862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb140052a0 0x7fbb1400b980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c070e50 tx=0x7fbb140057f0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.881+0000 7fbb2b7fe640 1 -- 172.21.15.16:0/1329662862 reap_dead start 2023-11-13T10:57:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.881+0000 7fbb2bfff640 1 -- 172.21.15.16:0/1329662862 shutdown_connections 2023-11-13T10:57:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.881+0000 7fbb2bfff640 1 -- 172.21.15.16:0/1329662862 >> 172.21.15.16:0/1329662862 conn(0x7fbb2c0b04a0 msgr2=0x7fbb2c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.882+0000 7fbb2bfff640 1 -- 172.21.15.16:0/1329662862 shutdown_connections 2023-11-13T10:57:17.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.882+0000 7fbb2bfff640 1 -- 172.21.15.16:0/1329662862 wait complete. 2023-11-13T10:57:17.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.883+0000 7fbb2bfff640 1 Processor -- start 2023-11-13T10:57:17.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.884+0000 7fbb2bfff640 1 -- start start 2023-11-13T10:57:17.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.884+0000 7fbb2bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c146e60 0x7fbb2c153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.884+0000 7fbb2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c147240 0x7fbb2c0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.884+0000 7fbb2bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c154730 0x7fbb2c157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.884+0000 7fbb2bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c0b1aa0 con 0x7fbb2c147240 2023-11-13T10:57:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.884+0000 7fbb2bfff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c1542a0 con 0x7fbb2c146e60 2023-11-13T10:57:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.884+0000 7fbb2bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c153690 con 0x7fbb2c154730 2023-11-13T10:57:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.885+0000 7fbb2a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c147240 0x7fbb2c0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.885+0000 7fbb2a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c147240 0x7fbb2c0b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52200/0 (socket says 172.21.15.16:52200) 2023-11-13T10:57:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.885+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/3117785703 learned_addr learned my addr 172.21.15.16:0/3117785703 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.885+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c154730 msgr2=0x7fbb2c157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.885+0000 7fbb2a7fc640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c154730 0x7fbb2c157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.885+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c146e60 msgr2=0x7fbb2c153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.885+0000 7fbb2a7fc640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c146e60 0x7fbb2c153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.885+0000 7fbb2a7fc640 1 -- 172.21.15.16:0/3117785703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c153970 con 0x7fbb2c147240 2023-11-13T10:57:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.886+0000 7fbb2a7fc640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c147240 0x7fbb2c0b1560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000eb20 tx=0x7fbb20001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.886+0000 7fbb13fff640 1 -- 172.21.15.16:0/3117785703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000c7f0 con 0x7fbb2c147240 2023-11-13T10:57:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.886+0000 7fbb13fff640 1 -- 172.21.15.16:0/3117785703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2000c950 con 0x7fbb2c147240 2023-11-13T10:57:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.886+0000 7fbb13fff640 1 -- 172.21.15.16:0/3117785703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000cce0 con 0x7fbb2c147240 2023-11-13T10:57:17.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.886+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c153c50 con 0x7fbb2c147240 2023-11-13T10:57:17.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.886+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c154bb0 con 0x7fbb2c147240 2023-11-13T10:57:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.887+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2c0715f0 con 0x7fbb2c147240 2023-11-13T10:57:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.888+0000 7fbb13fff640 1 -- 172.21.15.16:0/3117785703 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb2001d460 con 0x7fbb2c147240 2023-11-13T10:57:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.889+0000 7fbb13fff640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1806f250 0x7fbb18071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.890+0000 7fbb13fff640 1 -- 172.21.15.16:0/3117785703 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbb20089320 con 0x7fbb2c147240 2023-11-13T10:57:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.890+0000 7fbb2affd640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1806f250 0x7fbb18071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.891+0000 7fbb2affd640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1806f250 0x7fbb18071710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007fd0 tx=0x7fbb1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:17.894+0000 7fbb13fff640 1 -- 172.21.15.16:0/3117785703 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb200545d0 con 0x7fbb2c147240 2023-11-13T10:57:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.101+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbb2c0707c0 con 0x7fbb2c147240 2023-11-13T10:57:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.102+0000 7fbb13fff640 1 -- 172.21.15.16:0/3117785703 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbb20057bf0 con 0x7fbb2c147240 2023-11-13T10:57:18.106 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:18.106 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-13T10:57:18.106 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T10:57:18.106 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-13T10:57:18.107 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-13T10:57:18.107 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:18.107 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-13T10:57:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1806f250 msgr2=0x7fbb18071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:18.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1806f250 0x7fbb18071710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c007fd0 tx=0x7fbb1c009040 comp rx=0 tx=0).stop 2023-11-13T10:57:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c147240 msgr2=0x7fbb2c0b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c147240 0x7fbb2c0b1560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000eb20 tx=0x7fbb20001630 comp rx=0 tx=0).stop 2023-11-13T10:57:18.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 shutdown_connections 2023-11-13T10:57:18.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1806f250 0x7fbb18071710 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb2c154730 0x7fbb2c157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb2c147240 0x7fbb2c0b1560 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 --2- 172.21.15.16:0/3117785703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb2c146e60 0x7fbb2c153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 >> 172.21.15.16:0/3117785703 conn(0x7fbb2c0b04a0 msgr2=0x7fbb2c158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 shutdown_connections 2023-11-13T10:57:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.105+0000 7fbb2bfff640 1 -- 172.21.15.16:0/3117785703 wait complete. 2023-11-13T10:57:18.131 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-13T10:57:18.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-13T10:57:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.380+0000 7faeee1e2640 1 Processor -- start 2023-11-13T10:57:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.381+0000 7faeee1e2640 1 -- start start 2023-11-13T10:57:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.382+0000 7faeee1e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80cf580 0x7faee80cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.382+0000 7faeee1e2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faee80ce4c0 0x7faee80ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.382+0000 7faeee1e2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee80cfea0 con 0x7faee80d0e10 2023-11-13T10:57:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.382+0000 7faeee1e2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee80b59c0 con 0x7faee80ce4c0 2023-11-13T10:57:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.382+0000 7faeee1e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80cf1f0 con 0x7faee80cf580 2023-11-13T10:57:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faeed9e1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faee80d0e10 0x7faee80d12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60332/0 (socket says 172.21.15.16:60332) 2023-11-13T10:57:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faeed9e1640 1 -- 172.21.15.16:0/1920594246 learned_addr learned my addr 172.21.15.16:0/1920594246 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faeec9df640 1 --2- 172.21.15.16:0/1920594246 >> v2:172.21.15.16:3300/0 conn(0x7faee80ce4c0 0x7faee80ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faeed1e0640 1 --2- 172.21.15.16:0/1920594246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80cf580 0x7faee80cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faedffff640 1 -- 172.21.15.16:0/1920594246 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031389792 0 0) 0x7faee80cfea0 con 0x7faee80d0e10 2023-11-13T10:57:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faedffff640 1 -- 172.21.15.16:0/1920594246 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed4003520 con 0x7faee80d0e10 2023-11-13T10:57:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faeec9df640 1 -- 172.21.15.16:0/1920594246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80cf580 msgr2=0x7faee80cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faeec9df640 1 --2- 172.21.15.16:0/1920594246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80cf580 0x7faee80cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faeec9df640 1 -- 172.21.15.16:0/1920594246 >> v1:172.21.15.16:6789/0 conn(0x7faee80d0e10 legacy=0x7faee80d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faeec9df640 1 -- 172.21.15.16:0/1920594246 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80b5bd0 con 0x7faee80ce4c0 2023-11-13T10:57:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.383+0000 7faeed1e0640 1 --2- 172.21.15.16:0/1920594246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80cf580 0x7faee80cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:57:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.384+0000 7faeec9df640 1 --2- 172.21.15.16:0/1920594246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80ce4c0 0x7faee80ce920 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faed80097a0 tx=0x7faed802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11afe6b82025706 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faedffff640 1 -- 172.21.15.16:0/1920594246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed802d040 con 0x7faee80ce4c0 2023-11-13T10:57:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faedffff640 1 -- 172.21.15.16:0/1920594246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed8031810 con 0x7faee80ce4c0 2023-11-13T10:57:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faedffff640 1 -- 172.21.15.16:0/1920594246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8031ba0 con 0x7faee80ce4c0 2023-11-13T10:57:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faeee1e2640 1 -- 172.21.15.16:0/1920594246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80ce4c0 msgr2=0x7faee80ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faeee1e2640 1 --2- 172.21.15.16:0/1920594246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80ce4c0 0x7faee80ce920 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faed80097a0 tx=0x7faed802b8b0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faeee1e2640 1 -- 172.21.15.16:0/1920594246 shutdown_connections 2023-11-13T10:57:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faeee1e2640 1 --2- 172.21.15.16:0/1920594246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80ce4c0 0x7faee80ce920 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faeee1e2640 1 --2- 172.21.15.16:0/1920594246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80cf580 0x7faee80cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faeee1e2640 1 -- 172.21.15.16:0/1920594246 >> 172.21.15.16:0/1920594246 conn(0x7faee80d2b70 msgr2=0x7faee80d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:18.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faeee1e2640 1 -- 172.21.15.16:0/1920594246 shutdown_connections 2023-11-13T10:57:18.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.385+0000 7faeee1e2640 1 -- 172.21.15.16:0/1920594246 wait complete. 2023-11-13T10:57:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.387+0000 7faeee1e2640 1 Processor -- start 2023-11-13T10:57:18.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.387+0000 7faeee1e2640 1 -- start start 2023-11-13T10:57:18.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.388+0000 7faeee1e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee80ce4c0 0x7faee80af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:18.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.388+0000 7faeee1e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80cf580 0x7faee81e3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:18.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.388+0000 7faeee1e2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80d0e10 0x7faee81e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:18.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.388+0000 7faeee1e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee80cb370 con 0x7faee80cf580 2023-11-13T10:57:18.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.388+0000 7faeee1e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee806c620 con 0x7faee80ce4c0 2023-11-13T10:57:18.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.388+0000 7faeee1e2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80b3410 con 0x7faee80d0e10 2023-11-13T10:57:18.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.388+0000 7faeec9df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80cf580 0x7faee81e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:18.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.388+0000 7faeec9df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80cf580 0x7faee81e3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52210/0 (socket says 172.21.15.16:52210) 2023-11-13T10:57:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.388+0000 7faeec9df640 1 -- 172.21.15.16:0/3695125891 learned_addr learned my addr 172.21.15.16:0/3695125891 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.389+0000 7faeed1e0640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee80ce4c0 0x7faee80af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.389+0000 7faeed9e1640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80d0e10 0x7faee81e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.389+0000 7faeec9df640 1 -- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80d0e10 msgr2=0x7faee81e3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.389+0000 7faeec9df640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80d0e10 0x7faee81e3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.389+0000 7faeec9df640 1 -- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee80ce4c0 msgr2=0x7faee80af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.389+0000 7faeec9df640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee80ce4c0 0x7faee80af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.389+0000 7faeec9df640 1 -- 172.21.15.16:0/3695125891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8009400 con 0x7faee80cf580 2023-11-13T10:57:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.389+0000 7faeed1e0640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee80ce4c0 0x7faee80af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:18.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.389+0000 7faeec9df640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80cf580 0x7faee81e3630 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c00 tx=0x7faed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:18.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.390+0000 7faeddffb640 1 -- 172.21.15.16:0/3695125891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8006e10 con 0x7faee80cf580 2023-11-13T10:57:18.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.390+0000 7faeddffb640 1 -- 172.21.15.16:0/3695125891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed803ad60 con 0x7faee80cf580 2023-11-13T10:57:18.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.390+0000 7faeddffb640 1 -- 172.21.15.16:0/3695125891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8044630 con 0x7faee80cf580 2023-11-13T10:57:18.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.390+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee806bc80 con 0x7faee80cf580 2023-11-13T10:57:18.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.390+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee806c240 con 0x7faee80cf580 2023-11-13T10:57:18.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.391+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee806c3d0 con 0x7faee80cf580 2023-11-13T10:57:18.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.392+0000 7faeddffb640 1 -- 172.21.15.16:0/3695125891 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faed804d450 con 0x7faee80cf580 2023-11-13T10:57:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.393+0000 7faeddffb640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faecc06ae40 0x7faecc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.393+0000 7faeddffb640 1 -- 172.21.15.16:0/3695125891 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7faed80ab5e0 con 0x7faee80cf580 2023-11-13T10:57:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.393+0000 7faeed1e0640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faecc06ae40 0x7faecc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.394+0000 7faeed1e0640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faecc06ae40 0x7faecc06d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faee4009540 tx=0x7faee400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.397+0000 7faeddffb640 1 -- 172.21.15.16:0/3695125891 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faed80768a0 con 0x7faee80cf580 2023-11-13T10:57:18.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:18.584+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faee804e4d0 con 0x7faee80cf580 2023-11-13T10:57:18.872 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-13T10:57:18.873 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-13T10:57:19.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.032+0000 7faeddffb640 1 -- 172.21.15.16:0/3695125891 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7faed8079ec0 con 0x7faee80cf580 2023-11-13T10:57:19.037 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-13T10:57:19.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faecc06ae40 msgr2=0x7faecc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faecc06ae40 0x7faecc06d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faee4009540 tx=0x7faee400b040 comp rx=0 tx=0).stop 2023-11-13T10:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80cf580 msgr2=0x7faee81e3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80cf580 0x7faee81e3630 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c00 tx=0x7faed80046c0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 shutdown_connections 2023-11-13T10:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faecc06ae40 0x7faecc06d300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faee80d0e10 0x7faee81e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee80cf580 0x7faee81e3630 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 --2- 172.21.15.16:0/3695125891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faee80ce4c0 0x7faee80af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.036+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 >> 172.21.15.16:0/3695125891 conn(0x7faee80d2b70 msgr2=0x7faee806ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.037+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 shutdown_connections 2023-11-13T10:57:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.037+0000 7faeee1e2640 1 -- 172.21.15.16:0/3695125891 wait complete. 2023-11-13T10:57:19.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-13T10:57:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.285+0000 7f79ad813640 1 Processor -- start 2023-11-13T10:57:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.285+0000 7f79ad813640 1 -- start start 2023-11-13T10:57:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.287+0000 7f79ad813640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80ce9e0 0x7f79a806f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.287+0000 7f79ad813640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f79a80cca20 0x7f79a806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.287+0000 7f79ad813640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a806fe80 con 0x7f79a80adbe0 2023-11-13T10:57:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.287+0000 7f79ad813640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a80cc280 con 0x7f79a80cca20 2023-11-13T10:57:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.287+0000 7f79ad813640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8070950 con 0x7f79a80ce9e0 2023-11-13T10:57:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.288+0000 7f79ad012640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f79a80cca20 0x7f79a806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.288+0000 7f79ad012640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f79a80cca20 0x7f79a806bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:50278/0 (socket says 172.21.15.16:50278) 2023-11-13T10:57:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.288+0000 7f79ad012640 1 -- 172.21.15.16:0/95955552 learned_addr learned my addr 172.21.15.16:0/95955552 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.288+0000 7f79ac811640 1 --2- 172.21.15.16:0/95955552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80ce9e0 0x7f79a806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.289+0000 7f79ad012640 1 -- 172.21.15.16:0/95955552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80ce9e0 msgr2=0x7f79a806f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.289+0000 7f79ad012640 1 --2- 172.21.15.16:0/95955552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80ce9e0 0x7f79a806f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.289+0000 7f79ad012640 1 -- 172.21.15.16:0/95955552 >> v1:172.21.15.184:6789/0 conn(0x7f79a80adbe0 legacy=0x7f79a80cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.289+0000 7f79ad012640 1 -- 172.21.15.16:0/95955552 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a80cb590 con 0x7f79a80cca20 2023-11-13T10:57:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.289+0000 7f79ac811640 1 --2- 172.21.15.16:0/95955552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80ce9e0 0x7f79a806f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T10:57:19.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.290+0000 7f79ad012640 1 --2- 172.21.15.16:0/95955552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a80cca20 0x7f79a806bd10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79a00098a0 tx=0x7f79a002b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed849c438455201 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.290+0000 7f799f7fe640 1 -- 172.21.15.16:0/95955552 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0031640 con 0x7f79a80cca20 2023-11-13T10:57:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.291+0000 7f799f7fe640 1 -- 172.21.15.16:0/95955552 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a00317a0 con 0x7f79a80cca20 2023-11-13T10:57:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.291+0000 7f799f7fe640 1 -- 172.21.15.16:0/95955552 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0031b30 con 0x7f79a80cca20 2023-11-13T10:57:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.291+0000 7f79ad813640 1 -- 172.21.15.16:0/95955552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a80cca20 msgr2=0x7f79a806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.291+0000 7f79ad813640 1 --2- 172.21.15.16:0/95955552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a80cca20 0x7f79a806bd10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79a00098a0 tx=0x7f79a002b8e0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.291+0000 7f79ad813640 1 -- 172.21.15.16:0/95955552 shutdown_connections 2023-11-13T10:57:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.291+0000 7f79ad813640 1 --2- 172.21.15.16:0/95955552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a80cca20 0x7f79a806bd10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.291+0000 7f79ad813640 1 --2- 172.21.15.16:0/95955552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80ce9e0 0x7f79a806f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.291+0000 7f79ad813640 1 -- 172.21.15.16:0/95955552 >> 172.21.15.16:0/95955552 conn(0x7f79a806a370 msgr2=0x7f79a806a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:19.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.292+0000 7f79ad813640 1 -- 172.21.15.16:0/95955552 shutdown_connections 2023-11-13T10:57:19.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.292+0000 7f79ad813640 1 -- 172.21.15.16:0/95955552 wait complete. 2023-11-13T10:57:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.293+0000 7f79ad813640 1 Processor -- start 2023-11-13T10:57:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.294+0000 7f79ad813640 1 -- start start 2023-11-13T10:57:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.294+0000 7f79ad813640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a80adbe0 0x7f79a81472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f79ad813640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80cca20 0x7f79a8155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f79ad813640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a80ce9e0 0x7f79a8152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f79ad813640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8147830 con 0x7f79a80adbe0 2023-11-13T10:57:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f79ad813640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a81562e0 con 0x7f79a80ce9e0 2023-11-13T10:57:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f79ad813640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8154d00 con 0x7f79a80cca20 2023-11-13T10:57:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f79ac811640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a80adbe0 0x7f79a81472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f79ac811640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a80adbe0 0x7f79a81472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52226/0 (socket says 172.21.15.16:52226) 2023-11-13T10:57:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f79ac811640 1 -- 172.21.15.16:0/540586777 learned_addr learned my addr 172.21.15.16:0/540586777 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f79ad012640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a80ce9e0 0x7f79a8152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.295+0000 7f799ffff640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80cca20 0x7f79a8155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f79ac811640 1 -- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80cca20 msgr2=0x7f79a8155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f79ac811640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80cca20 0x7f79a8155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f79ac811640 1 -- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a80ce9e0 msgr2=0x7f79a8152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f79ac811640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a80ce9e0 0x7f79a8152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f79ac811640 1 -- 172.21.15.16:0/540586777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a4009a70 con 0x7f79a80adbe0 2023-11-13T10:57:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f79ac811640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a80adbe0 0x7f79a81472f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f79a400efc0 tx=0x7f79a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f799d7fa640 1 -- 172.21.15.16:0/540586777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4014680 con 0x7f79a80adbe0 2023-11-13T10:57:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f799d7fa640 1 -- 172.21.15.16:0/540586777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a400f040 con 0x7f79a80adbe0 2023-11-13T10:57:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f799d7fa640 1 -- 172.21.15.16:0/540586777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4014a00 con 0x7f79a80adbe0 2023-11-13T10:57:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a00094d0 con 0x7f79a80adbe0 2023-11-13T10:57:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.296+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a81552c0 con 0x7f79a80adbe0 2023-11-13T10:57:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.298+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a804e4d0 con 0x7f79a80adbe0 2023-11-13T10:57:19.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.299+0000 7f799d7fa640 1 -- 172.21.15.16:0/540586777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79a4014b60 con 0x7f79a80adbe0 2023-11-13T10:57:19.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.300+0000 7f799d7fa640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f799406ae40 0x7f799406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:19.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.300+0000 7f799d7fa640 1 -- 172.21.15.16:0/540586777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f79a408ac00 con 0x7f79a80adbe0 2023-11-13T10:57:19.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.301+0000 7f799ffff640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f799406ae40 0x7f799406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:19.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.302+0000 7f799ffff640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f799406ae40 0x7f799406d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f79a815a300 tx=0x7f7998009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:19.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.304+0000 7f799d7fa640 1 -- 172.21.15.16:0/540586777 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79a4055d30 con 0x7f79a80adbe0 2023-11-13T10:57:19.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:19.489+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f79a80cb0a0 con 0x7f79a80adbe0 2023-11-13T10:57:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.038+0000 7f799d7fa640 1 -- 172.21.15.16:0/540586777 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f79a4059350 con 0x7f79a80adbe0 2023-11-13T10:57:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-13T10:57:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.042+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f799406ae40 msgr2=0x7f799406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.042+0000 7f79ad813640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f799406ae40 0x7f799406d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f79a815a300 tx=0x7f7998009040 comp rx=0 tx=0).stop 2023-11-13T10:57:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.043+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a80adbe0 msgr2=0x7f79a81472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.043+0000 7f79ad813640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a80adbe0 0x7f79a81472f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f79a400efc0 tx=0x7f79a400c490 comp rx=0 tx=0).stop 2023-11-13T10:57:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.043+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 shutdown_connections 2023-11-13T10:57:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.043+0000 7f79ad813640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f799406ae40 0x7f799406d300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.043+0000 7f79ad813640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79a80ce9e0 0x7f79a8152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.043+0000 7f79ad813640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79a80cca20 0x7f79a8155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.043+0000 7f79ad813640 1 --2- 172.21.15.16:0/540586777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79a80adbe0 0x7f79a81472f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.043+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 >> 172.21.15.16:0/540586777 conn(0x7f79a806a370 msgr2=0x7f79a80b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.043+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 shutdown_connections 2023-11-13T10:57:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.044+0000 7f79ad813640 1 -- 172.21.15.16:0/540586777 wait complete. 2023-11-13T10:57:20.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-13T10:57:20.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.271+0000 7f18fb000640 1 Processor -- start 2023-11-13T10:57:20.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.271+0000 7f18fb000640 1 -- start start 2023-11-13T10:57:20.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.272+0000 7f18fb000640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f18f40ce550 0x7f18f40ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:20.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.272+0000 7f18fb000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18f40b19d0 0x7f18f40b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:20.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.272+0000 7f18fb000640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f40ceef0 con 0x7f18f4144570 2023-11-13T10:57:20.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.272+0000 7f18fb000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f4144950 con 0x7f18f40b19d0 2023-11-13T10:57:20.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.272+0000 7f18fb000640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f40ce300 con 0x7f18f40ce550 2023-11-13T10:57:20.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.273+0000 7f18f959d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18f40b19d0 0x7f18f40b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:20.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.273+0000 7f18f959d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18f40b19d0 0x7f18f40b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52240/0 (socket says 172.21.15.16:52240) 2023-11-13T10:57:20.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.273+0000 7f18f959d640 1 -- 172.21.15.16:0/1381371129 learned_addr learned my addr 172.21.15.16:0/1381371129 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:20.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.273+0000 7f18f3fff640 1 --2- 172.21.15.16:0/1381371129 >> v2:172.21.15.184:3300/0 conn(0x7f18f40ce550 0x7f18f40ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:20.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.273+0000 7f18f959d640 1 -- 172.21.15.16:0/1381371129 >> v2:172.21.15.184:3300/0 conn(0x7f18f40ce550 msgr2=0x7f18f40ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:20.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.273+0000 7f18f959d640 1 --2- 172.21.15.16:0/1381371129 >> v2:172.21.15.184:3300/0 conn(0x7f18f40ce550 0x7f18f40ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.273+0000 7f18f959d640 1 -- 172.21.15.16:0/1381371129 >> v1:172.21.15.184:6789/0 conn(0x7f18f4144570 legacy=0x7f18f415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:20.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.273+0000 7f18f959d640 1 -- 172.21.15.16:0/1381371129 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40cd260 con 0x7f18f40b19d0 2023-11-13T10:57:20.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.273+0000 7f18f3fff640 1 --2- 172.21.15.16:0/1381371129 >> v2:172.21.15.184:3300/0 conn(0x7f18f40ce550 0x7f18f40ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.274+0000 7f18f959d640 1 --2- 172.21.15.16:0/1381371129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40b19d0 0x7f18f40b1e40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f18f40cf5b0 tx=0x7f18ec02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6a9d8ed1753947 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.275+0000 7f18f37fe640 1 -- 172.21.15.16:0/1381371129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec02ca80 con 0x7f18f40b19d0 2023-11-13T10:57:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.275+0000 7f18f37fe640 1 -- 172.21.15.16:0/1381371129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec02cbe0 con 0x7f18f40b19d0 2023-11-13T10:57:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.275+0000 7f18f37fe640 1 -- 172.21.15.16:0/1381371129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec0343f0 con 0x7f18f40b19d0 2023-11-13T10:57:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.275+0000 7f18fb000640 1 -- 172.21.15.16:0/1381371129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40b19d0 msgr2=0x7f18f40b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.275+0000 7f18fb000640 1 --2- 172.21.15.16:0/1381371129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40b19d0 0x7f18f40b1e40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f18f40cf5b0 tx=0x7f18ec02bee0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.276+0000 7f18fb000640 1 -- 172.21.15.16:0/1381371129 shutdown_connections 2023-11-13T10:57:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.276+0000 7f18fb000640 1 --2- 172.21.15.16:0/1381371129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40b19d0 0x7f18f40b1e40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.276+0000 7f18fb000640 1 --2- 172.21.15.16:0/1381371129 >> v2:172.21.15.184:3300/0 conn(0x7f18f40ce550 0x7f18f40ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.276+0000 7f18fb000640 1 -- 172.21.15.16:0/1381371129 >> 172.21.15.16:0/1381371129 conn(0x7f18f406a190 msgr2=0x7f18f406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.276+0000 7f18fb000640 1 -- 172.21.15.16:0/1381371129 shutdown_connections 2023-11-13T10:57:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.276+0000 7f18fb000640 1 -- 172.21.15.16:0/1381371129 wait complete. 2023-11-13T10:57:20.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.277+0000 7f18fb000640 1 Processor -- start 2023-11-13T10:57:20.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.278+0000 7f18fb000640 1 -- start start 2023-11-13T10:57:20.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18fb000640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40b19d0 0x7f18f413ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:20.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18fb000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40ce550 0x7f18f413a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:20.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18fb000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f4144570 0x7f18f4157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18fb000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f413f630 con 0x7f18f40ce550 2023-11-13T10:57:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18fb000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f41f3620 con 0x7f18f4144570 2023-11-13T10:57:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18fb000640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f41f3900 con 0x7f18f40b19d0 2023-11-13T10:57:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40ce550 0x7f18f413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40ce550 0x7f18f413a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52254/0 (socket says 172.21.15.16:52254) 2023-11-13T10:57:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f3fff640 1 -- 172.21.15.16:0/378156033 learned_addr learned my addr 172.21.15.16:0/378156033 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f959d640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f4144570 0x7f18f4157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f8d9c640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40b19d0 0x7f18f413ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f3fff640 1 -- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40b19d0 msgr2=0x7f18f413ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f3fff640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40b19d0 0x7f18f413ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f3fff640 1 -- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f4144570 msgr2=0x7f18f4157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f3fff640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f4144570 0x7f18f4157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.279+0000 7f18f3fff640 1 -- 172.21.15.16:0/378156033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec009b30 con 0x7f18f40ce550 2023-11-13T10:57:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.280+0000 7f18f3fff640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40ce550 0x7f18f413a730 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f18e000c7f0 tx=0x7f18e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.280+0000 7f18f17fa640 1 -- 172.21.15.16:0/378156033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0007a30 con 0x7f18f40ce550 2023-11-13T10:57:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.280+0000 7f18f17fa640 1 -- 172.21.15.16:0/378156033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e0007b90 con 0x7f18f40ce550 2023-11-13T10:57:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.280+0000 7f18f17fa640 1 -- 172.21.15.16:0/378156033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e000ce40 con 0x7f18f40ce550 2023-11-13T10:57:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.280+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f41f3be0 con 0x7f18f40ce550 2023-11-13T10:57:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.280+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f413bf90 con 0x7f18f40ce550 2023-11-13T10:57:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.281+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f413f7e0 con 0x7f18f40ce550 2023-11-13T10:57:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.282+0000 7f18f17fa640 1 -- 172.21.15.16:0/378156033 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18e00113f0 con 0x7f18f40ce550 2023-11-13T10:57:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.283+0000 7f18f17fa640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.283+0000 7f18f17fa640 1 -- 172.21.15.16:0/378156033 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f18e0089960 con 0x7f18f40ce550 2023-11-13T10:57:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.284+0000 7f18f8d9c640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.285+0000 7f18f8d9c640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f18e800a750 tx=0x7f18e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.288+0000 7f18f17fa640 1 -- 172.21.15.16:0/378156033 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18f413f7e0 con 0x7f18f40ce550 2023-11-13T10:57:20.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:20.521+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f18f406a7a0 con 0x7f18f40ce550 2023-11-13T10:57:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.074+0000 7f18f17fa640 1 -- 172.21.15.16:0/378156033 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f18e001f0c0 con 0x7f18f40ce550 2023-11-13T10:57:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 5 and data pool 6 2023-11-13T10:57:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.078+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 msgr2=0x7f18d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.078+0000 7f18fb000640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f18e800a750 tx=0x7f18e8008040 comp rx=0 tx=0).stop 2023-11-13T10:57:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.078+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40ce550 msgr2=0x7f18f413a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.078+0000 7f18fb000640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40ce550 0x7f18f413a730 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f18e000c7f0 tx=0x7f18e000cbf0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.079+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 shutdown_connections 2023-11-13T10:57:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.079+0000 7f18fb000640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.079+0000 7f18fb000640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f4144570 0x7f18f4157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.079+0000 7f18fb000640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f40ce550 0x7f18f413a730 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.079+0000 7f18fb000640 1 --2- 172.21.15.16:0/378156033 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40b19d0 0x7f18f413ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.079+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 >> 172.21.15.16:0/378156033 conn(0x7f18f406a190 msgr2=0x7f18f4071020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.079+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 shutdown_connections 2023-11-13T10:57:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.079+0000 7f18fb000640 1 -- 172.21.15.16:0/378156033 wait complete. 2023-11-13T10:57:21.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:57:21.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.311+0000 7f3552f6c640 1 Processor -- start 2023-11-13T10:57:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.311+0000 7f3552f6c640 1 -- start start 2023-11-13T10:57:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.313+0000 7f3552f6c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f354c06a190 0x7f354c06a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.313+0000 7f3552f6c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c0b1840 con 0x7f354c06f970 2023-11-13T10:57:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.313+0000 7f3552f6c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c071600 con 0x7f354c161060 2023-11-13T10:57:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.313+0000 7f3552f6c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c06ab60 con 0x7f354c06a190 2023-11-13T10:57:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.313+0000 7f3551769640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f354c06f970 0x7f354c162900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60344/0 (socket says 172.21.15.16:60344) 2023-11-13T10:57:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3551769640 1 -- 172.21.15.16:0/35044046 learned_addr learned my addr 172.21.15.16:0/35044046 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f355276b640 1 --2- 172.21.15.16:0/35044046 >> v2:172.21.15.184:3300/0 conn(0x7f354c06a190 0x7f354c06a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433028138 0 0) 0x7f354c0b1840 con 0x7f354c06f970 2023-11-13T10:57:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3538003520 con 0x7f354c06f970 2023-11-13T10:57:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4264560676 0 0) 0x7f3538003520 con 0x7f354c06f970 2023-11-13T10:57:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 >> v2:172.21.15.184:3300/0 conn(0x7f354c06a190 msgr2=0x7f354c06a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3550f68640 1 --2- 172.21.15.16:0/35044046 >> v2:172.21.15.184:3300/0 conn(0x7f354c06a190 0x7f354c06a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 >> v1:172.21.15.184:6789/0 conn(0x7f354c161060 legacy=0x7f354c1621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0718e0 con 0x7f354c06f970 2023-11-13T10:57:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f353c001670 con 0x7f354c06f970 2023-11-13T10:57:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.314+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 >> v1:172.21.15.16:6789/0 conn(0x7f354c06f970 legacy=0x7f354c162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3550f68640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3538003970 0x7f3538003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3550f68640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35380047c0 0x7f3538004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3550f68640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538005180 0x7f353800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c06ab60 con 0x7f3538005180 2023-11-13T10:57:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353800c020 con 0x7f35380047c0 2023-11-13T10:57:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353800c1e0 con 0x7f3538003970 2023-11-13T10:57:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f355276b640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538005180 0x7f353800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f355276b640 1 -- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3538003970 msgr2=0x7f3538003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3551769640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35380047c0 0x7f3538004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3551f6a640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3538003970 0x7f3538003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f355276b640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3538003970 0x7f3538003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f355276b640 1 -- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35380047c0 msgr2=0x7f3538004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f355276b640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35380047c0 0x7f3538004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f355276b640 1 -- 172.21.15.16:0/35044046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3538003520 con 0x7f3538005180 2023-11-13T10:57:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3551769640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35380047c0 0x7f3538004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.315+0000 7f3551f6a640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3538003970 0x7f3538003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.316+0000 7f355276b640 1 -- 172.21.15.16:0/35044046 reap_dead start 2023-11-13T10:57:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.316+0000 7f355276b640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538005180 0x7f353800b980 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3544009fc0 tx=0x7f35380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ceb9d226d8472ca server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354402cc30 con 0x7f3538005180 2023-11-13T10:57:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354400f040 con 0x7f3538005180 2023-11-13T10:57:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3550f68640 1 -- 172.21.15.16:0/35044046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544003990 con 0x7f3538005180 2023-11-13T10:57:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3552f6c640 1 -- 172.21.15.16:0/35044046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538005180 msgr2=0x7f353800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3552f6c640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538005180 0x7f353800b980 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3544009fc0 tx=0x7f35380057f0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3552f6c640 1 -- 172.21.15.16:0/35044046 shutdown_connections 2023-11-13T10:57:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3552f6c640 1 --2- 172.21.15.16:0/35044046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538005180 0x7f353800b980 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3552f6c640 1 -- 172.21.15.16:0/35044046 >> 172.21.15.16:0/35044046 conn(0x7f354c0b04a0 msgr2=0x7f354c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3552f6c640 1 -- 172.21.15.16:0/35044046 shutdown_connections 2023-11-13T10:57:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.317+0000 7f3552f6c640 1 -- 172.21.15.16:0/35044046 wait complete. 2023-11-13T10:57:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.319+0000 7f3552f6c640 1 Processor -- start 2023-11-13T10:57:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.319+0000 7f3552f6c640 1 -- start start 2023-11-13T10:57:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.320+0000 7f3552f6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538005180 0x7f354c1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.320+0000 7f3552f6c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354c156c90 0x7f354c149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.320+0000 7f3552f6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354c152ab0 0x7f354c0ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.320+0000 7f3552f6c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c1587e0 con 0x7f354c152ab0 2023-11-13T10:57:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.320+0000 7f3552f6c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c149f50 con 0x7f3538005180 2023-11-13T10:57:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.320+0000 7f3552f6c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c157200 con 0x7f354c156c90 2023-11-13T10:57:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f355276b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354c152ab0 0x7f354c0ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f355276b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354c152ab0 0x7f354c0ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52262/0 (socket says 172.21.15.16:52262) 2023-11-13T10:57:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f355276b640 1 -- 172.21.15.16:0/290373369 learned_addr learned my addr 172.21.15.16:0/290373369 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f3551f6a640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538005180 0x7f354c1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f3551769640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354c156c90 0x7f354c149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f355276b640 1 -- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354c156c90 msgr2=0x7f354c149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f355276b640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354c156c90 0x7f354c149a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f355276b640 1 -- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538005180 msgr2=0x7f354c1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f355276b640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538005180 0x7f354c1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.321+0000 7f355276b640 1 -- 172.21.15.16:0/290373369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c1574e0 con 0x7f354c152ab0 2023-11-13T10:57:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.322+0000 7f355276b640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354c152ab0 0x7f354c0ae1a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f354400e200 tx=0x7f3544009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.322+0000 7f35427fc640 1 -- 172.21.15.16:0/290373369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354402cc30 con 0x7f354c152ab0 2023-11-13T10:57:21.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.322+0000 7f35427fc640 1 -- 172.21.15.16:0/290373369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3544003990 con 0x7f354c152ab0 2023-11-13T10:57:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.322+0000 7f35427fc640 1 -- 172.21.15.16:0/290373369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544003d80 con 0x7f354c152ab0 2023-11-13T10:57:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.322+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c1577c0 con 0x7f354c152ab0 2023-11-13T10:57:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.322+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c157d80 con 0x7f354c152ab0 2023-11-13T10:57:21.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.323+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c157fa0 con 0x7f354c152ab0 2023-11-13T10:57:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.324+0000 7f35427fc640 1 -- 172.21.15.16:0/290373369 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3544004030 con 0x7f354c152ab0 2023-11-13T10:57:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.325+0000 7f35427fc640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f353006ae40 0x7f353006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.326+0000 7f35427fc640 1 -- 172.21.15.16:0/290373369 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f35440ac1b0 con 0x7f354c152ab0 2023-11-13T10:57:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.326+0000 7f3551f6a640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f353006ae40 0x7f353006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.327+0000 7f3551f6a640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f353006ae40 0x7f353006d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3548000f60 tx=0x7f3548009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.330+0000 7f35427fc640 1 -- 172.21.15.16:0/290373369 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35440771a0 con 0x7f354c152ab0 2023-11-13T10:57:21.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.527+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f354c04e4d0 con 0x7f354c152ab0 2023-11-13T10:57:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.530+0000 7f35427fc640 1 -- 172.21.15.16:0/290373369 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16799 (secure 0 0 0) 0x7f354407a7c0 con 0x7f354c152ab0 2023-11-13T10:57:21.534 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:21.535 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:57:20.522917+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:57:18.637969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-13T10:57:19.541678+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:57:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f353006ae40 msgr2=0x7f353006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f353006ae40 0x7f353006d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3548000f60 tx=0x7f3548009040 comp rx=0 tx=0).stop 2023-11-13T10:57:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354c152ab0 msgr2=0x7f354c0ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354c152ab0 0x7f354c0ae1a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f354400e200 tx=0x7f3544009750 comp rx=0 tx=0).stop 2023-11-13T10:57:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 shutdown_connections 2023-11-13T10:57:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f353006ae40 0x7f353006d300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354c152ab0 0x7f354c0ae1a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f354c156c90 0x7f354c149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 --2- 172.21.15.16:0/290373369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3538005180 0x7f354c1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 >> 172.21.15.16:0/290373369 conn(0x7f354c0b04a0 msgr2=0x7f354c163d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 shutdown_connections 2023-11-13T10:57:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.533+0000 7f3552f6c640 1 -- 172.21.15.16:0/290373369 wait complete. 2023-11-13T10:57:21.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:57:21.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.783+0000 7f16a7c1a640 1 Processor -- start 2023-11-13T10:57:21.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.783+0000 7f16a7c1a640 1 -- start start 2023-11-13T10:57:21.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.784+0000 7f16a7c1a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f16a015cf20 0x7f16a006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.784+0000 7f16a7c1a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16a006bb90 0x7f16a00ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.784+0000 7f16a7c1a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16a00b2d90 0x7f16a00ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.784+0000 7f16a7c1a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a015d300 con 0x7f16a006bb90 2023-11-13T10:57:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.784+0000 7f16a7c1a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a00b2900 con 0x7f16a015cf20 2023-11-13T10:57:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a7c1a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00b1fd0 con 0x7f16a00b2d90 2023-11-13T10:57:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a51b5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16a006bb90 0x7f16a00ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a51b5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16a006bb90 0x7f16a00ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52268/0 (socket says 172.21.15.16:52268) 2023-11-13T10:57:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a51b5640 1 -- 172.21.15.16:0/1505378886 learned_addr learned my addr 172.21.15.16:0/1505378886 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:21.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a61b7640 1 --2- 172.21.15.16:0/1505378886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16a00b2d90 0x7f16a00ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a59b6640 1 --2- 172.21.15.16:0/1505378886 >> v2:172.21.15.184:3300/0 conn(0x7f16a015cf20 0x7f16a006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a51b5640 1 -- 172.21.15.16:0/1505378886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16a00b2d90 msgr2=0x7f16a00ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a51b5640 1 --2- 172.21.15.16:0/1505378886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16a00b2d90 0x7f16a00ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a51b5640 1 -- 172.21.15.16:0/1505378886 >> v2:172.21.15.184:3300/0 conn(0x7f16a015cf20 msgr2=0x7f16a006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a51b5640 1 --2- 172.21.15.16:0/1505378886 >> v2:172.21.15.184:3300/0 conn(0x7f16a015cf20 0x7f16a006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a51b5640 1 -- 172.21.15.16:0/1505378886 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a00b22b0 con 0x7f16a006bb90 2023-11-13T10:57:21.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.785+0000 7f16a59b6640 1 --2- 172.21.15.16:0/1505378886 >> v2:172.21.15.184:3300/0 conn(0x7f16a015cf20 0x7f16a006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.787+0000 7f16a51b5640 1 --2- 172.21.15.16:0/1505378886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 0x7f16a00ce5b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f16a00cf9f0 tx=0x7f169002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817f8658b09e6f75 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:21.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.787+0000 7f16a49b4640 1 -- 172.21.15.16:0/1505378886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690031870 con 0x7f16a006bb90 2023-11-13T10:57:21.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.787+0000 7f16a49b4640 1 -- 172.21.15.16:0/1505378886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16900319d0 con 0x7f16a006bb90 2023-11-13T10:57:21.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.787+0000 7f16a49b4640 1 -- 172.21.15.16:0/1505378886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690031d60 con 0x7f16a006bb90 2023-11-13T10:57:21.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.788+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1505378886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 msgr2=0x7f16a00ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.788+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1505378886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 0x7f16a00ce5b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f16a00cf9f0 tx=0x7f169002b910 comp rx=0 tx=0).stop 2023-11-13T10:57:21.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.788+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1505378886 shutdown_connections 2023-11-13T10:57:21.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.788+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1505378886 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16a00b2d90 0x7f16a00ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.788+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1505378886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 0x7f16a00ce5b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.788+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1505378886 >> v2:172.21.15.184:3300/0 conn(0x7f16a015cf20 0x7f16a006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.788+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1505378886 >> 172.21.15.16:0/1505378886 conn(0x7f16a006a190 msgr2=0x7f16a006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.788+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1505378886 shutdown_connections 2023-11-13T10:57:21.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.788+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1505378886 wait complete. 2023-11-13T10:57:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.789+0000 7f16a7c1a640 1 Processor -- start 2023-11-13T10:57:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.790+0000 7f16a7c1a640 1 -- start start 2023-11-13T10:57:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a7c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 0x7f16a01452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a7c1a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16a00b2d90 0x7f16a0145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a7c1a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16a015cf20 0x7f16a015fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a7c1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01547f0 con 0x7f16a006bb90 2023-11-13T10:57:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a7c1a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0151fb0 con 0x7f16a015cf20 2023-11-13T10:57:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a7c1a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0152290 con 0x7f16a00b2d90 2023-11-13T10:57:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a59b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 0x7f16a01452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a59b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 0x7f16a01452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52276/0 (socket says 172.21.15.16:52276) 2023-11-13T10:57:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a59b6640 1 -- 172.21.15.16:0/1338866215 learned_addr learned my addr 172.21.15.16:0/1338866215 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a59b6640 1 -- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16a00b2d90 msgr2=0x7f16a0145dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a59b6640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16a00b2d90 0x7f16a0145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a59b6640 1 -- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16a015cf20 msgr2=0x7f16a015fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a59b6640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16a015cf20 0x7f16a015fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.791+0000 7f16a59b6640 1 -- 172.21.15.16:0/1338866215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16900094d0 con 0x7f16a006bb90 2023-11-13T10:57:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.792+0000 7f16a59b6640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 0x7f16a01452f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c7f0 tx=0x7f169c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.792+0000 7f16967fc640 1 -- 172.21.15.16:0/1338866215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c007820 con 0x7f16a006bb90 2023-11-13T10:57:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.792+0000 7f16967fc640 1 -- 172.21.15.16:0/1338866215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c007980 con 0x7f16a006bb90 2023-11-13T10:57:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.792+0000 7f16967fc640 1 -- 172.21.15.16:0/1338866215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c007cf0 con 0x7f16a006bb90 2023-11-13T10:57:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.792+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a0152570 con 0x7f16a006bb90 2023-11-13T10:57:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.792+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a0145830 con 0x7f16a006bb90 2023-11-13T10:57:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.794+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a0067090 con 0x7f16a006bb90 2023-11-13T10:57:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.794+0000 7f16967fc640 1 -- 172.21.15.16:0/1338866215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f169c0113f0 con 0x7f16a006bb90 2023-11-13T10:57:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.795+0000 7f16967fc640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f168406ae40 0x7f168406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.795+0000 7f16967fc640 1 -- 172.21.15.16:0/1338866215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f169c089bc0 con 0x7f16a006bb90 2023-11-13T10:57:21.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.796+0000 7f16a51b5640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f168406ae40 0x7f168406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.797+0000 7f16a51b5640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f168406ae40 0x7f168406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1690036000 tx=0x7f1690030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.800+0000 7f16967fc640 1 -- 172.21.15.16:0/1338866215 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f169c054a00 con 0x7f16a006bb90 2023-11-13T10:57:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:21.997+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16a00af490 con 0x7f16a006bb90 2023-11-13T10:57:22.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.000+0000 7f16967fc640 1 -- 172.21.15.16:0/1338866215 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16786 (secure 0 0 0) 0x7f169c01f020 con 0x7f16a006bb90 2023-11-13T10:57:22.004 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:22.004 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:57:21.822707+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:57:18.637969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-13T10:57:19.541678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:57:22.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f168406ae40 msgr2=0x7f168406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:22.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f168406ae40 0x7f168406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1690036000 tx=0x7f1690030000 comp rx=0 tx=0).stop 2023-11-13T10:57:22.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 msgr2=0x7f16a01452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:22.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 0x7f16a01452f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c7f0 tx=0x7f169c00ccb0 comp rx=0 tx=0).stop 2023-11-13T10:57:22.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 shutdown_connections 2023-11-13T10:57:22.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f168406ae40 0x7f168406d300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:22.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16a015cf20 0x7f16a015fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:22.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16a00b2d90 0x7f16a0145dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 --2- 172.21.15.16:0/1338866215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a006bb90 0x7f16a01452f0 secure :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c7f0 tx=0x7f169c00ccb0 comp rx=0 tx=0).stop 2023-11-13T10:57:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.004+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 >> 172.21.15.16:0/1338866215 conn(0x7f16a006a190 msgr2=0x7f16a015be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.005+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 shutdown_connections 2023-11-13T10:57:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.005+0000 7f16a7c1a640 1 -- 172.21.15.16:0/1338866215 wait complete. 2023-11-13T10:57:22.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-13T10:57:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.252+0000 7f78f68d4640 1 Processor -- start 2023-11-13T10:57:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.253+0000 7f78f68d4640 1 -- start start 2023-11-13T10:57:22.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.254+0000 7f78f68d4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78f00ce9b0 0x7f78f00b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:22.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.254+0000 7f78f68d4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f0144950 con 0x7f78f00ce550 2023-11-13T10:57:22.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.254+0000 7f78f68d4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f00ce300 con 0x7f78f0144570 2023-11-13T10:57:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.254+0000 7f78f68d4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f00cd260 con 0x7f78f00ce9b0 2023-11-13T10:57:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.255+0000 7f78f4e71640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78f00ce9b0 0x7f78f00b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.255+0000 7f78ef7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f78f00ce550 0x7f78f00b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60358/0 (socket says 172.21.15.16:60358) 2023-11-13T10:57:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.255+0000 7f78ef7fe640 1 -- 172.21.15.16:0/980371919 learned_addr learned my addr 172.21.15.16:0/980371919 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.255+0000 7f78f4e71640 1 -- 172.21.15.16:0/980371919 >> v1:172.21.15.184:6789/0 conn(0x7f78f0144570 legacy=0x7f78f015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.255+0000 7f78f4e71640 1 -- 172.21.15.16:0/980371919 >> v1:172.21.15.16:6789/0 conn(0x7f78f00ce550 legacy=0x7f78f00b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.255+0000 7f78f4e71640 1 -- 172.21.15.16:0/980371919 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f00cd540 con 0x7f78f00ce9b0 2023-11-13T10:57:22.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.255+0000 7f78eeffd640 1 -- 172.21.15.16:0/980371919 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42634777 0 0) 0x7f78f0144950 con 0x7f78f00ce550 2023-11-13T10:57:22.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.257+0000 7f78f4e71640 1 --2- 172.21.15.16:0/980371919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f00ce9b0 0x7f78f00b20e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f78e80097a0 tx=0x7f78e802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fd89400487cfd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:22.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.257+0000 7f78eeffd640 1 -- 172.21.15.16:0/980371919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8030610 con 0x7f78f00ce9b0 2023-11-13T10:57:22.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.257+0000 7f78eeffd640 1 -- 172.21.15.16:0/980371919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8030770 con 0x7f78f00ce9b0 2023-11-13T10:57:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.257+0000 7f78eeffd640 1 -- 172.21.15.16:0/980371919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8030b00 con 0x7f78f00ce9b0 2023-11-13T10:57:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.258+0000 7f78f68d4640 1 -- 172.21.15.16:0/980371919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f00ce9b0 msgr2=0x7f78f00b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.258+0000 7f78f68d4640 1 --2- 172.21.15.16:0/980371919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f00ce9b0 0x7f78f00b20e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f78e80097a0 tx=0x7f78e802b960 comp rx=0 tx=0).stop 2023-11-13T10:57:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.258+0000 7f78f68d4640 1 -- 172.21.15.16:0/980371919 shutdown_connections 2023-11-13T10:57:22.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.258+0000 7f78f68d4640 1 --2- 172.21.15.16:0/980371919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f00ce9b0 0x7f78f00b20e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.258+0000 7f78f68d4640 1 -- 172.21.15.16:0/980371919 >> 172.21.15.16:0/980371919 conn(0x7f78f006a190 msgr2=0x7f78f006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.258+0000 7f78f68d4640 1 -- 172.21.15.16:0/980371919 shutdown_connections 2023-11-13T10:57:22.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.258+0000 7f78f68d4640 1 -- 172.21.15.16:0/980371919 wait complete. 2023-11-13T10:57:22.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.260+0000 7f78f68d4640 1 Processor -- start 2023-11-13T10:57:22.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.260+0000 7f78f68d4640 1 -- start start 2023-11-13T10:57:22.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.261+0000 7f78f68d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f00ce550 0x7f78f01396c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:22.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.261+0000 7f78f68d4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f00ce9b0 0x7f78f0140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:22.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.261+0000 7f78f68d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f0144570 0x7f78f01562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:22.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.261+0000 7f78f68d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f013b060 con 0x7f78f0144570 2023-11-13T10:57:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.261+0000 7f78f68d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f0139dc0 con 0x7f78f00ce550 2023-11-13T10:57:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.261+0000 7f78f68d4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f013b2b0 con 0x7f78f00ce9b0 2023-11-13T10:57:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.261+0000 7f78f4e71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f0144570 0x7f78f01562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.261+0000 7f78f4e71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f0144570 0x7f78f01562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52298/0 (socket says 172.21.15.16:52298) 2023-11-13T10:57:22.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.261+0000 7f78f4e71640 1 -- 172.21.15.16:0/2037814687 learned_addr learned my addr 172.21.15.16:0/2037814687 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:22.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.262+0000 7f78effff640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f00ce550 0x7f78f01396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:22.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.262+0000 7f78ef7fe640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f00ce9b0 0x7f78f0140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:22.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.262+0000 7f78f4e71640 1 -- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f00ce9b0 msgr2=0x7f78f0140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:22.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.262+0000 7f78f4e71640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f00ce9b0 0x7f78f0140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:22.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.262+0000 7f78f4e71640 1 -- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f00ce550 msgr2=0x7f78f01396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:22.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.262+0000 7f78f4e71640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f00ce550 0x7f78f01396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:22.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.262+0000 7f78f4e71640 1 -- 172.21.15.16:0/2037814687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8009400 con 0x7f78f0144570 2023-11-13T10:57:22.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.262+0000 7f78f4e71640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f0144570 0x7f78f01562d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f78e80098c0 tx=0x7f78e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:22.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.263+0000 7f78ecff9640 1 -- 172.21.15.16:0/2037814687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e803e040 con 0x7f78f0144570 2023-11-13T10:57:22.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.263+0000 7f78ecff9640 1 -- 172.21.15.16:0/2037814687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8039c10 con 0x7f78f0144570 2023-11-13T10:57:22.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.263+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f013b590 con 0x7f78f0144570 2023-11-13T10:57:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.263+0000 7f78ecff9640 1 -- 172.21.15.16:0/2037814687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e80443f0 con 0x7f78f0144570 2023-11-13T10:57:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.263+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f0156810 con 0x7f78f0144570 2023-11-13T10:57:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.264+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f013bab0 con 0x7f78f0144570 2023-11-13T10:57:22.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.265+0000 7f78ecff9640 1 -- 172.21.15.16:0/2037814687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78e804d3b0 con 0x7f78f0144570 2023-11-13T10:57:22.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.266+0000 7f78ecff9640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f78d006ae40 0x7f78d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:22.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.266+0000 7f78ecff9640 1 -- 172.21.15.16:0/2037814687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f78e80abec0 con 0x7f78f0144570 2023-11-13T10:57:22.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.267+0000 7f78effff640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f78d006ae40 0x7f78d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:22.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.268+0000 7f78effff640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f78d006ae40 0x7f78d006d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f78e00012e0 tx=0x7f78e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:22.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.272+0000 7f78ecff9640 1 -- 172.21.15.16:0/2037814687 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78f013bab0 con 0x7f78f0144570 2023-11-13T10:57:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:22.505+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f78f013bab0 con 0x7f78f0144570 2023-11-13T10:57:23.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.098+0000 7f78ecff9640 1 -- 172.21.15.16:0/2037814687 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f78f013bab0 con 0x7f78f0144570 2023-11-13T10:57:23.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.101+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f78d006ae40 msgr2=0x7f78d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:23.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.101+0000 7f78f68d4640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f78d006ae40 0x7f78d006d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f78e00012e0 tx=0x7f78e0009040 comp rx=0 tx=0).stop 2023-11-13T10:57:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.101+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f0144570 msgr2=0x7f78f01562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.101+0000 7f78f68d4640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f0144570 0x7f78f01562d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f78e80098c0 tx=0x7f78e8004730 comp rx=0 tx=0).stop 2023-11-13T10:57:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.102+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 shutdown_connections 2023-11-13T10:57:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.102+0000 7f78f68d4640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f78d006ae40 0x7f78d006d300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.102+0000 7f78f68d4640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78f0144570 0x7f78f01562d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.102+0000 7f78f68d4640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78f00ce9b0 0x7f78f0140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.102+0000 7f78f68d4640 1 --2- 172.21.15.16:0/2037814687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78f00ce550 0x7f78f01396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.102+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 >> 172.21.15.16:0/2037814687 conn(0x7f78f006a190 msgr2=0x7f78f00adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.102+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 shutdown_connections 2023-11-13T10:57:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.102+0000 7f78f68d4640 1 -- 172.21.15.16:0/2037814687 wait complete. 2023-11-13T10:57:23.131 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-13T10:57:23.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-13T10:57:23.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.366+0000 7f002e606640 1 Processor -- start 2023-11-13T10:57:23.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.367+0000 7f002e606640 1 -- start start 2023-11-13T10:57:23.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.368+0000 7f002e606640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00280d1850 0x7f00280d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.368+0000 7f002e606640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00280d2d00 0x7f00280c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.368+0000 7f002e606640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00280ce0e0 con 0x7f002815ce70 2023-11-13T10:57:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.368+0000 7f002e606640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00280ce3c0 con 0x7f00280d2d00 2023-11-13T10:57:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.368+0000 7f002e606640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00280ce6a0 con 0x7f00280d1850 2023-11-13T10:57:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f002d604640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f002815ce70 0x7f00280cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57710/0 (socket says 172.21.15.16:57710) 2023-11-13T10:57:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f002d604640 1 -- 172.21.15.16:0/3668851392 learned_addr learned my addr 172.21.15.16:0/3668851392 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f002de05640 1 --2- 172.21.15.16:0/3668851392 >> v2:172.21.15.16:3300/0 conn(0x7f00280d2d00 0x7f00280c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f002ce03640 1 --2- 172.21.15.16:0/3668851392 >> v2:172.21.15.184:3300/0 conn(0x7f00280d1850 0x7f00280d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f0027fff640 1 -- 172.21.15.16:0/3668851392 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507826791 0 0) 0x7f00280ce0e0 con 0x7f002815ce70 2023-11-13T10:57:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f0027fff640 1 -- 172.21.15.16:0/3668851392 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0010003520 con 0x7f002815ce70 2023-11-13T10:57:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f002de05640 1 -- 172.21.15.16:0/3668851392 >> v2:172.21.15.184:3300/0 conn(0x7f00280d1850 msgr2=0x7f00280d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f002de05640 1 --2- 172.21.15.16:0/3668851392 >> v2:172.21.15.184:3300/0 conn(0x7f00280d1850 0x7f00280d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f002de05640 1 -- 172.21.15.16:0/3668851392 >> v1:172.21.15.16:6789/0 conn(0x7f002815ce70 legacy=0x7f00280cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.369+0000 7f002de05640 1 -- 172.21.15.16:0/3668851392 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00280ce980 con 0x7f00280d2d00 2023-11-13T10:57:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.371+0000 7f002de05640 1 --2- 172.21.15.16:0/3668851392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 0x7f00280c9c30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f00280d0680 tx=0x7f002002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999c901e8503ad6d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.371+0000 7f0027fff640 1 -- 172.21.15.16:0/3668851392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002002cc70 con 0x7f00280d2d00 2023-11-13T10:57:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.371+0000 7f0027fff640 1 -- 172.21.15.16:0/3668851392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002002cdd0 con 0x7f00280d2d00 2023-11-13T10:57:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.371+0000 7f0027fff640 1 -- 172.21.15.16:0/3668851392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020034660 con 0x7f00280d2d00 2023-11-13T10:57:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.371+0000 7f002e606640 1 -- 172.21.15.16:0/3668851392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 msgr2=0x7f00280c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.371+0000 7f002e606640 1 --2- 172.21.15.16:0/3668851392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 0x7f00280c9c30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f00280d0680 tx=0x7f002002bec0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.372+0000 7f002e606640 1 -- 172.21.15.16:0/3668851392 shutdown_connections 2023-11-13T10:57:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.372+0000 7f002e606640 1 --2- 172.21.15.16:0/3668851392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 0x7f00280c9c30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.372+0000 7f002e606640 1 --2- 172.21.15.16:0/3668851392 >> v2:172.21.15.184:3300/0 conn(0x7f00280d1850 0x7f00280d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.372+0000 7f002e606640 1 -- 172.21.15.16:0/3668851392 >> 172.21.15.16:0/3668851392 conn(0x7f002808c580 msgr2=0x7f002809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.372+0000 7f002e606640 1 -- 172.21.15.16:0/3668851392 shutdown_connections 2023-11-13T10:57:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.372+0000 7f002e606640 1 -- 172.21.15.16:0/3668851392 wait complete. 2023-11-13T10:57:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.373+0000 7f002e606640 1 Processor -- start 2023-11-13T10:57:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.374+0000 7f002e606640 1 -- start start 2023-11-13T10:57:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002e606640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00280d1850 0x7f00280a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002e606640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 0x7f002815b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002e606640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f002815ce70 0x7f002815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002e606640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002815c170 con 0x7f00280d2d00 2023-11-13T10:57:23.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002e606640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002809ec80 con 0x7f00280d1850 2023-11-13T10:57:23.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002e606640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002809eef0 con 0x7f002815ce70 2023-11-13T10:57:23.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002ce03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 0x7f002815b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:23.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002ce03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 0x7f002815b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36364/0 (socket says 172.21.15.16:36364) 2023-11-13T10:57:23.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002ce03640 1 -- 172.21.15.16:0/3348746578 learned_addr learned my addr 172.21.15.16:0/3348746578 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:23.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002d604640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00280d1850 0x7f00280a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:23.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002ce03640 1 -- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f002815ce70 msgr2=0x7f002815bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002ce03640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f002815ce70 0x7f002815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002ce03640 1 -- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00280d1850 msgr2=0x7f00280a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002ce03640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00280d1850 0x7f00280a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002ce03640 1 -- 172.21.15.16:0/3348746578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0020009a70 con 0x7f00280d2d00 2023-11-13T10:57:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.375+0000 7f002d604640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00280d1850 0x7f00280a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.376+0000 7f002ce03640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 0x7f002815b6f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f001c00d620 tx=0x7f001c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.376+0000 7f0025ffb640 1 -- 172.21.15.16:0/3348746578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00b620 con 0x7f00280d2d00 2023-11-13T10:57:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.376+0000 7f0025ffb640 1 -- 172.21.15.16:0/3348746578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c00b780 con 0x7f00280d2d00 2023-11-13T10:57:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.376+0000 7f0025ffb640 1 -- 172.21.15.16:0/3348746578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00bb30 con 0x7f00280d2d00 2023-11-13T10:57:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.376+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002809f1d0 con 0x7f00280d2d00 2023-11-13T10:57:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.376+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002809f9d0 con 0x7f00280d2d00 2023-11-13T10:57:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.377+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002804e4d0 con 0x7f00280d2d00 2023-11-13T10:57:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.378+0000 7f0025ffb640 1 -- 172.21.15.16:0/3348746578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f001c00bc90 con 0x7f00280d2d00 2023-11-13T10:57:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.379+0000 7f0025ffb640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f001406ae40 0x7f001406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.379+0000 7f0025ffb640 1 -- 172.21.15.16:0/3348746578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f001c089c30 con 0x7f00280d2d00 2023-11-13T10:57:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.380+0000 7f002d604640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f001406ae40 0x7f001406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.381+0000 7f002d604640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f001406ae40 0x7f001406d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00180011e0 tx=0x7f0018009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.383+0000 7f0025ffb640 1 -- 172.21.15.16:0/3348746578 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f001c054b90 con 0x7f00280d2d00 2023-11-13T10:57:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:23.615+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f00280d2360 con 0x7f00280d2d00 2023-11-13T10:57:23.868 INFO:tasks.ceph.mds.d.smithi184.stderr:2023-11-13T10:57:23.864+0000 7fc60078e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-13T10:57:24.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.109+0000 7f0025ffb640 1 -- 172.21.15.16:0/3348746578 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f001c0581b0 con 0x7f00280d2d00 2023-11-13T10:57:24.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f001406ae40 msgr2=0x7f001406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f001406ae40 0x7f001406d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00180011e0 tx=0x7f0018009040 comp rx=0 tx=0).stop 2023-11-13T10:57:24.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 msgr2=0x7f002815b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 0x7f002815b6f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f001c00d620 tx=0x7f001c00dae0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 shutdown_connections 2023-11-13T10:57:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f001406ae40 0x7f001406d300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f002815ce70 0x7f002815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00280d2d00 0x7f002815b6f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 --2- 172.21.15.16:0/3348746578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00280d1850 0x7f00280a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 >> 172.21.15.16:0/3348746578 conn(0x7f002808c580 msgr2=0x7f002806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 shutdown_connections 2023-11-13T10:57:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.113+0000 7f002e606640 1 -- 172.21.15.16:0/3348746578 wait complete. 2023-11-13T10:57:24.138 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-13T10:57:24.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:57:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.362+0000 7fcaab983640 1 Processor -- start 2023-11-13T10:57:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.363+0000 7fcaab983640 1 -- start start 2023-11-13T10:57:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.364+0000 7fcaab983640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcaa415d0b0 0x7fcaa40cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.364+0000 7fcaab983640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcaa40cddb0 0x7fcaa40ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.364+0000 7fcaab983640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa40cfda0 con 0x7fcaa40d0b80 2023-11-13T10:57:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.364+0000 7fcaab983640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40b29e0 con 0x7fcaa415d0b0 2023-11-13T10:57:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.364+0000 7fcaab983640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40b26b0 con 0x7fcaa40cddb0 2023-11-13T10:57:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.364+0000 7fcaa971f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcaa415d0b0 0x7fcaa40cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.365+0000 7fcaa971f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcaa415d0b0 0x7fcaa40cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36370/0 (socket says 172.21.15.16:36370) 2023-11-13T10:57:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.365+0000 7fcaa971f640 1 -- 172.21.15.16:0/3634111482 learned_addr learned my addr 172.21.15.16:0/3634111482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.365+0000 7fcaa8f1e640 1 --2- 172.21.15.16:0/3634111482 >> v2:172.21.15.184:3300/0 conn(0x7fcaa40cddb0 0x7fcaa40ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.365+0000 7fcaa971f640 1 -- 172.21.15.16:0/3634111482 >> v2:172.21.15.184:3300/0 conn(0x7fcaa40cddb0 msgr2=0x7fcaa40ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.365+0000 7fcaa971f640 1 --2- 172.21.15.16:0/3634111482 >> v2:172.21.15.184:3300/0 conn(0x7fcaa40cddb0 0x7fcaa40ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.365+0000 7fcaa971f640 1 -- 172.21.15.16:0/3634111482 >> v1:172.21.15.184:6789/0 conn(0x7fcaa40d0b80 legacy=0x7fcaa40d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.365+0000 7fcaa971f640 1 -- 172.21.15.16:0/3634111482 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40ce750 con 0x7fcaa415d0b0 2023-11-13T10:57:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.367+0000 7fcaa971f640 1 --2- 172.21.15.16:0/3634111482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa415d0b0 0x7fcaa40cd870 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcaa408d460 tx=0x7fca9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b29cd9d9e85bfd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.367+0000 7fcaa3fff640 1 -- 172.21.15.16:0/3634111482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94031870 con 0x7fcaa415d0b0 2023-11-13T10:57:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.367+0000 7fcaa3fff640 1 -- 172.21.15.16:0/3634111482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca940319d0 con 0x7fcaa415d0b0 2023-11-13T10:57:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.367+0000 7fcaa3fff640 1 -- 172.21.15.16:0/3634111482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94031d60 con 0x7fcaa415d0b0 2023-11-13T10:57:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.368+0000 7fcaab983640 1 -- 172.21.15.16:0/3634111482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa415d0b0 msgr2=0x7fcaa40cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.368+0000 7fcaab983640 1 --2- 172.21.15.16:0/3634111482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa415d0b0 0x7fcaa40cd870 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcaa408d460 tx=0x7fca9402b910 comp rx=0 tx=0).stop 2023-11-13T10:57:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.368+0000 7fcaab983640 1 -- 172.21.15.16:0/3634111482 shutdown_connections 2023-11-13T10:57:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.368+0000 7fcaab983640 1 --2- 172.21.15.16:0/3634111482 >> v2:172.21.15.184:3300/0 conn(0x7fcaa40cddb0 0x7fcaa40ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.368+0000 7fcaab983640 1 --2- 172.21.15.16:0/3634111482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa415d0b0 0x7fcaa40cd870 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.368+0000 7fcaab983640 1 -- 172.21.15.16:0/3634111482 >> 172.21.15.16:0/3634111482 conn(0x7fcaa40d2da0 msgr2=0x7fcaa40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.368+0000 7fcaab983640 1 -- 172.21.15.16:0/3634111482 shutdown_connections 2023-11-13T10:57:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.368+0000 7fcaab983640 1 -- 172.21.15.16:0/3634111482 wait complete. 2023-11-13T10:57:24.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.370+0000 7fcaab983640 1 Processor -- start 2023-11-13T10:57:24.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.370+0000 7fcaab983640 1 -- start start 2023-11-13T10:57:24.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.371+0000 7fcaab983640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa40cddb0 0x7fcaa415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.371+0000 7fcaab983640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa40d0b80 0x7fcaa415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.371+0000 7fcaab983640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa415d0b0 0x7fcaa4143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.371+0000 7fcaab983640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa415f850 con 0x7fcaa40d0b80 2023-11-13T10:57:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.371+0000 7fcaab983640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40c9e70 con 0x7fcaa415d0b0 2023-11-13T10:57:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.371+0000 7fcaab983640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40b8310 con 0x7fcaa40cddb0 2023-11-13T10:57:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa8f1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa40d0b80 0x7fcaa415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa8f1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa40d0b80 0x7fcaa415e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36376/0 (socket says 172.21.15.16:36376) 2023-11-13T10:57:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa8f1e640 1 -- 172.21.15.16:0/2349937477 learned_addr learned my addr 172.21.15.16:0/2349937477 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa971f640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa40cddb0 0x7fcaa415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa9f20640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa415d0b0 0x7fcaa4143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa8f1e640 1 -- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa40cddb0 msgr2=0x7fcaa415fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa8f1e640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa40cddb0 0x7fcaa415fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa8f1e640 1 -- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa415d0b0 msgr2=0x7fcaa4143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa8f1e640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa415d0b0 0x7fcaa4143e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa8f1e640 1 -- 172.21.15.16:0/2349937477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca940094d0 con 0x7fcaa40d0b80 2023-11-13T10:57:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.372+0000 7fcaa8f1e640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa40d0b80 0x7fcaa415e9f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d5f0 tx=0x7fca9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.373+0000 7fcaa1ffb640 1 -- 172.21.15.16:0/2349937477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca980125f0 con 0x7fcaa40d0b80 2023-11-13T10:57:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.373+0000 7fcaa1ffb640 1 -- 172.21.15.16:0/2349937477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca98012750 con 0x7fcaa40d0b80 2023-11-13T10:57:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.373+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa40ca080 con 0x7fcaa40d0b80 2023-11-13T10:57:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.373+0000 7fcaa1ffb640 1 -- 172.21.15.16:0/2349937477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98012b00 con 0x7fcaa40d0b80 2023-11-13T10:57:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.373+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa415f470 con 0x7fcaa40d0b80 2023-11-13T10:57:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.374+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa40d2510 con 0x7fcaa40d0b80 2023-11-13T10:57:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.375+0000 7fcaa1ffb640 1 -- 172.21.15.16:0/2349937477 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca98012c60 con 0x7fcaa40d0b80 2023-11-13T10:57:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.376+0000 7fcaa1ffb640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca9006ae40 0x7fca9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.376+0000 7fcaa1ffb640 1 -- 172.21.15.16:0/2349937477 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fca98089c00 con 0x7fcaa40d0b80 2023-11-13T10:57:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.377+0000 7fcaa971f640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca9006ae40 0x7fca9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.378+0000 7fcaa971f640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca9006ae40 0x7fca9006d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fca94036000 tx=0x7fca9403c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.380+0000 7fcaa1ffb640 1 -- 172.21.15.16:0/2349937477 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca98054b60 con 0x7fcaa40d0b80 2023-11-13T10:57:24.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.610+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaa4067090 con 0x7fcaa40d0b80 2023-11-13T10:57:24.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.611+0000 7fcaa1ffb640 1 -- 172.21.15.16:0/2349937477 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fca98058180 con 0x7fcaa40d0b80 2023-11-13T10:57:24.615 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:24.615 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3678615052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3678615052},{"type":"v1","addr":"172.21.15.16:6833","nonce":3678615052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3358463124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3358463124},{"type":"v1","addr":"172.21.15.16:6835","nonce":3358463124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2329286743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2329286743},{"type":"v1","addr":"172.21.15.184:6836","nonce":2329286743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:57:20.523743+0000","modified":"2023-11-13T10:57:23.616898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6837/3171288994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3171288994},{"type":"v1","addr":"172.21.15.184:6837","nonce":3171288994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-13T10:57:24.616 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2023-11-13T10:57:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca9006ae40 msgr2=0x7fca9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca9006ae40 0x7fca9006d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fca94036000 tx=0x7fca9403c040 comp rx=0 tx=0).stop 2023-11-13T10:57:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa40d0b80 msgr2=0x7fcaa415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa40d0b80 0x7fcaa415e9f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d5f0 tx=0x7fca9800dab0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 shutdown_connections 2023-11-13T10:57:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca9006ae40 0x7fca9006d300 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaa415d0b0 0x7fcaa4143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaa40d0b80 0x7fcaa415e9f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 --2- 172.21.15.16:0/2349937477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaa40cddb0 0x7fcaa415fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 >> 172.21.15.16:0/2349937477 conn(0x7fcaa40d2da0 msgr2=0x7fcaa40b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 shutdown_connections 2023-11-13T10:57:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.614+0000 7fcaab983640 1 -- 172.21.15.16:0/2349937477 wait complete. 2023-11-13T10:57:24.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T10:57:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.859+0000 7f2baf59e640 1 Processor -- start 2023-11-13T10:57:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.860+0000 7f2baf59e640 1 -- start start 2023-11-13T10:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.861+0000 7f2baf59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bb015aa90 0x7f2bb006a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.861+0000 7f2baf59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb015d630 con 0x7f2bb006a6d0 2023-11-13T10:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.861+0000 7f2baf59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb015d910 con 0x7f2bb0070a70 2023-11-13T10:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.861+0000 7f2baf59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb006ab50 con 0x7f2bb015aa90 2023-11-13T10:57:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.862+0000 7f2bae59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bb015aa90 0x7f2bb006a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.862+0000 7f2bae59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bb015aa90 0x7f2bb006a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36384/0 (socket says 172.21.15.16:36384) 2023-11-13T10:57:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.862+0000 7f2bae59c640 1 -- 172.21.15.16:0/2224054325 learned_addr learned my addr 172.21.15.16:0/2224054325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.862+0000 7f2ba5d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2bb006a6d0 0x7f2bb0070360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57724/0 (socket says 172.21.15.16:57724) 2023-11-13T10:57:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.862+0000 7f2bae59c640 1 -- 172.21.15.16:0/2224054325 >> v1:172.21.15.184:6789/0 conn(0x7f2bb0070a70 legacy=0x7f2bb006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.862+0000 7f2bae59c640 1 -- 172.21.15.16:0/2224054325 >> v1:172.21.15.16:6789/0 conn(0x7f2bb006a6d0 legacy=0x7f2bb0070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.862+0000 7f2bae59c640 1 -- 172.21.15.16:0/2224054325 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb015dbf0 con 0x7f2bb015aa90 2023-11-13T10:57:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.862+0000 7f2badd9b640 1 -- 172.21.15.16:0/2224054325 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946121929 0 0) 0x7f2bb015d630 con 0x7f2bb006a6d0 2023-11-13T10:57:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.863+0000 7f2bae59c640 1 --2- 172.21.15.16:0/2224054325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb015aa90 0x7f2bb006a190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000bb10 tx=0x7f2ba002e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa15c4504dfce2a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.864+0000 7f2badd9b640 1 -- 172.21.15.16:0/2224054325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba002ee80 con 0x7f2bb015aa90 2023-11-13T10:57:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.864+0000 7f2badd9b640 1 -- 172.21.15.16:0/2224054325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0036440 con 0x7f2bb015aa90 2023-11-13T10:57:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.864+0000 7f2badd9b640 1 -- 172.21.15.16:0/2224054325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00367b0 con 0x7f2bb015aa90 2023-11-13T10:57:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.864+0000 7f2baf59e640 1 -- 172.21.15.16:0/2224054325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb015aa90 msgr2=0x7f2bb006a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.864+0000 7f2baf59e640 1 --2- 172.21.15.16:0/2224054325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb015aa90 0x7f2bb006a190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000bb10 tx=0x7f2ba002e0d0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.865+0000 7f2baf59e640 1 -- 172.21.15.16:0/2224054325 shutdown_connections 2023-11-13T10:57:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.865+0000 7f2baf59e640 1 --2- 172.21.15.16:0/2224054325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb015aa90 0x7f2bb006a190 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.865+0000 7f2baf59e640 1 -- 172.21.15.16:0/2224054325 >> 172.21.15.16:0/2224054325 conn(0x7f2bb00b04a0 msgr2=0x7f2bb00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.865+0000 7f2baf59e640 1 -- 172.21.15.16:0/2224054325 shutdown_connections 2023-11-13T10:57:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.865+0000 7f2baf59e640 1 -- 172.21.15.16:0/2224054325 wait complete. 2023-11-13T10:57:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.866+0000 7f2baf59e640 1 Processor -- start 2023-11-13T10:57:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.867+0000 7f2baf59e640 1 -- start start 2023-11-13T10:57:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2baf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb006a6d0 0x7f2bb014d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2baf59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb0070a70 0x7f2bb0157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2baf59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb015aa90 0x7f2bb00c3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2baf59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb014d960 con 0x7f2bb006a6d0 2023-11-13T10:57:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2baf59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb0158640 con 0x7f2bb0070a70 2023-11-13T10:57:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2baf59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb0158920 con 0x7f2bb015aa90 2023-11-13T10:57:24.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2bae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb006a6d0 0x7f2bb014d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2bae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb006a6d0 0x7f2bb014d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36388/0 (socket says 172.21.15.16:36388) 2023-11-13T10:57:24.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2bae59c640 1 -- 172.21.15.16:0/616865141 learned_addr learned my addr 172.21.15.16:0/616865141 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:24.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2ba5d9b640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb0070a70 0x7f2bb0157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.868+0000 7f2baed9d640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb015aa90 0x7f2bb00c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2bae59c640 1 -- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb015aa90 msgr2=0x7f2bb00c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2bae59c640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb015aa90 0x7f2bb00c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2bae59c640 1 -- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb0070a70 msgr2=0x7f2bb0157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2bae59c640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb0070a70 0x7f2bb0157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2bae59c640 1 -- 172.21.15.16:0/616865141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba000b770 con 0x7f2bb006a6d0 2023-11-13T10:57:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2baed9d640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb015aa90 0x7f2bb00c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2bae59c640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb006a6d0 0x7f2bb014d420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000b2c0 tx=0x7f2ba0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2ba77fe640 1 -- 172.21.15.16:0/616865141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0034950 con 0x7f2bb006a6d0 2023-11-13T10:57:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2ba77fe640 1 -- 172.21.15.16:0/616865141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0034ad0 con 0x7f2bb006a6d0 2023-11-13T10:57:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2ba77fe640 1 -- 172.21.15.16:0/616865141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0034e80 con 0x7f2bb006a6d0 2023-11-13T10:57:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb0158c00 con 0x7f2bb006a6d0 2023-11-13T10:57:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.869+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb01591c0 con 0x7f2bb006a6d0 2023-11-13T10:57:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.871+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb004e4d0 con 0x7f2bb006a6d0 2023-11-13T10:57:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.872+0000 7f2ba77fe640 1 -- 172.21.15.16:0/616865141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ba0041480 con 0x7f2bb006a6d0 2023-11-13T10:57:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.873+0000 7f2ba77fe640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b9006ae40 0x7f2b9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.873+0000 7f2ba77fe640 1 -- 172.21.15.16:0/616865141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2ba00aca10 con 0x7f2bb006a6d0 2023-11-13T10:57:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.873+0000 7f2ba5d9b640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b9006ae40 0x7f2b9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.874+0000 7f2ba5d9b640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b9006ae40 0x7f2b9006d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2bb01f6090 tx=0x7f2b98007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:24.878+0000 7f2ba77fe640 1 -- 172.21.15.16:0/616865141 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ba00778e0 con 0x7f2bb006a6d0 2023-11-13T10:57:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.076+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bb004e4d0 con 0x7f2bb006a6d0 2023-11-13T10:57:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.078+0000 7f2ba77fe640 1 -- 172.21.15.16:0/616865141 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16786 (secure 0 0 0) 0x7f2ba003b090 con 0x7f2bb006a6d0 2023-11-13T10:57:25.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:25.083 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T10:57:21.822707+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:57:18.637969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-13T10:57:19.541678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T10:57:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b9006ae40 msgr2=0x7f2b9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b9006ae40 0x7f2b9006d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2bb01f6090 tx=0x7f2b98007c00 comp rx=0 tx=0).stop 2023-11-13T10:57:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb006a6d0 msgr2=0x7f2bb014d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb006a6d0 0x7f2bb014d420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000b2c0 tx=0x7f2ba0004730 comp rx=0 tx=0).stop 2023-11-13T10:57:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 shutdown_connections 2023-11-13T10:57:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b9006ae40 0x7f2b9006d300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb015aa90 0x7f2bb00c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb0070a70 0x7f2bb0157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 --2- 172.21.15.16:0/616865141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb006a6d0 0x7f2bb014d420 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.081+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 >> 172.21.15.16:0/616865141 conn(0x7f2bb00b04a0 msgr2=0x7f2bb015bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.082+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 shutdown_connections 2023-11-13T10:57:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.082+0000 7f2baf59e640 1 -- 172.21.15.16:0/616865141 wait complete. 2023-11-13T10:57:25.105 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-13T10:57:25.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T10:57:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.327+0000 7f5a9359e640 1 Processor -- start 2023-11-13T10:57:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.327+0000 7f5a9359e640 1 -- start start 2023-11-13T10:57:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.329+0000 7f5a9359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a94070a70 0x7f5a94164920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.329+0000 7f5a9359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9406d680 con 0x7f5a9406a8a0 2023-11-13T10:57:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.329+0000 7f5a9359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9415d630 con 0x7f5a9415aa90 2023-11-13T10:57:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.329+0000 7f5a9359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9415d910 con 0x7f5a94070a70 2023-11-13T10:57:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.329+0000 7f5a8bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a9406a8a0 0x7f5a94070360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57740/0 (socket says 172.21.15.16:57740) 2023-11-13T10:57:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.329+0000 7f5a8bfff640 1 -- 172.21.15.16:0/4137573453 learned_addr learned my addr 172.21.15.16:0/4137573453 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a92d9d640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a94070a70 0x7f5a94164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047666192 0 0) 0x7f5a9406d680 con 0x7f5a9406a8a0 2023-11-13T10:57:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a80005730 con 0x7f5a9406a8a0 2023-11-13T10:57:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2918962264 0 0) 0x7f5a80005730 con 0x7f5a9406a8a0 2023-11-13T10:57:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a94070a70 msgr2=0x7f5a94164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a94070a70 0x7f5a94164920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 >> v1:172.21.15.184:6789/0 conn(0x7f5a9415aa90 legacy=0x7f5a9406a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9415dbf0 con 0x7f5a9406a8a0 2023-11-13T10:57:25.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f5a7c001670 con 0x7f5a9406a8a0 2023-11-13T10:57:25.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 >> v1:172.21.15.16:6789/0 conn(0x7f5a9406a8a0 legacy=0x7f5a94070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a80005bb0 0x7f5a80005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a800069d0 0x7f5a80006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a800074b0 0x7f5a8000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9415d910 con 0x7f5a80005bb0 2023-11-13T10:57:25.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8000e2e0 con 0x7f5a800069d0 2023-11-13T10:57:25.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8000e4a0 con 0x7f5a800074b0 2023-11-13T10:57:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.330+0000 7f5a92d9d640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a94070a70 0x7f5a94164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T10:57:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.331+0000 7f5a9259c640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a80005bb0 0x7f5a80005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.331+0000 7f5a8bfff640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a800069d0 0x7f5a80006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.331+0000 7f5a92d9d640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a800074b0 0x7f5a8000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.331+0000 7f5a9259c640 1 -- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a800074b0 msgr2=0x7f5a8000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.331+0000 7f5a9259c640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a800074b0 0x7f5a8000db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.331+0000 7f5a9259c640 1 -- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a800069d0 msgr2=0x7f5a80006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.331+0000 7f5a9259c640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a800069d0 0x7f5a80006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.331+0000 7f5a9259c640 1 -- 172.21.15.16:0/4137573453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a80005730 con 0x7f5a80005bb0 2023-11-13T10:57:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.332+0000 7f5a9259c640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a80005bb0 0x7f5a80005fd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5a84001750 tx=0x7f5a80007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c6f14298fa307e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.333+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a840052d0 con 0x7f5a80005bb0 2023-11-13T10:57:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.333+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8402e050 con 0x7f5a80005bb0 2023-11-13T10:57:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.333+0000 7f5a91d9b640 1 -- 172.21.15.16:0/4137573453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a840035f0 con 0x7f5a80005bb0 2023-11-13T10:57:25.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.333+0000 7f5a9359e640 1 -- 172.21.15.16:0/4137573453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a80005bb0 msgr2=0x7f5a80005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.333+0000 7f5a9359e640 1 --2- 172.21.15.16:0/4137573453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a80005bb0 0x7f5a80005fd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5a84001750 tx=0x7f5a80007a00 comp rx=0 tx=0).stop 2023-11-13T10:57:25.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.333+0000 7f5a92d9d640 1 -- 172.21.15.16:0/4137573453 reap_dead start 2023-11-13T10:57:25.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.333+0000 7f5a9359e640 1 -- 172.21.15.16:0/4137573453 shutdown_connections 2023-11-13T10:57:25.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.333+0000 7f5a9359e640 1 -- 172.21.15.16:0/4137573453 >> 172.21.15.16:0/4137573453 conn(0x7f5a940b04a0 msgr2=0x7f5a940b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:25.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.333+0000 7f5a9359e640 1 -- 172.21.15.16:0/4137573453 shutdown_connections 2023-11-13T10:57:25.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.334+0000 7f5a9359e640 1 -- 172.21.15.16:0/4137573453 wait complete. 2023-11-13T10:57:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.335+0000 7f5a9359e640 1 Processor -- start 2023-11-13T10:57:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.336+0000 7f5a9359e640 1 -- start start 2023-11-13T10:57:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.336+0000 7f5a9359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a80005bb0 0x7f5a941f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a9359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940a84d0 0x7f5a940a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a9359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a940cc3c0 0x7f5a940cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a9359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94147b10 con 0x7f5a940a84d0 2023-11-13T10:57:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a9359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9414b000 con 0x7f5a80005bb0 2023-11-13T10:57:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a9359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94148340 con 0x7f5a940cc3c0 2023-11-13T10:57:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940a84d0 0x7f5a940a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940a84d0 0x7f5a940a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36410/0 (socket says 172.21.15.16:36410) 2023-11-13T10:57:25.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a8bfff640 1 -- 172.21.15.16:0/3753850016 learned_addr learned my addr 172.21.15.16:0/3753850016 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:25.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a8bfff640 1 -- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a940cc3c0 msgr2=0x7f5a940cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a8bfff640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a940cc3c0 0x7f5a940cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a8bfff640 1 -- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a80005bb0 msgr2=0x7f5a941f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a8bfff640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a80005bb0 0x7f5a941f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.337+0000 7f5a8bfff640 1 -- 172.21.15.16:0/3753850016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940a8e90 con 0x7f5a940a84d0 2023-11-13T10:57:25.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.338+0000 7f5a8bfff640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940a84d0 0x7f5a940a8950 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00e780 tx=0x7f5a7c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.338+0000 7f5a8affd640 1 -- 172.21.15.16:0/3753850016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00ee90 con 0x7f5a940a84d0 2023-11-13T10:57:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.338+0000 7f5a8affd640 1 -- 172.21.15.16:0/3753850016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7c00c7f0 con 0x7f5a940a84d0 2023-11-13T10:57:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.338+0000 7f5a8affd640 1 -- 172.21.15.16:0/3753850016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00cba0 con 0x7f5a940a84d0 2023-11-13T10:57:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.338+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a940cd160 con 0x7f5a940a84d0 2023-11-13T10:57:25.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.338+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a94147fd0 con 0x7f5a940a84d0 2023-11-13T10:57:25.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.339+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a94156430 con 0x7f5a940a84d0 2023-11-13T10:57:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.340+0000 7f5a8affd640 1 -- 172.21.15.16:0/3753850016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a7c00cd00 con 0x7f5a940a84d0 2023-11-13T10:57:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.342+0000 7f5a8affd640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a7406ae40 0x7f5a7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.342+0000 7f5a8affd640 1 -- 172.21.15.16:0/3753850016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5a7c089740 con 0x7f5a940a84d0 2023-11-13T10:57:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.343+0000 7f5a9259c640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a7406ae40 0x7f5a7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.344+0000 7f5a9259c640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a7406ae40 0x7f5a7406d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5a84033000 tx=0x7f5a8402f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.347+0000 7f5a8affd640 1 -- 172.21.15.16:0/3753850016 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a7c054610 con 0x7f5a940a84d0 2023-11-13T10:57:25.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.539+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a9415e3f0 con 0x7f5a7406ae40 2023-11-13T10:57:25.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.565+0000 7f5a8affd640 1 -- 172.21.15.16:0/3753850016 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276107 (secure 0 0 0) 0x7f5a840052a0 con 0x7f5a7406ae40 2023-11-13T10:57:25.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T10:57:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a7406ae40 msgr2=0x7f5a7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a7406ae40 0x7f5a7406d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5a84033000 tx=0x7f5a8402f000 comp rx=0 tx=0).stop 2023-11-13T10:57:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940a84d0 msgr2=0x7f5a940a8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940a84d0 0x7f5a940a8950 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00e780 tx=0x7f5a7c00ec40 comp rx=0 tx=0).stop 2023-11-13T10:57:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 shutdown_connections 2023-11-13T10:57:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5a7406ae40 0x7f5a7406d300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a940cc3c0 0x7f5a940cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940a84d0 0x7f5a940a8950 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 --2- 172.21.15.16:0/3753850016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a80005bb0 0x7f5a941f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 >> 172.21.15.16:0/3753850016 conn(0x7f5a940b04a0 msgr2=0x7f5a940b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 shutdown_connections 2023-11-13T10:57:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.571+0000 7f5a9359e640 1 -- 172.21.15.16:0/3753850016 wait complete. 2023-11-13T10:57:25.597 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2023-11-13T10:57:24.017208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237240,"kb_used_data":4380,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737480,"statfs":{"total":773094113280,"available":772851179520,"internally_reserved":0,"allocated":4485120,"data_stored":2542397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.979558"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007390+0000","last_change":"2023-11-13T10:57:20.865445+0000","last_active":"2023-11-13T10:57:22.007390+0000","last_peered":"2023-11-13T10:57:22.007390+0000","last_clean":"2023-11-13T10:57:22.007390+0000","last_became_active":"2023-11-13T10:57:20.864853+0000","last_became_peered":"2023-11-13T10:57:20.864853+0000","last_unstale":"2023-11-13T10:57:22.007390+0000","last_undegraded":"2023-11-13T10:57:22.007390+0000","last_fullsized":"2023-11-13T10:57:22.007390+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:29:15.574855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008820+0000","last_change":"2023-11-13T10:57:20.858960+0000","last_active":"2023-11-13T10:57:22.008820+0000","last_peered":"2023-11-13T10:57:22.008820+0000","last_clean":"2023-11-13T10:57:22.008820+0000","last_became_active":"2023-11-13T10:57:20.858697+0000","last_became_peered":"2023-11-13T10:57:20.858697+0000","last_unstale":"2023-11-13T10:57:22.008820+0000","last_undegraded":"2023-11-13T10:57:22.008820+0000","last_fullsized":"2023-11-13T10:57:22.008820+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:22.317429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008042+0000","last_change":"2023-11-13T10:57:20.852731+0000","last_active":"2023-11-13T10:57:22.008042+0000","last_peered":"2023-11-13T10:57:22.008042+0000","last_clean":"2023-11-13T10:57:22.008042+0000","last_became_active":"2023-11-13T10:57:20.851852+0000","last_became_peered":"2023-11-13T10:57:20.851852+0000","last_unstale":"2023-11-13T10:57:22.008042+0000","last_undegraded":"2023-11-13T10:57:22.008042+0000","last_fullsized":"2023-11-13T10:57:22.008042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:10:33.233178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.041528+0000","last_change":"2023-11-13T10:57:19.041528+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.041528+0000","last_undegraded":"2023-11-13T10:57:19.041528+0000","last_fullsized":"2023-11-13T10:57:19.041528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.041017+0000","last_change":"2023-11-13T10:57:19.041017+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.041017+0000","last_undegraded":"2023-11-13T10:57:19.041017+0000","last_fullsized":"2023-11-13T10:57:19.041017+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.053332+0000","last_change":"2023-11-13T10:57:19.053332+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.053332+0000","last_undegraded":"2023-11-13T10:57:19.053332+0000","last_fullsized":"2023-11-13T10:57:19.053332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.041115+0000","last_change":"2023-11-13T10:57:19.041115+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.041115+0000","last_undegraded":"2023-11-13T10:57:19.041115+0000","last_fullsized":"2023-11-13T10:57:19.041115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.041022+0000","last_change":"2023-11-13T10:57:19.041022+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.041022+0000","last_undegraded":"2023-11-13T10:57:19.041022+0000","last_fullsized":"2023-11-13T10:57:19.041022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.049898+0000","last_change":"2023-11-13T10:57:19.049898+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.049898+0000","last_undegraded":"2023-11-13T10:57:19.049898+0000","last_fullsized":"2023-11-13T10:57:19.049898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.051689+0000","last_change":"2023-11-13T10:57:19.051689+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.051689+0000","last_undegraded":"2023-11-13T10:57:19.051689+0000","last_fullsized":"2023-11-13T10:57:19.051689+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.050125+0000","last_change":"2023-11-13T10:57:19.050125+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.050125+0000","last_undegraded":"2023-11-13T10:57:19.050125+0000","last_fullsized":"2023-11-13T10:57:19.050125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.054138+0000","last_change":"2023-11-13T10:57:19.054138+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.054138+0000","last_undegraded":"2023-11-13T10:57:19.054138+0000","last_fullsized":"2023-11-13T10:57:19.054138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.046487+0000","last_change":"2023-11-13T10:57:19.046487+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.046487+0000","last_undegraded":"2023-11-13T10:57:19.046487+0000","last_fullsized":"2023-11-13T10:57:19.046487+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.045708+0000","last_change":"2023-11-13T10:57:19.045708+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.045708+0000","last_undegraded":"2023-11-13T10:57:19.045708+0000","last_fullsized":"2023-11-13T10:57:19.045708+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.051899+0000","last_change":"2023-11-13T10:57:19.051899+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.051899+0000","last_undegraded":"2023-11-13T10:57:19.051899+0000","last_fullsized":"2023-11-13T10:57:19.051899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.053311+0000","last_change":"2023-11-13T10:57:19.053311+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.053311+0000","last_undegraded":"2023-11-13T10:57:19.053311+0000","last_fullsized":"2023-11-13T10:57:19.053311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.040973+0000","last_change":"2023-11-13T10:57:19.040973+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.040973+0000","last_undegraded":"2023-11-13T10:57:19.040973+0000","last_fullsized":"2023-11-13T10:57:19.040973+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.046510+0000","last_change":"2023-11-13T10:57:19.046510+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.046510+0000","last_undegraded":"2023-11-13T10:57:19.046510+0000","last_fullsized":"2023-11-13T10:57:19.046510+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007775+0000","last_change":"2023-11-13T10:57:20.858413+0000","last_active":"2023-11-13T10:57:22.007775+0000","last_peered":"2023-11-13T10:57:22.007775+0000","last_clean":"2023-11-13T10:57:22.007775+0000","last_became_active":"2023-11-13T10:57:20.856053+0000","last_became_peered":"2023-11-13T10:57:20.856053+0000","last_unstale":"2023-11-13T10:57:22.007775+0000","last_undegraded":"2023-11-13T10:57:22.007775+0000","last_fullsized":"2023-11-13T10:57:22.007775+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:01:48.369767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.041130+0000","last_change":"2023-11-13T10:57:19.041130+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.041130+0000","last_undegraded":"2023-11-13T10:57:19.041130+0000","last_fullsized":"2023-11-13T10:57:19.041130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.050881+0000","last_change":"2023-11-13T10:57:19.050881+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.050881+0000","last_undegraded":"2023-11-13T10:57:19.050881+0000","last_fullsized":"2023-11-13T10:57:19.050881+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.045823+0000","last_change":"2023-11-13T10:57:19.045823+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.045823+0000","last_undegraded":"2023-11-13T10:57:19.045823+0000","last_fullsized":"2023-11-13T10:57:19.045823+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007495+0000","last_change":"2023-11-13T10:57:20.082004+0000","last_active":"2023-11-13T10:57:22.007495+0000","last_peered":"2023-11-13T10:57:22.007495+0000","last_clean":"2023-11-13T10:57:22.007495+0000","last_became_active":"2023-11-13T10:57:20.081685+0000","last_became_peered":"2023-11-13T10:57:20.081685+0000","last_unstale":"2023-11-13T10:57:22.007495+0000","last_undegraded":"2023-11-13T10:57:22.007495+0000","last_fullsized":"2023-11-13T10:57:22.007495+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:20:38.399944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.045294+0000","last_change":"2023-11-13T10:57:19.045294+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.045294+0000","last_undegraded":"2023-11-13T10:57:19.045294+0000","last_fullsized":"2023-11-13T10:57:19.045294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.047237+0000","last_change":"2023-11-13T10:57:19.047237+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.047237+0000","last_undegraded":"2023-11-13T10:57:19.047237+0000","last_fullsized":"2023-11-13T10:57:19.047237+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.042153+0000","last_change":"2023-11-13T10:57:19.042153+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.042153+0000","last_undegraded":"2023-11-13T10:57:19.042153+0000","last_fullsized":"2023-11-13T10:57:19.042153+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.050389+0000","last_change":"2023-11-13T10:57:19.050389+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.050389+0000","last_undegraded":"2023-11-13T10:57:19.050389+0000","last_fullsized":"2023-11-13T10:57:19.050389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.040764+0000","last_change":"2023-11-13T10:57:19.040764+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.040764+0000","last_undegraded":"2023-11-13T10:57:19.040764+0000","last_fullsized":"2023-11-13T10:57:19.040764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008404+0000","last_change":"2023-11-13T10:57:20.868321+0000","last_active":"2023-11-13T10:57:22.008404+0000","last_peered":"2023-11-13T10:57:22.008404+0000","last_clean":"2023-11-13T10:57:22.008404+0000","last_became_active":"2023-11-13T10:57:20.867963+0000","last_became_peered":"2023-11-13T10:57:20.867963+0000","last_unstale":"2023-11-13T10:57:22.008404+0000","last_undegraded":"2023-11-13T10:57:22.008404+0000","last_fullsized":"2023-11-13T10:57:22.008404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:28:06.363762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.049654+0000","last_change":"2023-11-13T10:57:19.049654+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.049654+0000","last_undegraded":"2023-11-13T10:57:19.049654+0000","last_fullsized":"2023-11-13T10:57:19.049654+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.103253+0000","last_change":"2023-11-13T10:57:20.077545+0000","last_active":"2023-11-13T10:57:22.103253+0000","last_peered":"2023-11-13T10:57:22.103253+0000","last_clean":"2023-11-13T10:57:22.103253+0000","last_became_active":"2023-11-13T10:57:20.077264+0000","last_became_peered":"2023-11-13T10:57:20.077264+0000","last_unstale":"2023-11-13T10:57:22.103253+0000","last_undegraded":"2023-11-13T10:57:22.103253+0000","last_fullsized":"2023-11-13T10:57:22.103253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:09.602632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007754+0000","last_change":"2023-11-13T10:57:20.080966+0000","last_active":"2023-11-13T10:57:22.007754+0000","last_peered":"2023-11-13T10:57:22.007754+0000","last_clean":"2023-11-13T10:57:22.007754+0000","last_became_active":"2023-11-13T10:57:20.079977+0000","last_became_peered":"2023-11-13T10:57:20.079977+0000","last_unstale":"2023-11-13T10:57:22.007754+0000","last_undegraded":"2023-11-13T10:57:22.007754+0000","last_fullsized":"2023-11-13T10:57:22.007754+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:43:57.351520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007182+0000","last_change":"2023-11-13T10:57:20.862228+0000","last_active":"2023-11-13T10:57:22.007182+0000","last_peered":"2023-11-13T10:57:22.007182+0000","last_clean":"2023-11-13T10:57:22.007182+0000","last_became_active":"2023-11-13T10:57:20.861934+0000","last_became_peered":"2023-11-13T10:57:20.861934+0000","last_unstale":"2023-11-13T10:57:22.007182+0000","last_undegraded":"2023-11-13T10:57:22.007182+0000","last_fullsized":"2023-11-13T10:57:22.007182+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:47:22.240659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.052109+0000","last_change":"2023-11-13T10:57:19.052109+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.052109+0000","last_undegraded":"2023-11-13T10:57:19.052109+0000","last_fullsized":"2023-11-13T10:57:19.052109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.055785+0000","last_change":"2023-11-13T10:57:19.055785+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.055785+0000","last_undegraded":"2023-11-13T10:57:19.055785+0000","last_fullsized":"2023-11-13T10:57:19.055785+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.045855+0000","last_change":"2023-11-13T10:57:19.045855+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.045855+0000","last_undegraded":"2023-11-13T10:57:19.045855+0000","last_fullsized":"2023-11-13T10:57:19.045855+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.051592+0000","last_change":"2023-11-13T10:57:19.051592+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.051592+0000","last_undegraded":"2023-11-13T10:57:19.051592+0000","last_fullsized":"2023-11-13T10:57:19.051592+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T10:57:20.031983+0000","last_change":"2023-11-13T10:57:20.031983+0000","last_active":"2023-11-13T10:57:20.031983+0000","last_peered":"2023-11-13T10:57:20.031983+0000","last_clean":"2023-11-13T10:57:20.031983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:20.031983+0000","last_undegraded":"2023-11-13T10:57:20.031983+0000","last_fullsized":"2023-11-13T10:57:20.031983+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-11-13T10:57:19.052061+0000","last_change":"2023-11-13T10:57:19.052061+0000","last_active":"2023-11-13T10:57:19.018776+0000","last_peered":"2023-11-13T10:57:19.018776+0000","last_clean":"2023-11-13T10:57:19.018776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T10:57:19.052061+0000","last_undegraded":"2023-11-13T10:57:19.052061+0000","last_fullsized":"2023-11-13T10:57:19.052061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":177,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-13T10:57:19.051478+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T10:57:19.051478+0000","last_peered":"2023-11-13T10:57:19.051478+0000","last_clean":"2023-11-13T10:57:19.051478+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T10:57:19.051478+0000","last_undegraded":"2023-11-13T10:57:19.051478+0000","last_fullsized":"2023-11-13T10:57:19.051478+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":166893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:56:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.80100000000000005}]},{"osd":1,"last update":"Mon Nov 13 10:56:19 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.5390000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.196}]},{"osd":2,"last update":"Mon Nov 13 10:56:19 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.452}]},{"osd":3,"last update":"Mon Nov 13 10:56:19 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.363},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.1479999999999999}]},{"osd":4,"last update":"Mon Nov 13 10:56:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":4.4740000000000002,"5min":4.4740000000000002,"15min":4.4740000000000002},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.635,"15min":1.635},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":4.1479999999999997,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.484}]},{"osd":5,"last update":"Mon Nov 13 10:56:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":1.171,"5min":1.171,"15min":1.171},"max":{"1min":3.286,"5min":3.286,"15min":3.286},"last":1.397},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"min":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":3.1949999999999998,"5min":3.1949999999999998,"15min":3.1949999999999998},"last":1.5109999999999999}]},{"osd":7,"last update":"Mon Nov 13 10:56:19 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.647,"15min":1.647},"min":{"1min":1.127,"5min":1.127,"15min":1.127},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.411},{"interface":"front","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.042}]}]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":757261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":0.98799999999999999,"15min":0.97399999999999998},"min":{"1min":0.68200000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.016},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1850000000000001,"15min":1.196},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5389999999999999,"5min":1.78,"15min":1.78},"last":1.1799999999999999}]},{"osd":1,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1919999999999999,"15min":1.202},"min":{"1min":0.71699999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.641,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.212,"15min":1.2310000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.4059999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.4059999999999999}]},{"osd":2,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.1859999999999999,"15min":1.2090000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.52,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.212,"5min":1.202,"15min":1.2},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.2270000000000001}]},{"osd":3,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.383,"15min":1.3959999999999999},"min":{"1min":0.90900000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.6950000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.379},{"interface":"front","average":{"1min":1.268,"5min":1.4490000000000001,"15min":1.4790000000000001},"min":{"1min":0.99399999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.698,"5min":2.911,"15min":2.911},"last":1.3080000000000001}]},{"osd":4,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2969999999999999,"15min":1.3109999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5569999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.5109999999999999,"15min":1.5449999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.702,"5min":2.133,"15min":2.133},"last":1.2709999999999999}]},{"osd":6,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.5720000000000001,"15min":1.6299999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5589999999999999,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.4930000000000001,"15min":1.5149999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6639999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.335}]},{"osd":7,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.6499999999999999,"15min":1.6850000000000001},"min":{"1min":1.026,"5min":1.026,"15min":1.026},"max":{"1min":1.75,"5min":2.23,"15min":2.23},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.665,"15min":1.718},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.633,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.464}]}]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":166893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":1.266},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.94699999999999995}]},{"osd":1,"last update":"Mon Nov 13 10:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.1080000000000001}]},{"osd":2,"last update":"Mon Nov 13 10:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.3959999999999999}]},{"osd":3,"last update":"Mon Nov 13 10:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Mon Nov 13 10:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.3049999999999999}]},{"osd":6,"last update":"Mon Nov 13 10:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.536},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":1.103,"5min":1.103,"15min":1.103},"max":{"1min":2.6480000000000001,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":1.204}]},{"osd":7,"last update":"Mon Nov 13 10:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":1.226,"5min":1.226,"15min":1.226},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.425}]}]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":405504,"data_stored":166893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.288},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.80800000000000005}]},{"osd":1,"last update":"Mon Nov 13 10:56:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":3.71,"5min":3.71,"15min":3.71},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.96899999999999997}]},{"osd":2,"last update":"Mon Nov 13 10:56:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.361,"15min":1.361},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.716}]},{"osd":4,"last update":"Mon Nov 13 10:56:18 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":3.5030000000000001,"5min":3.5030000000000001,"15min":3.5030000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.0640000000000001}]},{"osd":5,"last update":"Mon Nov 13 10:56:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.6339999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.4870000000000001}]},{"osd":6,"last update":"Mon Nov 13 10:56:18 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":1.345},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.6720000000000002,"5min":3.6720000000000002,"15min":3.6720000000000002},"last":1.5469999999999999}]},{"osd":7,"last update":"Mon Nov 13 10:56:18 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.395},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.611}]}]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":166893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.90700000000000003,"15min":0.93300000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97099999999999997,"5min":1.502,"15min":1.502},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.875,"15min":0.89400000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.391,"5min":1.492,"15min":1.492},"last":0.84699999999999998}]},{"osd":1,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.91300000000000003,"15min":0.93500000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.419,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.097,"15min":1.1240000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.94099999999999995}]},{"osd":3,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.2170000000000001,"15min":1.2689999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.972,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.27,"15min":1.29},"min":{"1min":0.60499999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.173,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.472}]},{"osd":4,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.4650000000000001,"15min":1.5},"min":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0249999999999999,"5min":2.581,"15min":2.581},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.383,"15min":1.429},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6000000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.3280000000000001}]},{"osd":5,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.456,"15min":1.5},"min":{"1min":0.77700000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.877,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.536,"15min":1.5629999999999999},"min":{"1min":0.86799999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.3210000000000002,"5min":2.496,"15min":2.496},"last":1.3660000000000001}]},{"osd":6,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.5249999999999999,"15min":1.5800000000000001},"min":{"1min":0.78500000000000003,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.673,"5min":4.7569999999999997,"15min":4.7569999999999997},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.5620000000000001,"15min":1.603},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.23,"5min":2.621,"15min":2.621},"last":1.5229999999999999}]},{"osd":7,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.579,"15min":1.641},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6020000000000001,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.5820000000000001,"15min":1.6259999999999999},"min":{"1min":0.85699999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.383,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.4139999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509467,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":757261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:56:16 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Nov 13 10:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.77600000000000002}]},{"osd":3,"last update":"Mon Nov 13 10:56:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.67600000000000005}]},{"osd":4,"last update":"Mon Nov 13 10:56:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.052},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":1.1200000000000001}]},{"osd":5,"last update":"Mon Nov 13 10:56:16 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.93100000000000005}]},{"osd":6,"last update":"Mon Nov 13 10:56:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.872}]},{"osd":7,"last update":"Mon Nov 13 10:56:16 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.018}]}]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":166893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.89500000000000002,"15min":0.92300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3320000000000001,"5min":1.456,"15min":1.456},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.0389999999999999,"15min":1.054},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.415,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.1399999999999999}]},{"osd":2,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.125,"15min":1.1339999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.5940000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.248},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.98899999999999999,"15min":1.0109999999999999},"min":{"1min":0.54100000000000004,"5min":0.371,"15min":0.371},"max":{"1min":1.4399999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.86799999999999999}]},{"osd":3,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0900000000000001,"15min":1.0960000000000001},"min":{"1min":0.48999999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.8560000000000001,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.319,"15min":1.343},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7929999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4079999999999999}]},{"osd":4,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3440000000000001,"15min":1.359},"min":{"1min":0.89300000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.8180000000000001,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.444,"15min":1.478},"min":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.073,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.3440000000000001}]},{"osd":5,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.5580000000000001,"15min":1.593},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.907,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":1.113},{"interface":"front","average":{"1min":1.306,"5min":1.488,"15min":1.518},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.258,"5min":2.347,"15min":2.347},"last":1.2709999999999999}]},{"osd":6,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.518,"15min":1.536},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9810000000000001,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.23},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.5649999999999999,"15min":1.583},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.1080000000000001,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":1.72}]},{"osd":7,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.677,"15min":1.7070000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9750000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.788},{"interface":"front","average":{"1min":1.488,"5min":1.6020000000000001,"15min":1.621},"min":{"1min":0.98499999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.0379999999999998,"5min":2.476,"15min":2.476},"last":1.319}]}]},{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":193410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.131,"15min":1.127},"min":{"1min":0.71599999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.829,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.54},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0760000000000001,"15min":1.0700000000000001},"min":{"1min":0.75900000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":0.878}]},{"osd":1,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0349999999999999,"15min":1.0209999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.252},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2310000000000001,"15min":1.226},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.8260000000000001}]},{"osd":2,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.306,"15min":1.3080000000000001},"min":{"1min":0.89200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.306},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.196,"15min":1.196},"min":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.4910000000000001}]},{"osd":3,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.24,"15min":1.2230000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.4470000000000001,"15min":1.4370000000000001},"min":{"1min":1.1519999999999999,"5min":1.036,"15min":1.036},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.7749999999999999}]},{"osd":4,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3169999999999999,"15min":1.3129999999999999},"min":{"1min":0.81499999999999995,"5min":0.745,"15min":0.745},"max":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":1.212},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.371,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.2530000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":1.738}]},{"osd":5,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"min":{"1min":0.90000000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.548,"5min":1.476,"15min":1.464},"min":{"1min":0.98199999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.3620000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.6379999999999999}]},{"osd":6,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5620000000000001,"15min":1.55},"min":{"1min":1.331,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.492,"15min":1.4850000000000001},"min":{"1min":0.98099999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":1.589}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T10:57:25.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T10:57:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.863+0000 7ffbbe32c640 1 Processor -- start 2023-11-13T10:57:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.863+0000 7ffbbe32c640 1 -- start start 2023-11-13T10:57:25.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.865+0000 7ffbbe32c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffbb8159170 0x7ffbb806cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.865+0000 7ffbbe32c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb80adf90 0x7ffbb815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.865+0000 7ffbbe32c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb806d700 con 0x7ffbb815d330 2023-11-13T10:57:25.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.865+0000 7ffbbe32c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb806df10 con 0x7ffbb8159170 2023-11-13T10:57:25.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.865+0000 7ffbbe32c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb815d760 con 0x7ffbb80adf90 2023-11-13T10:57:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.866+0000 7ffbaffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb80adf90 0x7ffbb815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.866+0000 7ffbb7fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffbb8159170 0x7ffbb806cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.866+0000 7ffbb7fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffbb8159170 0x7ffbb806cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36012/0 (socket says 172.21.15.16:36012) 2023-11-13T10:57:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.866+0000 7ffbb7fff640 1 -- 172.21.15.16:0/360140530 learned_addr learned my addr 172.21.15.16:0/360140530 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.867+0000 7ffbb7fff640 1 -- 172.21.15.16:0/360140530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb80adf90 msgr2=0x7ffbb815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.867+0000 7ffbb7fff640 1 --2- 172.21.15.16:0/360140530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb80adf90 0x7ffbb815cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.867+0000 7ffbb7fff640 1 -- 172.21.15.16:0/360140530 >> v1:172.21.15.184:6789/0 conn(0x7ffbb815d330 legacy=0x7ffbb80b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.867+0000 7ffbb7fff640 1 -- 172.21.15.16:0/360140530 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb806d950 con 0x7ffbb8159170 2023-11-13T10:57:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.868+0000 7ffbb7fff640 1 --2- 172.21.15.16:0/360140530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb8159170 0x7ffbb806cb90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009870 tx=0x7ffba802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c30442178d4590 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.869+0000 7ffbb77fe640 1 -- 172.21.15.16:0/360140530 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba80317c0 con 0x7ffbb8159170 2023-11-13T10:57:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.869+0000 7ffbb77fe640 1 -- 172.21.15.16:0/360140530 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba8031920 con 0x7ffbb8159170 2023-11-13T10:57:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.869+0000 7ffbb77fe640 1 -- 172.21.15.16:0/360140530 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8031cb0 con 0x7ffbb8159170 2023-11-13T10:57:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.870+0000 7ffbbe32c640 1 -- 172.21.15.16:0/360140530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb8159170 msgr2=0x7ffbb806cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.870+0000 7ffbbe32c640 1 --2- 172.21.15.16:0/360140530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb8159170 0x7ffbb806cb90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009870 tx=0x7ffba802b860 comp rx=0 tx=0).stop 2023-11-13T10:57:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.870+0000 7ffbbe32c640 1 -- 172.21.15.16:0/360140530 shutdown_connections 2023-11-13T10:57:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.870+0000 7ffbbe32c640 1 --2- 172.21.15.16:0/360140530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb80adf90 0x7ffbb815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.870+0000 7ffbbe32c640 1 --2- 172.21.15.16:0/360140530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb8159170 0x7ffbb806cb90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.870+0000 7ffbbe32c640 1 -- 172.21.15.16:0/360140530 >> 172.21.15.16:0/360140530 conn(0x7ffbb806a190 msgr2=0x7ffbb806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.870+0000 7ffbbe32c640 1 -- 172.21.15.16:0/360140530 shutdown_connections 2023-11-13T10:57:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.870+0000 7ffbbe32c640 1 -- 172.21.15.16:0/360140530 wait complete. 2023-11-13T10:57:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.873+0000 7ffbbe32c640 1 Processor -- start 2023-11-13T10:57:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.873+0000 7ffbbe32c640 1 -- start start 2023-11-13T10:57:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.874+0000 7ffbbe32c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb80adf90 0x7ffbb815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.874+0000 7ffbbe32c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb8159170 0x7ffbb815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.874+0000 7ffbbe32c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb815d330 0x7ffbb81f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.874+0000 7ffbbe32c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb815b520 con 0x7ffbb8159170 2023-11-13T10:57:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.874+0000 7ffbbe32c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb815b240 con 0x7ffbb80adf90 2023-11-13T10:57:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.874+0000 7ffbbe32c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb81f20a0 con 0x7ffbb815d330 2023-11-13T10:57:25.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb8159170 0x7ffbb815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb8159170 0x7ffbb815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36420/0 (socket says 172.21.15.16:36420) 2023-11-13T10:57:25.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbaffff640 1 -- 172.21.15.16:0/4129702098 learned_addr learned my addr 172.21.15.16:0/4129702098 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:25.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbb7fff640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb80adf90 0x7ffbb815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbbc8c9640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb815d330 0x7ffbb81f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbaffff640 1 -- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb815d330 msgr2=0x7ffbb81f0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbaffff640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb815d330 0x7ffbb81f0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbaffff640 1 -- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb80adf90 msgr2=0x7ffbb815a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:25.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbaffff640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb80adf90 0x7ffbb815a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:25.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbaffff640 1 -- 172.21.15.16:0/4129702098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80094d0 con 0x7ffbb8159170 2023-11-13T10:57:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.875+0000 7ffbbc8c9640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb815d330 0x7ffbb81f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.876+0000 7ffbaffff640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb8159170 0x7ffbb815aa10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ffba000e870 tx=0x7ffba000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.876+0000 7ffbb57fa640 1 -- 172.21.15.16:0/4129702098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba00094f0 con 0x7ffbb8159170 2023-11-13T10:57:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.876+0000 7ffbb57fa640 1 -- 172.21.15.16:0/4129702098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000c940 con 0x7ffbb8159170 2023-11-13T10:57:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.876+0000 7ffbb57fa640 1 -- 172.21.15.16:0/4129702098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000ccf0 con 0x7ffbb8159170 2023-11-13T10:57:25.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.876+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb81f2380 con 0x7ffbb8159170 2023-11-13T10:57:25.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.876+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb815af50 con 0x7ffbb8159170 2023-11-13T10:57:25.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.877+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb8067090 con 0x7ffbb8159170 2023-11-13T10:57:25.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.878+0000 7ffbb57fa640 1 -- 172.21.15.16:0/4129702098 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffba0013470 con 0x7ffbb8159170 2023-11-13T10:57:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.879+0000 7ffbb57fa640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb9806ae70 0x7ffb9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.879+0000 7ffbb57fa640 1 -- 172.21.15.16:0/4129702098 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffba0089810 con 0x7ffbb8159170 2023-11-13T10:57:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.880+0000 7ffbb7fff640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb9806ae70 0x7ffb9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.881+0000 7ffbb7fff640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb9806ae70 0x7ffb9806d330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffba8036000 tx=0x7ffba803d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:25.883+0000 7ffbb57fa640 1 -- 172.21.15.16:0/4129702098 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffba0054770 con 0x7ffbb8159170 2023-11-13T10:57:26.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.074+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbb806d0d0 con 0x7ffb9806ae70 2023-11-13T10:57:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.098+0000 7ffbb57fa640 1 -- 172.21.15.16:0/4129702098 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286760 (secure 0 0 0) 0x7ffba8031b10 con 0x7ffb9806ae70 2023-11-13T10:57:26.107 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T10:57:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb9806ae70 msgr2=0x7ffb9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb9806ae70 0x7ffb9806d330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffba8036000 tx=0x7ffba803d040 comp rx=0 tx=0).stop 2023-11-13T10:57:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb8159170 msgr2=0x7ffbb815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb8159170 0x7ffbb815aa10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ffba000e870 tx=0x7ffba000ed30 comp rx=0 tx=0).stop 2023-11-13T10:57:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 shutdown_connections 2023-11-13T10:57:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb9806ae70 0x7ffb9806d330 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbb815d330 0x7ffbb81f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb8159170 0x7ffbb815aa10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 --2- 172.21.15.16:0/4129702098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb80adf90 0x7ffbb815a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 >> 172.21.15.16:0/4129702098 conn(0x7ffbb806a190 msgr2=0x7ffbb81557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.105+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 shutdown_connections 2023-11-13T10:57:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.106+0000 7ffbbe32c640 1 -- 172.21.15.16:0/4129702098 wait complete. 2023-11-13T10:57:26.130 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2023-11-13T10:57:26.017847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237596,"kb_used_data":4776,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737124,"statfs":{"total":773094113280,"available":772850814976,"internally_reserved":0,"allocated":4890624,"data_stored":2732276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002157"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.834726+0000","last_change":"2023-11-13T10:57:20.859490+0000","last_active":"2023-11-13T10:57:21.834726+0000","last_peered":"2023-11-13T10:57:21.834726+0000","last_clean":"2023-11-13T10:57:21.834726+0000","last_became_active":"2023-11-13T10:57:20.859206+0000","last_became_peered":"2023-11-13T10:57:20.859206+0000","last_unstale":"2023-11-13T10:57:21.834726+0000","last_undegraded":"2023-11-13T10:57:21.834726+0000","last_fullsized":"2023-11-13T10:57:21.834726+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:02:20.891567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009817+0000","last_change":"2023-11-13T10:57:20.854442+0000","last_active":"2023-11-13T10:57:22.009817+0000","last_peered":"2023-11-13T10:57:22.009817+0000","last_clean":"2023-11-13T10:57:22.009817+0000","last_became_active":"2023-11-13T10:57:20.853720+0000","last_became_peered":"2023-11-13T10:57:20.853720+0000","last_unstale":"2023-11-13T10:57:22.009817+0000","last_undegraded":"2023-11-13T10:57:22.009817+0000","last_fullsized":"2023-11-13T10:57:22.009817+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:29:55.955602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.835542+0000","last_change":"2023-11-13T10:57:20.859282+0000","last_active":"2023-11-13T10:57:21.835542+0000","last_peered":"2023-11-13T10:57:21.835542+0000","last_clean":"2023-11-13T10:57:21.835542+0000","last_became_active":"2023-11-13T10:57:20.858959+0000","last_became_peered":"2023-11-13T10:57:20.858959+0000","last_unstale":"2023-11-13T10:57:21.835542+0000","last_undegraded":"2023-11-13T10:57:21.835542+0000","last_fullsized":"2023-11-13T10:57:21.835542+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:09:50.352098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.835156+0000","last_change":"2023-11-13T10:57:20.859267+0000","last_active":"2023-11-13T10:57:21.835156+0000","last_peered":"2023-11-13T10:57:21.835156+0000","last_clean":"2023-11-13T10:57:21.835156+0000","last_became_active":"2023-11-13T10:57:20.858902+0000","last_became_peered":"2023-11-13T10:57:20.858902+0000","last_unstale":"2023-11-13T10:57:21.835156+0000","last_undegraded":"2023-11-13T10:57:21.835156+0000","last_fullsized":"2023-11-13T10:57:21.835156+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:58:36.795899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007390+0000","last_change":"2023-11-13T10:57:20.865445+0000","last_active":"2023-11-13T10:57:22.007390+0000","last_peered":"2023-11-13T10:57:22.007390+0000","last_clean":"2023-11-13T10:57:22.007390+0000","last_became_active":"2023-11-13T10:57:20.864853+0000","last_became_peered":"2023-11-13T10:57:20.864853+0000","last_unstale":"2023-11-13T10:57:22.007390+0000","last_undegraded":"2023-11-13T10:57:22.007390+0000","last_fullsized":"2023-11-13T10:57:22.007390+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:29:15.574855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008226+0000","last_change":"2023-11-13T10:57:20.851434+0000","last_active":"2023-11-13T10:57:22.008226+0000","last_peered":"2023-11-13T10:57:22.008226+0000","last_clean":"2023-11-13T10:57:22.008226+0000","last_became_active":"2023-11-13T10:57:20.851168+0000","last_became_peered":"2023-11-13T10:57:20.851168+0000","last_unstale":"2023-11-13T10:57:22.008226+0000","last_undegraded":"2023-11-13T10:57:22.008226+0000","last_fullsized":"2023-11-13T10:57:22.008226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:22:45.520868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009356+0000","last_change":"2023-11-13T10:57:20.865140+0000","last_active":"2023-11-13T10:57:22.009356+0000","last_peered":"2023-11-13T10:57:22.009356+0000","last_clean":"2023-11-13T10:57:22.009356+0000","last_became_active":"2023-11-13T10:57:20.864666+0000","last_became_peered":"2023-11-13T10:57:20.864666+0000","last_unstale":"2023-11-13T10:57:22.009356+0000","last_undegraded":"2023-11-13T10:57:22.009356+0000","last_fullsized":"2023-11-13T10:57:22.009356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:24:18.661066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.833825+0000","last_change":"2023-11-13T10:57:20.869777+0000","last_active":"2023-11-13T10:57:21.833825+0000","last_peered":"2023-11-13T10:57:21.833825+0000","last_clean":"2023-11-13T10:57:21.833825+0000","last_became_active":"2023-11-13T10:57:20.869520+0000","last_became_peered":"2023-11-13T10:57:20.869520+0000","last_unstale":"2023-11-13T10:57:21.833825+0000","last_undegraded":"2023-11-13T10:57:21.833825+0000","last_fullsized":"2023-11-13T10:57:21.833825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:49:52.233703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008820+0000","last_change":"2023-11-13T10:57:20.858960+0000","last_active":"2023-11-13T10:57:22.008820+0000","last_peered":"2023-11-13T10:57:22.008820+0000","last_clean":"2023-11-13T10:57:22.008820+0000","last_became_active":"2023-11-13T10:57:20.858697+0000","last_became_peered":"2023-11-13T10:57:20.858697+0000","last_unstale":"2023-11-13T10:57:22.008820+0000","last_undegraded":"2023-11-13T10:57:22.008820+0000","last_fullsized":"2023-11-13T10:57:22.008820+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:46:22.317429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.010148+0000","last_change":"2023-11-13T10:57:20.867380+0000","last_active":"2023-11-13T10:57:22.010148+0000","last_peered":"2023-11-13T10:57:22.010148+0000","last_clean":"2023-11-13T10:57:22.010148+0000","last_became_active":"2023-11-13T10:57:20.866994+0000","last_became_peered":"2023-11-13T10:57:20.866994+0000","last_unstale":"2023-11-13T10:57:22.010148+0000","last_undegraded":"2023-11-13T10:57:22.010148+0000","last_fullsized":"2023-11-13T10:57:22.010148+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:49:49.301262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008017+0000","last_change":"2023-11-13T10:57:20.869965+0000","last_active":"2023-11-13T10:57:22.008017+0000","last_peered":"2023-11-13T10:57:22.008017+0000","last_clean":"2023-11-13T10:57:22.008017+0000","last_became_active":"2023-11-13T10:57:20.869508+0000","last_became_peered":"2023-11-13T10:57:20.869508+0000","last_unstale":"2023-11-13T10:57:22.008017+0000","last_undegraded":"2023-11-13T10:57:22.008017+0000","last_fullsized":"2023-11-13T10:57:22.008017+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:58:32.629744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008042+0000","last_change":"2023-11-13T10:57:20.852731+0000","last_active":"2023-11-13T10:57:22.008042+0000","last_peered":"2023-11-13T10:57:22.008042+0000","last_clean":"2023-11-13T10:57:22.008042+0000","last_became_active":"2023-11-13T10:57:20.851852+0000","last_became_peered":"2023-11-13T10:57:20.851852+0000","last_unstale":"2023-11-13T10:57:22.008042+0000","last_undegraded":"2023-11-13T10:57:22.008042+0000","last_fullsized":"2023-11-13T10:57:22.008042+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:10:33.233178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009594+0000","last_change":"2023-11-13T10:57:20.851272+0000","last_active":"2023-11-13T10:57:22.009594+0000","last_peered":"2023-11-13T10:57:22.009594+0000","last_clean":"2023-11-13T10:57:22.009594+0000","last_became_active":"2023-11-13T10:57:20.850585+0000","last_became_peered":"2023-11-13T10:57:20.850585+0000","last_unstale":"2023-11-13T10:57:22.009594+0000","last_undegraded":"2023-11-13T10:57:22.009594+0000","last_fullsized":"2023-11-13T10:57:22.009594+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:46:55.966748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.006409+0000","last_change":"2023-11-13T10:57:20.856194+0000","last_active":"2023-11-13T10:57:22.006409+0000","last_peered":"2023-11-13T10:57:22.006409+0000","last_clean":"2023-11-13T10:57:22.006409+0000","last_became_active":"2023-11-13T10:57:20.855882+0000","last_became_peered":"2023-11-13T10:57:20.855882+0000","last_unstale":"2023-11-13T10:57:22.006409+0000","last_undegraded":"2023-11-13T10:57:22.006409+0000","last_fullsized":"2023-11-13T10:57:22.006409+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:30:50.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.830093+0000","last_change":"2023-11-13T10:57:20.858831+0000","last_active":"2023-11-13T10:57:21.830093+0000","last_peered":"2023-11-13T10:57:21.830093+0000","last_clean":"2023-11-13T10:57:21.830093+0000","last_became_active":"2023-11-13T10:57:20.858480+0000","last_became_peered":"2023-11-13T10:57:20.858480+0000","last_unstale":"2023-11-13T10:57:21.830093+0000","last_undegraded":"2023-11-13T10:57:21.830093+0000","last_fullsized":"2023-11-13T10:57:21.830093+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:17:32.191181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009426+0000","last_change":"2023-11-13T10:57:20.865795+0000","last_active":"2023-11-13T10:57:22.009426+0000","last_peered":"2023-11-13T10:57:22.009426+0000","last_clean":"2023-11-13T10:57:22.009426+0000","last_became_active":"2023-11-13T10:57:20.864785+0000","last_became_peered":"2023-11-13T10:57:20.864785+0000","last_unstale":"2023-11-13T10:57:22.009426+0000","last_undegraded":"2023-11-13T10:57:22.009426+0000","last_fullsized":"2023-11-13T10:57:22.009426+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:08:35.129340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.831062+0000","last_change":"2023-11-13T10:57:20.866032+0000","last_active":"2023-11-13T10:57:21.831062+0000","last_peered":"2023-11-13T10:57:21.831062+0000","last_clean":"2023-11-13T10:57:21.831062+0000","last_became_active":"2023-11-13T10:57:20.865735+0000","last_became_peered":"2023-11-13T10:57:20.865735+0000","last_unstale":"2023-11-13T10:57:21.831062+0000","last_undegraded":"2023-11-13T10:57:21.831062+0000","last_fullsized":"2023-11-13T10:57:21.831062+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:51:34.452747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.831165+0000","last_change":"2023-11-13T10:57:20.869544+0000","last_active":"2023-11-13T10:57:21.831165+0000","last_peered":"2023-11-13T10:57:21.831165+0000","last_clean":"2023-11-13T10:57:21.831165+0000","last_became_active":"2023-11-13T10:57:20.868874+0000","last_became_peered":"2023-11-13T10:57:20.868874+0000","last_unstale":"2023-11-13T10:57:21.831165+0000","last_undegraded":"2023-11-13T10:57:21.831165+0000","last_fullsized":"2023-11-13T10:57:21.831165+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:42.905837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.833355+0000","last_change":"2023-11-13T10:57:20.858299+0000","last_active":"2023-11-13T10:57:21.833355+0000","last_peered":"2023-11-13T10:57:21.833355+0000","last_clean":"2023-11-13T10:57:21.833355+0000","last_became_active":"2023-11-13T10:57:20.858034+0000","last_became_peered":"2023-11-13T10:57:20.858034+0000","last_unstale":"2023-11-13T10:57:21.833355+0000","last_undegraded":"2023-11-13T10:57:21.833355+0000","last_fullsized":"2023-11-13T10:57:21.833355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:13:31.615655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007560+0000","last_change":"2023-11-13T10:57:20.858474+0000","last_active":"2023-11-13T10:57:22.007560+0000","last_peered":"2023-11-13T10:57:22.007560+0000","last_clean":"2023-11-13T10:57:22.007560+0000","last_became_active":"2023-11-13T10:57:20.858159+0000","last_became_peered":"2023-11-13T10:57:20.858159+0000","last_unstale":"2023-11-13T10:57:22.007560+0000","last_undegraded":"2023-11-13T10:57:22.007560+0000","last_fullsized":"2023-11-13T10:57:22.007560+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:25:46.864262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007625+0000","last_change":"2023-11-13T10:57:20.863379+0000","last_active":"2023-11-13T10:57:22.007625+0000","last_peered":"2023-11-13T10:57:22.007625+0000","last_clean":"2023-11-13T10:57:22.007625+0000","last_became_active":"2023-11-13T10:57:20.863011+0000","last_became_peered":"2023-11-13T10:57:20.863011+0000","last_unstale":"2023-11-13T10:57:22.007625+0000","last_undegraded":"2023-11-13T10:57:22.007625+0000","last_fullsized":"2023-11-13T10:57:22.007625+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:21.800764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007091+0000","last_change":"2023-11-13T10:57:20.857604+0000","last_active":"2023-11-13T10:57:22.007091+0000","last_peered":"2023-11-13T10:57:22.007091+0000","last_clean":"2023-11-13T10:57:22.007091+0000","last_became_active":"2023-11-13T10:57:20.856491+0000","last_became_peered":"2023-11-13T10:57:20.856491+0000","last_unstale":"2023-11-13T10:57:22.007091+0000","last_undegraded":"2023-11-13T10:57:22.007091+0000","last_fullsized":"2023-11-13T10:57:22.007091+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:31:59.660701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.830749+0000","last_change":"2023-11-13T10:57:20.863388+0000","last_active":"2023-11-13T10:57:21.830749+0000","last_peered":"2023-11-13T10:57:21.830749+0000","last_clean":"2023-11-13T10:57:21.830749+0000","last_became_active":"2023-11-13T10:57:20.863110+0000","last_became_peered":"2023-11-13T10:57:20.863110+0000","last_unstale":"2023-11-13T10:57:21.830749+0000","last_undegraded":"2023-11-13T10:57:21.830749+0000","last_fullsized":"2023-11-13T10:57:21.830749+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:04:16.073558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.833804+0000","last_change":"2023-11-13T10:57:20.858493+0000","last_active":"2023-11-13T10:57:21.833804+0000","last_peered":"2023-11-13T10:57:21.833804+0000","last_clean":"2023-11-13T10:57:21.833804+0000","last_became_active":"2023-11-13T10:57:20.858050+0000","last_became_peered":"2023-11-13T10:57:20.858050+0000","last_unstale":"2023-11-13T10:57:21.833804+0000","last_undegraded":"2023-11-13T10:57:21.833804+0000","last_fullsized":"2023-11-13T10:57:21.833804+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:29:37.554165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.006709+0000","last_change":"2023-11-13T10:57:20.851472+0000","last_active":"2023-11-13T10:57:22.006709+0000","last_peered":"2023-11-13T10:57:22.006709+0000","last_clean":"2023-11-13T10:57:22.006709+0000","last_became_active":"2023-11-13T10:57:20.850193+0000","last_became_peered":"2023-11-13T10:57:20.850193+0000","last_unstale":"2023-11-13T10:57:22.006709+0000","last_undegraded":"2023-11-13T10:57:22.006709+0000","last_fullsized":"2023-11-13T10:57:22.006709+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:49:49.301262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.834579+0000","last_change":"2023-11-13T10:57:20.864833+0000","last_active":"2023-11-13T10:57:21.834579+0000","last_peered":"2023-11-13T10:57:21.834579+0000","last_clean":"2023-11-13T10:57:21.834579+0000","last_became_active":"2023-11-13T10:57:20.864097+0000","last_became_peered":"2023-11-13T10:57:20.864097+0000","last_unstale":"2023-11-13T10:57:21.834579+0000","last_undegraded":"2023-11-13T10:57:21.834579+0000","last_fullsized":"2023-11-13T10:57:21.834579+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:46:43.875553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008184+0000","last_change":"2023-11-13T10:57:20.854956+0000","last_active":"2023-11-13T10:57:22.008184+0000","last_peered":"2023-11-13T10:57:22.008184+0000","last_clean":"2023-11-13T10:57:22.008184+0000","last_became_active":"2023-11-13T10:57:20.854668+0000","last_became_peered":"2023-11-13T10:57:20.854668+0000","last_unstale":"2023-11-13T10:57:22.008184+0000","last_undegraded":"2023-11-13T10:57:22.008184+0000","last_fullsized":"2023-11-13T10:57:22.008184+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:21:31.921695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.010009+0000","last_change":"2023-11-13T10:57:20.861997+0000","last_active":"2023-11-13T10:57:22.010009+0000","last_peered":"2023-11-13T10:57:22.010009+0000","last_clean":"2023-11-13T10:57:22.010009+0000","last_became_active":"2023-11-13T10:57:20.861702+0000","last_became_peered":"2023-11-13T10:57:20.861702+0000","last_unstale":"2023-11-13T10:57:22.010009+0000","last_undegraded":"2023-11-13T10:57:22.010009+0000","last_fullsized":"2023-11-13T10:57:22.010009+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:20:24.107721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.830393+0000","last_change":"2023-11-13T10:57:20.862860+0000","last_active":"2023-11-13T10:57:21.830393+0000","last_peered":"2023-11-13T10:57:21.830393+0000","last_clean":"2023-11-13T10:57:21.830393+0000","last_became_active":"2023-11-13T10:57:20.862490+0000","last_became_peered":"2023-11-13T10:57:20.862490+0000","last_unstale":"2023-11-13T10:57:21.830393+0000","last_undegraded":"2023-11-13T10:57:21.830393+0000","last_fullsized":"2023-11-13T10:57:21.830393+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:16:53.903743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.830423+0000","last_change":"2023-11-13T10:57:20.860588+0000","last_active":"2023-11-13T10:57:21.830423+0000","last_peered":"2023-11-13T10:57:21.830423+0000","last_clean":"2023-11-13T10:57:21.830423+0000","last_became_active":"2023-11-13T10:57:20.860118+0000","last_became_peered":"2023-11-13T10:57:20.860118+0000","last_unstale":"2023-11-13T10:57:21.830423+0000","last_undegraded":"2023-11-13T10:57:21.830423+0000","last_fullsized":"2023-11-13T10:57:21.830423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:21:11.024315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009559+0000","last_change":"2023-11-13T10:57:20.859938+0000","last_active":"2023-11-13T10:57:22.009559+0000","last_peered":"2023-11-13T10:57:22.009559+0000","last_clean":"2023-11-13T10:57:22.009559+0000","last_became_active":"2023-11-13T10:57:20.854826+0000","last_became_peered":"2023-11-13T10:57:20.854826+0000","last_unstale":"2023-11-13T10:57:22.009559+0000","last_undegraded":"2023-11-13T10:57:22.009559+0000","last_fullsized":"2023-11-13T10:57:22.009559+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:05:51.011533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.834204+0000","last_change":"2023-11-13T10:57:20.864634+0000","last_active":"2023-11-13T10:57:21.834204+0000","last_peered":"2023-11-13T10:57:21.834204+0000","last_clean":"2023-11-13T10:57:21.834204+0000","last_became_active":"2023-11-13T10:57:20.864328+0000","last_became_peered":"2023-11-13T10:57:20.864328+0000","last_unstale":"2023-11-13T10:57:21.834204+0000","last_undegraded":"2023-11-13T10:57:21.834204+0000","last_fullsized":"2023-11-13T10:57:21.834204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:45.246450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.830557+0000","last_change":"2023-11-13T10:57:20.858164+0000","last_active":"2023-11-13T10:57:21.830557+0000","last_peered":"2023-11-13T10:57:21.830557+0000","last_clean":"2023-11-13T10:57:21.830557+0000","last_became_active":"2023-11-13T10:57:20.857825+0000","last_became_peered":"2023-11-13T10:57:20.857825+0000","last_unstale":"2023-11-13T10:57:21.830557+0000","last_undegraded":"2023-11-13T10:57:21.830557+0000","last_fullsized":"2023-11-13T10:57:21.830557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:39:13.309122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.097760+0000","last_change":"2023-11-13T10:57:20.070075+0000","last_active":"2023-11-13T10:57:21.097760+0000","last_peered":"2023-11-13T10:57:21.097760+0000","last_clean":"2023-11-13T10:57:21.097760+0000","last_became_active":"2023-11-13T10:57:20.069796+0000","last_became_peered":"2023-11-13T10:57:20.069796+0000","last_unstale":"2023-11-13T10:57:21.097760+0000","last_undegraded":"2023-11-13T10:57:21.097760+0000","last_fullsized":"2023-11-13T10:57:21.097760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:46:54.953541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009400+0000","last_change":"2023-11-13T10:57:20.850525+0000","last_active":"2023-11-13T10:57:22.009400+0000","last_peered":"2023-11-13T10:57:22.009400+0000","last_clean":"2023-11-13T10:57:22.009400+0000","last_became_active":"2023-11-13T10:57:20.850137+0000","last_became_peered":"2023-11-13T10:57:20.850137+0000","last_unstale":"2023-11-13T10:57:22.009400+0000","last_undegraded":"2023-11-13T10:57:22.009400+0000","last_fullsized":"2023-11-13T10:57:22.009400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:53:03.831718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.095538+0000","last_change":"2023-11-13T10:57:20.072354+0000","last_active":"2023-11-13T10:57:21.095538+0000","last_peered":"2023-11-13T10:57:21.095538+0000","last_clean":"2023-11-13T10:57:21.095538+0000","last_became_active":"2023-11-13T10:57:20.072029+0000","last_became_peered":"2023-11-13T10:57:20.072029+0000","last_unstale":"2023-11-13T10:57:21.095538+0000","last_undegraded":"2023-11-13T10:57:21.095538+0000","last_fullsized":"2023-11-13T10:57:21.095538+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:13:30.602447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.831197+0000","last_change":"2023-11-13T10:57:20.869747+0000","last_active":"2023-11-13T10:57:21.831197+0000","last_peered":"2023-11-13T10:57:21.831197+0000","last_clean":"2023-11-13T10:57:21.831197+0000","last_became_active":"2023-11-13T10:57:20.869269+0000","last_became_peered":"2023-11-13T10:57:20.869269+0000","last_unstale":"2023-11-13T10:57:21.831197+0000","last_undegraded":"2023-11-13T10:57:21.831197+0000","last_fullsized":"2023-11-13T10:57:21.831197+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:06:32.885778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.090751+0000","last_change":"2023-11-13T10:57:20.075107+0000","last_active":"2023-11-13T10:57:21.090751+0000","last_peered":"2023-11-13T10:57:21.090751+0000","last_clean":"2023-11-13T10:57:21.090751+0000","last_became_active":"2023-11-13T10:57:20.074712+0000","last_became_peered":"2023-11-13T10:57:20.074712+0000","last_unstale":"2023-11-13T10:57:21.090751+0000","last_undegraded":"2023-11-13T10:57:21.090751+0000","last_fullsized":"2023-11-13T10:57:21.090751+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:05.062213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009520+0000","last_change":"2023-11-13T10:57:20.856792+0000","last_active":"2023-11-13T10:57:22.009520+0000","last_peered":"2023-11-13T10:57:22.009520+0000","last_clean":"2023-11-13T10:57:22.009520+0000","last_became_active":"2023-11-13T10:57:20.856533+0000","last_became_peered":"2023-11-13T10:57:20.856533+0000","last_unstale":"2023-11-13T10:57:22.009520+0000","last_undegraded":"2023-11-13T10:57:22.009520+0000","last_fullsized":"2023-11-13T10:57:22.009520+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:21.800764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.095812+0000","last_change":"2023-11-13T10:57:20.069464+0000","last_active":"2023-11-13T10:57:21.095812+0000","last_peered":"2023-11-13T10:57:21.095812+0000","last_clean":"2023-11-13T10:57:21.095812+0000","last_became_active":"2023-11-13T10:57:20.069144+0000","last_became_peered":"2023-11-13T10:57:20.069144+0000","last_unstale":"2023-11-13T10:57:21.095812+0000","last_undegraded":"2023-11-13T10:57:21.095812+0000","last_fullsized":"2023-11-13T10:57:21.095812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:05:49.998325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.834556+0000","last_change":"2023-11-13T10:57:20.853511+0000","last_active":"2023-11-13T10:57:21.834556+0000","last_peered":"2023-11-13T10:57:21.834556+0000","last_clean":"2023-11-13T10:57:21.834556+0000","last_became_active":"2023-11-13T10:57:20.852292+0000","last_became_peered":"2023-11-13T10:57:20.852292+0000","last_unstale":"2023-11-13T10:57:21.834556+0000","last_undegraded":"2023-11-13T10:57:21.834556+0000","last_fullsized":"2023-11-13T10:57:21.834556+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:53:27.866635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.095587+0000","last_change":"2023-11-13T10:57:20.074664+0000","last_active":"2023-11-13T10:57:21.095587+0000","last_peered":"2023-11-13T10:57:21.095587+0000","last_clean":"2023-11-13T10:57:21.095587+0000","last_became_active":"2023-11-13T10:57:20.074363+0000","last_became_peered":"2023-11-13T10:57:20.074363+0000","last_unstale":"2023-11-13T10:57:21.095587+0000","last_undegraded":"2023-11-13T10:57:21.095587+0000","last_fullsized":"2023-11-13T10:57:21.095587+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:24:17.647858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007163+0000","last_change":"2023-11-13T10:57:20.849827+0000","last_active":"2023-11-13T10:57:22.007163+0000","last_peered":"2023-11-13T10:57:22.007163+0000","last_clean":"2023-11-13T10:57:22.007163+0000","last_became_active":"2023-11-13T10:57:20.849190+0000","last_became_peered":"2023-11-13T10:57:20.849190+0000","last_unstale":"2023-11-13T10:57:22.007163+0000","last_undegraded":"2023-11-13T10:57:22.007163+0000","last_fullsized":"2023-11-13T10:57:22.007163+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:59:31.756353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.098387+0000","last_change":"2023-11-13T10:57:20.082733+0000","last_active":"2023-11-13T10:57:21.098387+0000","last_peered":"2023-11-13T10:57:21.098387+0000","last_clean":"2023-11-13T10:57:21.098387+0000","last_became_active":"2023-11-13T10:57:20.081295+0000","last_became_peered":"2023-11-13T10:57:20.081295+0000","last_unstale":"2023-11-13T10:57:21.098387+0000","last_undegraded":"2023-11-13T10:57:21.098387+0000","last_fullsized":"2023-11-13T10:57:21.098387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:30:04.784278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.830591+0000","last_change":"2023-11-13T10:57:20.866858+0000","last_active":"2023-11-13T10:57:21.830591+0000","last_peered":"2023-11-13T10:57:21.830591+0000","last_clean":"2023-11-13T10:57:21.830591+0000","last_became_active":"2023-11-13T10:57:20.866434+0000","last_became_peered":"2023-11-13T10:57:20.866434+0000","last_unstale":"2023-11-13T10:57:21.830591+0000","last_undegraded":"2023-11-13T10:57:21.830591+0000","last_fullsized":"2023-11-13T10:57:21.830591+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:42:19.392608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.850070+0000","last_change":"2023-11-13T10:57:20.066313+0000","last_active":"2023-11-13T10:57:20.850070+0000","last_peered":"2023-11-13T10:57:20.850070+0000","last_clean":"2023-11-13T10:57:20.850070+0000","last_became_active":"2023-11-13T10:57:20.066022+0000","last_became_peered":"2023-11-13T10:57:20.066022+0000","last_unstale":"2023-11-13T10:57:20.850070+0000","last_undegraded":"2023-11-13T10:57:20.850070+0000","last_fullsized":"2023-11-13T10:57:20.850070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:20.787556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009141+0000","last_change":"2023-11-13T10:57:20.849053+0000","last_active":"2023-11-13T10:57:22.009141+0000","last_peered":"2023-11-13T10:57:22.009141+0000","last_clean":"2023-11-13T10:57:22.009141+0000","last_became_active":"2023-11-13T10:57:20.848681+0000","last_became_peered":"2023-11-13T10:57:20.848681+0000","last_unstale":"2023-11-13T10:57:22.009141+0000","last_undegraded":"2023-11-13T10:57:22.009141+0000","last_fullsized":"2023-11-13T10:57:22.009141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:59:31.756353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.843311+0000","last_change":"2023-11-13T10:57:20.079814+0000","last_active":"2023-11-13T10:57:20.843311+0000","last_peered":"2023-11-13T10:57:20.843311+0000","last_clean":"2023-11-13T10:57:20.843311+0000","last_became_active":"2023-11-13T10:57:20.077244+0000","last_became_peered":"2023-11-13T10:57:20.077244+0000","last_unstale":"2023-11-13T10:57:20.843311+0000","last_undegraded":"2023-11-13T10:57:20.843311+0000","last_fullsized":"2023-11-13T10:57:20.843311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:47:26.711061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.835369+0000","last_change":"2023-11-13T10:57:20.858716+0000","last_active":"2023-11-13T10:57:21.835369+0000","last_peered":"2023-11-13T10:57:21.835369+0000","last_clean":"2023-11-13T10:57:21.835369+0000","last_became_active":"2023-11-13T10:57:20.858345+0000","last_became_peered":"2023-11-13T10:57:20.858345+0000","last_unstale":"2023-11-13T10:57:21.835369+0000","last_undegraded":"2023-11-13T10:57:21.835369+0000","last_fullsized":"2023-11-13T10:57:21.835369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:25:02.921922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.854937+0000","last_change":"2023-11-13T10:57:20.068908+0000","last_active":"2023-11-13T10:57:20.854937+0000","last_peered":"2023-11-13T10:57:20.854937+0000","last_clean":"2023-11-13T10:57:20.854937+0000","last_became_active":"2023-11-13T10:57:20.068610+0000","last_became_peered":"2023-11-13T10:57:20.068610+0000","last_unstale":"2023-11-13T10:57:20.854937+0000","last_undegraded":"2023-11-13T10:57:20.854937+0000","last_fullsized":"2023-11-13T10:57:20.854937+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:21:30.908487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.006607+0000","last_change":"2023-11-13T10:57:20.853880+0000","last_active":"2023-11-13T10:57:22.006607+0000","last_peered":"2023-11-13T10:57:22.006607+0000","last_clean":"2023-11-13T10:57:22.006607+0000","last_became_active":"2023-11-13T10:57:20.853557+0000","last_became_peered":"2023-11-13T10:57:20.853557+0000","last_unstale":"2023-11-13T10:57:22.006607+0000","last_undegraded":"2023-11-13T10:57:22.006607+0000","last_fullsized":"2023-11-13T10:57:22.006607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:35:52.127782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.849700+0000","last_change":"2023-11-13T10:57:20.074787+0000","last_active":"2023-11-13T10:57:20.849700+0000","last_peered":"2023-11-13T10:57:20.849700+0000","last_clean":"2023-11-13T10:57:20.849700+0000","last_became_active":"2023-11-13T10:57:20.074515+0000","last_became_peered":"2023-11-13T10:57:20.074515+0000","last_unstale":"2023-11-13T10:57:20.849700+0000","last_undegraded":"2023-11-13T10:57:20.849700+0000","last_fullsized":"2023-11-13T10:57:20.849700+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:46:42.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.834966+0000","last_change":"2023-11-13T10:57:20.869585+0000","last_active":"2023-11-13T10:57:21.834966+0000","last_peered":"2023-11-13T10:57:21.834966+0000","last_clean":"2023-11-13T10:57:21.834966+0000","last_became_active":"2023-11-13T10:57:20.869282+0000","last_became_peered":"2023-11-13T10:57:20.869282+0000","last_unstale":"2023-11-13T10:57:21.834966+0000","last_undegraded":"2023-11-13T10:57:21.834966+0000","last_fullsized":"2023-11-13T10:57:21.834966+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:01:58.846184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.097989+0000","last_change":"2023-11-13T10:57:20.073134+0000","last_active":"2023-11-13T10:57:21.097989+0000","last_peered":"2023-11-13T10:57:21.097989+0000","last_clean":"2023-11-13T10:57:21.097989+0000","last_became_active":"2023-11-13T10:57:20.072859+0000","last_became_peered":"2023-11-13T10:57:20.072859+0000","last_unstale":"2023-11-13T10:57:21.097989+0000","last_undegraded":"2023-11-13T10:57:21.097989+0000","last_fullsized":"2023-11-13T10:57:21.097989+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:19:44.233243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.010003+0000","last_change":"2023-11-13T10:57:20.854979+0000","last_active":"2023-11-13T10:57:22.010003+0000","last_peered":"2023-11-13T10:57:22.010003+0000","last_clean":"2023-11-13T10:57:22.010003+0000","last_became_active":"2023-11-13T10:57:20.854651+0000","last_became_peered":"2023-11-13T10:57:20.854651+0000","last_unstale":"2023-11-13T10:57:22.010003+0000","last_undegraded":"2023-11-13T10:57:22.010003+0000","last_fullsized":"2023-11-13T10:57:22.010003+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:30:50.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.100613+0000","last_change":"2023-11-13T10:57:20.061856+0000","last_active":"2023-11-13T10:57:21.100613+0000","last_peered":"2023-11-13T10:57:21.100613+0000","last_clean":"2023-11-13T10:57:21.100613+0000","last_became_active":"2023-11-13T10:57:20.061209+0000","last_became_peered":"2023-11-13T10:57:20.061209+0000","last_unstale":"2023-11-13T10:57:21.100613+0000","last_undegraded":"2023-11-13T10:57:21.100613+0000","last_fullsized":"2023-11-13T10:57:21.100613+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:58:31.616536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.835152+0000","last_change":"2023-11-13T10:57:20.852718+0000","last_active":"2023-11-13T10:57:21.835152+0000","last_peered":"2023-11-13T10:57:21.835152+0000","last_clean":"2023-11-13T10:57:21.835152+0000","last_became_active":"2023-11-13T10:57:20.851062+0000","last_became_peered":"2023-11-13T10:57:20.851062+0000","last_unstale":"2023-11-13T10:57:21.835152+0000","last_undegraded":"2023-11-13T10:57:21.835152+0000","last_fullsized":"2023-11-13T10:57:21.835152+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:36:19.810298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.091227+0000","last_change":"2023-11-13T10:57:20.077997+0000","last_active":"2023-11-13T10:57:21.091227+0000","last_peered":"2023-11-13T10:57:21.091227+0000","last_clean":"2023-11-13T10:57:21.091227+0000","last_became_active":"2023-11-13T10:57:20.077510+0000","last_became_peered":"2023-11-13T10:57:20.077510+0000","last_unstale":"2023-11-13T10:57:21.091227+0000","last_undegraded":"2023-11-13T10:57:21.091227+0000","last_fullsized":"2023-11-13T10:57:21.091227+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:25:45.851054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008368+0000","last_change":"2023-11-13T10:57:20.868451+0000","last_active":"2023-11-13T10:57:22.008368+0000","last_peered":"2023-11-13T10:57:22.008368+0000","last_clean":"2023-11-13T10:57:22.008368+0000","last_became_active":"2023-11-13T10:57:20.868084+0000","last_became_peered":"2023-11-13T10:57:20.868084+0000","last_unstale":"2023-11-13T10:57:22.008368+0000","last_undegraded":"2023-11-13T10:57:22.008368+0000","last_fullsized":"2023-11-13T10:57:22.008368+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:31:59.660701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.835934+0000","last_change":"2023-11-13T10:57:20.080534+0000","last_active":"2023-11-13T10:57:20.835934+0000","last_peered":"2023-11-13T10:57:20.835934+0000","last_clean":"2023-11-13T10:57:20.835934+0000","last_became_active":"2023-11-13T10:57:20.080270+0000","last_became_peered":"2023-11-13T10:57:20.080270+0000","last_unstale":"2023-11-13T10:57:20.835934+0000","last_undegraded":"2023-11-13T10:57:20.835934+0000","last_fullsized":"2023-11-13T10:57:20.835934+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:37:04.030842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007417+0000","last_change":"2023-11-13T10:57:20.859289+0000","last_active":"2023-11-13T10:57:22.007417+0000","last_peered":"2023-11-13T10:57:22.007417+0000","last_clean":"2023-11-13T10:57:22.007417+0000","last_became_active":"2023-11-13T10:57:20.858960+0000","last_became_peered":"2023-11-13T10:57:20.858960+0000","last_unstale":"2023-11-13T10:57:22.007417+0000","last_undegraded":"2023-11-13T10:57:22.007417+0000","last_fullsized":"2023-11-13T10:57:22.007417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:53:03.831718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.853715+0000","last_change":"2023-11-13T10:57:20.067092+0000","last_active":"2023-11-13T10:57:20.853715+0000","last_peered":"2023-11-13T10:57:20.853715+0000","last_clean":"2023-11-13T10:57:20.853715+0000","last_became_active":"2023-11-13T10:57:20.066419+0000","last_became_peered":"2023-11-13T10:57:20.066419+0000","last_unstale":"2023-11-13T10:57:20.853715+0000","last_undegraded":"2023-11-13T10:57:20.853715+0000","last_fullsized":"2023-11-13T10:57:20.853715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:25:01.908714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007775+0000","last_change":"2023-11-13T10:57:20.858413+0000","last_active":"2023-11-13T10:57:22.007775+0000","last_peered":"2023-11-13T10:57:22.007775+0000","last_clean":"2023-11-13T10:57:22.007775+0000","last_became_active":"2023-11-13T10:57:20.856053+0000","last_became_peered":"2023-11-13T10:57:20.856053+0000","last_unstale":"2023-11-13T10:57:22.007775+0000","last_undegraded":"2023-11-13T10:57:22.007775+0000","last_fullsized":"2023-11-13T10:57:22.007775+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:01:48.369767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.100845+0000","last_change":"2023-11-13T10:57:20.078159+0000","last_active":"2023-11-13T10:57:21.100845+0000","last_peered":"2023-11-13T10:57:21.100845+0000","last_clean":"2023-11-13T10:57:21.100845+0000","last_became_active":"2023-11-13T10:57:20.077775+0000","last_became_peered":"2023-11-13T10:57:20.077775+0000","last_unstale":"2023-11-13T10:57:21.100845+0000","last_undegraded":"2023-11-13T10:57:21.100845+0000","last_fullsized":"2023-11-13T10:57:21.100845+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:02:57.541955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009400+0000","last_change":"2023-11-13T10:57:20.863192+0000","last_active":"2023-11-13T10:57:22.009400+0000","last_peered":"2023-11-13T10:57:22.009400+0000","last_clean":"2023-11-13T10:57:22.009400+0000","last_became_active":"2023-11-13T10:57:20.861487+0000","last_became_peered":"2023-11-13T10:57:20.861487+0000","last_unstale":"2023-11-13T10:57:22.009400+0000","last_undegraded":"2023-11-13T10:57:22.009400+0000","last_fullsized":"2023-11-13T10:57:22.009400+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:25:46.864262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.098395+0000","last_change":"2023-11-13T10:57:20.082256+0000","last_active":"2023-11-13T10:57:21.098395+0000","last_peered":"2023-11-13T10:57:21.098395+0000","last_clean":"2023-11-13T10:57:21.098395+0000","last_became_active":"2023-11-13T10:57:20.081968+0000","last_became_peered":"2023-11-13T10:57:20.081968+0000","last_unstale":"2023-11-13T10:57:21.098395+0000","last_undegraded":"2023-11-13T10:57:21.098395+0000","last_fullsized":"2023-11-13T10:57:21.098395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:49:05.177038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.009976+0000","last_change":"2023-11-13T10:57:20.862113+0000","last_active":"2023-11-13T10:57:22.009976+0000","last_peered":"2023-11-13T10:57:22.009976+0000","last_clean":"2023-11-13T10:57:22.009976+0000","last_became_active":"2023-11-13T10:57:20.861719+0000","last_became_peered":"2023-11-13T10:57:20.861719+0000","last_unstale":"2023-11-13T10:57:22.009976+0000","last_undegraded":"2023-11-13T10:57:22.009976+0000","last_fullsized":"2023-11-13T10:57:22.009976+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:53:35.936315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.095269+0000","last_change":"2023-11-13T10:57:20.074658+0000","last_active":"2023-11-13T10:57:21.095269+0000","last_peered":"2023-11-13T10:57:21.095269+0000","last_clean":"2023-11-13T10:57:21.095269+0000","last_became_active":"2023-11-13T10:57:20.074160+0000","last_became_peered":"2023-11-13T10:57:20.074160+0000","last_unstale":"2023-11-13T10:57:21.095269+0000","last_undegraded":"2023-11-13T10:57:21.095269+0000","last_fullsized":"2023-11-13T10:57:21.095269+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:01:57.832976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.010129+0000","last_change":"2023-11-13T10:57:20.853511+0000","last_active":"2023-11-13T10:57:22.010129+0000","last_peered":"2023-11-13T10:57:22.010129+0000","last_clean":"2023-11-13T10:57:22.010129+0000","last_became_active":"2023-11-13T10:57:20.853221+0000","last_became_peered":"2023-11-13T10:57:20.853221+0000","last_unstale":"2023-11-13T10:57:22.010129+0000","last_undegraded":"2023-11-13T10:57:22.010129+0000","last_fullsized":"2023-11-13T10:57:22.010129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:35:52.127782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007495+0000","last_change":"2023-11-13T10:57:20.082004+0000","last_active":"2023-11-13T10:57:22.007495+0000","last_peered":"2023-11-13T10:57:22.007495+0000","last_clean":"2023-11-13T10:57:22.007495+0000","last_became_active":"2023-11-13T10:57:20.081685+0000","last_became_peered":"2023-11-13T10:57:20.081685+0000","last_unstale":"2023-11-13T10:57:22.007495+0000","last_undegraded":"2023-11-13T10:57:22.007495+0000","last_fullsized":"2023-11-13T10:57:22.007495+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:20:38.399944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.833763+0000","last_change":"2023-11-13T10:57:20.861363+0000","last_active":"2023-11-13T10:57:21.833763+0000","last_peered":"2023-11-13T10:57:21.833763+0000","last_clean":"2023-11-13T10:57:21.833763+0000","last_became_active":"2023-11-13T10:57:20.861082+0000","last_became_peered":"2023-11-13T10:57:20.861082+0000","last_unstale":"2023-11-13T10:57:21.833763+0000","last_undegraded":"2023-11-13T10:57:21.833763+0000","last_fullsized":"2023-11-13T10:57:21.833763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:31:59.660701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.834392+0000","last_change":"2023-11-13T10:57:20.078470+0000","last_active":"2023-11-13T10:57:20.834392+0000","last_peered":"2023-11-13T10:57:20.834392+0000","last_clean":"2023-11-13T10:57:20.834392+0000","last_became_active":"2023-11-13T10:57:20.078092+0000","last_became_peered":"2023-11-13T10:57:20.078092+0000","last_unstale":"2023-11-13T10:57:20.834392+0000","last_undegraded":"2023-11-13T10:57:20.834392+0000","last_fullsized":"2023-11-13T10:57:20.834392+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:37:04.030842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.830106+0000","last_change":"2023-11-13T10:57:20.858367+0000","last_active":"2023-11-13T10:57:21.830106+0000","last_peered":"2023-11-13T10:57:21.830106+0000","last_clean":"2023-11-13T10:57:21.830106+0000","last_became_active":"2023-11-13T10:57:20.858038+0000","last_became_peered":"2023-11-13T10:57:20.858038+0000","last_unstale":"2023-11-13T10:57:21.830106+0000","last_undegraded":"2023-11-13T10:57:21.830106+0000","last_fullsized":"2023-11-13T10:57:21.830106+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:23:01.725451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.092511+0000","last_change":"2023-11-13T10:57:20.075067+0000","last_active":"2023-11-13T10:57:21.092511+0000","last_peered":"2023-11-13T10:57:21.092511+0000","last_clean":"2023-11-13T10:57:21.092511+0000","last_became_active":"2023-11-13T10:57:20.074742+0000","last_became_peered":"2023-11-13T10:57:20.074742+0000","last_unstale":"2023-11-13T10:57:21.092511+0000","last_undegraded":"2023-11-13T10:57:21.092511+0000","last_fullsized":"2023-11-13T10:57:21.092511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:30:49.560575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.834389+0000","last_change":"2023-11-13T10:57:20.867188+0000","last_active":"2023-11-13T10:57:21.834389+0000","last_peered":"2023-11-13T10:57:21.834389+0000","last_clean":"2023-11-13T10:57:21.834389+0000","last_became_active":"2023-11-13T10:57:20.866944+0000","last_became_peered":"2023-11-13T10:57:20.866944+0000","last_unstale":"2023-11-13T10:57:21.834389+0000","last_undegraded":"2023-11-13T10:57:21.834389+0000","last_fullsized":"2023-11-13T10:57:21.834389+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:51:05.794231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.835722+0000","last_change":"2023-11-13T10:57:20.078804+0000","last_active":"2023-11-13T10:57:20.835722+0000","last_peered":"2023-11-13T10:57:20.835722+0000","last_clean":"2023-11-13T10:57:20.835722+0000","last_became_active":"2023-11-13T10:57:20.076520+0000","last_became_peered":"2023-11-13T10:57:20.076520+0000","last_unstale":"2023-11-13T10:57:20.835722+0000","last_undegraded":"2023-11-13T10:57:20.835722+0000","last_fullsized":"2023-11-13T10:57:20.835722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:10:30.392129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007153+0000","last_change":"2023-11-13T10:57:20.864892+0000","last_active":"2023-11-13T10:57:22.007153+0000","last_peered":"2023-11-13T10:57:22.007153+0000","last_clean":"2023-11-13T10:57:22.007153+0000","last_became_active":"2023-11-13T10:57:20.864471+0000","last_became_peered":"2023-11-13T10:57:20.864471+0000","last_unstale":"2023-11-13T10:57:22.007153+0000","last_undegraded":"2023-11-13T10:57:22.007153+0000","last_fullsized":"2023-11-13T10:57:22.007153+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:22:22.796439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.846149+0000","last_change":"2023-11-13T10:57:20.077960+0000","last_active":"2023-11-13T10:57:20.846149+0000","last_peered":"2023-11-13T10:57:20.846149+0000","last_clean":"2023-11-13T10:57:20.846149+0000","last_became_active":"2023-11-13T10:57:20.077652+0000","last_became_peered":"2023-11-13T10:57:20.077652+0000","last_unstale":"2023-11-13T10:57:20.846149+0000","last_undegraded":"2023-11-13T10:57:20.846149+0000","last_fullsized":"2023-11-13T10:57:20.846149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:32.579323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.834166+0000","last_change":"2023-11-13T10:57:20.865451+0000","last_active":"2023-11-13T10:57:21.834166+0000","last_peered":"2023-11-13T10:57:21.834166+0000","last_clean":"2023-11-13T10:57:21.834166+0000","last_became_active":"2023-11-13T10:57:20.865144+0000","last_became_peered":"2023-11-13T10:57:20.865144+0000","last_unstale":"2023-11-13T10:57:21.834166+0000","last_undegraded":"2023-11-13T10:57:21.834166+0000","last_fullsized":"2023-11-13T10:57:21.834166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:47:49.343456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.835427+0000","last_change":"2023-11-13T10:57:20.075640+0000","last_active":"2023-11-13T10:57:20.835427+0000","last_peered":"2023-11-13T10:57:20.835427+0000","last_clean":"2023-11-13T10:57:20.835427+0000","last_became_active":"2023-11-13T10:57:20.073202+0000","last_became_peered":"2023-11-13T10:57:20.073202+0000","last_unstale":"2023-11-13T10:57:20.835427+0000","last_undegraded":"2023-11-13T10:57:20.835427+0000","last_fullsized":"2023-11-13T10:57:20.835427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:49:48.288055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008404+0000","last_change":"2023-11-13T10:57:20.868321+0000","last_active":"2023-11-13T10:57:22.008404+0000","last_peered":"2023-11-13T10:57:22.008404+0000","last_clean":"2023-11-13T10:57:22.008404+0000","last_became_active":"2023-11-13T10:57:20.867963+0000","last_became_peered":"2023-11-13T10:57:20.867963+0000","last_unstale":"2023-11-13T10:57:22.008404+0000","last_undegraded":"2023-11-13T10:57:22.008404+0000","last_fullsized":"2023-11-13T10:57:22.008404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:28:06.363762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.098642+0000","last_change":"2023-11-13T10:57:20.076780+0000","last_active":"2023-11-13T10:57:21.098642+0000","last_peered":"2023-11-13T10:57:21.098642+0000","last_clean":"2023-11-13T10:57:21.098642+0000","last_became_active":"2023-11-13T10:57:20.076471+0000","last_became_peered":"2023-11-13T10:57:20.076471+0000","last_unstale":"2023-11-13T10:57:21.098642+0000","last_undegraded":"2023-11-13T10:57:21.098642+0000","last_fullsized":"2023-11-13T10:57:21.098642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:02:19.878359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008208+0000","last_change":"2023-11-13T10:57:20.869741+0000","last_active":"2023-11-13T10:57:22.008208+0000","last_peered":"2023-11-13T10:57:22.008208+0000","last_clean":"2023-11-13T10:57:22.008208+0000","last_became_active":"2023-11-13T10:57:20.869140+0000","last_became_peered":"2023-11-13T10:57:20.869140+0000","last_unstale":"2023-11-13T10:57:22.008208+0000","last_undegraded":"2023-11-13T10:57:22.008208+0000","last_fullsized":"2023-11-13T10:57:22.008208+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:58:06.075420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.103253+0000","last_change":"2023-11-13T10:57:20.077545+0000","last_active":"2023-11-13T10:57:22.103253+0000","last_peered":"2023-11-13T10:57:22.103253+0000","last_clean":"2023-11-13T10:57:22.103253+0000","last_became_active":"2023-11-13T10:57:20.077264+0000","last_became_peered":"2023-11-13T10:57:20.077264+0000","last_unstale":"2023-11-13T10:57:22.103253+0000","last_undegraded":"2023-11-13T10:57:22.103253+0000","last_fullsized":"2023-11-13T10:57:22.103253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:09.602632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.833401+0000","last_change":"2023-11-13T10:57:20.857845+0000","last_active":"2023-11-13T10:57:21.833401+0000","last_peered":"2023-11-13T10:57:21.833401+0000","last_clean":"2023-11-13T10:57:21.833401+0000","last_became_active":"2023-11-13T10:57:20.855876+0000","last_became_peered":"2023-11-13T10:57:20.855876+0000","last_unstale":"2023-11-13T10:57:21.833401+0000","last_undegraded":"2023-11-13T10:57:21.833401+0000","last_fullsized":"2023-11-13T10:57:21.833401+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:22:45.520868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007754+0000","last_change":"2023-11-13T10:57:20.080966+0000","last_active":"2023-11-13T10:57:22.007754+0000","last_peered":"2023-11-13T10:57:22.007754+0000","last_clean":"2023-11-13T10:57:22.007754+0000","last_became_active":"2023-11-13T10:57:20.079977+0000","last_became_peered":"2023-11-13T10:57:20.079977+0000","last_unstale":"2023-11-13T10:57:22.007754+0000","last_undegraded":"2023-11-13T10:57:22.007754+0000","last_fullsized":"2023-11-13T10:57:22.007754+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:43:57.351520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007182+0000","last_change":"2023-11-13T10:57:20.862228+0000","last_active":"2023-11-13T10:57:22.007182+0000","last_peered":"2023-11-13T10:57:22.007182+0000","last_clean":"2023-11-13T10:57:22.007182+0000","last_became_active":"2023-11-13T10:57:20.861934+0000","last_became_peered":"2023-11-13T10:57:20.861934+0000","last_unstale":"2023-11-13T10:57:22.007182+0000","last_undegraded":"2023-11-13T10:57:22.007182+0000","last_fullsized":"2023-11-13T10:57:22.007182+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:47:22.240659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.850661+0000","last_change":"2023-11-13T10:57:20.078267+0000","last_active":"2023-11-13T10:57:20.850661+0000","last_peered":"2023-11-13T10:57:20.850661+0000","last_clean":"2023-11-13T10:57:20.850661+0000","last_became_active":"2023-11-13T10:57:20.077927+0000","last_became_peered":"2023-11-13T10:57:20.077927+0000","last_unstale":"2023-11-13T10:57:20.850661+0000","last_undegraded":"2023-11-13T10:57:20.850661+0000","last_fullsized":"2023-11-13T10:57:20.850661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:29:54.942394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.008612+0000","last_change":"2023-11-13T10:57:20.856322+0000","last_active":"2023-11-13T10:57:22.008612+0000","last_peered":"2023-11-13T10:57:22.008612+0000","last_clean":"2023-11-13T10:57:22.008612+0000","last_became_active":"2023-11-13T10:57:20.856037+0000","last_became_peered":"2023-11-13T10:57:20.856037+0000","last_unstale":"2023-11-13T10:57:22.008612+0000","last_undegraded":"2023-11-13T10:57:22.008612+0000","last_fullsized":"2023-11-13T10:57:22.008612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:22:22.796439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.841057+0000","last_change":"2023-11-13T10:57:20.076390+0000","last_active":"2023-11-13T10:57:20.841057+0000","last_peered":"2023-11-13T10:57:20.841057+0000","last_clean":"2023-11-13T10:57:20.841057+0000","last_became_active":"2023-11-13T10:57:20.076112+0000","last_became_peered":"2023-11-13T10:57:20.076112+0000","last_unstale":"2023-11-13T10:57:20.841057+0000","last_undegraded":"2023-11-13T10:57:20.841057+0000","last_fullsized":"2023-11-13T10:57:20.841057+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:35:51.114574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.830672+0000","last_change":"2023-11-13T10:57:20.862811+0000","last_active":"2023-11-13T10:57:21.830672+0000","last_peered":"2023-11-13T10:57:21.830672+0000","last_clean":"2023-11-13T10:57:21.830672+0000","last_became_active":"2023-11-13T10:57:20.862438+0000","last_became_peered":"2023-11-13T10:57:20.862438+0000","last_unstale":"2023-11-13T10:57:21.830672+0000","last_undegraded":"2023-11-13T10:57:21.830672+0000","last_fullsized":"2023-11-13T10:57:21.830672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:55:50.653012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.853882+0000","last_change":"2023-11-13T10:57:20.076067+0000","last_active":"2023-11-13T10:57:20.853882+0000","last_peered":"2023-11-13T10:57:20.853882+0000","last_clean":"2023-11-13T10:57:20.853882+0000","last_became_active":"2023-11-13T10:57:20.075794+0000","last_became_peered":"2023-11-13T10:57:20.075794+0000","last_unstale":"2023-11-13T10:57:20.853882+0000","last_undegraded":"2023-11-13T10:57:20.853882+0000","last_fullsized":"2023-11-13T10:57:20.853882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:09:49.338890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:22.007772+0000","last_change":"2023-11-13T10:57:20.859586+0000","last_active":"2023-11-13T10:57:22.007772+0000","last_peered":"2023-11-13T10:57:22.007772+0000","last_clean":"2023-11-13T10:57:22.007772+0000","last_became_active":"2023-11-13T10:57:20.857466+0000","last_became_peered":"2023-11-13T10:57:20.857466+0000","last_unstale":"2023-11-13T10:57:22.007772+0000","last_undegraded":"2023-11-13T10:57:22.007772+0000","last_fullsized":"2023-11-13T10:57:22.007772+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:29:55.955602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.845767+0000","last_change":"2023-11-13T10:57:20.067040+0000","last_active":"2023-11-13T10:57:20.845767+0000","last_peered":"2023-11-13T10:57:20.845767+0000","last_clean":"2023-11-13T10:57:20.845767+0000","last_became_active":"2023-11-13T10:57:20.066158+0000","last_became_peered":"2023-11-13T10:57:20.066158+0000","last_unstale":"2023-11-13T10:57:20.845767+0000","last_undegraded":"2023-11-13T10:57:20.845767+0000","last_fullsized":"2023-11-13T10:57:20.845767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:22:21.783231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-13T10:57:21.830972+0000","last_change":"2023-11-13T10:57:20.863591+0000","last_active":"2023-11-13T10:57:21.830972+0000","last_peered":"2023-11-13T10:57:21.830972+0000","last_clean":"2023-11-13T10:57:21.830972+0000","last_became_active":"2023-11-13T10:57:20.863264+0000","last_became_peered":"2023-11-13T10:57:20.863264+0000","last_unstale":"2023-11-13T10:57:21.830972+0000","last_undegraded":"2023-11-13T10:57:21.830972+0000","last_fullsized":"2023-11-13T10:57:21.830972+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:20.031983+0000","last_clean_scrub_stamp":"2023-11-13T10:57:20.031983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:44:31.116977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.842759+0000","last_change":"2023-11-13T10:57:20.078528+0000","last_active":"2023-11-13T10:57:20.842759+0000","last_peered":"2023-11-13T10:57:20.842759+0000","last_clean":"2023-11-13T10:57:20.842759+0000","last_became_active":"2023-11-13T10:57:20.078149+0000","last_became_peered":"2023-11-13T10:57:20.078149+0000","last_unstale":"2023-11-13T10:57:20.842759+0000","last_undegraded":"2023-11-13T10:57:20.842759+0000","last_fullsized":"2023-11-13T10:57:20.842759+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:57:19.018776+0000","last_clean_scrub_stamp":"2023-11-13T10:57:19.018776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:57.851909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-13T10:57:20.843598+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T10:57:20.843598+0000","last_peered":"2023-11-13T10:57:20.843598+0000","last_clean":"2023-11-13T10:57:20.843598+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T10:57:20.843598+0000","last_undegraded":"2023-11-13T10:57:20.843598+0000","last_fullsized":"2023-11-13T10:57:20.843598+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":193374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1299999999999999,"15min":1.1379999999999999},"min":{"1min":0.73499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.569,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.093,"15min":1.095},"min":{"1min":0.751,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6319999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.198}]},{"osd":1,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1359999999999999,"15min":1.147},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4590000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.248,"15min":1.272},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5740000000000001,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.5740000000000001}]},{"osd":2,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.2330000000000001,"15min":1.2609999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.4810000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.718,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.718}]},{"osd":3,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.3360000000000001,"15min":1.359},"min":{"1min":0.751,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.675,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.675},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.4079999999999999,"15min":1.4339999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.5129999999999999,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.5049999999999999}]},{"osd":4,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.5680000000000001,"15min":1.6100000000000001},"min":{"1min":0.99099999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.899,"5min":4.4740000000000002,"15min":4.4740000000000002},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.5760000000000001,"15min":1.615},"min":{"1min":1.032,"5min":1.024,"15min":1.024},"max":{"1min":1.8260000000000001,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.8260000000000001}]},{"osd":5,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.631,"15min":1.667},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":1.804,"5min":3.286,"15min":3.286},"last":1.448},{"interface":"front","average":{"1min":1.472,"5min":1.7170000000000001,"15min":1.758},"min":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"max":{"1min":1.9970000000000001,"5min":3.1949999999999998,"15min":3.1949999999999998},"last":1.53}]},{"osd":7,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.597,"15min":1.6299999999999999},"min":{"1min":1.095,"5min":1.095,"15min":1.095},"max":{"1min":1.7490000000000001,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.607,"15min":1.647},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":1.833,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.6319999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":784290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":0.98799999999999999,"15min":0.97399999999999998},"min":{"1min":0.68200000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.26},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1850000000000001,"15min":1.196},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5389999999999999,"5min":1.78,"15min":1.78},"last":1.1200000000000001}]},{"osd":1,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1919999999999999,"15min":1.202},"min":{"1min":0.71699999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.641,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.212,"15min":1.2310000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.4059999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.3200000000000001}]},{"osd":2,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.1859999999999999,"15min":1.2090000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.52,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.202,"15min":1.2},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.81699999999999995}]},{"osd":3,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.383,"15min":1.3959999999999999},"min":{"1min":0.90900000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.6950000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.4490000000000001,"15min":1.4790000000000001},"min":{"1min":0.99399999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.698,"5min":2.911,"15min":2.911},"last":1.446}]},{"osd":4,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2969999999999999,"15min":1.3109999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5569999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.5109999999999999,"15min":1.5449999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.702,"5min":2.133,"15min":2.133},"last":1.1719999999999999}]},{"osd":6,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.5720000000000001,"15min":1.6299999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5589999999999999,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.4930000000000001,"15min":1.5149999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6639999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.042}]},{"osd":7,"last update":"Mon Nov 13 10:57:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.6499999999999999,"15min":1.6850000000000001},"min":{"1min":1.026,"5min":1.026,"15min":1.026},"max":{"1min":1.75,"5min":2.23,"15min":2.23},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.665,"15min":1.718},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.633,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.538}]}]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":475136,"data_stored":195076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.071,"15min":1.0529999999999999},"min":{"1min":0.73299999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.0489999999999999,"15min":1.0569999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.399,"5min":1.639,"15min":1.639},"last":0.45500000000000002}]},{"osd":1,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1140000000000001,"15min":1.1240000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.681,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.177,"5min":1.234,"15min":1.244},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5429999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.83699999999999997}]},{"osd":2,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.212,"15min":1.216},"min":{"1min":0.89000000000000001,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.643,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.339,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.002}]},{"osd":3,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3839999999999999,"15min":1.389},"min":{"1min":0.93100000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.28,"15min":1.29},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.522,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.70099999999999996}]},{"osd":5,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.478,"15min":1.504},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.21,"5min":1.3839999999999999,"15min":1.413},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.5}]},{"osd":6,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.605,"15min":1.629},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.1970000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.5149999999999999,"15min":1.5489999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.992,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":0.624}]},{"osd":7,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.5800000000000001,"15min":1.6120000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0609999999999999,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.599,"15min":1.619},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.145,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.78700000000000003}]}]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":458752,"data_stored":193430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.84199999999999997,"15min":0.86199999999999999},"min":{"1min":0.46500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.288,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.022,"15min":1.03},"min":{"1min":0.65200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.526,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.5920000000000001}]},{"osd":1,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.56100000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.5680000000000001,"5min":3.71,"15min":3.71},"last":1.026},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0880000000000001,"15min":1.093},"min":{"1min":0.58099999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.5940000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.80600000000000005}]},{"osd":2,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.077,"15min":1.0629999999999999},"min":{"1min":0.78000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.7290000000000001,"5min":1.913,"15min":1.913},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.3160000000000001,"15min":1.3460000000000001},"min":{"1min":0.68500000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.716,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.421}]},{"osd":4,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.4399999999999999,"15min":1.4610000000000001},"min":{"1min":1.0109999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.726,"5min":3.5030000000000001,"15min":3.5030000000000001},"last":1.53},{"interface":"front","average":{"1min":1.23,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.86599999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.581,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.911}]},{"osd":5,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.4810000000000001,"15min":1.51},"min":{"1min":0.90400000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6919999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.331},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.5089999999999999,"15min":1.544},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.788,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.371}]},{"osd":6,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.474,"15min":1.4950000000000001},"min":{"1min":0.98399999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.8260000000000001,"5min":2.585,"15min":2.585},"last":2.0619999999999998},{"interface":"front","average":{"1min":1.405,"5min":1.607,"15min":1.641},"min":{"1min":0.91300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7569999999999999,"5min":3.6720000000000002,"15min":3.6720000000000002},"last":1.472}]},{"osd":7,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.603,"15min":1.643},"min":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.5329999999999999,"15min":1.5429999999999999},"min":{"1min":0.96499999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.7589999999999999,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":2.1859999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":471040,"data_stored":195534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.90700000000000003,"15min":0.93300000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97099999999999997,"5min":1.502,"15min":1.502},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.875,"15min":0.89400000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.391,"5min":1.492,"15min":1.492},"last":1.2569999999999999}]},{"osd":1,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.91300000000000003,"15min":0.93500000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.419,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.325},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.097,"15min":1.1240000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.0029999999999999}]},{"osd":3,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.2170000000000001,"15min":1.2689999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.972,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.27,"15min":1.29},"min":{"1min":0.60499999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.173,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.125}]},{"osd":4,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.4650000000000001,"15min":1.5},"min":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0249999999999999,"5min":2.581,"15min":2.581},"last":1.401},{"interface":"front","average":{"1min":1.111,"5min":1.383,"15min":1.429},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6000000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.153}]},{"osd":5,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.456,"15min":1.5},"min":{"1min":0.77700000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.877,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.76},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.536,"15min":1.5629999999999999},"min":{"1min":0.86799999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.3210000000000002,"5min":2.496,"15min":2.496},"last":1.5609999999999999}]},{"osd":6,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.5249999999999999,"15min":1.5800000000000001},"min":{"1min":0.78500000000000003,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.673,"5min":4.7569999999999997,"15min":4.7569999999999997},"last":1.514},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.5620000000000001,"15min":1.603},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.23,"5min":2.621,"15min":2.621},"last":1.5820000000000001}]},{"osd":7,"last update":"Mon Nov 13 10:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.579,"15min":1.641},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6020000000000001,"5min":4.9370000000000003,"15min":4.9370000000000003},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.5820000000000001,"15min":1.6259999999999999},"min":{"1min":0.85699999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.383,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.6859999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":783730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.97599999999999998,"15min":1.0009999999999999},"min":{"1min":0.45700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.458,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.93300000000000005,"15min":0.95299999999999996},"min":{"1min":0.54400000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6140000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.66800000000000004}]},{"osd":2,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.97099999999999997,"15min":0.97599999999999998},"min":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.97299999999999998,"15min":0.98199999999999998},"min":{"1min":0.59899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0489999999999999}]},{"osd":3,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.1459999999999999,"15min":1.1639999999999999},"min":{"1min":0.64500000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0609999999999999,"15min":1.0780000000000001},"min":{"1min":0.57399999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5489999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.76800000000000002}]},{"osd":4,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.3009999999999999,"15min":1.3120000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.7230000000000001,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.198},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3580000000000001,"15min":1.3740000000000001},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.8640000000000001,"5min":1.881,"15min":1.881},"last":1.2809999999999999}]},{"osd":5,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3959999999999999,"15min":1.419},"min":{"1min":0.84999999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.141},{"interface":"front","average":{"1min":1.292,"5min":1.4039999999999999,"15min":1.423},"min":{"1min":0.82699999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.7490000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.2350000000000001}]},{"osd":6,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.4159999999999999,"15min":1.448},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9399999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.4850000000000001,"15min":1.516},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.577,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.512}]},{"osd":7,"last update":"Mon Nov 13 10:57:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.4930000000000001,"15min":1.5129999999999999},"min":{"1min":0.86799999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7250000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.4490000000000001,"15min":1.4710000000000001},"min":{"1min":0.95299999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.036,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.4470000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":462848,"data_stored":193432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.89500000000000002,"15min":0.92300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3320000000000001,"5min":1.456,"15min":1.456},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.0389999999999999,"15min":1.054},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.415,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.85099999999999998}]},{"osd":2,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.125,"15min":1.1339999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.5940000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.98899999999999999,"15min":1.0109999999999999},"min":{"1min":0.54100000000000004,"5min":0.371,"15min":0.371},"max":{"1min":1.4399999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.1679999999999999}]},{"osd":3,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0900000000000001,"15min":1.0960000000000001},"min":{"1min":0.48999999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.8560000000000001,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.119},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.319,"15min":1.343},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7929999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.5509999999999999}]},{"osd":4,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3440000000000001,"15min":1.359},"min":{"1min":0.89300000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.8180000000000001,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.444,"15min":1.478},"min":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.073,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.288}]},{"osd":5,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.5580000000000001,"15min":1.593},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.907,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.488,"15min":1.518},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.258,"5min":2.347,"15min":2.347},"last":1.4490000000000001}]},{"osd":6,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.518,"15min":1.536},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9810000000000001,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.345},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.5649999999999999,"15min":1.583},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.1080000000000001,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":1.3939999999999999}]},{"osd":7,"last update":"Mon Nov 13 10:57:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.677,"15min":1.7070000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9750000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.232},{"interface":"front","average":{"1min":1.488,"5min":1.6020000000000001,"15min":1.621},"min":{"1min":0.98499999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.0379999999999998,"5min":2.476,"15min":2.476},"last":1.5840000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":193410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.131,"15min":1.127},"min":{"1min":0.71599999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.829,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.54},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0760000000000001,"15min":1.0700000000000001},"min":{"1min":0.75900000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":0.878}]},{"osd":1,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0349999999999999,"15min":1.0209999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.252},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2310000000000001,"15min":1.226},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.8260000000000001}]},{"osd":2,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.306,"15min":1.3080000000000001},"min":{"1min":0.89200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.306},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.196,"15min":1.196},"min":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.4910000000000001}]},{"osd":3,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.24,"15min":1.2230000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.4470000000000001,"15min":1.4370000000000001},"min":{"1min":1.1519999999999999,"5min":1.036,"15min":1.036},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.7749999999999999}]},{"osd":4,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3169999999999999,"15min":1.3129999999999999},"min":{"1min":0.81499999999999995,"5min":0.745,"15min":0.745},"max":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":1.212},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.371,"15min":1.357},"min":{"1min":0.84199999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.2530000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":1.738}]},{"osd":5,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"min":{"1min":0.90000000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.548,"5min":1.476,"15min":1.464},"min":{"1min":0.98199999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.3620000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.6379999999999999}]},{"osd":6,"last update":"Mon Nov 13 10:57:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5620000000000001,"15min":1.55},"min":{"1min":1.331,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.492,"15min":1.4850000000000001},"min":{"1min":0.98099999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":1.589}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T10:57:26.140 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-13T10:57:26.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-13T10:57:26.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.412+0000 7fca8d07c640 1 Processor -- start 2023-11-13T10:57:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.412+0000 7fca8d07c640 1 -- start start 2023-11-13T10:57:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.414+0000 7fca8d07c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca88143780 0x7fca880b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.414+0000 7fca8d07c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 0x7fca8806bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.414+0000 7fca8d07c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8806c4e0 con 0x7fca8806c050 2023-11-13T10:57:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.414+0000 7fca8d07c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8806dd10 con 0x7fca88143780 2023-11-13T10:57:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.414+0000 7fca8d07c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca880b2850 con 0x7fca880b2240 2023-11-13T10:57:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.414+0000 7fca86d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca88143780 0x7fca880b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.414+0000 7fca86d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca88143780 0x7fca880b1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36430/0 (socket says 172.21.15.16:36430) 2023-11-13T10:57:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.414+0000 7fca86d9d640 1 -- 172.21.15.16:0/3173901693 learned_addr learned my addr 172.21.15.16:0/3173901693 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:26.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.414+0000 7fca8659c640 1 --2- 172.21.15.16:0/3173901693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 0x7fca8806bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.415+0000 7fca86d9d640 1 -- 172.21.15.16:0/3173901693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 msgr2=0x7fca8806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:26.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.415+0000 7fca86d9d640 1 --2- 172.21.15.16:0/3173901693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 0x7fca8806bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.415+0000 7fca86d9d640 1 -- 172.21.15.16:0/3173901693 >> v1:172.21.15.184:6789/0 conn(0x7fca8806c050 legacy=0x7fca880cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:26.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.415+0000 7fca86d9d640 1 -- 172.21.15.16:0/3173901693 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca880b2b30 con 0x7fca88143780 2023-11-13T10:57:26.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.416+0000 7fca86d9d640 1 --2- 172.21.15.16:0/3173901693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 0x7fca880b1d00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fca78009870 tx=0x7fca7802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e04e83f704463c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:26.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.416+0000 7fca85d9b640 1 -- 172.21.15.16:0/3173901693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78031740 con 0x7fca88143780 2023-11-13T10:57:26.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.416+0000 7fca85d9b640 1 -- 172.21.15.16:0/3173901693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca780318a0 con 0x7fca88143780 2023-11-13T10:57:26.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.416+0000 7fca85d9b640 1 -- 172.21.15.16:0/3173901693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78031c30 con 0x7fca88143780 2023-11-13T10:57:26.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.417+0000 7fca8d07c640 1 -- 172.21.15.16:0/3173901693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 msgr2=0x7fca880b1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.417+0000 7fca8d07c640 1 --2- 172.21.15.16:0/3173901693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 0x7fca880b1d00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fca78009870 tx=0x7fca7802b7e0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.417+0000 7fca8d07c640 1 -- 172.21.15.16:0/3173901693 shutdown_connections 2023-11-13T10:57:26.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.417+0000 7fca8d07c640 1 --2- 172.21.15.16:0/3173901693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 0x7fca8806bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.417+0000 7fca8d07c640 1 --2- 172.21.15.16:0/3173901693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 0x7fca880b1d00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.417+0000 7fca8d07c640 1 -- 172.21.15.16:0/3173901693 >> 172.21.15.16:0/3173901693 conn(0x7fca880b06d0 msgr2=0x7fca880b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:26.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.417+0000 7fca8d07c640 1 -- 172.21.15.16:0/3173901693 shutdown_connections 2023-11-13T10:57:26.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.417+0000 7fca8d07c640 1 -- 172.21.15.16:0/3173901693 wait complete. 2023-11-13T10:57:26.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.419+0000 7fca8d07c640 1 Processor -- start 2023-11-13T10:57:26.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.419+0000 7fca8d07c640 1 -- start start 2023-11-13T10:57:26.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8d07c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8806c050 0x7fca88167440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8d07c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 0x7fca8812e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8d07c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 0x7fca88165a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8d07c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca88167020 con 0x7fca88143780 2023-11-13T10:57:26.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8d07c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca88167980 con 0x7fca8806c050 2023-11-13T10:57:26.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8d07c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8812e6e0 con 0x7fca880b2240 2023-11-13T10:57:26.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 0x7fca88165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 0x7fca88165a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36446/0 (socket says 172.21.15.16:36446) 2023-11-13T10:57:26.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8759e640 1 -- 172.21.15.16:0/140422206 learned_addr learned my addr 172.21.15.16:0/140422206 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:26.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca86d9d640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8806c050 0x7fca88167440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8659c640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 0x7fca8812e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8759e640 1 -- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 msgr2=0x7fca8812e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8759e640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 0x7fca8812e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8759e640 1 -- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8806c050 msgr2=0x7fca88167440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8759e640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8806c050 0x7fca88167440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.420+0000 7fca8759e640 1 -- 172.21.15.16:0/140422206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca780094d0 con 0x7fca88143780 2023-11-13T10:57:26.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.421+0000 7fca8759e640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 0x7fca88165a30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fca8000c780 tx=0x7fca8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:26.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.421+0000 7fca6f7fe640 1 -- 172.21.15.16:0/140422206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80004b60 con 0x7fca88143780 2023-11-13T10:57:26.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.421+0000 7fca6f7fe640 1 -- 172.21.15.16:0/140422206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8000ce90 con 0x7fca88143780 2023-11-13T10:57:26.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.421+0000 7fca6f7fe640 1 -- 172.21.15.16:0/140422206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8000a7f0 con 0x7fca88143780 2023-11-13T10:57:26.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.421+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca881663a0 con 0x7fca88143780 2023-11-13T10:57:26.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.421+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca88166960 con 0x7fca88143780 2023-11-13T10:57:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.423+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca88167160 con 0x7fca88143780 2023-11-13T10:57:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.424+0000 7fca6f7fe640 1 -- 172.21.15.16:0/140422206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca8000a950 con 0x7fca88143780 2023-11-13T10:57:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.425+0000 7fca6f7fe640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca7406ae40 0x7fca7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.425+0000 7fca6f7fe640 1 -- 172.21.15.16:0/140422206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fca80089970 con 0x7fca88143780 2023-11-13T10:57:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.425+0000 7fca86d9d640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca7406ae40 0x7fca7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.426+0000 7fca86d9d640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca7406ae40 0x7fca7406d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca78035000 tx=0x7fca78008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.429+0000 7fca6f7fe640 1 -- 172.21.15.16:0/140422206 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca88167160 con 0x7fca88143780 2023-11-13T10:57:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.703+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fca88167160 con 0x7fca88143780 2023-11-13T10:57:26.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.708+0000 7fca6f7fe640 1 -- 172.21.15.16:0/140422206 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7fca88167160 con 0x7fca88143780 2023-11-13T10:57:26.716 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-13T10:57:26.716 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBdAFJl9p66LRAAYRThccLplmNpsr+W647dOA== 2023-11-13T10:57:26.716 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2023-11-13T10:57:26.716 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2023-11-13T10:57:26.717 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-13T10:57:26.718 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-11-13T10:57:26.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca7406ae40 msgr2=0x7fca7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca7406ae40 0x7fca7406d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca78035000 tx=0x7fca78008030 comp rx=0 tx=0).stop 2023-11-13T10:57:26.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 msgr2=0x7fca88165a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 0x7fca88165a30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fca8000c780 tx=0x7fca8000cc40 comp rx=0 tx=0).stop 2023-11-13T10:57:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 shutdown_connections 2023-11-13T10:57:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca7406ae40 0x7fca7406d300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca88143780 0x7fca88165a30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca880b2240 0x7fca8812e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 --2- 172.21.15.16:0/140422206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca8806c050 0x7fca88167440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 >> 172.21.15.16:0/140422206 conn(0x7fca880b06d0 msgr2=0x7fca880ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 shutdown_connections 2023-11-13T10:57:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.711+0000 7fca8d07c640 1 -- 172.21.15.16:0/140422206 wait complete. 2023-11-13T10:57:26.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T10:57:26.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.951+0000 7f9375bb8640 1 Processor -- start 2023-11-13T10:57:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.951+0000 7f9375bb8640 1 -- start start 2023-11-13T10:57:26.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.953+0000 7f9375bb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f937015cc90 0x7f937006a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.953+0000 7f9375bb8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9370070cf0 0x7f937006a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.953+0000 7f9375bb8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937006fc50 con 0x7f9370071320 2023-11-13T10:57:26.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.953+0000 7f9375bb8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937006ff30 con 0x7f9370070cf0 2023-11-13T10:57:26.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.953+0000 7f9375bb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9370070210 con 0x7f937015cc90 2023-11-13T10:57:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.954+0000 7f936ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9370070cf0 0x7f937006a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.954+0000 7f9374bb6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f937015cc90 0x7f937006a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.954+0000 7f936ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9370070cf0 0x7f937006a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:36034/0 (socket says 172.21.15.16:36034) 2023-11-13T10:57:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.954+0000 7f936ffff640 1 -- 172.21.15.16:0/775903300 learned_addr learned my addr 172.21.15.16:0/775903300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.955+0000 7f936ffff640 1 -- 172.21.15.16:0/775903300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f937015cc90 msgr2=0x7f937006a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.955+0000 7f936ffff640 1 --2- 172.21.15.16:0/775903300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f937015cc90 0x7f937006a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.955+0000 7f936ffff640 1 -- 172.21.15.16:0/775903300 >> v1:172.21.15.184:6789/0 conn(0x7f9370071320 legacy=0x7f937006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.955+0000 7f936ffff640 1 -- 172.21.15.16:0/775903300 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93700704f0 con 0x7f9370070cf0 2023-11-13T10:57:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.956+0000 7f936ffff640 1 --2- 172.21.15.16:0/775903300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9370070cf0 0x7f937006a6d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f935c009870 tx=0x7f935c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=556825a5fb603230 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.957+0000 7f936f7fe640 1 -- 172.21.15.16:0/775903300 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c030840 con 0x7f9370070cf0 2023-11-13T10:57:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.957+0000 7f936f7fe640 1 -- 172.21.15.16:0/775903300 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c0309a0 con 0x7f9370070cf0 2023-11-13T10:57:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.957+0000 7f936f7fe640 1 -- 172.21.15.16:0/775903300 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c030d30 con 0x7f9370070cf0 2023-11-13T10:57:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.958+0000 7f9375bb8640 1 -- 172.21.15.16:0/775903300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9370070cf0 msgr2=0x7f937006a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.958+0000 7f9375bb8640 1 --2- 172.21.15.16:0/775903300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9370070cf0 0x7f937006a6d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f935c009870 tx=0x7f935c02b980 comp rx=0 tx=0).stop 2023-11-13T10:57:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.958+0000 7f9375bb8640 1 -- 172.21.15.16:0/775903300 shutdown_connections 2023-11-13T10:57:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.958+0000 7f9375bb8640 1 --2- 172.21.15.16:0/775903300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9370070cf0 0x7f937006a6d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.958+0000 7f9375bb8640 1 --2- 172.21.15.16:0/775903300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f937015cc90 0x7f937006a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.958+0000 7f9375bb8640 1 -- 172.21.15.16:0/775903300 >> 172.21.15.16:0/775903300 conn(0x7f93700b04a0 msgr2=0x7f93700b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.958+0000 7f9375bb8640 1 -- 172.21.15.16:0/775903300 shutdown_connections 2023-11-13T10:57:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.958+0000 7f9375bb8640 1 -- 172.21.15.16:0/775903300 wait complete. 2023-11-13T10:57:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.960+0000 7f9375bb8640 1 Processor -- start 2023-11-13T10:57:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.960+0000 7f9375bb8640 1 -- start start 2023-11-13T10:57:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.961+0000 7f9375bb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9370070cf0 0x7f9370153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.961+0000 7f9375bb8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9370071320 0x7f9370155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.961+0000 7f9375bb8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f937015cc90 0x7f9370156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.961+0000 7f9375bb8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937015ec50 con 0x7f9370070cf0 2023-11-13T10:57:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.961+0000 7f9375bb8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937015dfd0 con 0x7f937015cc90 2023-11-13T10:57:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.961+0000 7f9375bb8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937015e2b0 con 0x7f9370071320 2023-11-13T10:57:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.961+0000 7f9374bb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9370070cf0 0x7f9370153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f9374bb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9370070cf0 0x7f9370153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36456/0 (socket says 172.21.15.16:36456) 2023-11-13T10:57:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f9374bb6640 1 -- 172.21.15.16:0/4103014258 learned_addr learned my addr 172.21.15.16:0/4103014258 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f936ffff640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9370071320 0x7f9370155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f9374bb6640 1 -- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9370071320 msgr2=0x7f9370155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f9374bb6640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9370071320 0x7f9370155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f93753b7640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f937015cc90 0x7f9370156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f9374bb6640 1 -- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f937015cc90 msgr2=0x7f9370156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:26.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f9374bb6640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f937015cc90 0x7f9370156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:26.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f9374bb6640 1 -- 172.21.15.16:0/4103014258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0094d0 con 0x7f9370070cf0 2023-11-13T10:57:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f936ffff640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9370071320 0x7f9370155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f93753b7640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f937015cc90 0x7f9370156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.962+0000 7f9374bb6640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9370070cf0 0x7f9370153d80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f936400e9b0 tx=0x7f936400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.963+0000 7f936d7fa640 1 -- 172.21.15.16:0/4103014258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364009670 con 0x7f9370070cf0 2023-11-13T10:57:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.963+0000 7f936d7fa640 1 -- 172.21.15.16:0/4103014258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936400c9a0 con 0x7f9370070cf0 2023-11-13T10:57:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.963+0000 7f936d7fa640 1 -- 172.21.15.16:0/4103014258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400cd50 con 0x7f9370070cf0 2023-11-13T10:57:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.963+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937015e590 con 0x7f9370070cf0 2023-11-13T10:57:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.963+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93701542c0 con 0x7f9370070cf0 2023-11-13T10:57:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.964+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937004e4d0 con 0x7f9370070cf0 2023-11-13T10:57:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.965+0000 7f936d7fa640 1 -- 172.21.15.16:0/4103014258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9364013470 con 0x7f9370070cf0 2023-11-13T10:57:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.966+0000 7f936d7fa640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f935406ae40 0x7f935406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.966+0000 7f936d7fa640 1 -- 172.21.15.16:0/4103014258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9364089a40 con 0x7f9370070cf0 2023-11-13T10:57:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.967+0000 7f936ffff640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f935406ae40 0x7f935406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.968+0000 7f936ffff640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f935406ae40 0x7f935406d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f935c036000 tx=0x7f935c03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:26.970+0000 7f936d7fa640 1 -- 172.21.15.16:0/4103014258 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9364054910 con 0x7f9370070cf0 2023-11-13T10:57:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.206+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9370072c80 con 0x7f9370070cf0 2023-11-13T10:57:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.207+0000 7f936d7fa640 1 -- 172.21.15.16:0/4103014258 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f9364017090 con 0x7f9370070cf0 2023-11-13T10:57:27.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T10:57:27.213 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3678615052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3678615052},{"type":"v1","addr":"172.21.15.16:6833","nonce":3678615052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3358463124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3358463124},{"type":"v1","addr":"172.21.15.16:6835","nonce":3358463124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2329286743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2329286743},{"type":"v1","addr":"172.21.15.184:6836","nonce":2329286743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T10:57:20.523743+0000","modified":"2023-11-13T10:57:23.616898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6837/3171288994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3171288994},{"type":"v1","addr":"172.21.15.184:6837","nonce":3171288994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-13T10:57:27.213 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2023-11-13T10:57:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f935406ae40 msgr2=0x7f935406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f935406ae40 0x7f935406d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f935c036000 tx=0x7f935c03b040 comp rx=0 tx=0).stop 2023-11-13T10:57:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9370070cf0 msgr2=0x7f9370153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9370070cf0 0x7f9370153d80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f936400e9b0 tx=0x7f936400ee70 comp rx=0 tx=0).stop 2023-11-13T10:57:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 shutdown_connections 2023-11-13T10:57:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f935406ae40 0x7f935406d300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f937015cc90 0x7f9370156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9370071320 0x7f9370155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 --2- 172.21.15.16:0/4103014258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9370070cf0 0x7f9370153d80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.210+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 >> 172.21.15.16:0/4103014258 conn(0x7f93700b04a0 msgr2=0x7f93700cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.211+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 shutdown_connections 2023-11-13T10:57:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.211+0000 7f9375bb8640 1 -- 172.21.15.16:0/4103014258 wait complete. 2023-11-13T10:57:27.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-13T10:57:20.523743+0000', 'modified': '2023-11-13T10:57:23.616898+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4990': {'gid': 4990, 'name': 'd', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.184:6837/3171288994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6835', 'nonce': 3171288994}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 3171288994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-13T10:57:27.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-13T10:57:27.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-13T10:57:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.870+0000 7fbb2e421640 1 Processor -- start 2023-11-13T10:57:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.870+0000 7fbb2e421640 1 -- start start 2023-11-13T10:57:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.872+0000 7fbb2e421640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb28155410 0x7fbb280ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.872+0000 7fbb2e421640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb28071b50 0x7fbb280cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.872+0000 7fbb2e421640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2815b500 con 0x7fbb280ce990 2023-11-13T10:57:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.872+0000 7fbb2e421640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2815b7e0 con 0x7fbb28155410 2023-11-13T10:57:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.872+0000 7fbb2e421640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2815bac0 con 0x7fbb28071b50 2023-11-13T10:57:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.872+0000 7fbb27fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb28155410 0x7fbb280ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.872+0000 7fbb277fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb280ce990 0x7fbb28071440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57744/0 (socket says 172.21.15.16:57744) 2023-11-13T10:57:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.872+0000 7fbb277fe640 1 -- 172.21.15.16:0/798018584 learned_addr learned my addr 172.21.15.16:0/798018584 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T10:57:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.873+0000 7fbb2c9be640 1 --2- 172.21.15.16:0/798018584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb28071b50 0x7fbb280cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.873+0000 7fbb26ffd640 1 -- 172.21.15.16:0/798018584 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893862496 0 0) 0x7fbb2815b500 con 0x7fbb280ce990 2023-11-13T10:57:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.873+0000 7fbb26ffd640 1 -- 172.21.15.16:0/798018584 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb10003520 con 0x7fbb280ce990 2023-11-13T10:57:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.873+0000 7fbb27fff640 1 -- 172.21.15.16:0/798018584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb28071b50 msgr2=0x7fbb280cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.873+0000 7fbb27fff640 1 --2- 172.21.15.16:0/798018584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb28071b50 0x7fbb280cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.873+0000 7fbb27fff640 1 -- 172.21.15.16:0/798018584 >> v1:172.21.15.16:6789/0 conn(0x7fbb280ce990 legacy=0x7fbb28071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.873+0000 7fbb27fff640 1 -- 172.21.15.16:0/798018584 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2815bda0 con 0x7fbb28155410 2023-11-13T10:57:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.875+0000 7fbb27fff640 1 --2- 172.21.15.16:0/798018584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28155410 0x7fbb280ccc60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097a0 tx=0x7fbb1802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8757940cf1c24c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.875+0000 7fbb26ffd640 1 -- 172.21.15.16:0/798018584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18031830 con 0x7fbb28155410 2023-11-13T10:57:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.875+0000 7fbb26ffd640 1 -- 172.21.15.16:0/798018584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18031990 con 0x7fbb28155410 2023-11-13T10:57:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.875+0000 7fbb26ffd640 1 -- 172.21.15.16:0/798018584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18031d20 con 0x7fbb28155410 2023-11-13T10:57:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.875+0000 7fbb2e421640 1 -- 172.21.15.16:0/798018584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28155410 msgr2=0x7fbb280ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.875+0000 7fbb2e421640 1 --2- 172.21.15.16:0/798018584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28155410 0x7fbb280ccc60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097a0 tx=0x7fbb1802b8d0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.876+0000 7fbb2e421640 1 -- 172.21.15.16:0/798018584 shutdown_connections 2023-11-13T10:57:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.876+0000 7fbb2e421640 1 --2- 172.21.15.16:0/798018584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb28071b50 0x7fbb280cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.876+0000 7fbb2e421640 1 --2- 172.21.15.16:0/798018584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28155410 0x7fbb280ccc60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.876+0000 7fbb2e421640 1 -- 172.21.15.16:0/798018584 >> 172.21.15.16:0/798018584 conn(0x7fbb2806a190 msgr2=0x7fbb2806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.876+0000 7fbb2e421640 1 -- 172.21.15.16:0/798018584 shutdown_connections 2023-11-13T10:57:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.876+0000 7fbb2e421640 1 -- 172.21.15.16:0/798018584 wait complete. 2023-11-13T10:57:27.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.877+0000 7fbb2e421640 1 Processor -- start 2023-11-13T10:57:27.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.878+0000 7fbb2e421640 1 -- start start 2023-11-13T10:57:27.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.878+0000 7fbb2e421640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28071b50 0x7fbb28157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.878+0000 7fbb2e421640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb280ce990 0x7fbb28165180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb2e421640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb28155410 0x7fbb280aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb2e421640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb281575c0 con 0x7fbb28071b50 2023-11-13T10:57:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb2e421640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2814ad10 con 0x7fbb28155410 2023-11-13T10:57:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb2e421640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28157ec0 con 0x7fbb280ce990 2023-11-13T10:57:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28071b50 0x7fbb28157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28071b50 0x7fbb28157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36484/0 (socket says 172.21.15.16:36484) 2023-11-13T10:57:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb27fff640 1 -- 172.21.15.16:0/2305492785 learned_addr learned my addr 172.21.15.16:0/2305492785 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb27fff640 1 -- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb280ce990 msgr2=0x7fbb28165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb27fff640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb280ce990 0x7fbb28165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb27fff640 1 -- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb28155410 msgr2=0x7fbb280aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb27fff640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb28155410 0x7fbb280aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.879+0000 7fbb27fff640 1 -- 172.21.15.16:0/2305492785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18009400 con 0x7fbb28071b50 2023-11-13T10:57:27.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.880+0000 7fbb27fff640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28071b50 0x7fbb28157980 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000c00 tx=0x7fbb180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:27.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.880+0000 7fbb24ff9640 1 -- 172.21.15.16:0/2305492785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1803a830 con 0x7fbb28071b50 2023-11-13T10:57:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.880+0000 7fbb24ff9640 1 -- 172.21.15.16:0/2305492785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1803a9b0 con 0x7fbb28071b50 2023-11-13T10:57:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.880+0000 7fbb24ff9640 1 -- 172.21.15.16:0/2305492785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1803ad60 con 0x7fbb28071b50 2023-11-13T10:57:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.880+0000 7fbb2e421640 1 -- 172.21.15.16:0/2305492785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb28159300 con 0x7fbb28071b50 2023-11-13T10:57:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.880+0000 7fbb2e421640 1 -- 172.21.15.16:0/2305492785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb281598c0 con 0x7fbb28071b50 2023-11-13T10:57:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.882+0000 7fbb24ff9640 1 -- 172.21.15.16:0/2305492785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb1804d450 con 0x7fbb28071b50 2023-11-13T10:57:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.883+0000 7fbb24ff9640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1406ae90 0x7fbb1406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.883+0000 7fbb24ff9640 1 -- 172.21.15.16:0/2305492785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbb180ab850 con 0x7fbb28071b50 2023-11-13T10:57:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.884+0000 7fbb30fbc740 1 Processor -- start 2023-11-13T10:57:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.884+0000 7fbb30fbc740 1 -- start start 2023-11-13T10:57:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.884+0000 7fbb277fe640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1406ae90 0x7fbb1406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57b9881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b982ae0 0x55a57b986fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57b9cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57b8e6880 con 0x55a57b987de0 2023-11-13T10:57:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57b8f5ae0 con 0x55a57b982ae0 2023-11-13T10:57:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a57b8f31d0 con 0x55a57b987520 2023-11-13T10:57:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57b9881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57b9881c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36492/0 (socket says 172.21.15.16:36492) 2023-11-13T10:57:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb27fff640 1 -- 172.21.15.16:0/3028265555 learned_addr learned my addr 172.21.15.16:0/3028265555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb27fff640 1 -- 172.21.15.16:0/3028265555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 msgr2=0x55a57b9cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb27fff640 1 --2- 172.21.15.16:0/3028265555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57b9cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb27fff640 1 -- 172.21.15.16:0/3028265555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b982ae0 msgr2=0x55a57b986fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb27fff640 1 --2- 172.21.15.16:0/3028265555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b982ae0 0x55a57b986fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb277fe640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1406ae90 0x7fbb1406d350 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c000f60 tx=0x7fbb1c0078f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.885+0000 7fbb27fff640 1 -- 172.21.15.16:0/3028265555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a57b8b6860 con 0x55a57b987de0 2023-11-13T10:57:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.886+0000 7fbb27fff640 1 --2- 172.21.15.16:0/3028265555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57b9881c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbb18082230 tx=0x7fbb1807ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b960fd643a895f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.886+0000 7fbb0e7fc640 1 -- 172.21.15.16:0/3028265555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18082760 con 0x55a57b987de0 2023-11-13T10:57:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.886+0000 7fbb0e7fc640 1 -- 172.21.15.16:0/3028265555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18078800 con 0x55a57b987de0 2023-11-13T10:57:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.886+0000 7fbb0e7fc640 1 -- 172.21.15.16:0/3028265555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1807a6c0 con 0x55a57b987de0 2023-11-13T10:57:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 -- 172.21.15.16:0/3028265555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 msgr2=0x55a57b9881c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/3028265555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57b9881c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbb18082230 tx=0x7fbb1807ab60 comp rx=0 tx=0).stop 2023-11-13T10:57:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 -- 172.21.15.16:0/3028265555 shutdown_connections 2023-11-13T10:57:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/3028265555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57b9cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/3028265555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b982ae0 0x55a57b986fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/3028265555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57b9881c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 -- 172.21.15.16:0/3028265555 >> 172.21.15.16:0/3028265555 conn(0x55a57b949bb0 msgr2=0x55a57b9cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 -- 172.21.15.16:0/3028265555 shutdown_connections 2023-11-13T10:57:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 -- 172.21.15.16:0/3028265555 wait complete. 2023-11-13T10:57:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.887+0000 7fbb30fbc740 1 -- start start 2023-11-13T10:57:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.888+0000 7fbb30fbc740 1 Processor -- start 2023-11-13T10:57:27.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b982ae0 0x55a57b9acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57b9d5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 0x55a57b9dd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57b8eeac0 con 0x55a57b982ae0 2023-11-13T10:57:27.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57b88b2f0 con 0x55a57b987de0 2023-11-13T10:57:27.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a57b8a8c70 con 0x55a57b987520 2023-11-13T10:57:27.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb30fbc740 10 client.0 _ll_register_callbacks cb 0x55a57b8ad3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T10:57:27.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b982ae0 0x55a57b9acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b982ae0 0x55a57b9acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36498/0 (socket says 172.21.15.16:36498) 2023-11-13T10:57:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb27fff640 1 -- 172.21.15.16:0/2386026451 learned_addr learned my addr 172.21.15.16:0/2386026451 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb27fff640 1 -- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 msgr2=0x55a57b9d5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb27fff640 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57b9d5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb27fff640 1 -- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 msgr2=0x55a57b9dd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb27fff640 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 0x55a57b9dd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.889+0000 7fbb27fff640 1 -- 172.21.15.16:0/2386026451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb281598c0 con 0x55a57b982ae0 2023-11-13T10:57:27.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb27fff640 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b982ae0 0x55a57b9acb80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbb1807a620 tx=0x7fbb18009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4258ab889022fc server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:27.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T10:57:27.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2386026451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18079b50 con 0x55a57b982ae0 2023-11-13T10:57:27.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2386026451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbb1807a6c0 con 0x55a57b982ae0 2023-11-13T10:57:27.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2386026451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a57b8e7aa0 con 0x55a57b982ae0 2023-11-13T10:57:27.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2386026451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb1807a880 con 0x55a57b982ae0 2023-11-13T10:57:27.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb30fbc740 10 client.5679 fetch_fsmap learned FSMap version 18 2023-11-13T10:57:27.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2386026451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55a57b8ed140 con 0x55a57b982ae0 2023-11-13T10:57:27.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2386026451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1809b080 con 0x55a57b982ae0 2023-11-13T10:57:27.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.890+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2386026451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb18091070 con 0x55a57b982ae0 2023-11-13T10:57:27.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.891+0000 7fbb30fbc740 10 client.5679 fetch_fsmap finished waiting for FSMap version 18 2023-11-13T10:57:27.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.891+0000 7fbb30fbc740 10 client.5679 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-13T10:57:27.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.891+0000 7fbb30fbc740 20 client.5679 populate_metadata read hostname 'smithi016' 2023-11-13T10:57:27.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.891+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x55a57b9d0910 0x55a57b9d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.891+0000 7fbb30fbc740 4 client.5679 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T10:57:27.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.891+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2386026451 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a57b8e77c0 con 0x55a57b9d0910 2023-11-13T10:57:27.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.891+0000 7fbb277fe640 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x55a57b9d0910 0x55a57b9d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.892+0000 7fbb277fe640 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x55a57b9d0910 0x55a57b9d2620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec3167558e55eeb server_cookie=7ef8436e3e46d3ed in_seq=0 out_seq=0 2023-11-13T10:57:27.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.892+0000 7fbb06ffd640 10 client.5679 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T10:57:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.894+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2386026451 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a57b8e77c0 con 0x55a57b9d0910 2023-11-13T10:57:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.894+0000 7fbb06ffd640 10 client.5679 handle_command_reply: tid=0 2023-11-13T10:57:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.894+0000 7fbb30fbc740 1 client.5679 shutdown 2023-11-13T10:57:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.894+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2386026451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b982ae0 msgr2=0x55a57b9acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.894+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b982ae0 0x55a57b9acb80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbb1807a620 tx=0x7fbb18009110 comp rx=0 tx=0).stop 2023-11-13T10:57:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2386026451 shutdown_connections 2023-11-13T10:57:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 0x55a57b9dd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57b9d5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x55a57b9d0910 0x55a57b9d2620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2386026451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b982ae0 0x55a57b9acb80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb06ffd640 0 client.5679 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T10:57:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2386026451 >> 172.21.15.16:0/2386026451 conn(0x55a57b949bb0 msgr2=0x55a57b981ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2386026451 shutdown_connections 2023-11-13T10:57:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2386026451 wait complete. 2023-11-13T10:57:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.895+0000 7fbb30fbc740 20 client.5679 trim_cache size 0 max 16384 2023-11-13T10:57:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.912+0000 7fbb30fbc740 1 Processor -- start 2023-11-13T10:57:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.912+0000 7fbb30fbc740 1 -- start start 2023-11-13T10:57:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.913+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b982ae0 0x55a57b9a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.913+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57ba3ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.913+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57ba3b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.913+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57b651f20 con 0x55a57b987de0 2023-11-13T10:57:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.913+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57b8f0a40 con 0x55a57b982ae0 2023-11-13T10:57:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.913+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a57b9cd220 con 0x55a57b987520 2023-11-13T10:57:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.913+0000 7fbb2c9be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57ba3b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.913+0000 7fbb2c9be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57ba3b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36512/0 (socket says 172.21.15.16:36512) 2023-11-13T10:57:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.913+0000 7fbb2c9be640 1 -- 172.21.15.16:0/539674179 learned_addr learned my addr 172.21.15.16:0/539674179 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.914+0000 7fbb27fff640 1 --2- 172.21.15.16:0/539674179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b982ae0 0x55a57b9a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.914+0000 7fbb277fe640 1 --2- 172.21.15.16:0/539674179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57ba3ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.914+0000 7fbb2c9be640 1 -- 172.21.15.16:0/539674179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 msgr2=0x55a57ba3ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.914+0000 7fbb2c9be640 1 --2- 172.21.15.16:0/539674179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57ba3ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.914+0000 7fbb2c9be640 1 -- 172.21.15.16:0/539674179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b982ae0 msgr2=0x55a57b9a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.914+0000 7fbb2c9be640 1 --2- 172.21.15.16:0/539674179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b982ae0 0x55a57b9a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.914+0000 7fbb2c9be640 1 -- 172.21.15.16:0/539674179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a57b88ab80 con 0x55a57b987de0 2023-11-13T10:57:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.914+0000 7fbb2c9be640 1 --2- 172.21.15.16:0/539674179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57ba3b280 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009e20 tx=0x7fbb200075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510b527ab87be0a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb0effd640 1 -- 172.21.15.16:0/539674179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb200135f0 con 0x55a57b987de0 2023-11-13T10:57:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb0effd640 1 -- 172.21.15.16:0/539674179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2000f040 con 0x55a57b987de0 2023-11-13T10:57:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb0effd640 1 -- 172.21.15.16:0/539674179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20013990 con 0x55a57b987de0 2023-11-13T10:57:27.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb30fbc740 1 -- 172.21.15.16:0/539674179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 msgr2=0x55a57ba3b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/539674179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57ba3b280 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009e20 tx=0x7fbb200075c0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb30fbc740 1 -- 172.21.15.16:0/539674179 shutdown_connections 2023-11-13T10:57:27.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/539674179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987de0 0x55a57ba3b280 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/539674179 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b987520 0x55a57ba3ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/539674179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b982ae0 0x55a57b9a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb30fbc740 1 -- 172.21.15.16:0/539674179 >> 172.21.15.16:0/539674179 conn(0x55a57b949bb0 msgr2=0x55a57b97e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:27.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb30fbc740 1 -- 172.21.15.16:0/539674179 shutdown_connections 2023-11-13T10:57:27.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.915+0000 7fbb30fbc740 1 -- 172.21.15.16:0/539674179 wait complete. 2023-11-13T10:57:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.916+0000 7fbb30fbc740 1 -- start start 2023-11-13T10:57:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.916+0000 7fbb30fbc740 1 Processor -- start 2023-11-13T10:57:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b982ae0 0x55a57ba3f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987520 0x55a57b981ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb30fbc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 0x55a57b982000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57b9d35a0 con 0x55a57b987520 2023-11-13T10:57:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57b9a8f50 con 0x55a57b987de0 2023-11-13T10:57:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb30fbc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a57ba3fcb0 con 0x55a57b982ae0 2023-11-13T10:57:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb30fbc740 10 client.0 _ll_register_callbacks cb 0x7fbb1807a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T10:57:27.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987520 0x55a57b981ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987520 0x55a57b981ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36514/0 (socket says 172.21.15.16:36514) 2023-11-13T10:57:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.917+0000 7fbb277fe640 1 -- 172.21.15.16:0/2990208622 learned_addr learned my addr 172.21.15.16:0/2990208622 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb27fff640 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b982ae0 0x55a57ba3f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb2c9be640 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 0x55a57b982000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb277fe640 1 -- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b982ae0 msgr2=0x55a57ba3f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb277fe640 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b982ae0 0x55a57ba3f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb277fe640 1 -- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 msgr2=0x55a57b982000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb277fe640 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 0x55a57b982000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb277fe640 1 -- 172.21.15.16:0/2990208622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb20009ad0 con 0x55a57b987520 2023-11-13T10:57:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb2c9be640 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 0x55a57b982000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb277fe640 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987520 0x55a57b981ac0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0011e0 tx=0x7fbb1c005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0aa0eac76acf3b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T10:57:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2990208622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0076a0 con 0x55a57b987520 2023-11-13T10:57:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2990208622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbb1c012740 con 0x55a57b987520 2023-11-13T10:57:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.918+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2990208622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb18078800 con 0x55a57b987520 2023-11-13T10:57:27.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2990208622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb1c012900 con 0x55a57b987520 2023-11-13T10:57:27.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb30fbc740 10 client.5685 fetch_fsmap learned FSMap version 18 2023-11-13T10:57:27.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2990208622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55a57b8b0170 con 0x55a57b987520 2023-11-13T10:57:27.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2990208622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18078800 con 0x55a57b987520 2023-11-13T10:57:27.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2990208622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb1c01fd20 con 0x55a57b987520 2023-11-13T10:57:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb30fbc740 10 client.5685 fetch_fsmap finished waiting for FSMap version 18 2023-11-13T10:57:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb30fbc740 10 client.5685 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-13T10:57:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb30fbc740 20 client.5685 populate_metadata read hostname 'smithi016' 2023-11-13T10:57:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x55a57b9c3ca0 0x55a57b9d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb30fbc740 4 client.5685 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-13T10:57:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.919+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2990208622 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a57b5ef6c0 con 0x55a57b9c3ca0 2023-11-13T10:57:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.920+0000 7fbb27fff640 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x55a57b9c3ca0 0x55a57b9d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.921+0000 7fbb27fff640 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x55a57b9c3ca0 0x55a57b9d2260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6931f8167a66ee27 server_cookie=23787bc2e828f567 in_seq=0 out_seq=0 2023-11-13T10:57:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.921+0000 7fbb06ffd640 10 client.5685 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T10:57:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.921+0000 7fbb06ffd640 1 -- 172.21.15.16:0/2990208622 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55a57b5ef6c0 con 0x55a57b9c3ca0 2023-11-13T10:57:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.921+0000 7fbb06ffd640 10 client.5685 handle_command_reply: tid=0 2023-11-13T10:57:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.921+0000 7fbb30fbc740 1 client.5685 shutdown 2023-11-13T10:57:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.921+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2990208622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987520 msgr2=0x55a57b981ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.921+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987520 0x55a57b981ac0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0011e0 tx=0x7fbb1c005c60 comp rx=0 tx=0).stop 2023-11-13T10:57:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2990208622 shutdown_connections 2023-11-13T10:57:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a57b987de0 0x55a57b982000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x55a57b9c3ca0 0x55a57b9d2260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a57b987520 0x55a57b981ac0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb30fbc740 1 --2- 172.21.15.16:0/2990208622 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a57b982ae0 0x55a57ba3f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb06ffd640 0 client.5685 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T10:57:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2990208622 >> 172.21.15.16:0/2990208622 conn(0x55a57b949bb0 msgr2=0x55a57b9cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2990208622 shutdown_connections 2023-11-13T10:57:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb30fbc740 1 -- 172.21.15.16:0/2990208622 wait complete. 2023-11-13T10:57:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb30fbc740 20 client.5685 trim_cache size 0 max 16384 2023-11-13T10:57:27.981 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T10:57:27.981 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9", 2023-11-13T10:57:27.981 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-13T10:57:27.982 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4990, 2023-11-13T10:57:27.982 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-13T10:57:27.982 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-13T10:57:27.982 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-13T10:57:27.982 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.8450714990000003, 2023-11-13T10:57:27.983 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 18, 2023-11-13T10:57:27.983 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 40, 2023-11-13T10:57:27.983 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 40, 2023-11-13T10:57:27.983 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.82709382 2023-11-13T10:57:27.983 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T10:57:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb2e421640 1 -- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1406ae90 msgr2=0x7fbb1406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb2e421640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1406ae90 0x7fbb1406d350 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c000f60 tx=0x7fbb1c0078f0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb2e421640 1 -- 172.21.15.16:0/2305492785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28071b50 msgr2=0x7fbb28157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.922+0000 7fbb2e421640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28071b50 0x7fbb28157980 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000c00 tx=0x7fbb180046c0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.923+0000 7fbb2e421640 1 -- 172.21.15.16:0/2305492785 shutdown_connections 2023-11-13T10:57:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.923+0000 7fbb2e421640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbb1406ae90 0x7fbb1406d350 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.923+0000 7fbb2e421640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb28155410 0x7fbb280aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.923+0000 7fbb2e421640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb280ce990 0x7fbb28165180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.923+0000 7fbb2e421640 1 --2- 172.21.15.16:0/2305492785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb28071b50 0x7fbb28157980 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.923+0000 7fbb2e421640 1 -- 172.21.15.16:0/2305492785 >> 172.21.15.16:0/2305492785 conn(0x7fbb2806a190 msgr2=0x7fbb280b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.923+0000 7fbb2e421640 1 -- 172.21.15.16:0/2305492785 shutdown_connections 2023-11-13T10:57:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:27.923+0000 7fbb2e421640 1 -- 172.21.15.16:0/2305492785 wait complete. 2023-11-13T10:57:27.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-13T10:57:28.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.236+0000 7f8aaaaf7640 1 Processor -- start 2023-11-13T10:57:28.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.236+0000 7f8aaaaf7640 1 -- start start 2023-11-13T10:57:28.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.238+0000 7f8aaaaf7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8aa415d9f0 0x7f8aa406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:28.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.238+0000 7f8aaaaf7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8aa415ddd0 0x7f8aa415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:28.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.238+0000 7f8aaaaf7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 0x7f8aa40b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:28.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.238+0000 7f8aaaaf7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa415b070 con 0x7f8aa415ddd0 2023-11-13T10:57:28.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.238+0000 7f8aaaaf7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa415aab0 con 0x7f8aa415d9f0 2023-11-13T10:57:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.238+0000 7f8aaaaf7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa415ad90 con 0x7f8aa415a610 2023-11-13T10:57:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8a9bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8aa415ddd0 0x7f8aa415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8a9bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8aa415ddd0 0x7f8aa415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36524/0 (socket says 172.21.15.16:36524) 2023-11-13T10:57:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8a9bfff640 1 -- 172.21.15.16:0/4149386582 learned_addr learned my addr 172.21.15.16:0/4149386582 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8aa8893640 1 --2- 172.21.15.16:0/4149386582 >> v2:172.21.15.184:3300/0 conn(0x7f8aa415d9f0 0x7f8aa406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8aa9094640 1 --2- 172.21.15.16:0/4149386582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 0x7f8aa40b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8a9bfff640 1 -- 172.21.15.16:0/4149386582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 msgr2=0x7f8aa40b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8a9bfff640 1 --2- 172.21.15.16:0/4149386582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 0x7f8aa40b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8a9bfff640 1 -- 172.21.15.16:0/4149386582 >> v2:172.21.15.184:3300/0 conn(0x7f8aa415d9f0 msgr2=0x7f8aa406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8a9bfff640 1 --2- 172.21.15.16:0/4149386582 >> v2:172.21.15.184:3300/0 conn(0x7f8aa415d9f0 0x7f8aa406d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8a9bfff640 1 -- 172.21.15.16:0/4149386582 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa406cca0 con 0x7f8aa415ddd0 2023-11-13T10:57:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.239+0000 7f8aa8893640 1 --2- 172.21.15.16:0/4149386582 >> v2:172.21.15.184:3300/0 conn(0x7f8aa415d9f0 0x7f8aa406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T10:57:28.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.240+0000 7f8a9bfff640 1 --2- 172.21.15.16:0/4149386582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415ddd0 0x7f8aa415a0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40b2600 tx=0x7f8a9402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b237fd822ca9c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:28.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.240+0000 7f8a9b7fe640 1 -- 172.21.15.16:0/4149386582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a940317e0 con 0x7f8aa415ddd0 2023-11-13T10:57:28.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8a9b7fe640 1 -- 172.21.15.16:0/4149386582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a94031940 con 0x7f8aa415ddd0 2023-11-13T10:57:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8a9b7fe640 1 -- 172.21.15.16:0/4149386582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94031cd0 con 0x7f8aa415ddd0 2023-11-13T10:57:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/4149386582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415ddd0 msgr2=0x7f8aa415a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/4149386582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415ddd0 0x7f8aa415a0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40b2600 tx=0x7f8a9402b880 comp rx=0 tx=0).stop 2023-11-13T10:57:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/4149386582 shutdown_connections 2023-11-13T10:57:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/4149386582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 0x7f8aa40b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/4149386582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415ddd0 0x7f8aa415a0d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/4149386582 >> v2:172.21.15.184:3300/0 conn(0x7f8aa415d9f0 0x7f8aa406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/4149386582 >> 172.21.15.16:0/4149386582 conn(0x7f8aa406a190 msgr2=0x7f8aa406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/4149386582 shutdown_connections 2023-11-13T10:57:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.241+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/4149386582 wait complete. 2023-11-13T10:57:28.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.243+0000 7f8aaaaf7640 1 Processor -- start 2023-11-13T10:57:28.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.243+0000 7f8aaaaf7640 1 -- start start 2023-11-13T10:57:28.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.244+0000 7f8aaaaf7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 0x7f8aa4166d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:28.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.244+0000 7f8aaaaf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415d9f0 0x7f8aa4147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.244+0000 7f8aaaaf7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8aa415ddd0 0x7f8aa41663a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.244+0000 7f8aaaaf7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa4147210 con 0x7f8aa415d9f0 2023-11-13T10:57:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.244+0000 7f8aaaaf7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa41668e0 con 0x7f8aa415ddd0 2023-11-13T10:57:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.244+0000 7f8aaaaf7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa4146870 con 0x7f8aa415a610 2023-11-13T10:57:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.244+0000 7f8a9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415d9f0 0x7f8aa4147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.244+0000 7f8a9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415d9f0 0x7f8aa4147630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36532/0 (socket says 172.21.15.16:36532) 2023-11-13T10:57:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.245+0000 7f8a9bfff640 1 -- 172.21.15.16:0/1867833669 learned_addr learned my addr 172.21.15.16:0/1867833669 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.245+0000 7f8aa8893640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 0x7f8aa4166d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.245+0000 7f8aa9094640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8aa415ddd0 0x7f8aa41663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.245+0000 7f8a9bfff640 1 -- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 msgr2=0x7f8aa4166d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.245+0000 7f8a9bfff640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 0x7f8aa4166d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.245+0000 7f8a9bfff640 1 -- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8aa415ddd0 msgr2=0x7f8aa41663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.245+0000 7f8a9bfff640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8aa415ddd0 0x7f8aa41663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.245+0000 7f8a9bfff640 1 -- 172.21.15.16:0/1867833669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a940094d0 con 0x7f8aa415d9f0 2023-11-13T10:57:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.245+0000 7f8a9bfff640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415d9f0 0x7f8aa4147630 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8a9402b8c0 tx=0x7f8a940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.246+0000 7f8a997fa640 1 -- 172.21.15.16:0/1867833669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94032880 con 0x7f8aa415d9f0 2023-11-13T10:57:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.246+0000 7f8a997fa640 1 -- 172.21.15.16:0/1867833669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a94032a20 con 0x7f8aa415d9f0 2023-11-13T10:57:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.246+0000 7f8a997fa640 1 -- 172.21.15.16:0/1867833669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94032dd0 con 0x7f8aa415d9f0 2023-11-13T10:57:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.246+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa4146b50 con 0x7f8aa415d9f0 2023-11-13T10:57:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.246+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa4145eb0 con 0x7f8aa415d9f0 2023-11-13T10:57:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.247+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa4147350 con 0x7f8aa415d9f0 2023-11-13T10:57:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.249+0000 7f8a997fa640 1 -- 172.21.15.16:0/1867833669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a940453f0 con 0x7f8aa415d9f0 2023-11-13T10:57:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.250+0000 7f8a997fa640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a9006ae40 0x7f8a9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.250+0000 7f8a997fa640 1 -- 172.21.15.16:0/1867833669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8a940ab8c0 con 0x7f8aa415d9f0 2023-11-13T10:57:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.250+0000 7f8aa8893640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a9006ae40 0x7f8a9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.252+0000 7f8aa8893640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a9006ae40 0x7f8a9006d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00075b0 tx=0x7f8aa0007540 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.255+0000 7f8a997fa640 1 -- 172.21.15.16:0/1867833669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8aa4147350 con 0x7f8aa415d9f0 2023-11-13T10:57:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.489+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8aa4147350 con 0x7f8aa415d9f0 2023-11-13T10:57:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.490+0000 7f8a997fa640 1 -- 172.21.15.16:0/1867833669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8aa4147350 con 0x7f8aa415d9f0 2023-11-13T10:57:28.496 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-13T10:57:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a9006ae40 msgr2=0x7f8a9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a9006ae40 0x7f8a9006d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00075b0 tx=0x7f8aa0007540 comp rx=0 tx=0).stop 2023-11-13T10:57:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415d9f0 msgr2=0x7f8aa4147630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415d9f0 0x7f8aa4147630 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8a9402b8c0 tx=0x7f8a940046c0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 shutdown_connections 2023-11-13T10:57:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a9006ae40 0x7f8a9006d300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8aa415ddd0 0x7f8aa41663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa415d9f0 0x7f8aa4147630 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 --2- 172.21.15.16:0/1867833669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8aa415a610 0x7f8aa4166d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 >> 172.21.15.16:0/1867833669 conn(0x7f8aa406a190 msgr2=0x7f8aa4153510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 shutdown_connections 2023-11-13T10:57:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:28.493+0000 7f8aaaaf7640 1 -- 172.21.15.16:0/1867833669 wait complete. 2023-11-13T10:57:28.516 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-13T10:57:28.516 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-13T10:57:28.517 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-13T10:57:28.517 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-13T10:57:28.518 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-13T10:57:28.519 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-13T10:57:28.519 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-13T10:57:28.519 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-13T10:57:28.520 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-13T10:57:28.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:28.521 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-13T10:57:28.575 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T10:57:28.576 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T10:57:28.576 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T10:57:28.576 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:57:28.576 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-13T10:57:28.576 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:57:28.577 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:57:28.577 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:57:28.577 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-13T10:57:28.577 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:57:28.578 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:57:28.578 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-13T10:57:28.578 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-13T10:57:28.578 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:57:28.578 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-13T10:57:28.579 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-13T10:57:28.579 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42194sec preferred_lft 42194sec 2023-11-13T10:57:28.579 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-13T10:57:28.579 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:57:28.580 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T10:57:28.580 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:57:28.580 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-13T10:57:28.581 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-13T10:57:28.581 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1a:84:8e:80:25:76 brd ff:ff:ff:ff:ff:ff 2023-11-13T10:57:28.581 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-13T10:57:28.581 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:57:28.582 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5426:dff:feb9:4760/64 scope link 2023-11-13T10:57:28.582 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:57:28.583 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-13T10:57:28.584 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1a:84:8e:80:25:76 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:57:28.584 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::1884:8eff:fe80:2576/64 scope link 2023-11-13T10:57:28.584 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:57:28.584 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-13T10:57:28.584 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether ae:42:49:3f:88:fc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:57:28.585 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ac42:49ff:fe3f:88fc/64 scope link 2023-11-13T10:57:28.585 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T10:57:28.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:28.587 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-13T10:57:28.684 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-13T10:57:28.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:28.691 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-13T10:57:28.750 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-13T10:57:28.751 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-13T10:57:28.751 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-13T10:57:28.751 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-13T10:57:28.751 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-13T10:57:28.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:28.752 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T10:57:28.752 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T10:57:28.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-13T10:57:28.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-13T10:57:28.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-13T10:57:28.754 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T10:57:28.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:28.962 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-13T10:57:29.020 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-13T10:57:29.021 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-13T10:57:29.021 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-13T10:57:29.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:29.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-13T10:57:29.117 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-13T10:57:29.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:57:29.118 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-13T10:57:29.178 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-13T10:57:29.178 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T10:57:29.179 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-13T10:57:29.230 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T10:57:29.328 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:57:29.394 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T10:57:29.394 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T10:57:29.394 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T10:57:29.395 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T10:57:29.395 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T10:57:29.395 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T10:57:29.395 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T10:57:29.395 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T10:57:29.396 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T10:57:29.396 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T10:57:29.396 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T10:57:29.396 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:57:29.396 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T10:57:29.397 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T10:57:29.397 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T10:57:29.397 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T10:57:29.398 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:57:29.398 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T10:57:29.398 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T10:57:29.398 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:57:29.399 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:57:29.399 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:57:29.399 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T10:57:29.399 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T10:57:29.400 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T10:57:29.400 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T10:57:29.400 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T10:57:29.400 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T10:57:29.400 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T10:57:29.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:57:29.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:57:29.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:57:29.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:57:29.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:29.402 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T10:57:29.461 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-13T10:57:29.461 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-13T10:57:29.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T10:57:29.600 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:57:29.655 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T10:57:29.655 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T10:57:29.655 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T10:57:29.656 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T10:57:29.656 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T10:57:29.656 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T10:57:29.657 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T10:57:29.657 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T10:57:29.657 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T10:57:29.657 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T10:57:29.658 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T10:57:29.658 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:57:29.658 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T10:57:29.658 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T10:57:29.658 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T10:57:29.659 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T10:57:29.659 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:57:29.659 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T10:57:29.659 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T10:57:29.660 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:57:29.660 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:57:29.660 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:57:29.660 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T10:57:29.661 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T10:57:29.661 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T10:57:29.666 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T10:57:29.667 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T10:57:29.667 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T10:57:29.667 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T10:57:29.667 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:57:29.667 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:57:29.668 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:57:29.668 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:57:29.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:29.669 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T10:57:29.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-13T10:57:29.712+0000 7fc12b047480 -1 init, newargv = 0x55a8e1c9b7e0 newargc=15 2023-11-13T10:57:29.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[102488]: starting ceph client 2023-11-13T10:57:29.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[102488]: starting fuse 2023-11-13T10:57:30.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T10:57:30.939 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T10:57:31.003 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T10:57:31.003 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T10:57:31.003 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T10:57:31.004 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T10:57:31.004 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T10:57:31.004 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T10:57:31.004 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T10:57:31.004 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T10:57:31.005 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T10:57:31.005 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T10:57:31.005 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T10:57:31.005 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T10:57:31.005 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T10:57:31.006 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T10:57:31.006 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T10:57:31.006 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T10:57:31.006 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T10:57:31.006 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T10:57:31.007 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T10:57:31.007 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T10:57:31.007 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T10:57:31.007 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T10:57:31.007 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T10:57:31.008 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T10:57:31.008 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T10:57:31.008 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T10:57:31.008 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T10:57:31.008 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T10:57:31.008 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T10:57:31.009 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:57:31.010 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T10:57:31.010 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:57:31.010 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T10:57:31.010 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-13T10:57:31.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:31.012 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T10:57:31.069 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-13T10:57:31.069 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-13T10:57:31.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-13T10:57:31.070 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-13T10:57:31.072 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-13T10:57:31.073 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T10:57:31.073 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-13T10:57:31.074 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:57:31.074 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-13T10:57:31.074 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-13T10:57:31.075 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-13T10:57:31.075 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-13T10:57:31.076 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:57:31.076 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-13T10:57:31.077 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-13T10:57:31.077 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-13T10:57:31.078 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-13T10:57:31.078 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:57:31.078 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-13T10:57:31.079 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-13T10:57:31.079 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-13T10:57:31.079 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-13T10:57:31.080 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-13T10:57:31.088 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-13T10:57:31.089 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-13T10:57:31.092 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-13T10:57:31.099 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:57:31.100 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-13T10:57:31.100 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T10:57:31.295 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.102488.asok 2023-11-13T10:57:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T10:57:31.314 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.102488.asok 2023-11-13T10:57:31.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:31.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.102488.asok status 2023-11-13T10:57:31.568 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T10:57:31.569 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-13T10:57:31.569 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "35f0debb3cc28955b7132ef4c509cd138a09d56b", 2023-11-13T10:57:31.569 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7288-g35f0debb (35f0debb3cc28955b7132ef4c509cd138a09d56b) reef (dev)", 2023-11-13T10:57:31.569 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-13T10:57:31.570 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-13T10:57:31.570 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-13T10:57:31.570 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "102488", 2023-11-13T10:57:31.570 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-13T10:57:31.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:31.571 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-13T10:57:31.571 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-13T10:57:31.571 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5700, 2023-11-13T10:57:31.571 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-13T10:57:31.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-13T10:57:31.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-13T10:57:31.572 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5700 2023-11-13T10:57:31.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:31.572 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T10:57:31.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T10:57:31.573 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T10:57:31.573 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 783295856 2023-11-13T10:57:31.573 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T10:57:31.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:31.574 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T10:57:31.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T10:57:31.574 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T10:57:31.574 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 783295856 2023-11-13T10:57:31.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T10:57:31.575 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5700 192.168.0.1:0/783295856", 2023-11-13T10:57:31.575 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/783295856", 2023-11-13T10:57:31.575 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-13T10:57:31.576 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 18, 2023-11-13T10:57:31.576 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 40, 2023-11-13T10:57:31.576 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-13T10:57:31.576 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-13T10:57:31.576 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-13T10:57:31.577 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T10:57:31.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:31.584 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-13T10:57:31.585 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-13T10:57:31.647 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-13T10:57:31.648 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-13T10:57:31.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:31.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-13T10:57:31.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-11-13T10:57:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.995+0000 7fad264f8640 1 Processor -- start 2023-11-13T10:57:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.995+0000 7fad264f8640 1 -- start start 2023-11-13T10:57:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.997+0000 7fad264f8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad200b2a50 0x7fad200ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.997+0000 7fad264f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad200cb3b0 0x7fad200adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.997+0000 7fad264f8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad200b3d50 con 0x7fad20157720 2023-11-13T10:57:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.997+0000 7fad264f8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad20157b00 con 0x7fad200b2a50 2023-11-13T10:57:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.997+0000 7fad264f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad200cdac0 con 0x7fad200cb3b0 2023-11-13T10:57:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.997+0000 7fad24cf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad200b2a50 0x7fad200ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.997+0000 7fad24cf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad200b2a50 0x7fad200ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36534/0 (socket says 172.21.15.16:36534) 2023-11-13T10:57:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.998+0000 7fad24cf5640 1 -- 172.21.15.16:0/353339509 learned_addr learned my addr 172.21.15.16:0/353339509 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:32.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.998+0000 7fad24cf5640 1 -- 172.21.15.16:0/353339509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad200cb3b0 msgr2=0x7fad200adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T10:57:32.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.998+0000 7fad24cf5640 1 --2- 172.21.15.16:0/353339509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad200cb3b0 0x7fad200adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.998+0000 7fad24cf5640 1 -- 172.21.15.16:0/353339509 >> v1:172.21.15.16:6789/0 conn(0x7fad20157720 legacy=0x7fad200b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.998+0000 7fad1ffff640 1 -- 172.21.15.16:0/353339509 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753075167 0 0) 0x7fad10001490 con 0x7fad20157720 2023-11-13T10:57:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.998+0000 7fad24cf5640 1 -- 172.21.15.16:0/353339509 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200b19b0 con 0x7fad200b2a50 2023-11-13T10:57:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:31.999+0000 7fad24cf5640 1 --2- 172.21.15.16:0/353339509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad200b2a50 0x7fad200ad730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fad1400bb10 tx=0x7fad1402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65472db4b3302b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:32.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.000+0000 7fad1ffff640 1 -- 172.21.15.16:0/353339509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1402ee70 con 0x7fad200b2a50 2023-11-13T10:57:32.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.000+0000 7fad1ffff640 1 -- 172.21.15.16:0/353339509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad14036440 con 0x7fad200b2a50 2023-11-13T10:57:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.000+0000 7fad1ffff640 1 -- 172.21.15.16:0/353339509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad140367d0 con 0x7fad200b2a50 2023-11-13T10:57:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.000+0000 7fad264f8640 1 -- 172.21.15.16:0/353339509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad200b2a50 msgr2=0x7fad200ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.000+0000 7fad264f8640 1 --2- 172.21.15.16:0/353339509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad200b2a50 0x7fad200ad730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fad1400bb10 tx=0x7fad1402e0c0 comp rx=0 tx=0).stop 2023-11-13T10:57:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.000+0000 7fad264f8640 1 -- 172.21.15.16:0/353339509 shutdown_connections 2023-11-13T10:57:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.000+0000 7fad264f8640 1 --2- 172.21.15.16:0/353339509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad200cb3b0 0x7fad200adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:32.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.001+0000 7fad264f8640 1 --2- 172.21.15.16:0/353339509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad200b2a50 0x7fad200ad730 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:32.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.001+0000 7fad264f8640 1 -- 172.21.15.16:0/353339509 >> 172.21.15.16:0/353339509 conn(0x7fad200b0970 msgr2=0x7fad200b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:32.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.001+0000 7fad264f8640 1 -- 172.21.15.16:0/353339509 shutdown_connections 2023-11-13T10:57:32.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.001+0000 7fad264f8640 1 -- 172.21.15.16:0/353339509 wait complete. 2023-11-13T10:57:32.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.003+0000 7fad264f8640 1 Processor -- start 2023-11-13T10:57:32.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.003+0000 7fad264f8640 1 -- start start 2023-11-13T10:57:32.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad264f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad200b2a50 0x7fad200af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:32.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad264f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad200cb3b0 0x7fad201522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:32.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad264f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20157720 0x7fad2013ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:32.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad264f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2015bbb0 con 0x7fad20157720 2023-11-13T10:57:32.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad264f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2015be90 con 0x7fad200cb3b0 2023-11-13T10:57:32.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad264f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2015c170 con 0x7fad200b2a50 2023-11-13T10:57:32.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad25cf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20157720 0x7fad2013ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:32.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad25cf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20157720 0x7fad2013ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36550/0 (socket says 172.21.15.16:36550) 2023-11-13T10:57:32.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad25cf7640 1 -- 172.21.15.16:0/3836639570 learned_addr learned my addr 172.21.15.16:0/3836639570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T10:57:32.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad254f6640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad200b2a50 0x7fad200af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:32.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.004+0000 7fad24cf5640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad200cb3b0 0x7fad201522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:32.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad25cf7640 1 -- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad200b2a50 msgr2=0x7fad200af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:32.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad25cf7640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad200b2a50 0x7fad200af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:32.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad25cf7640 1 -- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad200cb3b0 msgr2=0x7fad201522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:32.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad25cf7640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad200cb3b0 0x7fad201522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad25cf7640 1 -- 172.21.15.16:0/3836639570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1400b770 con 0x7fad20157720 2023-11-13T10:57:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad25cf7640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20157720 0x7fad2013ec90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fad1800b3f0 tx=0x7fad1800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad1dffb640 1 -- 172.21.15.16:0/3836639570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18011830 con 0x7fad20157720 2023-11-13T10:57:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad1dffb640 1 -- 172.21.15.16:0/3836639570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18011990 con 0x7fad20157720 2023-11-13T10:57:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad1dffb640 1 -- 172.21.15.16:0/3836639570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18011d40 con 0x7fad20157720 2023-11-13T10:57:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2015c450 con 0x7fad20157720 2023-11-13T10:57:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.005+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad200afcb0 con 0x7fad20157720 2023-11-13T10:57:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.006+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2015c970 con 0x7fad20157720 2023-11-13T10:57:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.008+0000 7fad1dffb640 1 -- 172.21.15.16:0/3836639570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad1801a400 con 0x7fad20157720 2023-11-13T10:57:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.009+0000 7fad1dffb640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad0c06ae40 0x7fad0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T10:57:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.009+0000 7fad1dffb640 1 -- 172.21.15.16:0/3836639570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fad180899e0 con 0x7fad20157720 2023-11-13T10:57:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.009+0000 7fad254f6640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad0c06ae40 0x7fad0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T10:57:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.010+0000 7fad254f6640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad0c06ae40 0x7fad0c06d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fad10007fd0 tx=0x7fad10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T10:57:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.013+0000 7fad1dffb640 1 -- 172.21.15.16:0/3836639570 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad2015c970 con 0x7fad20157720 2023-11-13T10:57:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:32.251+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fad2015c970 con 0x7fad20157720 2023-11-13T10:57:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.143+0000 7fad1dffb640 1 -- 172.21.15.16:0/3836639570 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v19) v1 ==== 167+0+0 (secure 0 0 0) 0x7fad2015c970 con 0x7fad20157720 2023-11-13T10:57:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:enabled new snapshots 2023-11-13T10:57:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.146+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad0c06ae40 msgr2=0x7fad0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.146+0000 7fad264f8640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad0c06ae40 0x7fad0c06d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fad10007fd0 tx=0x7fad10009040 comp rx=0 tx=0).stop 2023-11-13T10:57:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.146+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20157720 msgr2=0x7fad2013ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T10:57:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.146+0000 7fad264f8640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20157720 0x7fad2013ec90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fad1800b3f0 tx=0x7fad1800b8b0 comp rx=0 tx=0).stop 2023-11-13T10:57:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.147+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 shutdown_connections 2023-11-13T10:57:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.147+0000 7fad264f8640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad0c06ae40 0x7fad0c06d300 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.147+0000 7fad264f8640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20157720 0x7fad2013ec90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.147+0000 7fad264f8640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad200cb3b0 0x7fad201522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.147+0000 7fad264f8640 1 --2- 172.21.15.16:0/3836639570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad200b2a50 0x7fad200af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T10:57:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.147+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 >> 172.21.15.16:0/3836639570 conn(0x7fad200b0970 msgr2=0x7fad200703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T10:57:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.147+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 shutdown_connections 2023-11-13T10:57:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:57:33.147+0000 7fad264f8640 1 -- 172.21.15.16:0/3836639570 wait complete. 2023-11-13T10:57:33.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:57:33.174 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2023-11-13T10:57:33.243 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T10:57:33.244 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T10:57:33.303 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2023-11-13T10:57:33.303 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T10:57:33.304 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2023-11-13T10:57:33.304 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:57:33.304 DEBUG:teuthology.orchestra.run.smithi016:> n = 2000 2023-11-13T10:57:33.305 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2023-11-13T10:57:33.305 DEBUG:teuthology.orchestra.run.smithi016:> if 3 > 0: 2023-11-13T10:57:33.305 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2023-11-13T10:57:33.306 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2023-11-13T10:57:33.306 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:57:33.307 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2023-11-13T10:57:33.307 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2023-11-13T10:57:33.308 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2023-11-13T10:57:33.309 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:57:33.310 DEBUG:teuthology.orchestra.run.smithi016:> try: 2023-11-13T10:57:33.310 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-11-13T10:57:33.311 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T10:57:33.311 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2023-11-13T10:57:33.312 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-11-13T10:57:33.312 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2023-11-13T10:57:33.312 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2023-11-13T10:57:33.313 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2023-11-13T10:57:33.313 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2023-11-13T10:57:33.313 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2023-11-13T10:57:33.314 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2023-11-13T10:57:33.314 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2023-11-13T10:57:33.316 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2023-11-13T10:57:33.318 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2023-11-13T10:57:33.318 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2023-11-13T10:57:33.319 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(3): 2023-11-13T10:57:33.321 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-11-13T10:57:33.321 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2023-11-13T10:57:33.323 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2023-11-13T10:57:33.323 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2023-11-13T10:57:33.324 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2023-11-13T10:57:33.324 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T10:57:47.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:57:47.261 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:58:17.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:58:17.391 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:58:47.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:58:47.561 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:59:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T10:59:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T10:59:13.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:59:13.370 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2023-11-13T10:59:13.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T10:59:13.526 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2023-11-13T10:59:17.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:59:17.709 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:59:47.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T10:59:47.883 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:00:18.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:00:18.028 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:00:43.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:00:43.743 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2023-11-13T11:00:43.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2023-11-13T11:00:44.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.101+0000 7f4d04dce640 1 Processor -- start 2023-11-13T11:00:44.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.102+0000 7f4d04dce640 1 -- start start 2023-11-13T11:00:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.103+0000 7f4d04dce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d00144570 0x7f4d0015f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.103+0000 7f4d04dce640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4d000ce9e0 0x7f4d0006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.103+0000 7f4d04dce640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d00144950 con 0x7f4d000ce550 2023-11-13T11:00:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.103+0000 7f4d04dce640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0015f740 con 0x7f4d00144570 2023-11-13T11:00:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.103+0000 7f4d04dce640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d000ce300 con 0x7f4d000ce9e0 2023-11-13T11:00:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.103+0000 7f4cfe59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d00144570 0x7f4d0015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.103+0000 7f4cfe59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d00144570 0x7f4d0015f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46340/0 (socket says 172.21.15.16:46340) 2023-11-13T11:00:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.103+0000 7f4cfe59c640 1 -- 172.21.15.16:0/113405795 learned_addr learned my addr 172.21.15.16:0/113405795 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.103+0000 7f4cf7fff640 1 --1- 172.21.15.16:0/113405795 >> v1:172.21.15.16:6789/0 conn(0x7f4d000ce550 0x7f4d0006bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40876/0 (socket says 172.21.15.16:40876) 2023-11-13T11:00:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.104+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/113405795 >> v2:172.21.15.184:3300/0 conn(0x7f4d000ce9e0 0x7f4d0006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.104+0000 7f4cfe59c640 1 -- 172.21.15.16:0/113405795 >> v2:172.21.15.184:3300/0 conn(0x7f4d000ce9e0 msgr2=0x7f4d0006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.104+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/113405795 >> v2:172.21.15.184:3300/0 conn(0x7f4d000ce9e0 0x7f4d0006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.104+0000 7f4cfe59c640 1 -- 172.21.15.16:0/113405795 >> v1:172.21.15.16:6789/0 conn(0x7f4d000ce550 legacy=0x7f4d0006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.104+0000 7f4cfe59c640 1 -- 172.21.15.16:0/113405795 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d000cee50 con 0x7f4d00144570 2023-11-13T11:00:44.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.104+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/113405795 >> v2:172.21.15.184:3300/0 conn(0x7f4d000ce9e0 0x7f4d0006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:00:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.104+0000 7f4cfdd9b640 1 -- 172.21.15.16:0/113405795 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200701055 0 0) 0x7f4d00144950 con 0x7f4d000ce550 2023-11-13T11:00:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.105+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/113405795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 0x7f4d0015f200 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4d0006f890 tx=0x7f4cf002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3554d625d41c12fd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.105+0000 7f4cfdd9b640 1 -- 172.21.15.16:0/113405795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0030860 con 0x7f4d00144570 2023-11-13T11:00:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.105+0000 7f4cfdd9b640 1 -- 172.21.15.16:0/113405795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf00309c0 con 0x7f4d00144570 2023-11-13T11:00:44.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.105+0000 7f4cfdd9b640 1 -- 172.21.15.16:0/113405795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0030d50 con 0x7f4d00144570 2023-11-13T11:00:44.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.106+0000 7f4d04dce640 1 -- 172.21.15.16:0/113405795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 msgr2=0x7f4d0015f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.106+0000 7f4d04dce640 1 --2- 172.21.15.16:0/113405795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 0x7f4d0015f200 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4d0006f890 tx=0x7f4cf002b9a0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.106+0000 7f4d04dce640 1 -- 172.21.15.16:0/113405795 shutdown_connections 2023-11-13T11:00:44.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.106+0000 7f4d04dce640 1 --2- 172.21.15.16:0/113405795 >> v2:172.21.15.184:3300/0 conn(0x7f4d000ce9e0 0x7f4d0006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.106+0000 7f4d04dce640 1 --2- 172.21.15.16:0/113405795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 0x7f4d0015f200 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.106+0000 7f4d04dce640 1 -- 172.21.15.16:0/113405795 >> 172.21.15.16:0/113405795 conn(0x7f4d0006a190 msgr2=0x7f4d0006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.106+0000 7f4d04dce640 1 -- 172.21.15.16:0/113405795 shutdown_connections 2023-11-13T11:00:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.106+0000 7f4d04dce640 1 -- 172.21.15.16:0/113405795 wait complete. 2023-11-13T11:00:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.108+0000 7f4d04dce640 1 Processor -- start 2023-11-13T11:00:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.108+0000 7f4d04dce640 1 -- start start 2023-11-13T11:00:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.109+0000 7f4d04dce640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d000ce550 0x7f4d001f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.109+0000 7f4d04dce640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d000ce9e0 0x7f4d00158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.109+0000 7f4d04dce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 0x7f4d00159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.109+0000 7f4d04dce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0013bdd0 con 0x7f4d00144570 2023-11-13T11:00:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.109+0000 7f4d04dce640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0013c0b0 con 0x7f4d000ce9e0 2023-11-13T11:00:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.109+0000 7f4d04dce640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0013c390 con 0x7f4d000ce550 2023-11-13T11:00:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.109+0000 7f4cfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 0x7f4d00159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.109+0000 7f4cfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 0x7f4d00159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46350/0 (socket says 172.21.15.16:46350) 2023-11-13T11:00:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.109+0000 7f4cfed9d640 1 -- 172.21.15.16:0/3680154901 learned_addr learned my addr 172.21.15.16:0/3680154901 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d000ce9e0 0x7f4d00158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d000ce550 0x7f4d001f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cfed9d640 1 -- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d000ce550 msgr2=0x7f4d001f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d000ce550 0x7f4d001f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cfed9d640 1 -- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d000ce9e0 msgr2=0x7f4d00158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d000ce9e0 0x7f4d00158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cfed9d640 1 -- 172.21.15.16:0/3680154901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf00094d0 con 0x7f4d00144570 2023-11-13T11:00:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d000ce9e0 0x7f4d00158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:00:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d000ce550 0x7f4d001f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:00:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 0x7f4d00159150 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800cef0 tx=0x7f4cf800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cf6ffd640 1 -- 172.21.15.16:0/3680154901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800d840 con 0x7f4d00144570 2023-11-13T11:00:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cf6ffd640 1 -- 172.21.15.16:0/3680154901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf800d9c0 con 0x7f4d00144570 2023-11-13T11:00:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4cf6ffd640 1 -- 172.21.15.16:0/3680154901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800dd70 con 0x7f4d00144570 2023-11-13T11:00:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4d04dce640 1 -- 172.21.15.16:0/3680154901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0013c670 con 0x7f4d00144570 2023-11-13T11:00:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.110+0000 7f4d04dce640 1 -- 172.21.15.16:0/3680154901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d001f6cb0 con 0x7f4d00144570 2023-11-13T11:00:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.113+0000 7f4cf6ffd640 1 -- 172.21.15.16:0/3680154901 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4cf801e3b0 con 0x7f4d00144570 2023-11-13T11:00:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.114+0000 7f4cf6ffd640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ce006ae40 0x7f4ce006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.114+0000 7f4cf6ffd640 1 -- 172.21.15.16:0/3680154901 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f4cf808a2d0 con 0x7f4d00144570 2023-11-13T11:00:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.114+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ce006ae40 0x7f4ce006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.114+0000 7f4d07963740 1 Processor -- start 2023-11-13T11:00:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.115+0000 7f4d07963740 1 -- start start 2023-11-13T11:00:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.115+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ce006ae40 0x7f4ce006d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0036000 tx=0x7f4cf003b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.115+0000 7f4d07963740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.115+0000 7f4d07963740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accef3c80 0x557accef5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.115+0000 7f4d07963740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef0120 0x557accf1ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.115+0000 7f4d07963740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accdfa180 con 0x557accf3dc00 2023-11-13T11:00:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4d07963740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acce62dd0 con 0x557accef3c80 2023-11-13T11:00:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4d07963740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557acce604c0 con 0x557accef0120 2023-11-13T11:00:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cfe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cfe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46360/0 (socket says 172.21.15.16:46360) 2023-11-13T11:00:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cfe59c640 1 -- 172.21.15.16:0/2642976798 learned_addr learned my addr 172.21.15.16:0/2642976798 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/2642976798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef0120 0x557accf1ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/2642976798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accef3c80 0x557accef5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cfe59c640 1 -- 172.21.15.16:0/2642976798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef0120 msgr2=0x557accf1ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/2642976798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef0120 0x557accf1ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cfe59c640 1 -- 172.21.15.16:0/2642976798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accef3c80 msgr2=0x557accef5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/2642976798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accef3c80 0x557accef5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.116+0000 7f4cfe59c640 1 -- 172.21.15.16:0/2642976798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557acce1e950 con 0x557accf3dc00 2023-11-13T11:00:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.117+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/2642976798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef5230 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4cf002b9e0 tx=0x7f4cf0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3bee6ade5781d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.117+0000 7f4cdffff640 1 -- 172.21.15.16:0/2642976798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf003e040 con 0x557accf3dc00 2023-11-13T11:00:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.117+0000 7f4cdffff640 1 -- 172.21.15.16:0/2642976798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf004a040 con 0x557accf3dc00 2023-11-13T11:00:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.117+0000 7f4cdffff640 1 -- 172.21.15.16:0/2642976798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0044610 con 0x557accf3dc00 2023-11-13T11:00:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.118+0000 7f4d07963740 1 -- 172.21.15.16:0/2642976798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 msgr2=0x557accef5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.118+0000 7f4d07963740 1 --2- 172.21.15.16:0/2642976798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef5230 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4cf002b9e0 tx=0x7f4cf0030000 comp rx=0 tx=0).stop 2023-11-13T11:00:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.118+0000 7f4d07963740 1 -- 172.21.15.16:0/2642976798 shutdown_connections 2023-11-13T11:00:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.118+0000 7f4d07963740 1 --2- 172.21.15.16:0/2642976798 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef0120 0x557accf1ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.118+0000 7f4d07963740 1 --2- 172.21.15.16:0/2642976798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accef3c80 0x557accef5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.118+0000 7f4d07963740 1 --2- 172.21.15.16:0/2642976798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef5230 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.118+0000 7f4d07963740 1 -- 172.21.15.16:0/2642976798 >> 172.21.15.16:0/2642976798 conn(0x557accef2390 msgr2=0x557accf3af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.118+0000 7f4d07963740 1 -- 172.21.15.16:0/2642976798 shutdown_connections 2023-11-13T11:00:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.118+0000 7f4d07963740 1 -- 172.21.15.16:0/2642976798 wait complete. 2023-11-13T11:00:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.119+0000 7f4d07963740 1 -- start start 2023-11-13T11:00:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.119+0000 7f4d07963740 1 Processor -- start 2023-11-13T11:00:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4d07963740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf3c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4d07963740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 0x557accf3fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4d07963740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf04a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4d07963740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557acce5bdb0 con 0x557accf40da0 2023-11-13T11:00:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4d07963740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accdf8460 con 0x557accf3dc00 2023-11-13T11:00:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4d07963740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557acce15c30 con 0x557accef3c80 2023-11-13T11:00:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4d07963740 10 client.0 _ll_register_callbacks cb 0x557accb6dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:00:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4cfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf04a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4cfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf04a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46366/0 (socket says 172.21.15.16:46366) 2023-11-13T11:00:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.120+0000 7f4cfed9d640 1 -- 172.21.15.16:0/1540926882 learned_addr learned my addr 172.21.15.16:0/1540926882 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf3c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 0x557accf3fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cfed9d640 1 -- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 msgr2=0x557accf3c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf3c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cfed9d640 1 -- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 msgr2=0x557accf3fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 0x557accf3fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cfed9d640 1 -- 172.21.15.16:0/1540926882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d000cee50 con 0x557accf40da0 2023-11-13T11:00:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf04a50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8055420 tx=0x7f4cf8058fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1047882ecf5dbcf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:00:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1540926882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8058410 con 0x557accf40da0 2023-11-13T11:00:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1540926882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f4cf8058570 con 0x557accf40da0 2023-11-13T11:00:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.121+0000 7f4d07963740 1 -- 172.21.15.16:0/1540926882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557acce54d90 con 0x557accf40da0 2023-11-13T11:00:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1540926882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cf8088250 con 0x557accf40da0 2023-11-13T11:00:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4d07963740 10 client.5724 fetch_fsmap learned FSMap version 19 2023-11-13T11:00:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4d07963740 1 -- 172.21.15.16:0/1540926882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x557acce5a430 con 0x557accf40da0 2023-11-13T11:00:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1540926882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557acce54d90 con 0x557accf40da0 2023-11-13T11:00:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1540926882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4cf8008040 con 0x557accf40da0 2023-11-13T11:00:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4d07963740 10 client.5724 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:00:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4d07963740 10 client.5724 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:00:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4d07963740 20 client.5724 populate_metadata read hostname 'smithi016' 2023-11-13T11:00:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4d07963740 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x557accf184d0 0x557accf426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4d07963740 4 client.5724 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:00:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.122+0000 7f4d07963740 1 -- 172.21.15.16:0/1540926882 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557acce54ab0 con 0x557accf184d0 2023-11-13T11:00:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.123+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x557accf184d0 0x557accf426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.123+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x557accf184d0 0x557accf426e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5304aecde4cf4d4d server_cookie=5d1c8dbc86c9d418 in_seq=0 out_seq=0 2023-11-13T11:00:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.123+0000 7f4cdd7fa640 10 client.5724 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.124+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1540926882 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557acce54ab0 con 0x557accf184d0 2023-11-13T11:00:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.124+0000 7f4cdd7fa640 10 client.5724 handle_command_reply: tid=0 2023-11-13T11:00:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 client.5724 shutdown 2023-11-13T11:00:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 -- 172.21.15.16:0/1540926882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 msgr2=0x557accf04a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf04a50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8055420 tx=0x7f4cf8058fe0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 -- 172.21.15.16:0/1540926882 shutdown_connections 2023-11-13T11:00:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x557accf184d0 0x557accf426e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf04a50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 0x557accf3fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 --2- 172.21.15.16:0/1540926882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf3c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4cdd7fa640 0 client.5724 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 -- 172.21.15.16:0/1540926882 >> 172.21.15.16:0/1540926882 conn(0x557accef0120 msgr2=0x557accef4470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 -- 172.21.15.16:0/1540926882 shutdown_connections 2023-11-13T11:00:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 1 -- 172.21.15.16:0/1540926882 wait complete. 2023-11-13T11:00:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.125+0000 7f4d07963740 20 client.5724 trim_cache size 0 max 16384 2023-11-13T11:00:44.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.141+0000 7f4d07963740 1 Processor -- start 2023-11-13T11:00:44.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.141+0000 7f4d07963740 1 -- start start 2023-11-13T11:00:44.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.142+0000 7f4d07963740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf3f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.142+0000 7f4d07963740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.142+0000 7f4d07963740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf40da0 0x557accef52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.142+0000 7f4d07963740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accbbf040 con 0x557accf3dc00 2023-11-13T11:00:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.142+0000 7f4d07963740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accf176e0 con 0x557accf40da0 2023-11-13T11:00:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.142+0000 7f4d07963740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557accf4e240 con 0x557accef3c80 2023-11-13T11:00:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.143+0000 7f4cf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.143+0000 7f4cf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46382/0 (socket says 172.21.15.16:46382) 2023-11-13T11:00:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.143+0000 7f4cf7fff640 1 -- 172.21.15.16:0/1377368750 learned_addr learned my addr 172.21.15.16:0/1377368750 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.143+0000 7f4cf7fff640 1 -- 172.21.15.16:0/1377368750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 msgr2=0x557accf3f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.143+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/1377368750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf3f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.143+0000 7f4cf7fff640 1 -- 172.21.15.16:0/1377368750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf40da0 msgr2=0x557accef52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.143+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/1377368750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf40da0 0x557accef52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.143+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/1377368750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf40da0 0x557accef52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.143+0000 7f4cf7fff640 1 -- 172.21.15.16:0/1377368750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557accdf7cf0 con 0x557accf3dc00 2023-11-13T11:00:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.144+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/1377368750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef4da0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8007710 tx=0x7f4ce8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30229f2221711a80 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.144+0000 7f4cf4ff9640 1 -- 172.21.15.16:0/1377368750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8005e10 con 0x557accf3dc00 2023-11-13T11:00:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.144+0000 7f4cf4ff9640 1 -- 172.21.15.16:0/1377368750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce800d040 con 0x557accf3dc00 2023-11-13T11:00:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.144+0000 7f4cf4ff9640 1 -- 172.21.15.16:0/1377368750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8001c60 con 0x557accf3dc00 2023-11-13T11:00:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.144+0000 7f4d07963740 1 -- 172.21.15.16:0/1377368750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 msgr2=0x557accef4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.144+0000 7f4d07963740 1 --2- 172.21.15.16:0/1377368750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef4da0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8007710 tx=0x7f4ce8005bc0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.145+0000 7f4d07963740 1 -- 172.21.15.16:0/1377368750 shutdown_connections 2023-11-13T11:00:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.145+0000 7f4d07963740 1 --2- 172.21.15.16:0/1377368750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf40da0 0x557accef52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.145+0000 7f4d07963740 1 --2- 172.21.15.16:0/1377368750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf3dc00 0x557accef4da0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.145+0000 7f4d07963740 1 --2- 172.21.15.16:0/1377368750 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf3f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.145+0000 7f4d07963740 1 -- 172.21.15.16:0/1377368750 >> 172.21.15.16:0/1377368750 conn(0x557accef0120 msgr2=0x557accf23ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.145+0000 7f4d07963740 1 -- 172.21.15.16:0/1377368750 shutdown_connections 2023-11-13T11:00:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.145+0000 7f4d07963740 1 -- 172.21.15.16:0/1377368750 wait complete. 2023-11-13T11:00:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.145+0000 7f4d07963740 1 -- start start 2023-11-13T11:00:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.146+0000 7f4d07963740 1 Processor -- start 2023-11-13T11:00:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4d07963740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf5e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4d07963740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 0x557accf17870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4d07963740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf5c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4d07963740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accf5cdc0 con 0x557accf40da0 2023-11-13T11:00:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4d07963740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557accf5d1f0 con 0x557accf3dc00 2023-11-13T11:00:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4d07963740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557acceeb8e0 con 0x557accef3c80 2023-11-13T11:00:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4d07963740 10 client.0 _ll_register_callbacks cb 0x7f4cf00096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:00:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4cfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf5c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4cfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf5c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46388/0 (socket says 172.21.15.16:46388) 2023-11-13T11:00:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.147+0000 7f4cfed9d640 1 -- 172.21.15.16:0/1795044909 learned_addr learned my addr 172.21.15.16:0/1795044909 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cf7fff640 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 0x557accf17870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cfed9d640 1 -- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 msgr2=0x557accf5e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf5e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cfed9d640 1 -- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 msgr2=0x557accf17870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 0x557accf17870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cfed9d640 1 -- 172.21.15.16:0/1795044909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce8007390 con 0x557accf40da0 2023-11-13T11:00:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cfed9d640 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf5c880 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800ae50 tx=0x7f4cf80884f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88885d0fbf0c2c92 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:00:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1795044909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf80572c0 con 0x557accf40da0 2023-11-13T11:00:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1795044909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f4cf8054d10 con 0x557accf40da0 2023-11-13T11:00:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.148+0000 7f4d07963740 1 -- 172.21.15.16:0/1795044909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cf004a040 con 0x557accf40da0 2023-11-13T11:00:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.150+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1795044909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cf8054f10 con 0x557accf40da0 2023-11-13T11:00:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.150+0000 7f4d07963740 10 client.5730 fetch_fsmap learned FSMap version 19 2023-11-13T11:00:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.150+0000 7f4d07963740 1 -- 172.21.15.16:0/1795044909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x557acce1d1b0 con 0x557accf40da0 2023-11-13T11:00:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.151+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1795044909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf004a040 con 0x557accf40da0 2023-11-13T11:00:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.151+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1795044909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4cf8082080 con 0x557accf40da0 2023-11-13T11:00:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.151+0000 7f4d07963740 10 client.5730 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:00:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.151+0000 7f4d07963740 10 client.5730 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:00:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.151+0000 7f4d07963740 20 client.5730 populate_metadata read hostname 'smithi016' 2023-11-13T11:00:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.151+0000 7f4d07963740 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x557accf389e0 0x557accf38df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.151+0000 7f4d07963740 4 client.5730 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2023-11-13T11:00:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.151+0000 7f4d07963740 1 -- 172.21.15.16:0/1795044909 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x557accb5c7e0 con 0x557accf389e0 2023-11-13T11:00:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.151+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x557accf389e0 0x557accf38df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.152+0000 7f4cfe59c640 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x557accf389e0 0x557accf38df0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9d6923404d9aad8 server_cookie=65f8ab55b98ff80e in_seq=0 out_seq=0 2023-11-13T11:00:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.152+0000 7f4cdd7fa640 10 client.5730 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.153+0000 7f4cdd7fa640 1 -- 172.21.15.16:0/1795044909 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17911 (crc 0 0 0) 0x557accb5c7e0 con 0x557accf389e0 2023-11-13T11:00:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4cdd7fa640 10 client.5730 handle_command_reply: tid=0 2023-11-13T11:00:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 client.5730 shutdown 2023-11-13T11:00:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 -- 172.21.15.16:0/1795044909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 msgr2=0x557accf5c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf5c880 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800ae50 tx=0x7f4cf80884f0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 -- 172.21.15.16:0/1795044909 shutdown_connections 2023-11-13T11:00:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x557accf389e0 0x557accf38df0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557accf40da0 0x557accf5c880 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557accf3dc00 0x557accf17870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 --2- 172.21.15.16:0/1795044909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557accef3c80 0x557accf5e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4cdd7fa640 0 client.5730 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 -- 172.21.15.16:0/1795044909 >> 172.21.15.16:0/1795044909 conn(0x557accef0120 msgr2=0x557accef5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 -- 172.21.15.16:0/1795044909 shutdown_connections 2023-11-13T11:00:44.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 1 -- 172.21.15.16:0/1795044909 wait complete. 2023-11-13T11:00:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.154+0000 7f4d07963740 20 client.5730 trim_cache size 0 max 16384 2023-11-13T11:00:44.170 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:00:44.170 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2023-11-13T11:00:44.171 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4135, 2023-11-13T11:00:44.171 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4136, 2023-11-13T11:00:44.171 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1221398, 2023-11-13T11:00:44.171 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 41369285, 2023-11-13T11:00:44.171 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 9, 2023-11-13T11:00:44.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2023-11-13T11:00:44.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 1.001926052, 2023-11-13T11:00:44.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.507865325, 2023-11-13T11:00:44.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.189976209, 2023-11-13T11:00:44.172 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.224445100, 2023-11-13T11:00:44.173 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2023-11-13T11:00:44.173 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4135, 2023-11-13T11:00:44.173 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.696166862, 2023-11-13T11:00:44.173 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000168359 2023-11-13T11:00:44.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.173 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2023-11-13T11:00:44.174 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2, 2023-11-13T11:00:44.174 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000269, 2023-11-13T11:00:44.174 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000134 2023-11-13T11:00:44.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.174 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-13T11:00:44.175 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2023-11-13T11:00:44.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.175 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2023-11-13T11:00:44.175 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 34739, 2023-11-13T11:00:44.175 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 60482, 2023-11-13T11:00:44.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 12166449, 2023-11-13T11:00:44.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 49443308, 2023-11-13T11:00:44.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2023-11-13T11:00:44.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2023-11-13T11:00:44.176 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 11.301427645, 2023-11-13T11:00:44.177 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 5.144333222, 2023-11-13T11:00:44.177 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 6.501188273, 2023-11-13T11:00:44.177 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.347287335, 2023-11-13T11:00:44.177 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2023-11-13T11:00:44.177 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 60481, 2023-11-13T11:00:44.178 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 4.821397419, 2023-11-13T11:00:44.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000079717 2023-11-13T11:00:44.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.178 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2023-11-13T11:00:44.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 88310, 2023-11-13T11:00:44.179 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.050015692, 2023-11-13T11:00:44.179 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000566 2023-11-13T11:00:44.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.179 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 127600, 2023-11-13T11:00:44.179 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 41296 2023-11-13T11:00:44.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.180 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2023-11-13T11:00:44.180 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2660, 2023-11-13T11:00:44.180 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2703, 2023-11-13T11:00:44.180 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 661265, 2023-11-13T11:00:44.180 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 21161504, 2023-11-13T11:00:44.181 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2023-11-13T11:00:44.181 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2023-11-13T11:00:44.181 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.616287554, 2023-11-13T11:00:44.181 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.275533539, 2023-11-13T11:00:44.181 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.126215353, 2023-11-13T11:00:44.182 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.154069760, 2023-11-13T11:00:44.182 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2023-11-13T11:00:44.182 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2702, 2023-11-13T11:00:44.182 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.434222721, 2023-11-13T11:00:44.182 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000160704 2023-11-13T11:00:44.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.183 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2023-11-13T11:00:44.183 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.183 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.183 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.184 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1728, 2023-11-13T11:00:44.184 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 159632 2023-11-13T11:00:44.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.184 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2023-11-13T11:00:44.184 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2023-11-13T11:00:44.185 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2023-11-13T11:00:44.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.185 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2023-11-13T11:00:44.185 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2023-11-13T11:00:44.185 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2023-11-13T11:00:44.186 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 9407, 2023-11-13T11:00:44.186 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 29.826380417, 2023-11-13T11:00:44.186 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003170658 2023-11-13T11:00:44.186 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.187 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2023-11-13T11:00:44.187 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2023-11-13T11:00:44.187 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2023-11-13T11:00:44.187 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2262, 2023-11-13T11:00:44.187 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.271803412, 2023-11-13T11:00:44.188 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000562247 2023-11-13T11:00:44.188 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.188 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2023-11-13T11:00:44.188 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 28340, 2023-11-13T11:00:44.189 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 28340, 2023-11-13T11:00:44.189 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2023-11-13T11:00:44.189 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 28340, 2023-11-13T11:00:44.189 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 514.930655561, 2023-11-13T11:00:44.189 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.018169747 2023-11-13T11:00:44.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.190 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2023-11-13T11:00:44.190 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2023-11-13T11:00:44.190 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2023-11-13T11:00:44.190 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2023-11-13T11:00:44.191 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 12, 2023-11-13T11:00:44.191 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2023-11-13T11:00:44.191 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2023-11-13T11:00:44.191 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 19902, 2023-11-13T11:00:44.191 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2023-11-13T11:00:44.191 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 19902, 2023-11-13T11:00:44.192 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2023-11-13T11:00:44.192 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 19902, 2023-11-13T11:00:44.192 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2023-11-13T11:00:44.192 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 190, 2023-11-13T11:00:44.192 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 190, 2023-11-13T11:00:44.193 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 2, 2023-11-13T11:00:44.193 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 46704, 2023-11-13T11:00:44.193 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 46247, 2023-11-13T11:00:44.193 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2023-11-13T11:00:44.193 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2023-11-13T11:00:44.194 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2023-11-13T11:00:44.194 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2023-11-13T11:00:44.194 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 133, 2023-11-13T11:00:44.194 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 216162, 2023-11-13T11:00:44.194 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2023-11-13T11:00:44.195 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2023-11-13T11:00:44.195 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2023-11-13T11:00:44.195 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2023-11-13T11:00:44.195 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2023-11-13T11:00:44.195 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2023-11-13T11:00:44.195 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2023-11-13T11:00:44.196 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2023-11-13T11:00:44.196 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2023-11-13T11:00:44.196 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2023-11-13T11:00:44.196 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2023-11-13T11:00:44.196 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2023-11-13T11:00:44.197 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2023-11-13T11:00:44.197 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2023-11-13T11:00:44.197 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2023-11-13T11:00:44.197 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2023-11-13T11:00:44.197 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2023-11-13T11:00:44.198 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2023-11-13T11:00:44.198 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2023-11-13T11:00:44.198 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2023-11-13T11:00:44.198 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 6178, 2023-11-13T11:00:44.198 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 6006, 2023-11-13T11:00:44.198 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2023-11-13T11:00:44.199 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-13T11:00:44.199 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-13T11:00:44.199 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 6178, 2023-11-13T11:00:44.199 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 2000, 2023-11-13T11:00:44.199 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 68, 2023-11-13T11:00:44.200 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 17846 2023-11-13T11:00:44.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.200 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2023-11-13T11:00:44.200 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2023-11-13T11:00:44.200 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2023-11-13T11:00:44.201 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2023-11-13T11:00:44.201 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2023-11-13T11:00:44.201 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2023-11-13T11:00:44.201 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 1895, 2023-11-13T11:00:44.201 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 42, 2023-11-13T11:00:44.201 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2023-11-13T11:00:44.202 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 4176, 2023-11-13T11:00:44.202 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 1812, 2023-11-13T11:00:44.202 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 2253, 2023-11-13T11:00:44.202 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2023-11-13T11:00:44.202 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2023-11-13T11:00:44.203 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2023-11-13T11:00:44.203 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2023-11-13T11:00:44.203 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2023-11-13T11:00:44.203 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2023-11-13T11:00:44.203 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2023-11-13T11:00:44.204 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2023-11-13T11:00:44.204 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2023-11-13T11:00:44.204 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2023-11-13T11:00:44.204 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2023-11-13T11:00:44.204 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2023-11-13T11:00:44.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.205 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2023-11-13T11:00:44.205 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2023-11-13T11:00:44.205 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 22082, 2023-11-13T11:00:44.205 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2023-11-13T11:00:44.205 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2023-11-13T11:00:44.206 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 22082, 2023-11-13T11:00:44.206 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2023-11-13T11:00:44.206 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2023-11-13T11:00:44.206 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 28, 2023-11-13T11:00:44.206 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2023-11-13T11:00:44.207 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2023-11-13T11:00:44.207 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 28, 2023-11-13T11:00:44.207 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 3, 2023-11-13T11:00:44.207 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2023-11-13T11:00:44.207 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2023-11-13T11:00:44.207 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2023-11-13T11:00:44.208 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 82107121, 2023-11-13T11:00:44.208 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 0, 2023-11-13T11:00:44.208 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2023-11-13T11:00:44.208 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6596, 2023-11-13T11:00:44.208 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 130.469772093, 2023-11-13T11:00:44.209 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.019780135 2023-11-13T11:00:44.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.209 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 0 2023-11-13T11:00:44.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.209 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2023-11-13T11:00:44.210 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 2014, 2023-11-13T11:00:44.210 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 2014, 2023-11-13T11:00:44.210 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2023-11-13T11:00:44.210 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 13, 2023-11-13T11:00:44.210 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 13, 2023-11-13T11:00:44.210 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2023-11-13T11:00:44.211 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 19902, 2023-11-13T11:00:44.211 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 19902, 2023-11-13T11:00:44.211 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2023-11-13T11:00:44.211 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 148, 2023-11-13T11:00:44.211 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 2002, 2023-11-13T11:00:44.212 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 1854, 2023-11-13T11:00:44.212 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 100388, 2023-11-13T11:00:44.212 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 207196 2023-11-13T11:00:44.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.212 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2023-11-13T11:00:44.212 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 37588, 2023-11-13T11:00:44.213 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2023-11-13T11:00:44.213 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 28340, 2023-11-13T11:00:44.213 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 12, 2023-11-13T11:00:44.213 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2023-11-13T11:00:44.213 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2023-11-13T11:00:44.214 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2000, 2023-11-13T11:00:44.214 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3.713455889, 2023-11-13T11:00:44.214 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001856727 2023-11-13T11:00:44.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.214 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2023-11-13T11:00:44.215 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2013, 2023-11-13T11:00:44.215 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.772931577, 2023-11-13T11:00:44.215 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000880740 2023-11-13T11:00:44.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.215 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2023-11-13T11:00:44.216 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.216 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.216 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.216 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2023-11-13T11:00:44.217 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6000, 2023-11-13T11:00:44.217 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 28.088464230, 2023-11-13T11:00:44.217 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.004681410 2023-11-13T11:00:44.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.217 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2023-11-13T11:00:44.218 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8061, 2023-11-13T11:00:44.218 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 20.079647353, 2023-11-13T11:00:44.218 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002490962 2023-11-13T11:00:44.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.218 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2023-11-13T11:00:44.219 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.219 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.219 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.219 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2023-11-13T11:00:44.219 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.220 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.220 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.220 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2023-11-13T11:00:44.220 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.221 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.221 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.221 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2023-11-13T11:00:44.221 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.221 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.222 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.222 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2023-11-13T11:00:44.222 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2023-11-13T11:00:44.222 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016119678, 2023-11-13T11:00:44.223 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.005373226 2023-11-13T11:00:44.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.223 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2023-11-13T11:00:44.223 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.223 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.224 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.224 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2023-11-13T11:00:44.224 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2023-11-13T11:00:44.224 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.006430661, 2023-11-13T11:00:44.225 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.006430661 2023-11-13T11:00:44.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.225 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2023-11-13T11:00:44.225 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.225 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.225 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.226 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2023-11-13T11:00:44.226 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2023-11-13T11:00:44.226 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.015617211, 2023-11-13T11:00:44.226 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.015617211 2023-11-13T11:00:44.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.227 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2023-11-13T11:00:44.227 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.227 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.227 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.228 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2023-11-13T11:00:44.228 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6, 2023-11-13T11:00:44.228 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.247721739, 2023-11-13T11:00:44.228 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.041286956 2023-11-13T11:00:44.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.229 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2023-11-13T11:00:44.229 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2253, 2023-11-13T11:00:44.229 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 424.334683844, 2023-11-13T11:00:44.229 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.188342070 2023-11-13T11:00:44.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.230 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2023-11-13T11:00:44.230 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.230 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.230 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.231 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2023-11-13T11:00:44.231 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.231 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.231 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.232 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2023-11-13T11:00:44.232 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.232 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.232 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.233 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2023-11-13T11:00:44.233 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2023-11-13T11:00:44.233 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.010406493, 2023-11-13T11:00:44.233 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.010406493 2023-11-13T11:00:44.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.234 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2023-11-13T11:00:44.234 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.234 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.234 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.234 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2023-11-13T11:00:44.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2023-11-13T11:00:44.235 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000976048, 2023-11-13T11:00:44.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000976048 2023-11-13T11:00:44.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.235 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2023-11-13T11:00:44.236 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.236 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.236 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.236 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2023-11-13T11:00:44.237 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.237 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.237 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.237 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2023-11-13T11:00:44.238 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.238 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.238 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.238 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2023-11-13T11:00:44.239 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.239 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.239 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.239 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2023-11-13T11:00:44.239 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.240 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.240 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.240 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2023-11-13T11:00:44.240 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8000, 2023-11-13T11:00:44.241 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 36.644200838, 2023-11-13T11:00:44.241 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.004580525 2023-11-13T11:00:44.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.241 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2023-11-13T11:00:44.241 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2023-11-13T11:00:44.242 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2023-11-13T11:00:44.242 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.242 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.242 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.242 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2023-11-13T11:00:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2023-11-13T11:00:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2023-11-13T11:00:44.243 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2023-11-13T11:00:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2023-11-13T11:00:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2023-11-13T11:00:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 12077, 2023-11-13T11:00:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 12077, 2023-11-13T11:00:44.244 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 192, 2023-11-13T11:00:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2023-11-13T11:00:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2023-11-13T11:00:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2023-11-13T11:00:44.245 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2023-11-13T11:00:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2023-11-13T11:00:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2023-11-13T11:00:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2023-11-13T11:00:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2023-11-13T11:00:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-13T11:00:44.246 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2023-11-13T11:00:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-13T11:00:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2023-11-13T11:00:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2023-11-13T11:00:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2023-11-13T11:00:44.247 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-13T11:00:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2023-11-13T11:00:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2023-11-13T11:00:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2023-11-13T11:00:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2023-11-13T11:00:44.248 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2023-11-13T11:00:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-13T11:00:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2023-11-13T11:00:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-13T11:00:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2023-11-13T11:00:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2023-11-13T11:00:44.249 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2023-11-13T11:00:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2023-11-13T11:00:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2023-11-13T11:00:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-13T11:00:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2023-11-13T11:00:44.250 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2023-11-13T11:00:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2023-11-13T11:00:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2023-11-13T11:00:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2023-11-13T11:00:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2023-11-13T11:00:44.251 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2023-11-13T11:00:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2023-11-13T11:00:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2023-11-13T11:00:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 18087738, 2023-11-13T11:00:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 4566, 2023-11-13T11:00:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2023-11-13T11:00:44.252 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2023-11-13T11:00:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2023-11-13T11:00:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2023-11-13T11:00:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2023-11-13T11:00:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2023-11-13T11:00:44.253 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2023-11-13T11:00:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2023-11-13T11:00:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 5904, 2023-11-13T11:00:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 86, 2023-11-13T11:00:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2023-11-13T11:00:44.254 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2023-11-13T11:00:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2023-11-13T11:00:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2023-11-13T11:00:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 21007742, 2023-11-13T11:00:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 818512, 2023-11-13T11:00:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2023-11-13T11:00:44.255 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2023-11-13T11:00:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2023-11-13T11:00:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2023-11-13T11:00:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2023-11-13T11:00:44.256 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2023-11-13T11:00:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2023-11-13T11:00:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 8860, 2023-11-13T11:00:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 78068232, 2023-11-13T11:00:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 31, 2023-11-13T11:00:44.257 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 8860, 2023-11-13T11:00:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2023-11-13T11:00:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8860, 2023-11-13T11:00:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 58.697764930, 2023-11-13T11:00:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.006625029 2023-11-13T11:00:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.258 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2023-11-13T11:00:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2023-11-13T11:00:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8860, 2023-11-13T11:00:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 8913 2023-11-13T11:00:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.259 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 8860, 2023-11-13T11:00:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 137, 2023-11-13T11:00:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 8723, 2023-11-13T11:00:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2023-11-13T11:00:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2023-11-13T11:00:44.260 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 0, 2023-11-13T11:00:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 11, 2023-11-13T11:00:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 137, 2023-11-13T11:00:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 6733, 2023-11-13T11:00:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 88, 2023-11-13T11:00:44.261 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2023-11-13T11:00:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2023-11-13T11:00:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 1707, 2023-11-13T11:00:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2023-11-13T11:00:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 135, 2023-11-13T11:00:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2023-11-13T11:00:44.262 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2023-11-13T11:00:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2023-11-13T11:00:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 0, 2023-11-13T11:00:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 22, 2023-11-13T11:00:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2023-11-13T11:00:44.263 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2023-11-13T11:00:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2023-11-13T11:00:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2023-11-13T11:00:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2023-11-13T11:00:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2023-11-13T11:00:44.264 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2023-11-13T11:00:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2023-11-13T11:00:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2023-11-13T11:00:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2023-11-13T11:00:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2023-11-13T11:00:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2023-11-13T11:00:44.265 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2023-11-13T11:00:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2023-11-13T11:00:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2023-11-13T11:00:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2023-11-13T11:00:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2023-11-13T11:00:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2023-11-13T11:00:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2023-11-13T11:00:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2023-11-13T11:00:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2023-11-13T11:00:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2023-11-13T11:00:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2023-11-13T11:00:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2023-11-13T11:00:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2023-11-13T11:00:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2023-11-13T11:00:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 42, 2023-11-13T11:00:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2023-11-13T11:00:44.268 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 3, 2023-11-13T11:00:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 8, 2023-11-13T11:00:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 8, 2023-11-13T11:00:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2023-11-13T11:00:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2023-11-13T11:00:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 69, 2023-11-13T11:00:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 0, 2023-11-13T11:00:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 11 2023-11-13T11:00:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2023-11-13T11:00:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 1, 2023-11-13T11:00:44.270 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2023-11-13T11:00:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2023-11-13T11:00:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2023-11-13T11:00:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2023-11-13T11:00:44.271 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2023-11-13T11:00:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2023-11-13T11:00:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2023-11-13T11:00:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 32, 2023-11-13T11:00:44.272 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2023-11-13T11:00:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 32, 2023-11-13T11:00:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 1812, 2023-11-13T11:00:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2023-11-13T11:00:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-13T11:00:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2023-11-13T11:00:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2023-11-13T11:00:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:00:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 41530, 2023-11-13T11:00:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 10304748, 2023-11-13T11:00:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:00:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 41530, 2023-11-13T11:00:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2023-11-13T11:00:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2023-11-13T11:00:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 41530, 2023-11-13T11:00:44.275 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 10304748, 2023-11-13T11:00:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:00:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2023-11-13T11:00:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2023-11-13T11:00:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2023-11-13T11:00:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:00:44.277 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2023-11-13T11:00:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2023-11-13T11:00:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:00:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2023-11-13T11:00:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 8860, 2023-11-13T11:00:44.278 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 78264900, 2023-11-13T11:00:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 8860, 2023-11-13T11:00:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 78264900, 2023-11-13T11:00:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:00:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.279 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2023-11-13T11:00:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2023-11-13T11:00:44.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2023-11-13T11:00:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:00:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2023-11-13T11:00:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2023-11-13T11:00:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:00:44.281 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2023-11-13T11:00:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 8860, 2023-11-13T11:00:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 8860, 2023-11-13T11:00:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 8860, 2023-11-13T11:00:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 8860, 2023-11-13T11:00:44.282 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:00:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.283 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2023-11-13T11:00:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2023-11-13T11:00:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2023-11-13T11:00:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:00:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 1812, 2023-11-13T11:00:44.284 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 196668, 2023-11-13T11:00:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:00:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 1812, 2023-11-13T11:00:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2023-11-13T11:00:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2023-11-13T11:00:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 137, 2023-11-13T11:00:44.285 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 196668, 2023-11-13T11:00:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:00:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.286 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:00:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55f93add66e0": { 2023-11-13T11:00:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 62979, 2023-11-13T11:00:44.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2023-11-13T11:00:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:00:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 22082, 2023-11-13T11:00:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 77912817, 2023-11-13T11:00:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:00:44.288 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 22082, 2023-11-13T11:00:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2023-11-13T11:00:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2023-11-13T11:00:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 6596, 2023-11-13T11:00:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 77849838, 2023-11-13T11:00:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:00:44.289 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:00:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:00:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:00:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.290 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:00:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 -- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ce006ae40 msgr2=0x7f4ce006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ce006ae40 0x7f4ce006d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0036000 tx=0x7f4cf003b040 comp rx=0 tx=0).stop 2023-11-13T11:00:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 -- 172.21.15.16:0/3680154901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 msgr2=0x7f4d00159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 0x7f4d00159150 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800cef0 tx=0x7f4cf800a710 comp rx=0 tx=0).stop 2023-11-13T11:00:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 -- 172.21.15.16:0/3680154901 shutdown_connections 2023-11-13T11:00:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ce006ae40 0x7f4ce006d300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d00144570 0x7f4d00159150 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d000ce9e0 0x7f4d00158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 --2- 172.21.15.16:0/3680154901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4d000ce550 0x7f4d001f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 -- 172.21.15.16:0/3680154901 >> 172.21.15.16:0/3680154901 conn(0x7f4d0006a190 msgr2=0x7f4d000b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 -- 172.21.15.16:0/3680154901 shutdown_connections 2023-11-13T11:00:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.155+0000 7f4d04dce640 1 -- 172.21.15.16:0/3680154901 wait complete. 2023-11-13T11:00:44.295 INFO:tasks.cephfs.test_scrub_checks:num of strays: 1895 2023-11-13T11:00:44.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2023-11-13T11:00:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.530+0000 7f0700d83640 1 Processor -- start 2023-11-13T11:00:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.530+0000 7f0700d83640 1 -- start start 2023-11-13T11:00:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.532+0000 7f0700d83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0cdb80 0x7f06fc165b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.532+0000 7f0700d83640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc0cc980 con 0x7f06fc0ce160 2023-11-13T11:00:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.532+0000 7f0700d83640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc06df70 con 0x7f06fc15ae90 2023-11-13T11:00:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.532+0000 7f0700d83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc06dd60 con 0x7f06fc0cdb80 2023-11-13T11:00:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.532+0000 7f06f3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f06fc0ce160 0x7f06fc0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40886/0 (socket says 172.21.15.16:40886) 2023-11-13T11:00:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.533+0000 7f06f3fff640 1 -- 172.21.15.16:0/2083099986 learned_addr learned my addr 172.21.15.16:0/2083099986 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:00:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.533+0000 7f06fad9d640 1 --2- 172.21.15.16:0/2083099986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0cdb80 0x7f06fc165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.533+0000 7f06f9d9b640 1 -- 172.21.15.16:0/2083099986 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682361941 0 0) 0x7f06fc0cc980 con 0x7f06fc0ce160 2023-11-13T11:00:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.533+0000 7f06f9d9b640 1 -- 172.21.15.16:0/2083099986 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06e8003520 con 0x7f06fc0ce160 2023-11-13T11:00:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.534+0000 7f06f9d9b640 1 -- 172.21.15.16:0/2083099986 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525016919 0 0) 0x7f06fc06df70 con 0x7f06fc15ae90 2023-11-13T11:00:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.534+0000 7f06f9d9b640 1 -- 172.21.15.16:0/2083099986 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06fc0cc980 con 0x7f06fc15ae90 2023-11-13T11:00:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.534+0000 7f06fad9d640 1 -- 172.21.15.16:0/2083099986 >> v1:172.21.15.184:6789/0 conn(0x7f06fc15ae90 legacy=0x7f06fc0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.534+0000 7f06fad9d640 1 -- 172.21.15.16:0/2083099986 >> v1:172.21.15.16:6789/0 conn(0x7f06fc0ce160 legacy=0x7f06fc0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.534+0000 7f06fad9d640 1 -- 172.21.15.16:0/2083099986 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc06e250 con 0x7f06fc0cdb80 2023-11-13T11:00:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.535+0000 7f06fad9d640 1 --2- 172.21.15.16:0/2083099986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0cdb80 0x7f06fc165b10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009dc0 tx=0x7f06f402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6b48d2d148bf80 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.536+0000 7f06f9d9b640 1 -- 172.21.15.16:0/2083099986 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f402ccd0 con 0x7f06fc0cdb80 2023-11-13T11:00:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.536+0000 7f06f9d9b640 1 -- 172.21.15.16:0/2083099986 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f402ce30 con 0x7f06fc0cdb80 2023-11-13T11:00:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.536+0000 7f06f9d9b640 1 -- 172.21.15.16:0/2083099986 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4034660 con 0x7f06fc0cdb80 2023-11-13T11:00:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.536+0000 7f0700d83640 1 -- 172.21.15.16:0/2083099986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0cdb80 msgr2=0x7f06fc165b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.536+0000 7f0700d83640 1 --2- 172.21.15.16:0/2083099986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0cdb80 0x7f06fc165b10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009dc0 tx=0x7f06f402bf20 comp rx=0 tx=0).stop 2023-11-13T11:00:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.536+0000 7f0700d83640 1 -- 172.21.15.16:0/2083099986 shutdown_connections 2023-11-13T11:00:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.536+0000 7f0700d83640 1 --2- 172.21.15.16:0/2083099986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0cdb80 0x7f06fc165b10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.536+0000 7f0700d83640 1 -- 172.21.15.16:0/2083099986 >> 172.21.15.16:0/2083099986 conn(0x7f06fc0d0b10 msgr2=0x7f06fc0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.537+0000 7f0700d83640 1 -- 172.21.15.16:0/2083099986 shutdown_connections 2023-11-13T11:00:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.537+0000 7f0700d83640 1 -- 172.21.15.16:0/2083099986 wait complete. 2023-11-13T11:00:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.539+0000 7f0700d83640 1 Processor -- start 2023-11-13T11:00:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.539+0000 7f0700d83640 1 -- start start 2023-11-13T11:00:44.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.540+0000 7f0700d83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc0cdb80 0x7f06fc15d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.540+0000 7f0700d83640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0ce160 0x7f06fc0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.541+0000 7f0700d83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc15ae90 0x7f06fc152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.541+0000 7f0700d83640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0b51d0 con 0x7f06fc15ae90 2023-11-13T11:00:44.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.541+0000 7f0700d83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0b54b0 con 0x7f06fc0cdb80 2023-11-13T11:00:44.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.541+0000 7f0700d83640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0b5790 con 0x7f06fc0ce160 2023-11-13T11:00:44.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.541+0000 7f06fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc15ae90 0x7f06fc152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.541+0000 7f06fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc15ae90 0x7f06fc152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46402/0 (socket says 172.21.15.16:46402) 2023-11-13T11:00:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.541+0000 7f06fad9d640 1 -- 172.21.15.16:0/3150092248 learned_addr learned my addr 172.21.15.16:0/3150092248 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.541+0000 7f06fa59c640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc0cdb80 0x7f06fc15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.541+0000 7f06f3fff640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0ce160 0x7f06fc0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06fad9d640 1 -- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0ce160 msgr2=0x7f06fc0b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06fad9d640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0ce160 0x7f06fc0b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06fad9d640 1 -- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc0cdb80 msgr2=0x7f06fc15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06fad9d640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc0cdb80 0x7f06fc15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06fad9d640 1 -- 172.21.15.16:0/3150092248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f4009a70 con 0x7f06fc15ae90 2023-11-13T11:00:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06fa59c640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc0cdb80 0x7f06fc15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:00:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06fad9d640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc15ae90 0x7f06fc152b80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f06f402dfd0 tx=0x7f06f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06f2ffd640 1 -- 172.21.15.16:0/3150092248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f403d600 con 0x7f06fc15ae90 2023-11-13T11:00:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06f2ffd640 1 -- 172.21.15.16:0/3150092248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f403d760 con 0x7f06fc15ae90 2023-11-13T11:00:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f06f2ffd640 1 -- 172.21.15.16:0/3150092248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f403db10 con 0x7f06fc15ae90 2023-11-13T11:00:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.542+0000 7f0700d83640 1 -- 172.21.15.16:0/3150092248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc0b5a70 con 0x7f06fc15ae90 2023-11-13T11:00:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.543+0000 7f0700d83640 1 -- 172.21.15.16:0/3150092248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc0b6250 con 0x7f06fc15ae90 2023-11-13T11:00:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.545+0000 7f06f2ffd640 1 -- 172.21.15.16:0/3150092248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06f403b040 con 0x7f06fc15ae90 2023-11-13T11:00:44.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.546+0000 7f06f2ffd640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06dc06ae70 0x7f06dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.546+0000 7f06f2ffd640 1 -- 172.21.15.16:0/3150092248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f06f40ab9e0 con 0x7f06fc15ae90 2023-11-13T11:00:44.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.547+0000 7f06fa59c640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06dc06ae70 0x7f06dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.547+0000 7f07039f1740 1 Processor -- start 2023-11-13T11:00:44.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.547+0000 7f07039f1740 1 -- start start 2023-11-13T11:00:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f07039f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b37b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f07039f1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b34b5c0 0x56321b378810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f07039f1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b34f120 0x56321b378d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f07039f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321b30cb30 con 0x56321b37b5a0 2023-11-13T11:00:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f07039f1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321b2bee30 con 0x56321b34f120 2023-11-13T11:00:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f07039f1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56321b2bc520 con 0x56321b34b5c0 2023-11-13T11:00:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f06fa59c640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06dc06ae70 0x7f06dc06d330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0011e0 tx=0x7f06ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f06fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b37b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f06fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b37b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46404/0 (socket says 172.21.15.16:46404) 2023-11-13T11:00:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f06fa59c640 1 -- 172.21.15.16:0/2950161610 learned_addr learned my addr 172.21.15.16:0/2950161610 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f06fad9d640 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b34f120 0x56321b378d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.548+0000 7f06f3fff640 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b34b5c0 0x56321b378810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06fa59c640 1 -- 172.21.15.16:0/2950161610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b34b5c0 msgr2=0x56321b378810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06fa59c640 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b34b5c0 0x56321b378810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06fa59c640 1 -- 172.21.15.16:0/2950161610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b34f120 msgr2=0x56321b378d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06fa59c640 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b34f120 0x56321b378d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06fa59c640 1 -- 172.21.15.16:0/2950161610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56321b27a760 con 0x56321b37b5a0 2023-11-13T11:00:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06f3fff640 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b34b5c0 0x56321b378810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:00:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06fa59c640 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b37b980 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0050d0 tx=0x7f06ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24967b6994d2252 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06dbfff640 1 -- 172.21.15.16:0/2950161610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec00ee20 con 0x56321b37b5a0 2023-11-13T11:00:44.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06dbfff640 1 -- 172.21.15.16:0/2950161610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ec003800 con 0x56321b37b5a0 2023-11-13T11:00:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.549+0000 7f06dbfff640 1 -- 172.21.15.16:0/2950161610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec003b70 con 0x56321b37b5a0 2023-11-13T11:00:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.550+0000 7f07039f1740 1 -- 172.21.15.16:0/2950161610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 msgr2=0x56321b37b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.550+0000 7f07039f1740 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b37b980 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0050d0 tx=0x7f06ec011000 comp rx=0 tx=0).stop 2023-11-13T11:00:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.550+0000 7f07039f1740 1 -- 172.21.15.16:0/2950161610 shutdown_connections 2023-11-13T11:00:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.550+0000 7f07039f1740 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b34f120 0x56321b378d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.550+0000 7f07039f1740 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b34b5c0 0x56321b378810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.550+0000 7f07039f1740 1 --2- 172.21.15.16:0/2950161610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b37b980 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.550+0000 7f07039f1740 1 -- 172.21.15.16:0/2950161610 >> 172.21.15.16:0/2950161610 conn(0x56321b34d830 msgr2=0x56321b3963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.550+0000 7f07039f1740 1 -- 172.21.15.16:0/2950161610 shutdown_connections 2023-11-13T11:00:44.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.550+0000 7f07039f1740 1 -- 172.21.15.16:0/2950161610 wait complete. 2023-11-13T11:00:44.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.551+0000 7f07039f1740 1 -- start start 2023-11-13T11:00:44.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.551+0000 7f07039f1740 1 Processor -- start 2023-11-13T11:00:44.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.552+0000 7f07039f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b34f120 0x56321b395440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.552+0000 7f07039f1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b37b5a0 0x56321b3718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.552+0000 7f07039f1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b3a78a0 0x56321b375500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.552+0000 7f07039f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321b2b7e10 con 0x56321b34f120 2023-11-13T11:00:44.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.552+0000 7f07039f1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321b254080 con 0x56321b3a78a0 2023-11-13T11:00:44.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.552+0000 7f07039f1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56321b271a00 con 0x56321b37b5a0 2023-11-13T11:00:44.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.552+0000 7f07039f1740 10 client.0 _ll_register_callbacks cb 0x56321b274060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:00:44.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.552+0000 7f06fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b34f120 0x56321b395440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.553+0000 7f06fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b34f120 0x56321b395440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46406/0 (socket says 172.21.15.16:46406) 2023-11-13T11:00:44.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.553+0000 7f06fa59c640 1 -- 172.21.15.16:0/1111082554 learned_addr learned my addr 172.21.15.16:0/1111082554 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.553+0000 7f06fa59c640 1 -- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b37b5a0 msgr2=0x56321b3718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.553+0000 7f06fa59c640 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b37b5a0 0x56321b3718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.553+0000 7f06fa59c640 1 -- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b3a78a0 msgr2=0x56321b375500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.553+0000 7f06fa59c640 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b3a78a0 0x56321b375500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.553+0000 7f06fa59c640 1 -- 172.21.15.16:0/1111082554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ec00f040 con 0x56321b34f120 2023-11-13T11:00:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.553+0000 7f06fa59c640 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b34f120 0x56321b395440 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f06ec003ba0 tx=0x7f06ec000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71675910aa777d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.553+0000 7f06d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:00:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f06d97fa640 1 -- 172.21.15.16:0/1111082554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec027070 con 0x56321b34f120 2023-11-13T11:00:44.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f06d97fa640 1 -- 172.21.15.16:0/1111082554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f06ec02dca0 con 0x56321b34f120 2023-11-13T11:00:44.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f07039f1740 1 -- 172.21.15.16:0/1111082554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56321b2b0df0 con 0x56321b34f120 2023-11-13T11:00:44.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f06d97fa640 1 -- 172.21.15.16:0/1111082554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06ec02de60 con 0x56321b34f120 2023-11-13T11:00:44.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f07039f1740 10 client.5742 fetch_fsmap learned FSMap version 19 2023-11-13T11:00:44.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f07039f1740 1 -- 172.21.15.16:0/1111082554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x56321b2b6490 con 0x56321b34f120 2023-11-13T11:00:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f06d97fa640 1 -- 172.21.15.16:0/1111082554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56321b2b0df0 con 0x56321b34f120 2023-11-13T11:00:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f06d97fa640 1 -- 172.21.15.16:0/1111082554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f06ec021450 con 0x56321b34f120 2023-11-13T11:00:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f07039f1740 10 client.5742 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:00:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f07039f1740 10 client.5742 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:00:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.554+0000 7f07039f1740 20 client.5742 populate_metadata read hostname 'smithi016' 2023-11-13T11:00:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.555+0000 7f07039f1740 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x56321b394cd0 0x56321b370520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.555+0000 7f07039f1740 4 client.5742 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:00:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.555+0000 7f07039f1740 1 -- 172.21.15.16:0/1111082554 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56321b2b0b10 con 0x56321b394cd0 2023-11-13T11:00:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.555+0000 7f06f3fff640 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x56321b394cd0 0x56321b370520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.556+0000 7f06f3fff640 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x56321b394cd0 0x56321b370520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96b30f85b0b674dc server_cookie=8bdd314e1e2d858e in_seq=0 out_seq=0 2023-11-13T11:00:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.556+0000 7f06d97fa640 10 client.5742 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f06d97fa640 1 -- 172.21.15.16:0/1111082554 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56321b2b0b10 con 0x56321b394cd0 2023-11-13T11:00:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f06d97fa640 10 client.5742 handle_command_reply: tid=0 2023-11-13T11:00:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 client.5742 shutdown 2023-11-13T11:00:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 -- 172.21.15.16:0/1111082554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b34f120 msgr2=0x56321b395440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b34f120 0x56321b395440 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f06ec003ba0 tx=0x7f06ec000ed0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 -- 172.21.15.16:0/1111082554 shutdown_connections 2023-11-13T11:00:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b3a78a0 0x56321b375500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b37b5a0 0x56321b3718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x56321b394cd0 0x56321b370520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 --2- 172.21.15.16:0/1111082554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b34f120 0x56321b395440 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 -- 172.21.15.16:0/1111082554 >> 172.21.15.16:0/1111082554 conn(0x56321b34b5c0 msgr2=0x56321b37f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f06d97fa640 0 client.5742 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 -- 172.21.15.16:0/1111082554 shutdown_connections 2023-11-13T11:00:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.558+0000 7f07039f1740 1 -- 172.21.15.16:0/1111082554 wait complete. 2023-11-13T11:00:44.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.559+0000 7f07039f1740 20 client.5742 trim_cache size 0 max 16384 2023-11-13T11:00:44.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.575+0000 7f07039f1740 1 Processor -- start 2023-11-13T11:00:44.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.575+0000 7f07039f1740 1 -- start start 2023-11-13T11:00:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.576+0000 7f07039f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b3a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.576+0000 7f07039f1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b3a78a0 0x56321b370ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.576+0000 7f07039f1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b377840 0x56321b371410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.576+0000 7f07039f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321b01b180 con 0x56321b37b5a0 2023-11-13T11:00:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.576+0000 7f07039f1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321b3734c0 con 0x56321b3a78a0 2023-11-13T11:00:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.576+0000 7f07039f1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56321b396fe0 con 0x56321b377840 2023-11-13T11:00:44.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.576+0000 7f06fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b3a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b3a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46408/0 (socket says 172.21.15.16:46408) 2023-11-13T11:00:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06fa59c640 1 -- 172.21.15.16:0/2958333147 learned_addr learned my addr 172.21.15.16:0/2958333147 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06fad9d640 1 --2- 172.21.15.16:0/2958333147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b377840 0x56321b371410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06fa59c640 1 -- 172.21.15.16:0/2958333147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b377840 msgr2=0x56321b371410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06f3fff640 1 --2- 172.21.15.16:0/2958333147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b3a78a0 0x56321b370ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06fa59c640 1 --2- 172.21.15.16:0/2958333147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b377840 0x56321b371410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06fa59c640 1 -- 172.21.15.16:0/2958333147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b3a78a0 msgr2=0x56321b370ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06fa59c640 1 --2- 172.21.15.16:0/2958333147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b3a78a0 0x56321b370ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06fa59c640 1 -- 172.21.15.16:0/2958333147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56321b253910 con 0x56321b37b5a0 2023-11-13T11:00:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.577+0000 7f06fa59c640 1 --2- 172.21.15.16:0/2958333147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b3a6f00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000f10 tx=0x7f06ec027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a00b8bf02d59ec server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f06f0ff9640 1 -- 172.21.15.16:0/2958333147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec03b070 con 0x56321b37b5a0 2023-11-13T11:00:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f06f0ff9640 1 -- 172.21.15.16:0/2958333147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ec019070 con 0x56321b37b5a0 2023-11-13T11:00:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f06f0ff9640 1 -- 172.21.15.16:0/2958333147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec003bd0 con 0x56321b37b5a0 2023-11-13T11:00:44.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f07039f1740 1 -- 172.21.15.16:0/2958333147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 msgr2=0x56321b3a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f07039f1740 1 --2- 172.21.15.16:0/2958333147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b3a6f00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000f10 tx=0x7f06ec027000 comp rx=0 tx=0).stop 2023-11-13T11:00:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f07039f1740 1 -- 172.21.15.16:0/2958333147 shutdown_connections 2023-11-13T11:00:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f07039f1740 1 --2- 172.21.15.16:0/2958333147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b377840 0x56321b371410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f07039f1740 1 --2- 172.21.15.16:0/2958333147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b3a78a0 0x56321b370ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f07039f1740 1 --2- 172.21.15.16:0/2958333147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b37b5a0 0x56321b3a6f00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f07039f1740 1 -- 172.21.15.16:0/2958333147 >> 172.21.15.16:0/2958333147 conn(0x56321b34b5c0 msgr2=0x56321b3b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f07039f1740 1 -- 172.21.15.16:0/2958333147 shutdown_connections 2023-11-13T11:00:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.578+0000 7f07039f1740 1 -- 172.21.15.16:0/2958333147 wait complete. 2023-11-13T11:00:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.579+0000 7f07039f1740 1 -- start start 2023-11-13T11:00:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.579+0000 7f07039f1740 1 Processor -- start 2023-11-13T11:00:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.580+0000 7f07039f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b377840 0x56321b3f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.580+0000 7f07039f1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b37b5a0 0x56321b376600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.580+0000 7f07039f1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b3a78a0 0x56321b376b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.580+0000 7f07039f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321b3f3370 con 0x56321b377840 2023-11-13T11:00:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.580+0000 7f07039f1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321b3f5210 con 0x56321b37b5a0 2023-11-13T11:00:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.580+0000 7f07039f1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56321b3f5460 con 0x56321b3a78a0 2023-11-13T11:00:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.580+0000 7f07039f1740 10 client.0 _ll_register_callbacks cb 0x7f06ec001dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:00:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b377840 0x56321b3f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b377840 0x56321b3f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46414/0 (socket says 172.21.15.16:46414) 2023-11-13T11:00:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06fa59c640 1 -- 172.21.15.16:0/2406700138 learned_addr learned my addr 172.21.15.16:0/2406700138 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06fa59c640 1 -- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b3a78a0 msgr2=0x56321b376b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06fa59c640 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b3a78a0 0x56321b376b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06fa59c640 1 -- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b37b5a0 msgr2=0x56321b376600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06fa59c640 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b37b5a0 0x56321b376600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06fa59c640 1 -- 172.21.15.16:0/2406700138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56321b2b6490 con 0x56321b377840 2023-11-13T11:00:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06fa59c640 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b377840 0x56321b3f0f40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f06ec003fd0 tx=0x7f06ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172b990701e92a71 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.581+0000 7f06d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:00:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f06d97fa640 1 -- 172.21.15.16:0/2406700138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec027070 con 0x56321b377840 2023-11-13T11:00:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f06d97fa640 1 -- 172.21.15.16:0/2406700138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f06ec007c80 con 0x56321b377840 2023-11-13T11:00:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f07039f1740 1 -- 172.21.15.16:0/2406700138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06ec003800 con 0x56321b377840 2023-11-13T11:00:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f06d97fa640 1 -- 172.21.15.16:0/2406700138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06ec03b040 con 0x56321b377840 2023-11-13T11:00:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f07039f1740 10 client.5748 fetch_fsmap learned FSMap version 19 2023-11-13T11:00:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f07039f1740 1 -- 172.21.15.16:0/2406700138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x56321b278fc0 con 0x56321b377840 2023-11-13T11:00:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f06d97fa640 1 -- 172.21.15.16:0/2406700138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ec003800 con 0x56321b377840 2023-11-13T11:00:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f06d97fa640 1 -- 172.21.15.16:0/2406700138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f06ec036470 con 0x56321b377840 2023-11-13T11:00:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f07039f1740 10 client.5748 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:00:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.582+0000 7f07039f1740 10 client.5748 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:00:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.583+0000 7f07039f1740 20 client.5748 populate_metadata read hostname 'smithi016' 2023-11-13T11:00:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.583+0000 7f07039f1740 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x56321b374600 0x56321b3fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.583+0000 7f07039f1740 4 client.5748 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2023-11-13T11:00:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.583+0000 7f07039f1740 1 -- 172.21.15.16:0/2406700138 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x56321afb8920 con 0x56321b374600 2023-11-13T11:00:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.583+0000 7f06f3fff640 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x56321b374600 0x56321b3fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.584+0000 7f06f3fff640 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x56321b374600 0x56321b3fc2b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696198b2675b2f70 server_cookie=e692a82732744eeb in_seq=0 out_seq=0 2023-11-13T11:00:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.584+0000 7f06d97fa640 10 client.5748 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f06d97fa640 1 -- 172.21.15.16:0/2406700138 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56321afb8920 con 0x56321b374600 2023-11-13T11:00:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f06d97fa640 10 client.5748 handle_command_reply: tid=0 2023-11-13T11:00:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f07039f1740 1 client.5748 shutdown 2023-11-13T11:00:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f07039f1740 1 -- 172.21.15.16:0/2406700138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b377840 msgr2=0x56321b3f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f07039f1740 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b377840 0x56321b3f0f40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f06ec003fd0 tx=0x7f06ec012000 comp rx=0 tx=0).stop 2023-11-13T11:00:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f07039f1740 1 -- 172.21.15.16:0/2406700138 shutdown_connections 2023-11-13T11:00:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f07039f1740 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56321b3a78a0 0x56321b376b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f07039f1740 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56321b37b5a0 0x56321b376600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f07039f1740 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x56321b374600 0x56321b3fc2b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f07039f1740 1 --2- 172.21.15.16:0/2406700138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56321b377840 0x56321b3f0f40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f07039f1740 1 -- 172.21.15.16:0/2406700138 >> 172.21.15.16:0/2406700138 conn(0x56321b34b5c0 msgr2=0x56321b370ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.602+0000 7f06d97fa640 0 client.5748 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.603+0000 7f07039f1740 1 -- 172.21.15.16:0/2406700138 shutdown_connections 2023-11-13T11:00:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.603+0000 7f07039f1740 1 -- 172.21.15.16:0/2406700138 wait complete. 2023-11-13T11:00:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.603+0000 7f07039f1740 20 client.5748 trim_cache size 0 max 16384 2023-11-13T11:00:44.613 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:00:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-11-13T11:00:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "7a5a8d84-56f5-41cd-abb3-70f6c9cc035f", 2023-11-13T11:00:44.614 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-11-13T11:00:44.615 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:00:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 -- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06dc06ae70 msgr2=0x7f06dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06dc06ae70 0x7f06dc06d330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0011e0 tx=0x7f06ec009040 comp rx=0 tx=0).stop 2023-11-13T11:00:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 -- 172.21.15.16:0/3150092248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc15ae90 msgr2=0x7f06fc152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc15ae90 0x7f06fc152b80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f06f402dfd0 tx=0x7f06f4004a40 comp rx=0 tx=0).stop 2023-11-13T11:00:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 -- 172.21.15.16:0/3150092248 shutdown_connections 2023-11-13T11:00:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06dc06ae70 0x7f06dc06d330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc15ae90 0x7f06fc152b80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06fc0ce160 0x7f06fc0b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 --2- 172.21.15.16:0/3150092248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06fc0cdb80 0x7f06fc15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 -- 172.21.15.16:0/3150092248 >> 172.21.15.16:0/3150092248 conn(0x7f06fc0d0b10 msgr2=0x7f06fc159610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 -- 172.21.15.16:0/3150092248 shutdown_connections 2023-11-13T11:00:44.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.604+0000 7f0700d83640 1 -- 172.21.15.16:0/3150092248 wait complete. 2023-11-13T11:00:44.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-11-13T11:00:44.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.879+0000 7fba72cad640 1 Processor -- start 2023-11-13T11:00:44.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.879+0000 7fba72cad640 1 -- start start 2023-11-13T11:00:44.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba72cad640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fba6c15b8e0 0x7fba6c06c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba72cad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba6c15bcc0 0x7fba6c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba72cad640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c0ce090 0x7fba6c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba72cad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6c0ce510 con 0x7fba6c15bcc0 2023-11-13T11:00:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba72cad640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6c06b860 con 0x7fba6c15b8e0 2023-11-13T11:00:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba72cad640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c06bb40 con 0x7fba6c0ce090 2023-11-13T11:00:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba714aa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba6c15bcc0 0x7fba6c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba714aa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba6c15bcc0 0x7fba6c0cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46424/0 (socket says 172.21.15.16:46424) 2023-11-13T11:00:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba714aa640 1 -- 172.21.15.16:0/2787431067 learned_addr learned my addr 172.21.15.16:0/2787431067 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba71cab640 1 --2- 172.21.15.16:0/2787431067 >> v2:172.21.15.184:3300/0 conn(0x7fba6c15b8e0 0x7fba6c06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.881+0000 7fba724ac640 1 --2- 172.21.15.16:0/2787431067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c0ce090 0x7fba6c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.882+0000 7fba714aa640 1 -- 172.21.15.16:0/2787431067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c0ce090 msgr2=0x7fba6c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.882+0000 7fba714aa640 1 --2- 172.21.15.16:0/2787431067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c0ce090 0x7fba6c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.882+0000 7fba714aa640 1 -- 172.21.15.16:0/2787431067 >> v2:172.21.15.184:3300/0 conn(0x7fba6c15b8e0 msgr2=0x7fba6c06c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.882+0000 7fba714aa640 1 --2- 172.21.15.16:0/2787431067 >> v2:172.21.15.184:3300/0 conn(0x7fba6c15b8e0 0x7fba6c06c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.882+0000 7fba714aa640 1 -- 172.21.15.16:0/2787431067 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c06be20 con 0x7fba6c15bcc0 2023-11-13T11:00:44.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.883+0000 7fba714aa640 1 --2- 172.21.15.16:0/2787431067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c15bcc0 0x7fba6c0cdb50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fba6c071d70 tx=0x7fba5c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908b84e2ad56b0c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.884+0000 7fba70ca9640 1 -- 172.21.15.16:0/2787431067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c031860 con 0x7fba6c15bcc0 2023-11-13T11:00:44.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.884+0000 7fba70ca9640 1 -- 172.21.15.16:0/2787431067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c0319c0 con 0x7fba6c15bcc0 2023-11-13T11:00:44.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.884+0000 7fba70ca9640 1 -- 172.21.15.16:0/2787431067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c031d50 con 0x7fba6c15bcc0 2023-11-13T11:00:44.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.884+0000 7fba72cad640 1 -- 172.21.15.16:0/2787431067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c15bcc0 msgr2=0x7fba6c0cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.884+0000 7fba72cad640 1 --2- 172.21.15.16:0/2787431067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c15bcc0 0x7fba6c0cdb50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fba6c071d70 tx=0x7fba5c02b900 comp rx=0 tx=0).stop 2023-11-13T11:00:44.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.885+0000 7fba72cad640 1 -- 172.21.15.16:0/2787431067 shutdown_connections 2023-11-13T11:00:44.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.885+0000 7fba72cad640 1 --2- 172.21.15.16:0/2787431067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c0ce090 0x7fba6c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.885+0000 7fba72cad640 1 --2- 172.21.15.16:0/2787431067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c15bcc0 0x7fba6c0cdb50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.885+0000 7fba72cad640 1 --2- 172.21.15.16:0/2787431067 >> v2:172.21.15.184:3300/0 conn(0x7fba6c15b8e0 0x7fba6c06c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.885+0000 7fba72cad640 1 -- 172.21.15.16:0/2787431067 >> 172.21.15.16:0/2787431067 conn(0x7fba6c0b0970 msgr2=0x7fba6c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.885+0000 7fba72cad640 1 -- 172.21.15.16:0/2787431067 shutdown_connections 2023-11-13T11:00:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.885+0000 7fba72cad640 1 -- 172.21.15.16:0/2787431067 wait complete. 2023-11-13T11:00:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.887+0000 7fba72cad640 1 Processor -- start 2023-11-13T11:00:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.887+0000 7fba72cad640 1 -- start start 2023-11-13T11:00:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.888+0000 7fba72cad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0ce090 0x7fba6c155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.888+0000 7fba72cad640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c15b8e0 0x7fba6c071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.888+0000 7fba72cad640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba6c06fef0 0x7fba6c156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.888+0000 7fba72cad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c070490 con 0x7fba6c0ce090 2023-11-13T11:00:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.888+0000 7fba72cad640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c070770 con 0x7fba6c06fef0 2023-11-13T11:00:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.888+0000 7fba72cad640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c070a50 con 0x7fba6c15b8e0 2023-11-13T11:00:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.888+0000 7fba71cab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0ce090 0x7fba6c155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba71cab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0ce090 0x7fba6c155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46440/0 (socket says 172.21.15.16:46440) 2023-11-13T11:00:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba71cab640 1 -- 172.21.15.16:0/355025667 learned_addr learned my addr 172.21.15.16:0/355025667 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba714aa640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c15b8e0 0x7fba6c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba724ac640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba6c06fef0 0x7fba6c156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba71cab640 1 -- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c15b8e0 msgr2=0x7fba6c071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba71cab640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c15b8e0 0x7fba6c071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba71cab640 1 -- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba6c06fef0 msgr2=0x7fba6c156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba71cab640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba6c06fef0 0x7fba6c156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba71cab640 1 -- 172.21.15.16:0/355025667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0094d0 con 0x7fba6c0ce090 2023-11-13T11:00:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.889+0000 7fba71cab640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0ce090 0x7fba6c155980 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fba6800e9b0 tx=0x7fba6800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.890+0000 7fba627fc640 1 -- 172.21.15.16:0/355025667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68009670 con 0x7fba6c0ce090 2023-11-13T11:00:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.890+0000 7fba627fc640 1 -- 172.21.15.16:0/355025667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6800c9a0 con 0x7fba6c0ce090 2023-11-13T11:00:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.890+0000 7fba627fc640 1 -- 172.21.15.16:0/355025667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6800cd50 con 0x7fba6c0ce090 2023-11-13T11:00:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.890+0000 7fba72cad640 1 -- 172.21.15.16:0/355025667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6c070d30 con 0x7fba6c0ce090 2023-11-13T11:00:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.890+0000 7fba72cad640 1 -- 172.21.15.16:0/355025667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6c152690 con 0x7fba6c0ce090 2023-11-13T11:00:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.893+0000 7fba627fc640 1 -- 172.21.15.16:0/355025667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba68013470 con 0x7fba6c0ce090 2023-11-13T11:00:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.894+0000 7fba627fc640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba5006ae90 0x7fba5006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.894+0000 7fba627fc640 1 -- 172.21.15.16:0/355025667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fba68089850 con 0x7fba6c0ce090 2023-11-13T11:00:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.895+0000 7fba714aa640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba5006ae90 0x7fba5006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.895+0000 7fba76b7b740 1 Processor -- start 2023-11-13T11:00:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.895+0000 7fba76b7b740 1 -- start start 2023-11-13T11:00:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.896+0000 7fba714aa640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba5006ae90 0x7fba5006d350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fba5c036000 tx=0x7fba5c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.896+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a5628210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.896+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 0x5647a5658000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.896+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5628750 0x5647a5658540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.896+0000 7fba76b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a559ae90 con 0x5647a5655650 2023-11-13T11:00:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.896+0000 7fba76b7b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a5598580 con 0x5647a5655270 2023-11-13T11:00:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.896+0000 7fba76b7b740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647a5593e70 con 0x5647a5628750 2023-11-13T11:00:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba71cab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a5628210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba714aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 0x5647a5658000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba71cab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a5628210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44304/0 (socket says 172.21.15.16:44304) 2023-11-13T11:00:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba71cab640 1 -- 172.21.15.16:0/1736124757 learned_addr learned my addr 172.21.15.16:0/1736124757 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba724ac640 1 --2- 172.21.15.16:0/1736124757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5628750 0x5647a5658540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba714aa640 1 -- 172.21.15.16:0/1736124757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5628750 msgr2=0x5647a5658540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba714aa640 1 --2- 172.21.15.16:0/1736124757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5628750 0x5647a5658540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba714aa640 1 -- 172.21.15.16:0/1736124757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 msgr2=0x5647a5628210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba714aa640 1 --2- 172.21.15.16:0/1736124757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a5628210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba714aa640 1 -- 172.21.15.16:0/1736124757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a555e020 con 0x5647a5655650 2023-11-13T11:00:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.897+0000 7fba724ac640 1 --2- 172.21.15.16:0/1736124757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5628750 0x5647a5658540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:00:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.898+0000 7fba714aa640 1 --2- 172.21.15.16:0/1736124757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 0x5647a5658000 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fba6c06dd00 tx=0x7fba5c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74f7e20ea88c5e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.898+0000 7fba4f7fe640 1 -- 172.21.15.16:0/1736124757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c049070 con 0x5647a5655650 2023-11-13T11:00:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.898+0000 7fba4f7fe640 1 -- 172.21.15.16:0/1736124757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c044b10 con 0x5647a5655650 2023-11-13T11:00:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.898+0000 7fba4f7fe640 1 -- 172.21.15.16:0/1736124757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c044e80 con 0x5647a5655650 2023-11-13T11:00:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.899+0000 7fba76b7b740 1 -- 172.21.15.16:0/1736124757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 msgr2=0x5647a5658000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.899+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1736124757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 0x5647a5658000 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fba6c06dd00 tx=0x7fba5c030000 comp rx=0 tx=0).stop 2023-11-13T11:00:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.899+0000 7fba76b7b740 1 -- 172.21.15.16:0/1736124757 shutdown_connections 2023-11-13T11:00:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.899+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1736124757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5628750 0x5647a5658540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.899+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1736124757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 0x5647a5658000 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.899+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1736124757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a5628210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.899+0000 7fba76b7b740 1 -- 172.21.15.16:0/1736124757 >> 172.21.15.16:0/1736124757 conn(0x5647a562a480 msgr2=0x5647a5673000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.899+0000 7fba76b7b740 1 -- 172.21.15.16:0/1736124757 shutdown_connections 2023-11-13T11:00:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.899+0000 7fba76b7b740 1 -- 172.21.15.16:0/1736124757 wait complete. 2023-11-13T11:00:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.900+0000 7fba76b7b740 1 -- start start 2023-11-13T11:00:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.901+0000 7fba76b7b740 1 Processor -- start 2023-11-13T11:00:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.902+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a56508e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.902+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a56538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.902+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5653e00 0x5647a5644d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.902+0000 7fba76b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a5530480 con 0x5647a5653e00 2023-11-13T11:00:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.902+0000 7fba76b7b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a554ddc0 con 0x5647a5655270 2023-11-13T11:00:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.902+0000 7fba76b7b740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647a528cf90 con 0x5647a5655650 2023-11-13T11:00:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.902+0000 7fba76b7b740 10 client.0 _ll_register_callbacks cb 0x5647a52a5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:00:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba714aa640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a56538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba71cab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a56508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba724ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5653e00 0x5647a5644d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba714aa640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a56538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:37742/0 (socket says 172.21.15.16:37742) 2023-11-13T11:00:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba71cab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a56508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44316/0 (socket says 172.21.15.16:44316) 2023-11-13T11:00:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba714aa640 1 -- 172.21.15.16:0/1284742909 learned_addr learned my addr 172.21.15.16:0/1284742909 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba714aa640 1 -- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 msgr2=0x5647a56508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba714aa640 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a56508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba714aa640 1 -- 172.21.15.16:0/1284742909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5653e00 msgr2=0x5647a5644d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba714aa640 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5653e00 0x5647a5644d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba714aa640 1 -- 172.21.15.16:0/1284742909 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6c06be20 con 0x5647a5655650 2023-11-13T11:00:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.903+0000 7fba724ac640 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5653e00 0x5647a5644d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T11:00:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.904+0000 7fba714aa640 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a56538c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba5c031fd0 tx=0x7fba5c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738fd5030195daa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.904+0000 7fba4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.184:3301/0 2023-11-13T11:00:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.904+0000 7fba4cff9640 1 -- 172.21.15.16:0/1284742909 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c049040 con 0x5647a5655650 2023-11-13T11:00:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.904+0000 7fba76b7b740 1 -- 172.21.15.16:0/1284742909 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647a558ce50 con 0x5647a5655650 2023-11-13T11:00:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.904+0000 7fba4cff9640 1 -- 172.21.15.16:0/1284742909 <== mon.2 v2:172.21.15.184:3301/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fba5c03f070 con 0x5647a5655650 2023-11-13T11:00:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.904+0000 7fba4cff9640 1 -- 172.21.15.16:0/1284742909 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba5c04ee40 con 0x5647a5655650 2023-11-13T11:00:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba76b7b740 10 client.4862 fetch_fsmap learned FSMap version 19 2023-11-13T11:00:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba76b7b740 1 -- 172.21.15.16:0/1284742909 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x5647a55924f0 con 0x5647a5655650 2023-11-13T11:00:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba4cff9640 1 -- 172.21.15.16:0/1284742909 <== mon.2 v2:172.21.15.184:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5647a558ce50 con 0x5647a5655650 2023-11-13T11:00:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba4cff9640 1 -- 172.21.15.16:0/1284742909 <== mon.2 v2:172.21.15.184:3301/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fba5c059640 con 0x5647a5655650 2023-11-13T11:00:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba76b7b740 10 client.4862 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:00:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba76b7b740 10 client.4862 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:00:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba76b7b740 20 client.4862 populate_metadata read hostname 'smithi016' 2023-11-13T11:00:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5647a563eb40 0x5647a563ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba76b7b740 4 client.4862 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:00:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.905+0000 7fba76b7b740 1 -- 172.21.15.16:0/1284742909 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647a558cb70 con 0x5647a563eb40 2023-11-13T11:00:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.906+0000 7fba71cab640 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5647a563eb40 0x5647a563ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.907+0000 7fba71cab640 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5647a563eb40 0x5647a563ef50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ad40778811d91f9 server_cookie=1156cee2eef64a0e in_seq=0 out_seq=0 2023-11-13T11:00:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.907+0000 7fba4cff9640 10 client.4862 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba4cff9640 1 -- 172.21.15.16:0/1284742909 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647a558cb70 con 0x5647a563eb40 2023-11-13T11:00:44.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba4cff9640 10 client.4862 handle_command_reply: tid=0 2023-11-13T11:00:44.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba76b7b740 1 client.4862 shutdown 2023-11-13T11:00:44.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba76b7b740 1 -- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 msgr2=0x5647a56538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a56538c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba5c031fd0 tx=0x7fba5c004730 comp rx=0 tx=0).stop 2023-11-13T11:00:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba76b7b740 1 -- 172.21.15.16:0/1284742909 shutdown_connections 2023-11-13T11:00:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5647a563eb40 0x5647a563ef50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5653e00 0x5647a5644d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a56538c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba76b7b740 1 --2- 172.21.15.16:0/1284742909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5655270 0x5647a56508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba4cff9640 0 client.4862 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.908+0000 7fba76b7b740 1 -- 172.21.15.16:0/1284742909 >> 172.21.15.16:0/1284742909 conn(0x5647a5628750 msgr2=0x5647a5654930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.909+0000 7fba76b7b740 1 -- 172.21.15.16:0/1284742909 shutdown_connections 2023-11-13T11:00:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.909+0000 7fba76b7b740 1 -- 172.21.15.16:0/1284742909 wait complete. 2023-11-13T11:00:44.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.909+0000 7fba76b7b740 20 client.4862 trim_cache size 0 max 16384 2023-11-13T11:00:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.932+0000 7fba76b7b740 1 Processor -- start 2023-11-13T11:00:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.932+0000 7fba76b7b740 1 -- start start 2023-11-13T11:00:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.933+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a5675640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.933+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655270 0x5647a5658ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.934+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a5673610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.934+0000 7fba76b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a5627210 con 0x5647a5655270 2023-11-13T11:00:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.934+0000 7fba76b7b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a562c410 con 0x5647a5653e00 2023-11-13T11:00:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.934+0000 7fba76b7b740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647a562dce0 con 0x5647a5655650 2023-11-13T11:00:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.934+0000 7fba714aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655270 0x5647a5658ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.934+0000 7fba71cab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a5675640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.934+0000 7fba71cab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a5675640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44324/0 (socket says 172.21.15.16:44324) 2023-11-13T11:00:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.934+0000 7fba71cab640 1 -- 172.21.15.16:0/3062125468 learned_addr learned my addr 172.21.15.16:0/3062125468 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.934+0000 7fba724ac640 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a5673610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.935+0000 7fba71cab640 1 -- 172.21.15.16:0/3062125468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 msgr2=0x5647a5673610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.935+0000 7fba71cab640 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a5673610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.935+0000 7fba71cab640 1 -- 172.21.15.16:0/3062125468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655270 msgr2=0x5647a5658ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.935+0000 7fba71cab640 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655270 0x5647a5658ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.935+0000 7fba71cab640 1 -- 172.21.15.16:0/3062125468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a552fd10 con 0x5647a5653e00 2023-11-13T11:00:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.935+0000 7fba714aa640 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655270 0x5647a5658ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T11:00:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.935+0000 7fba724ac640 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a5673610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:00:44.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.935+0000 7fba71cab640 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a5675640 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fba68058290 tx=0x7fba68089040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bdd60b61680b6b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.936+0000 7fba4ffff640 1 -- 172.21.15.16:0/3062125468 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68087cf0 con 0x5647a5653e00 2023-11-13T11:00:44.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.936+0000 7fba4ffff640 1 -- 172.21.15.16:0/3062125468 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba68054ca0 con 0x5647a5653e00 2023-11-13T11:00:44.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.936+0000 7fba4ffff640 1 -- 172.21.15.16:0/3062125468 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68060550 con 0x5647a5653e00 2023-11-13T11:00:44.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.937+0000 7fba76b7b740 1 -- 172.21.15.16:0/3062125468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 msgr2=0x5647a5675640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.937+0000 7fba76b7b740 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a5675640 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fba68058290 tx=0x7fba68089040 comp rx=0 tx=0).stop 2023-11-13T11:00:44.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.937+0000 7fba76b7b740 1 -- 172.21.15.16:0/3062125468 shutdown_connections 2023-11-13T11:00:44.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.937+0000 7fba76b7b740 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655650 0x5647a5673610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.937+0000 7fba76b7b740 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655270 0x5647a5658ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.937+0000 7fba76b7b740 1 --2- 172.21.15.16:0/3062125468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a5675640 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.937+0000 7fba76b7b740 1 -- 172.21.15.16:0/3062125468 >> 172.21.15.16:0/3062125468 conn(0x5647a5628750 msgr2=0x5647a565b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.937+0000 7fba76b7b740 1 -- 172.21.15.16:0/3062125468 shutdown_connections 2023-11-13T11:00:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.937+0000 7fba76b7b740 1 -- 172.21.15.16:0/3062125468 wait complete. 2023-11-13T11:00:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.938+0000 7fba76b7b740 1 -- start start 2023-11-13T11:00:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.938+0000 7fba76b7b740 1 Processor -- start 2023-11-13T11:00:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.939+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a562e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.939+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655270 0x5647a5657a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.939+0000 7fba76b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 0x5647a56400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.939+0000 7fba76b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a563fc80 con 0x5647a5655650 2023-11-13T11:00:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.939+0000 7fba76b7b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a5658360 con 0x5647a5653e00 2023-11-13T11:00:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.939+0000 7fba76b7b740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647a56508c0 con 0x5647a5655270 2023-11-13T11:00:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.939+0000 7fba76b7b740 10 client.0 _ll_register_callbacks cb 0x7fba5c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:00:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba71cab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a562e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba71cab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a562e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:44328/0 (socket says 172.21.15.16:44328) 2023-11-13T11:00:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba71cab640 1 -- 172.21.15.16:0/727022570 learned_addr learned my addr 172.21.15.16:0/727022570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:00:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba714aa640 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655270 0x5647a5657a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba724ac640 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 0x5647a56400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba71cab640 1 -- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655270 msgr2=0x5647a5657a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba71cab640 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655270 0x5647a5657a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba71cab640 1 -- 172.21.15.16:0/727022570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 msgr2=0x5647a56400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:00:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba71cab640 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 0x5647a56400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.940+0000 7fba71cab640 1 -- 172.21.15.16:0/727022570 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647a55924f0 con 0x5647a5653e00 2023-11-13T11:00:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.941+0000 7fba71cab640 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a562e600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba68054a90 tx=0x7fba5c009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=865dfa136e249982 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:00:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.941+0000 7fba4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-11-13T11:00:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.941+0000 7fba4cff9640 1 -- 172.21.15.16:0/727022570 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68060550 con 0x5647a5653e00 2023-11-13T11:00:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.941+0000 7fba4cff9640 1 -- 172.21.15.16:0/727022570 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fba68081bc0 con 0x5647a5653e00 2023-11-13T11:00:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.941+0000 7fba76b7b740 1 -- 172.21.15.16:0/727022570 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba5c044b10 con 0x5647a5653e00 2023-11-13T11:00:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.941+0000 7fba4cff9640 1 -- 172.21.15.16:0/727022570 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba68081d80 con 0x5647a5653e00 2023-11-13T11:00:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.941+0000 7fba76b7b740 10 client.5119 fetch_fsmap learned FSMap version 19 2023-11-13T11:00:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.941+0000 7fba76b7b740 1 -- 172.21.15.16:0/727022570 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x7fba6c152690 con 0x5647a5653e00 2023-11-13T11:00:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.942+0000 7fba4cff9640 1 -- 172.21.15.16:0/727022570 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c044b10 con 0x5647a5653e00 2023-11-13T11:00:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.942+0000 7fba4cff9640 1 -- 172.21.15.16:0/727022570 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fba68076030 con 0x5647a5653e00 2023-11-13T11:00:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.942+0000 7fba76b7b740 10 client.5119 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:00:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.942+0000 7fba76b7b740 10 client.5119 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:00:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.942+0000 7fba76b7b740 20 client.5119 populate_metadata read hostname 'smithi016' 2023-11-13T11:00:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.942+0000 7fba76b7b740 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5647a56405e0 0x5647a5673610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:00:44.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.942+0000 7fba76b7b740 4 client.5119 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-13T11:00:44.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.942+0000 7fba76b7b740 1 -- 172.21.15.16:0/727022570 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5647a52947e0 con 0x5647a56405e0 2023-11-13T11:00:44.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.942+0000 7fba714aa640 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5647a56405e0 0x5647a5673610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:00:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.943+0000 7fba714aa640 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5647a56405e0 0x5647a5673610 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a985a3aa66fd2a server_cookie=c750b5816a3918df in_seq=0 out_seq=0 2023-11-13T11:00:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.943+0000 7fba4cff9640 10 client.5119 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba4cff9640 1 -- 172.21.15.16:0/727022570 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+274 (crc 0 0 0) 0x5647a52947e0 con 0x5647a56405e0 2023-11-13T11:00:44.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba4cff9640 10 client.5119 handle_command_reply: tid=0 2023-11-13T11:00:44.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 client.5119 shutdown 2023-11-13T11:00:44.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 -- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 msgr2=0x5647a562e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a562e600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba68054a90 tx=0x7fba5c009240 comp rx=0 tx=0).stop 2023-11-13T11:00:44.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 -- 172.21.15.16:0/727022570 shutdown_connections 2023-11-13T11:00:44.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5647a56405e0 0x5647a5673610 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5647a5655650 0x5647a56400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5647a5655270 0x5647a5657a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 --2- 172.21.15.16:0/727022570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5647a5653e00 0x5647a562e600 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 -- 172.21.15.16:0/727022570 >> 172.21.15.16:0/727022570 conn(0x5647a5628750 msgr2=0x5647a5676510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba4cff9640 0 client.5119 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:00:44.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 -- 172.21.15.16:0/727022570 shutdown_connections 2023-11-13T11:00:44.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 1 -- 172.21.15.16:0/727022570 wait complete. 2023-11-13T11:00:44.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.944+0000 7fba76b7b740 20 client.5119 trim_cache size 0 max 16384 2023-11-13T11:00:44.967 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:00:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "scrub active (480 inodes in the stack)", 2023-11-13T11:00:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2023-11-13T11:00:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "7a5a8d84-56f5-41cd-abb3-70f6c9cc035f": { 2023-11-13T11:00:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2023-11-13T11:00:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "7a5a8d84-56f5-41cd-abb3-70f6c9cc035f", 2023-11-13T11:00:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,scrub_mdsdir" 2023-11-13T11:00:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:00:44.969 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:00:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 -- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba5006ae90 msgr2=0x7fba5006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba5006ae90 0x7fba5006d350 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fba5c036000 tx=0x7fba5c03c040 comp rx=0 tx=0).stop 2023-11-13T11:00:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 -- 172.21.15.16:0/355025667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0ce090 msgr2=0x7fba6c155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:00:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0ce090 0x7fba6c155980 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fba6800e9b0 tx=0x7fba6800ee70 comp rx=0 tx=0).stop 2023-11-13T11:00:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 -- 172.21.15.16:0/355025667 shutdown_connections 2023-11-13T11:00:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba5006ae90 0x7fba5006d350 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba6c06fef0 0x7fba6c156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba6c15b8e0 0x7fba6c071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 --2- 172.21.15.16:0/355025667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0ce090 0x7fba6c155980 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:00:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 -- 172.21.15.16:0/355025667 >> 172.21.15.16:0/355025667 conn(0x7fba6c0b0970 msgr2=0x7fba6c0b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:00:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.945+0000 7fba72cad640 1 -- 172.21.15.16:0/355025667 shutdown_connections 2023-11-13T11:00:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:00:44.946+0000 7fba72cad640 1 -- 172.21.15.16:0/355025667 wait complete. 2023-11-13T11:00:44.973 INFO:tasks.cephfs.filesystem:scrub status for tag:7a5a8d84-56f5-41cd-abb3-70f6c9cc035f - {'path': '/', 'tag': '7a5a8d84-56f5-41cd-abb3-70f6c9cc035f', 'options': 'recursive,scrub_mdsdir'} 2023-11-13T11:00:48.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:00:48.972 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:01:14.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-11-13T11:01:15.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.213+0000 7ff9d40c6640 1 Processor -- start 2023-11-13T11:01:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.214+0000 7ff9d40c6640 1 -- start start 2023-11-13T11:01:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.215+0000 7ff9d40c6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff9cc15d1c0 0x7ff9cc06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.215+0000 7ff9d40c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc0ccad0 0x7ff9cc06e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.215+0000 7ff9d40c6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc15d5a0 con 0x7ff9cc06e4b0 2023-11-13T11:01:15.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.215+0000 7ff9d40c6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc0d00a0 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.215+0000 7ff9d40c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0cfae0 con 0x7ff9cc0ccad0 2023-11-13T11:01:15.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.216+0000 7ff9d1e62640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff9cc15d1c0 0x7ff9cc06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.216+0000 7ff9d2663640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc0ccad0 0x7ff9cc06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.216+0000 7ff9d1e62640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff9cc15d1c0 0x7ff9cc06df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:41610/0 (socket says 172.21.15.16:41610) 2023-11-13T11:01:15.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.216+0000 7ff9d1e62640 1 -- 172.21.15.16:0/4204915207 learned_addr learned my addr 172.21.15.16:0/4204915207 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.216+0000 7ff9d2663640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc0ccad0 0x7ff9cc06e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:58186/0 (socket says 172.21.15.16:58186) 2023-11-13T11:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.216+0000 7ff9d1661640 1 --1- 172.21.15.16:0/4204915207 >> v1:172.21.15.184:6789/0 conn(0x7ff9cc06e4b0 0x7ff9cc0cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:55066/0 (socket says 172.21.15.16:55066) 2023-11-13T11:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.217+0000 7ff9d1e62640 1 -- 172.21.15.16:0/4204915207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc0ccad0 msgr2=0x7ff9cc06e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.217+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/4204915207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc0ccad0 0x7ff9cc06e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.217+0000 7ff9d1e62640 1 -- 172.21.15.16:0/4204915207 >> v1:172.21.15.184:6789/0 conn(0x7ff9cc06e4b0 legacy=0x7ff9cc0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.217+0000 7ff9d1e62640 1 -- 172.21.15.16:0/4204915207 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc0cfdc0 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.218+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/4204915207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc15d1c0 0x7ff9cc06df70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009870 tx=0x7ff9c802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afee2e783ca39856 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.219+0000 7ff9d0e60640 1 -- 172.21.15.16:0/4204915207 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8030840 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.219+0000 7ff9d0e60640 1 -- 172.21.15.16:0/4204915207 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c80309a0 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.219+0000 7ff9d0e60640 1 -- 172.21.15.16:0/4204915207 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8030d30 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.220+0000 7ff9d40c6640 1 -- 172.21.15.16:0/4204915207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc15d1c0 msgr2=0x7ff9cc06df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.220+0000 7ff9d40c6640 1 --2- 172.21.15.16:0/4204915207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc15d1c0 0x7ff9cc06df70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009870 tx=0x7ff9c802b980 comp rx=0 tx=0).stop 2023-11-13T11:01:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.220+0000 7ff9d40c6640 1 -- 172.21.15.16:0/4204915207 shutdown_connections 2023-11-13T11:01:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.220+0000 7ff9d40c6640 1 --2- 172.21.15.16:0/4204915207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc0ccad0 0x7ff9cc06e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.220+0000 7ff9d40c6640 1 --2- 172.21.15.16:0/4204915207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc15d1c0 0x7ff9cc06df70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.220+0000 7ff9d40c6640 1 -- 172.21.15.16:0/4204915207 >> 172.21.15.16:0/4204915207 conn(0x7ff9cc0b2600 msgr2=0x7ff9cc0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.220+0000 7ff9d40c6640 1 -- 172.21.15.16:0/4204915207 shutdown_connections 2023-11-13T11:01:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.220+0000 7ff9d40c6640 1 -- 172.21.15.16:0/4204915207 wait complete. 2023-11-13T11:01:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.223+0000 7ff9d40c6640 1 Processor -- start 2023-11-13T11:01:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.223+0000 7ff9d40c6640 1 -- start start 2023-11-13T11:01:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.224+0000 7ff9d40c6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc06e4b0 0x7ff9cc0c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.224+0000 7ff9d40c6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc0ccad0 0x7ff9cc06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d40c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc15d1c0 0x7ff9cc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d40c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0c7ba0 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d40c6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0c53b0 con 0x7ff9cc0ccad0 2023-11-13T11:01:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d40c6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0b3ac0 con 0x7ff9cc06e4b0 2023-11-13T11:01:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d2663640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc15d1c0 0x7ff9cc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d2663640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc15d1c0 0x7ff9cc06c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41792/0 (socket says 172.21.15.16:41792) 2023-11-13T11:01:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d2663640 1 -- 172.21.15.16:0/2413066790 learned_addr learned my addr 172.21.15.16:0/2413066790 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc06e4b0 0x7ff9cc0c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d1661640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc0ccad0 0x7ff9cc06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d2663640 1 -- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc06e4b0 msgr2=0x7ff9cc0c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d2663640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc06e4b0 0x7ff9cc0c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d2663640 1 -- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc0ccad0 msgr2=0x7ff9cc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d2663640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc0ccad0 0x7ff9cc06c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d2663640 1 -- 172.21.15.16:0/2413066790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c80094d0 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.225+0000 7ff9d1661640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc0ccad0 0x7ff9cc06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.226+0000 7ff9d2663640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc15d1c0 0x7ff9cc06c920 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400efc0 tx=0x7ff9c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.226+0000 7ff9c27fc640 1 -- 172.21.15.16:0/2413066790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4009ba0 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.226+0000 7ff9c27fc640 1 -- 172.21.15.16:0/2413066790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c400f040 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.226+0000 7ff9d40c6640 1 -- 172.21.15.16:0/2413066790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc0b3da0 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.226+0000 7ff9c27fc640 1 -- 172.21.15.16:0/2413066790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40146c0 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.226+0000 7ff9d40c6640 1 -- 172.21.15.16:0/2413066790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc0b4360 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.228+0000 7ff9c27fc640 1 -- 172.21.15.16:0/2413066790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9c4014820 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.230+0000 7ff9c27fc640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff9b006ae40 0x7ff9b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.230+0000 7ff9c27fc640 1 -- 172.21.15.16:0/2413066790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7ff9c408a670 con 0x7ff9cc15d1c0 2023-11-13T11:01:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.231+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff9b006ae40 0x7ff9b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.231+0000 7ff9d6c58740 1 Processor -- start 2023-11-13T11:01:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.231+0000 7ff9d6c58740 1 -- start start 2023-11-13T11:01:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff9b006ae40 0x7ff9b006d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8036000 tx=0x7ff9c8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1c84030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1cb3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1c84570 0x5607c1cb4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c1bf6e90 con 0x5607c1cb1090 2023-11-13T11:01:15.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c1bf4580 con 0x5607c1cb1470 2023-11-13T11:01:15.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607c1befe70 con 0x5607c1c84570 2023-11-13T11:01:15.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d1e62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1c84030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d1e62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1c84030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41804/0 (socket says 172.21.15.16:41804) 2023-11-13T11:01:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.232+0000 7ff9d1e62640 1 -- 172.21.15.16:0/3730321900 learned_addr learned my addr 172.21.15.16:0/3730321900 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9d1e62640 1 -- 172.21.15.16:0/3730321900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1c84570 msgr2=0x5607c1cb4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9d2663640 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1c84570 0x5607c1cb4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9d1661640 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1cb3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1c84570 0x5607c1cb4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9d1e62640 1 -- 172.21.15.16:0/3730321900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 msgr2=0x5607c1cb3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1cb3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9d1e62640 1 -- 172.21.15.16:0/3730321900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607c1bba020 con 0x5607c1cb1090 2023-11-13T11:01:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9d1661640 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1cb3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1c84030 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009960 tx=0x7ff9c8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53176ee78287e13f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9af7fe640 1 -- 172.21.15.16:0/3730321900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8045e40 con 0x5607c1cb1090 2023-11-13T11:01:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9af7fe640 1 -- 172.21.15.16:0/3730321900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8039450 con 0x5607c1cb1090 2023-11-13T11:01:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.233+0000 7ff9af7fe640 1 -- 172.21.15.16:0/3730321900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80397e0 con 0x5607c1cb1090 2023-11-13T11:01:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.234+0000 7ff9d6c58740 1 -- 172.21.15.16:0/3730321900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 msgr2=0x5607c1c84030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.234+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1c84030 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009960 tx=0x7ff9c8030000 comp rx=0 tx=0).stop 2023-11-13T11:01:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.234+0000 7ff9d6c58740 1 -- 172.21.15.16:0/3730321900 shutdown_connections 2023-11-13T11:01:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.234+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1c84570 0x5607c1cb4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.234+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1cb3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.234+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/3730321900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1c84030 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.234+0000 7ff9d6c58740 1 -- 172.21.15.16:0/3730321900 >> 172.21.15.16:0/3730321900 conn(0x5607c1c862a0 msgr2=0x5607c1ccee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.234+0000 7ff9d6c58740 1 -- 172.21.15.16:0/3730321900 shutdown_connections 2023-11-13T11:01:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.234+0000 7ff9d6c58740 1 -- 172.21.15.16:0/3730321900 wait complete. 2023-11-13T11:01:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.235+0000 7ff9d6c58740 1 -- start start 2023-11-13T11:01:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.235+0000 7ff9d6c58740 1 Processor -- start 2023-11-13T11:01:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.236+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1ccf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.236+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1470 0x5607c1ca1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.236+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1ca16a0 0x5607c1cd2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.236+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c1b8c480 con 0x5607c1cb1090 2023-11-13T11:01:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.236+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c1ba9dc0 con 0x5607c1ca16a0 2023-11-13T11:01:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.236+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607c18e8f90 con 0x5607c1cb1470 2023-11-13T11:01:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.236+0000 7ff9d6c58740 10 client.0 _ll_register_callbacks cb 0x5607c1901be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:01:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1e62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1ccf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1e62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1ccf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41806/0 (socket says 172.21.15.16:41806) 2023-11-13T11:01:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1e62640 1 -- 172.21.15.16:0/1711197374 learned_addr learned my addr 172.21.15.16:0/1711197374 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1e62640 1 -- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1470 msgr2=0x5607c1ca1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1661640 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1470 0x5607c1ca1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1470 0x5607c1ca1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d2663640 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1ca16a0 0x5607c1cd2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1e62640 1 -- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1ca16a0 msgr2=0x5607c1cd2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1ca16a0 0x5607c1cd2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1e62640 1 -- 172.21.15.16:0/1711197374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc0cfdc0 con 0x5607c1cb1090 2023-11-13T11:01:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1ccf190 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009990 tx=0x7ff9c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946c10c933d65a89 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.237+0000 7ff9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:01:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9acff9640 1 -- 172.21.15.16:0/1711197374 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c805a470 con 0x5607c1cb1090 2023-11-13T11:01:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9acff9640 1 -- 172.21.15.16:0/1711197374 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7ff9c803e040 con 0x5607c1cb1090 2023-11-13T11:01:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1711197374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607c1be8e50 con 0x5607c1cb1090 2023-11-13T11:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9acff9640 1 -- 172.21.15.16:0/1711197374 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c805a5d0 con 0x5607c1cb1090 2023-11-13T11:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9d6c58740 10 client.5775 fetch_fsmap learned FSMap version 19 2023-11-13T11:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1711197374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x5607c1bee4f0 con 0x5607c1cb1090 2023-11-13T11:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9acff9640 1 -- 172.21.15.16:0/1711197374 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5607c1be8e50 con 0x5607c1cb1090 2023-11-13T11:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9acff9640 1 -- 172.21.15.16:0/1711197374 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff9c8042050 con 0x5607c1cb1090 2023-11-13T11:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9d6c58740 10 client.5775 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.238+0000 7ff9d6c58740 10 client.5775 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.239+0000 7ff9d6c58740 20 client.5775 populate_metadata read hostname 'smithi016' 2023-11-13T11:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.239+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5607c1cb6330 0x5607c1cb6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.239+0000 7ff9d6c58740 4 client.5775 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.239+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1711197374 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607c1be8b70 con 0x5607c1cb6330 2023-11-13T11:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.239+0000 7ff9d1661640 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5607c1cb6330 0x5607c1cb6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.240+0000 7ff9d1661640 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5607c1cb6330 0x5607c1cb6710 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473f09fb5a55a33b server_cookie=f72047aaf405080e in_seq=0 out_seq=0 2023-11-13T11:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.240+0000 7ff9acff9640 10 client.5775 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9acff9640 1 -- 172.21.15.16:0/1711197374 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607c1be8b70 con 0x5607c1cb6330 2023-11-13T11:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9acff9640 10 client.5775 handle_command_reply: tid=0 2023-11-13T11:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 client.5775 shutdown 2023-11-13T11:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1711197374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 msgr2=0x5607c1ccf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1ccf190 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009990 tx=0x7ff9c8004730 comp rx=0 tx=0).stop 2023-11-13T11:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1711197374 shutdown_connections 2023-11-13T11:01:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1ca16a0 0x5607c1cd2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1470 0x5607c1ca1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5607c1cb6330 0x5607c1cb6710 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1711197374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1090 0x5607c1ccf190 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1711197374 >> 172.21.15.16:0/1711197374 conn(0x5607c1c84570 msgr2=0x5607c1cb3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9acff9640 0 client.5775 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:01:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1711197374 shutdown_connections 2023-11-13T11:01:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.242+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1711197374 wait complete. 2023-11-13T11:01:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.243+0000 7ff9d6c58740 20 client.5775 trim_cache size 0 max 16384 2023-11-13T11:01:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.259+0000 7ff9d6c58740 1 Processor -- start 2023-11-13T11:01:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.259+0000 7ff9d6c58740 1 -- start start 2023-11-13T11:01:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.260+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1ca16a0 0x5607c1cb4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.260+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1090 0x5607c1d309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.260+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1470 0x5607c1cd14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.260+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c1c9ebc0 con 0x5607c1cb1470 2023-11-13T11:01:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.260+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c1cb0050 con 0x5607c1cb1090 2023-11-13T11:01:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.260+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607c1cb0880 con 0x5607c1ca16a0 2023-11-13T11:01:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.260+0000 7ff9d2663640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1470 0x5607c1cd14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d2663640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1470 0x5607c1cd14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41818/0 (socket says 172.21.15.16:41818) 2023-11-13T11:01:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d2663640 1 -- 172.21.15.16:0/1308578036 learned_addr learned my addr 172.21.15.16:0/1308578036 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1ca16a0 0x5607c1cb4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d1661640 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1090 0x5607c1d309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d2663640 1 -- 172.21.15.16:0/1308578036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1ca16a0 msgr2=0x5607c1cb4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d2663640 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1ca16a0 0x5607c1cb4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d2663640 1 -- 172.21.15.16:0/1308578036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1090 msgr2=0x5607c1d309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d2663640 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1090 0x5607c1d309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d2663640 1 -- 172.21.15.16:0/1308578036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607c1b8bd10 con 0x5607c1cb1470 2023-11-13T11:01:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d1661640 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1090 0x5607c1d309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9d2663640 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1470 0x5607c1cd14c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40590e0 tx=0x7ff9c405f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825319a1b2255e68 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9affff640 1 -- 172.21.15.16:0/1308578036 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4070070 con 0x5607c1cb1470 2023-11-13T11:01:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9affff640 1 -- 172.21.15.16:0/1308578036 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c4059150 con 0x5607c1cb1470 2023-11-13T11:01:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.261+0000 7ff9affff640 1 -- 172.21.15.16:0/1308578036 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c406cea0 con 0x5607c1cb1470 2023-11-13T11:01:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.262+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1308578036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1470 msgr2=0x5607c1cd14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.262+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1470 0x5607c1cd14c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40590e0 tx=0x7ff9c405f040 comp rx=0 tx=0).stop 2023-11-13T11:01:15.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.262+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1308578036 shutdown_connections 2023-11-13T11:01:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.262+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1cb1470 0x5607c1cd14c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.262+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1090 0x5607c1d309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.262+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/1308578036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1ca16a0 0x5607c1cb4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.262+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1308578036 >> 172.21.15.16:0/1308578036 conn(0x5607c1c84570 msgr2=0x5607c1c8b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.262+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1308578036 shutdown_connections 2023-11-13T11:01:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.262+0000 7ff9d6c58740 1 -- 172.21.15.16:0/1308578036 wait complete. 2023-11-13T11:01:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.263+0000 7ff9d6c58740 1 -- start start 2023-11-13T11:01:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.263+0000 7ff9d6c58740 1 Processor -- start 2023-11-13T11:01:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1ca16a0 0x5607c1cb79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1090 0x5607c1ca0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d6c58740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1d310a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c1d31980 con 0x5607c1ca16a0 2023-11-13T11:01:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c1d33da0 con 0x5607c1cb1470 2023-11-13T11:01:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d6c58740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607c1d2d5c0 con 0x5607c1cb1090 2023-11-13T11:01:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d6c58740 10 client.0 _ll_register_callbacks cb 0x7ff9c40095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:01:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d1e62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1ca16a0 0x5607c1cb79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d1e62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1ca16a0 0x5607c1cb79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41834/0 (socket says 172.21.15.16:41834) 2023-11-13T11:01:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.264+0000 7ff9d1e62640 1 -- 172.21.15.16:0/472202741 learned_addr learned my addr 172.21.15.16:0/472202741 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d2663640 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1d310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d1661640 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1090 0x5607c1ca0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d1e62640 1 -- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1090 msgr2=0x5607c1ca0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1090 0x5607c1ca0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d1e62640 1 -- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 msgr2=0x5607c1d310a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1d310a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d1e62640 1 -- 172.21.15.16:0/472202741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc0b4360 con 0x5607c1ca16a0 2023-11-13T11:01:15.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d2663640 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1d310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:15.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d1e62640 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1ca16a0 0x5607c1cb79b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009770 tx=0x7ff9c8047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0948927ad3a8e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:01:15.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9acff9640 1 -- 172.21.15.16:0/472202741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8045910 con 0x5607c1ca16a0 2023-11-13T11:01:15.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9acff9640 1 -- 172.21.15.16:0/472202741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7ff9c8030760 con 0x5607c1ca16a0 2023-11-13T11:01:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.265+0000 7ff9d6c58740 1 -- 172.21.15.16:0/472202741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9c8039450 con 0x5607c1ca16a0 2023-11-13T11:01:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9acff9640 1 -- 172.21.15.16:0/472202741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c805a680 con 0x5607c1ca16a0 2023-11-13T11:01:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9d6c58740 10 client.5781 fetch_fsmap learned FSMap version 19 2023-11-13T11:01:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9d6c58740 1 -- 172.21.15.16:0/472202741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x5607c1bb1300 con 0x5607c1ca16a0 2023-11-13T11:01:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9acff9640 1 -- 172.21.15.16:0/472202741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8039450 con 0x5607c1ca16a0 2023-11-13T11:01:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9acff9640 1 -- 172.21.15.16:0/472202741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff9c805a7e0 con 0x5607c1ca16a0 2023-11-13T11:01:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9d6c58740 10 client.5781 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:01:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9d6c58740 10 client.5781 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:01:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9d6c58740 20 client.5781 populate_metadata read hostname 'smithi016' 2023-11-13T11:01:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5607c1cd1bd0 0x5607c1d2c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9d6c58740 4 client.5781 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-13T11:01:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.266+0000 7ff9d6c58740 1 -- 172.21.15.16:0/472202741 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5607c18f07e0 con 0x5607c1cd1bd0 2023-11-13T11:01:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.267+0000 7ff9d1661640 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5607c1cd1bd0 0x5607c1d2c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.268+0000 7ff9d1661640 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5607c1cd1bd0 0x5607c1d2c6b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d5f7456b63cec8 server_cookie=146dae3e87deb02c in_seq=0 out_seq=0 2023-11-13T11:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.268+0000 7ff9acff9640 10 client.5781 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.268+0000 7ff9acff9640 1 -- 172.21.15.16:0/472202741 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5607c18f07e0 con 0x5607c1cd1bd0 2023-11-13T11:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.268+0000 7ff9acff9640 10 client.5781 handle_command_reply: tid=0 2023-11-13T11:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.268+0000 7ff9d6c58740 1 client.5781 shutdown 2023-11-13T11:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 -- 172.21.15.16:0/472202741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1ca16a0 msgr2=0x5607c1cb79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1ca16a0 0x5607c1cb79b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009770 tx=0x7ff9c8047040 comp rx=0 tx=0).stop 2023-11-13T11:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 -- 172.21.15.16:0/472202741 shutdown_connections 2023-11-13T11:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5607c1cb1470 0x5607c1d310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5607c1cb1090 0x5607c1ca0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5607c1cd1bd0 0x5607c1d2c6b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 --2- 172.21.15.16:0/472202741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607c1ca16a0 0x5607c1cb79b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9acff9640 0 client.5781 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 -- 172.21.15.16:0/472202741 >> 172.21.15.16:0/472202741 conn(0x5607c1c84570 msgr2=0x5607c1cd14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 -- 172.21.15.16:0/472202741 shutdown_connections 2023-11-13T11:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 1 -- 172.21.15.16:0/472202741 wait complete. 2023-11-13T11:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.269+0000 7ff9d6c58740 20 client.5781 trim_cache size 0 max 16384 2023-11-13T11:01:15.291 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:01:15.292 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2023-11-13T11:01:15.292 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2023-11-13T11:01:15.292 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 -- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff9b006ae40 msgr2=0x7ff9b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff9b006ae40 0x7ff9b006d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8036000 tx=0x7ff9c8008030 comp rx=0 tx=0).stop 2023-11-13T11:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 -- 172.21.15.16:0/2413066790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc15d1c0 msgr2=0x7ff9cc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc15d1c0 0x7ff9cc06c920 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400efc0 tx=0x7ff9c400c490 comp rx=0 tx=0).stop 2023-11-13T11:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 -- 172.21.15.16:0/2413066790 shutdown_connections 2023-11-13T11:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff9b006ae40 0x7ff9b006d300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9cc15d1c0 0x7ff9cc06c920 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9cc0ccad0 0x7ff9cc06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 --2- 172.21.15.16:0/2413066790 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9cc06e4b0 0x7ff9cc0c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 -- 172.21.15.16:0/2413066790 >> 172.21.15.16:0/2413066790 conn(0x7ff9cc0b2600 msgr2=0x7ff9cc0cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 -- 172.21.15.16:0/2413066790 shutdown_connections 2023-11-13T11:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.270+0000 7ff9d40c6640 1 -- 172.21.15.16:0/2413066790 wait complete. 2023-11-13T11:01:15.296 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-13T11:01:15.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2023-11-13T11:01:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.514+0000 7f62db59e640 1 Processor -- start 2023-11-13T11:01:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.514+0000 7f62db59e640 1 -- start start 2023-11-13T11:01:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.516+0000 7f62db59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f62dc15b5c0 0x7f62dc164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.516+0000 7f62db59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62dc161640 0x7f62dc0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.516+0000 7f62db59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62dc165b40 con 0x7f62dc0b2970 2023-11-13T11:01:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.516+0000 7f62db59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc0b2de0 con 0x7f62dc161640 2023-11-13T11:01:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.516+0000 7f62db59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc1652a0 con 0x7f62dc15b5c0 2023-11-13T11:01:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.516+0000 7f62d3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62dc161640 0x7f62dc0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.516+0000 7f62d3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62dc161640 0x7f62dc0b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41840/0 (socket says 172.21.15.16:41840) 2023-11-13T11:01:15.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.516+0000 7f62d3fff640 1 -- 172.21.15.16:0/75173163 learned_addr learned my addr 172.21.15.16:0/75173163 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.517+0000 7f62da59c640 1 --2- 172.21.15.16:0/75173163 >> v2:172.21.15.184:3300/0 conn(0x7f62dc15b5c0 0x7f62dc164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.517+0000 7f62d3fff640 1 -- 172.21.15.16:0/75173163 >> v2:172.21.15.184:3300/0 conn(0x7f62dc15b5c0 msgr2=0x7f62dc164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.517+0000 7f62d3fff640 1 --2- 172.21.15.16:0/75173163 >> v2:172.21.15.184:3300/0 conn(0x7f62dc15b5c0 0x7f62dc164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.517+0000 7f62d3fff640 1 -- 172.21.15.16:0/75173163 >> v1:172.21.15.184:6789/0 conn(0x7f62dc0b2970 legacy=0x7f62dc0ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.517+0000 7f62d3fff640 1 -- 172.21.15.16:0/75173163 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc165580 con 0x7f62dc161640 2023-11-13T11:01:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.518+0000 7f62d3fff640 1 --2- 172.21.15.16:0/75173163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc161640 0x7f62dc0b2430 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f62dc06d360 tx=0x7f62c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e0cca903b80d3c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.518+0000 7f62d9d9b640 1 -- 172.21.15.16:0/75173163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4031870 con 0x7f62dc161640 2023-11-13T11:01:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.519+0000 7f62d9d9b640 1 -- 172.21.15.16:0/75173163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c40319d0 con 0x7f62dc161640 2023-11-13T11:01:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.519+0000 7f62d9d9b640 1 -- 172.21.15.16:0/75173163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4031d60 con 0x7f62dc161640 2023-11-13T11:01:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.519+0000 7f62db59e640 1 -- 172.21.15.16:0/75173163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc161640 msgr2=0x7f62dc0b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.519+0000 7f62db59e640 1 --2- 172.21.15.16:0/75173163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc161640 0x7f62dc0b2430 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f62dc06d360 tx=0x7f62c402b910 comp rx=0 tx=0).stop 2023-11-13T11:01:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.519+0000 7f62db59e640 1 -- 172.21.15.16:0/75173163 shutdown_connections 2023-11-13T11:01:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.519+0000 7f62db59e640 1 --2- 172.21.15.16:0/75173163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc161640 0x7f62dc0b2430 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.519+0000 7f62db59e640 1 --2- 172.21.15.16:0/75173163 >> v2:172.21.15.184:3300/0 conn(0x7f62dc15b5c0 0x7f62dc164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.519+0000 7f62db59e640 1 -- 172.21.15.16:0/75173163 >> 172.21.15.16:0/75173163 conn(0x7f62dc0b0970 msgr2=0x7f62dc0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.519+0000 7f62db59e640 1 -- 172.21.15.16:0/75173163 shutdown_connections 2023-11-13T11:01:15.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.520+0000 7f62db59e640 1 -- 172.21.15.16:0/75173163 wait complete. 2023-11-13T11:01:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.522+0000 7f62db59e640 1 Processor -- start 2023-11-13T11:01:15.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.522+0000 7f62db59e640 1 -- start start 2023-11-13T11:01:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.523+0000 7f62db59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc0b2970 0x7f62dc159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.523+0000 7f62db59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62dc15b5c0 0x7f62dc15c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.523+0000 7f62db59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62dc161640 0x7f62dc1ecc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.523+0000 7f62db59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15f9d0 con 0x7f62dc0b2970 2023-11-13T11:01:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.523+0000 7f62db59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15fcb0 con 0x7f62dc161640 2023-11-13T11:01:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.523+0000 7f62db59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15ff90 con 0x7f62dc15b5c0 2023-11-13T11:01:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc0b2970 0x7f62dc159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc0b2970 0x7f62dc159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41854/0 (socket says 172.21.15.16:41854) 2023-11-13T11:01:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62da59c640 1 -- 172.21.15.16:0/2681319859 learned_addr learned my addr 172.21.15.16:0/2681319859 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62da59c640 1 -- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62dc15b5c0 msgr2=0x7f62dc15c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62da59c640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62dc15b5c0 0x7f62dc15c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62da59c640 1 -- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62dc161640 msgr2=0x7f62dc1ecc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62dad9d640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62dc161640 0x7f62dc1ecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62da59c640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62dc161640 0x7f62dc1ecc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62da59c640 1 -- 172.21.15.16:0/2681319859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c40094d0 con 0x7f62dc0b2970 2023-11-13T11:01:15.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.524+0000 7f62dad9d640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62dc161640 0x7f62dc1ecc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:15.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.525+0000 7f62da59c640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc0b2970 0x7f62dc159a60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00d5f0 tx=0x7f62cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.525+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2681319859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0125f0 con 0x7f62dc0b2970 2023-11-13T11:01:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.525+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2681319859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc012750 con 0x7f62dc0b2970 2023-11-13T11:01:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.525+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2681319859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc012b00 con 0x7f62dc0b2970 2023-11-13T11:01:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.525+0000 7f62db59e640 1 -- 172.21.15.16:0/2681319859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc160270 con 0x7f62dc0b2970 2023-11-13T11:01:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.525+0000 7f62db59e640 1 -- 172.21.15.16:0/2681319859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc15cb90 con 0x7f62dc0b2970 2023-11-13T11:01:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.527+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2681319859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62cc012c60 con 0x7f62dc0b2970 2023-11-13T11:01:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.529+0000 7f62d2ffd640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f62bc06ae70 0x7f62bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.529+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2681319859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f62cc089ca0 con 0x7f62dc0b2970 2023-11-13T11:01:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.530+0000 7f62d3fff640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f62bc06ae70 0x7f62bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.530+0000 7f62e3564740 1 Processor -- start 2023-11-13T11:01:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.530+0000 7f62e3564740 1 -- start start 2023-11-13T11:01:15.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62d3fff640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f62bc06ae70 0x7f62bc06d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f62c4036000 tx=0x7f62c4030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62e3564740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7cde3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62e3564740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7cde930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62e3564740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7cd92e0 0x5580f7d090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62e3564740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f7be3180 con 0x5580f7d26dc0 2023-11-13T11:01:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62e3564740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f7c4bdd0 con 0x5580f7cdce40 2023-11-13T11:01:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62e3564740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580f7c494c0 con 0x5580f7cd92e0 2023-11-13T11:01:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7cde3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7cde3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41856/0 (socket says 172.21.15.16:41856) 2023-11-13T11:01:15.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.531+0000 7f62da59c640 1 -- 172.21.15.16:0/88268018 learned_addr learned my addr 172.21.15.16:0/88268018 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62d3fff640 1 --2- 172.21.15.16:0/88268018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7cde930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62dad9d640 1 --2- 172.21.15.16:0/88268018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7cd92e0 0x5580f7d090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62da59c640 1 -- 172.21.15.16:0/88268018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7cd92e0 msgr2=0x5580f7d090d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62da59c640 1 --2- 172.21.15.16:0/88268018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7cd92e0 0x5580f7d090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62da59c640 1 -- 172.21.15.16:0/88268018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 msgr2=0x5580f7cde930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62da59c640 1 --2- 172.21.15.16:0/88268018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7cde930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62da59c640 1 -- 172.21.15.16:0/88268018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580f7c07950 con 0x5580f7d26dc0 2023-11-13T11:01:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62da59c640 1 --2- 172.21.15.16:0/88268018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7cde3f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f62cc054d10 tx=0x7f62cc058710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53779d99563a7716 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62bbfff640 1 -- 172.21.15.16:0/88268018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc060b10 con 0x5580f7d26dc0 2023-11-13T11:01:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62bbfff640 1 -- 172.21.15.16:0/88268018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc056bb0 con 0x5580f7d26dc0 2023-11-13T11:01:15.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.532+0000 7f62bbfff640 1 -- 172.21.15.16:0/88268018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc057d00 con 0x5580f7d26dc0 2023-11-13T11:01:15.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.533+0000 7f62e3564740 1 -- 172.21.15.16:0/88268018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 msgr2=0x5580f7cde3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.533+0000 7f62e3564740 1 --2- 172.21.15.16:0/88268018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7cde3f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f62cc054d10 tx=0x7f62cc058710 comp rx=0 tx=0).stop 2023-11-13T11:01:15.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.533+0000 7f62e3564740 1 -- 172.21.15.16:0/88268018 shutdown_connections 2023-11-13T11:01:15.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.533+0000 7f62e3564740 1 --2- 172.21.15.16:0/88268018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7cd92e0 0x5580f7d090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.533+0000 7f62e3564740 1 --2- 172.21.15.16:0/88268018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7cde930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.533+0000 7f62e3564740 1 --2- 172.21.15.16:0/88268018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7cde3f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.533+0000 7f62e3564740 1 -- 172.21.15.16:0/88268018 >> 172.21.15.16:0/88268018 conn(0x5580f7cdb550 msgr2=0x5580f7d240d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.533+0000 7f62e3564740 1 -- 172.21.15.16:0/88268018 shutdown_connections 2023-11-13T11:01:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.533+0000 7f62e3564740 1 -- 172.21.15.16:0/88268018 wait complete. 2023-11-13T11:01:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.534+0000 7f62e3564740 1 -- start start 2023-11-13T11:01:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.534+0000 7f62e3564740 1 Processor -- start 2023-11-13T11:01:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.535+0000 7f62e3564740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.535+0000 7f62e3564740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7d00ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.535+0000 7f62e3564740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d01010 0x5580f7d07330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.535+0000 7f62e3564740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f7c44db0 con 0x5580f7d26dc0 2023-11-13T11:01:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.535+0000 7f62e3564740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f7be1460 con 0x5580f7cdce40 2023-11-13T11:01:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.535+0000 7f62e3564740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580f7bfec30 con 0x5580f7d01010 2023-11-13T11:01:15.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62e3564740 10 client.0 _ll_register_callbacks cb 0x5580f7956be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:01:15.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7d00ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7d00ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41868/0 (socket says 172.21.15.16:41868) 2023-11-13T11:01:15.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62d3fff640 1 -- 172.21.15.16:0/815955941 learned_addr learned my addr 172.21.15.16:0/815955941 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62da59c640 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62d3fff640 1 -- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d01010 msgr2=0x5580f7d07330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62dad9d640 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d01010 0x5580f7d07330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62d3fff640 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d01010 0x5580f7d07330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62d3fff640 1 -- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 msgr2=0x5580f7d259f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62d3fff640 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d259f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62d3fff640 1 -- 172.21.15.16:0/815955941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc15cb90 con 0x5580f7d26dc0 2023-11-13T11:01:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62da59c640 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d259f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.536+0000 7f62dad9d640 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d01010 0x5580f7d07330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:15.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.537+0000 7f62d3fff640 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7d00ad0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009960 tx=0x7f62c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4602055ac5542a11 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.537+0000 7f62b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.537+0000 7f62b97fa640 1 -- 172.21.15.16:0/815955941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c403a830 con 0x5580f7d26dc0 2023-11-13T11:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.537+0000 7f62b97fa640 1 -- 172.21.15.16:0/815955941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f62c402d040 con 0x5580f7d26dc0 2023-11-13T11:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.537+0000 7f62e3564740 1 -- 172.21.15.16:0/815955941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580f7c3dd90 con 0x5580f7d26dc0 2023-11-13T11:01:15.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.537+0000 7f62b97fa640 1 -- 172.21.15.16:0/815955941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c403a990 con 0x5580f7d26dc0 2023-11-13T11:01:15.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.537+0000 7f62e3564740 10 client.5793 fetch_fsmap learned FSMap version 19 2023-11-13T11:01:15.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.537+0000 7f62e3564740 1 -- 172.21.15.16:0/815955941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x5580f7c43430 con 0x5580f7d26dc0 2023-11-13T11:01:15.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.538+0000 7f62b97fa640 1 -- 172.21.15.16:0/815955941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580f7c3dd90 con 0x5580f7d26dc0 2023-11-13T11:01:15.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.538+0000 7f62b97fa640 1 -- 172.21.15.16:0/815955941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f62c403aaf0 con 0x5580f7d26dc0 2023-11-13T11:01:15.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.538+0000 7f62e3564740 10 client.5793 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:01:15.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.538+0000 7f62e3564740 10 client.5793 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:01:15.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.538+0000 7f62e3564740 20 client.5793 populate_metadata read hostname 'smithi016' 2023-11-13T11:01:15.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.538+0000 7f62e3564740 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5580f7d28c70 0x5580f7d29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.538+0000 7f62e3564740 4 client.5793 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:01:15.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.538+0000 7f62e3564740 1 -- 172.21.15.16:0/815955941 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580f7c3dab0 con 0x5580f7d28c70 2023-11-13T11:01:15.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.538+0000 7f62da59c640 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5580f7d28c70 0x5580f7d29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.539+0000 7f62da59c640 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5580f7d28c70 0x5580f7d29080 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7830eb75c6d9518 server_cookie=c0839a74fe17dac8 in_seq=0 out_seq=0 2023-11-13T11:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.539+0000 7f62b97fa640 10 client.5793 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62b97fa640 1 -- 172.21.15.16:0/815955941 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5580f7c3dab0 con 0x5580f7d28c70 2023-11-13T11:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62b97fa640 10 client.5793 handle_command_reply: tid=0 2023-11-13T11:01:15.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62e3564740 1 client.5793 shutdown 2023-11-13T11:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62e3564740 1 -- 172.21.15.16:0/815955941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 msgr2=0x5580f7d00ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62e3564740 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7d00ad0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009960 tx=0x7f62c4004730 comp rx=0 tx=0).stop 2023-11-13T11:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62e3564740 1 -- 172.21.15.16:0/815955941 shutdown_connections 2023-11-13T11:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62e3564740 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d01010 0x5580f7d07330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62e3564740 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5580f7d28c70 0x5580f7d29080 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62e3564740 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d26dc0 0x5580f7d00ad0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62e3564740 1 --2- 172.21.15.16:0/815955941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d259f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62b97fa640 0 client.5793 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.541+0000 7f62e3564740 1 -- 172.21.15.16:0/815955941 >> 172.21.15.16:0/815955941 conn(0x5580f7cd92e0 msgr2=0x5580f7cd8150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.542+0000 7f62e3564740 1 -- 172.21.15.16:0/815955941 shutdown_connections 2023-11-13T11:01:15.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.542+0000 7f62e3564740 1 -- 172.21.15.16:0/815955941 wait complete. 2023-11-13T11:01:15.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.542+0000 7f62e3564740 20 client.5793 trim_cache size 0 max 16384 2023-11-13T11:01:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.558+0000 7f62e3564740 1 Processor -- start 2023-11-13T11:01:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.558+0000 7f62e3564740 1 -- start start 2023-11-13T11:01:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.558+0000 7f62e3564740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7cd8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62e3564740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7cd8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62e3564740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d26dc0 0x5580f7cf1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62e3564740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f79a8040 con 0x5580f7d01010 2023-11-13T11:01:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62e3564740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f7cda940 con 0x5580f7cdce40 2023-11-13T11:01:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62e3564740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580f7d2c000 con 0x5580f7d26dc0 2023-11-13T11:01:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7cd8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7cd8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41880/0 (socket says 172.21.15.16:41880) 2023-11-13T11:01:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62d3fff640 1 -- 172.21.15.16:0/3069700221 learned_addr learned my addr 172.21.15.16:0/3069700221 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62d3fff640 1 -- 172.21.15.16:0/3069700221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d26dc0 msgr2=0x5580f7cf1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62d3fff640 1 --2- 172.21.15.16:0/3069700221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d26dc0 0x5580f7cf1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62d3fff640 1 -- 172.21.15.16:0/3069700221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 msgr2=0x5580f7cd8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62d3fff640 1 --2- 172.21.15.16:0/3069700221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7cd8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62d3fff640 1 -- 172.21.15.16:0/3069700221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580f7be0cf0 con 0x5580f7d01010 2023-11-13T11:01:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.559+0000 7f62d3fff640 1 --2- 172.21.15.16:0/3069700221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7cd8690 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f62c4006a90 tx=0x7f62c4049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2a4bf77d1e692 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62d0ff9640 1 -- 172.21.15.16:0/3069700221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4004240 con 0x5580f7d01010 2023-11-13T11:01:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62d0ff9640 1 -- 172.21.15.16:0/3069700221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c40043a0 con 0x5580f7d01010 2023-11-13T11:01:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62d0ff9640 1 -- 172.21.15.16:0/3069700221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c403a830 con 0x5580f7d01010 2023-11-13T11:01:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62e3564740 1 -- 172.21.15.16:0/3069700221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 msgr2=0x5580f7cd8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62e3564740 1 --2- 172.21.15.16:0/3069700221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7cd8690 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f62c4006a90 tx=0x7f62c4049040 comp rx=0 tx=0).stop 2023-11-13T11:01:15.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62e3564740 1 -- 172.21.15.16:0/3069700221 shutdown_connections 2023-11-13T11:01:15.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62e3564740 1 --2- 172.21.15.16:0/3069700221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d26dc0 0x5580f7cf1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62e3564740 1 --2- 172.21.15.16:0/3069700221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7cd8690 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62e3564740 1 --2- 172.21.15.16:0/3069700221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7cd8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62e3564740 1 -- 172.21.15.16:0/3069700221 >> 172.21.15.16:0/3069700221 conn(0x5580f7cd92e0 msgr2=0x5580f7d07100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.560+0000 7f62e3564740 1 -- 172.21.15.16:0/3069700221 shutdown_connections 2023-11-13T11:01:15.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.561+0000 7f62e3564740 1 -- 172.21.15.16:0/3069700221 wait complete. 2023-11-13T11:01:15.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.561+0000 7f62e3564740 1 -- start start 2023-11-13T11:01:15.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.562+0000 7f62e3564740 1 Processor -- start 2023-11-13T11:01:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62e3564740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d8fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62e3564740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7d92960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62e3564740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d26dc0 0x5580f7cd8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62e3564740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f7d8f340 con 0x5580f7d01010 2023-11-13T11:01:15.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62e3564740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f7d00dd0 con 0x5580f7cdce40 2023-11-13T11:01:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62e3564740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580f7d8f8d0 con 0x5580f7d26dc0 2023-11-13T11:01:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62e3564740 10 client.0 _ll_register_callbacks cb 0x7f62cc012f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:01:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7d92960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7d92960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41884/0 (socket says 172.21.15.16:41884) 2023-11-13T11:01:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62d3fff640 1 -- 172.21.15.16:0/567051085 learned_addr learned my addr 172.21.15.16:0/567051085 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62da59c640 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d8fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62dad9d640 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d26dc0 0x5580f7cd8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62d3fff640 1 -- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d26dc0 msgr2=0x5580f7cd8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62d3fff640 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d26dc0 0x5580f7cd8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62d3fff640 1 -- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 msgr2=0x5580f7d8fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62d3fff640 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d8fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62d3fff640 1 -- 172.21.15.16:0/567051085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580f7c43430 con 0x5580f7d01010 2023-11-13T11:01:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.563+0000 7f62da59c640 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d8fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.564+0000 7f62d3fff640 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7d92960 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f62c402b950 tx=0x7f62c403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5e286affe83ad2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.564+0000 7f62b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:01:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.564+0000 7f62b97fa640 1 -- 172.21.15.16:0/567051085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4003810 con 0x5580f7d01010 2023-11-13T11:01:15.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.564+0000 7f62b97fa640 1 -- 172.21.15.16:0/567051085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f62c403abf0 con 0x5580f7d01010 2023-11-13T11:01:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.564+0000 7f62e3564740 1 -- 172.21.15.16:0/567051085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62cc056bb0 con 0x5580f7d01010 2023-11-13T11:01:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.564+0000 7f62b97fa640 1 -- 172.21.15.16:0/567051085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c4031e50 con 0x5580f7d01010 2023-11-13T11:01:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62e3564740 10 client.5799 fetch_fsmap learned FSMap version 19 2023-11-13T11:01:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62e3564740 1 -- 172.21.15.16:0/567051085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x5580f7c061b0 con 0x5580f7d01010 2023-11-13T11:01:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62b97fa640 1 -- 172.21.15.16:0/567051085 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc056bb0 con 0x5580f7d01010 2023-11-13T11:01:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62b97fa640 1 -- 172.21.15.16:0/567051085 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f62c4003970 con 0x5580f7d01010 2023-11-13T11:01:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62e3564740 10 client.5799 fetch_fsmap finished waiting for FSMap version 19 2023-11-13T11:01:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62e3564740 10 client.5799 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2023-11-13T11:01:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62e3564740 20 client.5799 populate_metadata read hostname 'smithi016' 2023-11-13T11:01:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62e3564740 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5580f7d92ea0 0x5580f7cd8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62e3564740 4 client.5799 mds_command: new command op to 4990 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2023-11-13T11:01:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.565+0000 7f62e3564740 1 -- 172.21.15.16:0/567051085 --> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5580f79457e0 con 0x5580f7d92ea0 2023-11-13T11:01:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.566+0000 7f62da59c640 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5580f7d92ea0 0x5580f7cd8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:15.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.566+0000 7f62da59c640 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5580f7d92ea0 0x5580f7cd8690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17952f3aa84b31e6 server_cookie=da04dd5e02b2874b in_seq=0 out_seq=0 2023-11-13T11:01:15.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.566+0000 7f62b97fa640 10 client.5799 ms_handle_connect on v2:172.21.15.184:6835/3171288994 2023-11-13T11:01:15.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.568+0000 7f62b97fa640 1 -- 172.21.15.16:0/567051085 <== mds.0 v2:172.21.15.184:6835/3171288994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17926 (crc 0 0 0) 0x5580f79457e0 con 0x5580f7d92ea0 2023-11-13T11:01:15.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.568+0000 7f62b97fa640 10 client.5799 handle_command_reply: tid=0 2023-11-13T11:01:15.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.568+0000 7f62e3564740 1 client.5799 shutdown 2023-11-13T11:01:15.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 -- 172.21.15.16:0/567051085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 msgr2=0x5580f7d92960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7d92960 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f62c402b950 tx=0x7f62c403d000 comp rx=0 tx=0).stop 2023-11-13T11:01:15.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 -- 172.21.15.16:0/567051085 shutdown_connections 2023-11-13T11:01:15.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5580f7d26dc0 0x5580f7cd8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:6835/3171288994,v1:172.21.15.184:6837/3171288994] conn(0x5580f7d92ea0 0x5580f7cd8690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f7d01010 0x5580f7d92960 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 --2- 172.21.15.16:0/567051085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5580f7cdce40 0x5580f7d8fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62b97fa640 0 client.5799 ms_handle_reset on v2:172.21.15.184:6835/3171288994 2023-11-13T11:01:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 -- 172.21.15.16:0/567051085 >> 172.21.15.16:0/567051085 conn(0x5580f7cd92e0 msgr2=0x5580f7cdd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 -- 172.21.15.16:0/567051085 shutdown_connections 2023-11-13T11:01:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 1 -- 172.21.15.16:0/567051085 wait complete. 2023-11-13T11:01:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.569+0000 7f62e3564740 20 client.5799 trim_cache size 0 max 16384 2023-11-13T11:01:15.588 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:01:15.588 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2023-11-13T11:01:15.588 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6053, 2023-11-13T11:01:15.589 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 6054, 2023-11-13T11:01:15.589 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1899825, 2023-11-13T11:01:15.589 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 43069427, 2023-11-13T11:01:15.589 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 17, 2023-11-13T11:01:15.590 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 6, 2023-11-13T11:01:15.590 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 1.209992963, 2023-11-13T11:01:15.590 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.595096900, 2023-11-13T11:01:15.590 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.239790215, 2023-11-13T11:01:15.590 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.279021665, 2023-11-13T11:01:15.591 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2023-11-13T11:01:15.591 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6053, 2023-11-13T11:01:15.591 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.887496228, 2023-11-13T11:01:15.592 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000146620 2023-11-13T11:01:15.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.592 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2023-11-13T11:01:15.592 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 10, 2023-11-13T11:01:15.593 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000001621, 2023-11-13T11:01:15.593 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000162 2023-11-13T11:01:15.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.593 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-13T11:01:15.593 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2023-11-13T11:01:15.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.594 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2023-11-13T11:01:15.594 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 35484, 2023-11-13T11:01:15.594 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 61196, 2023-11-13T11:01:15.595 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 12364505, 2023-11-13T11:01:15.595 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 49675148, 2023-11-13T11:01:15.595 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2023-11-13T11:01:15.595 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2023-11-13T11:01:15.595 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 11.382321922, 2023-11-13T11:01:15.596 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 5.173632931, 2023-11-13T11:01:15.596 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 6.519769187, 2023-11-13T11:01:15.596 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.370713709, 2023-11-13T11:01:15.596 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2023-11-13T11:01:15.596 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 61195, 2023-11-13T11:01:15.596 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 4.888297390, 2023-11-13T11:01:15.597 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000079880 2023-11-13T11:01:15.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.597 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2023-11-13T11:01:15.598 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 88346, 2023-11-13T11:01:15.598 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.050024392, 2023-11-13T11:01:15.598 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000566 2023-11-13T11:01:15.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 131568, 2023-11-13T11:01:15.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 48144 2023-11-13T11:01:15.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.599 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2023-11-13T11:01:15.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 3880, 2023-11-13T11:01:15.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 3929, 2023-11-13T11:01:15.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 972043, 2023-11-13T11:01:15.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 21565115, 2023-11-13T11:01:15.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2023-11-13T11:01:15.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2023-11-13T11:01:15.601 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.733595732, 2023-11-13T11:01:15.601 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.322787622, 2023-11-13T11:01:15.601 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.153900338, 2023-11-13T11:01:15.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.186887605, 2023-11-13T11:01:15.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2023-11-13T11:01:15.602 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3928, 2023-11-13T11:01:15.602 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.551671533, 2023-11-13T11:01:15.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000140445 2023-11-13T11:01:15.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.603 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2023-11-13T11:01:15.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.603 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.604 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.604 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1728, 2023-11-13T11:01:15.604 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 170288 2023-11-13T11:01:15.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.605 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2023-11-13T11:01:15.605 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2023-11-13T11:01:15.605 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2023-11-13T11:01:15.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.606 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2023-11-13T11:01:15.606 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2023-11-13T11:01:15.606 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2023-11-13T11:01:15.607 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 10932, 2023-11-13T11:01:15.607 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 30.273753903, 2023-11-13T11:01:15.607 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002769278 2023-11-13T11:01:15.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:15.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.608 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2023-11-13T11:01:15.608 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2023-11-13T11:01:15.608 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2023-11-13T11:01:15.609 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4438, 2023-11-13T11:01:15.609 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.539955916, 2023-11-13T11:01:15.609 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000346993 2023-11-13T11:01:15.609 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:15.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.610 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2023-11-13T11:01:15.610 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 28340, 2023-11-13T11:01:15.610 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 28340, 2023-11-13T11:01:15.610 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2023-11-13T11:01:15.610 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 28340, 2023-11-13T11:01:15.611 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 514.930655561, 2023-11-13T11:01:15.611 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.018169747 2023-11-13T11:01:15.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.611 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2023-11-13T11:01:15.612 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2023-11-13T11:01:15.612 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2023-11-13T11:01:15.612 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2023-11-13T11:01:15.612 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 12, 2023-11-13T11:01:15.612 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2023-11-13T11:01:15.613 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2023-11-13T11:01:15.613 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 17902, 2023-11-13T11:01:15.613 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2023-11-13T11:01:15.613 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 17902, 2023-11-13T11:01:15.614 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2023-11-13T11:01:15.614 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 17902, 2023-11-13T11:01:15.614 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2023-11-13T11:01:15.614 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 2, 2023-11-13T11:01:15.614 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 2, 2023-11-13T11:01:15.615 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 2, 2023-11-13T11:01:15.615 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 46705, 2023-11-13T11:01:15.615 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 46248, 2023-11-13T11:01:15.615 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2023-11-13T11:01:15.615 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2023-11-13T11:01:15.615 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2023-11-13T11:01:15.616 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2023-11-13T11:01:15.616 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 133, 2023-11-13T11:01:15.616 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 4198, 2023-11-13T11:01:15.616 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2023-11-13T11:01:15.616 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2023-11-13T11:01:15.617 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2023-11-13T11:01:15.617 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2023-11-13T11:01:15.617 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2023-11-13T11:01:15.617 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2023-11-13T11:01:15.617 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 1908, 2023-11-13T11:01:15.618 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2023-11-13T11:01:15.618 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2023-11-13T11:01:15.619 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2023-11-13T11:01:15.619 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2023-11-13T11:01:15.619 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 1908, 2023-11-13T11:01:15.619 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2023-11-13T11:01:15.620 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2023-11-13T11:01:15.620 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2023-11-13T11:01:15.620 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 13, 2023-11-13T11:01:15.620 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 2, 2023-11-13T11:01:15.620 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 11, 2023-11-13T11:01:15.621 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 1895, 2023-11-13T11:01:15.621 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2023-11-13T11:01:15.621 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 6178, 2023-11-13T11:01:15.621 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 6006, 2023-11-13T11:01:15.621 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2023-11-13T11:01:15.622 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-13T11:01:15.622 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-13T11:01:15.623 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 6178, 2023-11-13T11:01:15.623 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 2000, 2023-11-13T11:01:15.623 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 69, 2023-11-13T11:01:15.623 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 17846 2023-11-13T11:01:15.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.624 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2023-11-13T11:01:15.624 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2023-11-13T11:01:15.624 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2023-11-13T11:01:15.624 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2023-11-13T11:01:15.624 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2023-11-13T11:01:15.625 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2023-11-13T11:01:15.625 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2023-11-13T11:01:15.625 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2023-11-13T11:01:15.625 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2023-11-13T11:01:15.626 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 4176, 2023-11-13T11:01:15.626 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 3707, 2023-11-13T11:01:15.626 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 2253, 2023-11-13T11:01:15.626 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2023-11-13T11:01:15.626 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2023-11-13T11:01:15.627 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2023-11-13T11:01:15.627 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2023-11-13T11:01:15.627 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2023-11-13T11:01:15.627 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2023-11-13T11:01:15.627 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 1, 2023-11-13T11:01:15.628 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2023-11-13T11:01:15.628 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2023-11-13T11:01:15.628 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2023-11-13T11:01:15.628 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2023-11-13T11:01:15.628 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2023-11-13T11:01:15.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.629 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2023-11-13T11:01:15.629 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2023-11-13T11:01:15.630 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 23983, 2023-11-13T11:01:15.630 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2023-11-13T11:01:15.630 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2023-11-13T11:01:15.630 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 23983, 2023-11-13T11:01:15.630 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2023-11-13T11:01:15.631 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2023-11-13T11:01:15.631 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 30, 2023-11-13T11:01:15.631 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2023-11-13T11:01:15.631 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2023-11-13T11:01:15.631 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 30, 2023-11-13T11:01:15.632 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 3, 2023-11-13T11:01:15.632 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2023-11-13T11:01:15.632 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2023-11-13T11:01:15.632 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2023-11-13T11:01:15.633 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 83246670, 2023-11-13T11:01:15.633 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 0, 2023-11-13T11:01:15.633 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2023-11-13T11:01:15.633 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6600, 2023-11-13T11:01:15.633 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 130.543515507, 2023-11-13T11:01:15.634 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.019779320 2023-11-13T11:01:15.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.634 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 0 2023-11-13T11:01:15.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.634 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2023-11-13T11:01:15.635 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 14, 2023-11-13T11:01:15.635 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 2016, 2023-11-13T11:01:15.635 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 2002, 2023-11-13T11:01:15.635 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 13, 2023-11-13T11:01:15.635 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 13, 2023-11-13T11:01:15.636 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2023-11-13T11:01:15.636 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 17902, 2023-11-13T11:01:15.636 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 19902, 2023-11-13T11:01:15.636 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 2000, 2023-11-13T11:01:15.637 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 2, 2023-11-13T11:01:15.637 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 2002, 2023-11-13T11:01:15.637 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 2000, 2023-11-13T11:01:15.637 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 92604, 2023-11-13T11:01:15.637 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 207196 2023-11-13T11:01:15.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.638 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2023-11-13T11:01:15.638 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 37588, 2023-11-13T11:01:15.638 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2023-11-13T11:01:15.638 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 28340, 2023-11-13T11:01:15.638 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 14, 2023-11-13T11:01:15.639 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2023-11-13T11:01:15.639 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2023-11-13T11:01:15.639 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2000, 2023-11-13T11:01:15.640 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3.713455889, 2023-11-13T11:01:15.640 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001856727 2023-11-13T11:01:15.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.640 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2023-11-13T11:01:15.640 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2013, 2023-11-13T11:01:15.641 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.772931577, 2023-11-13T11:01:15.641 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000880740 2023-11-13T11:01:15.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.641 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2023-11-13T11:01:15.641 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.642 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.642 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.642 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2023-11-13T11:01:15.643 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6000, 2023-11-13T11:01:15.643 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 28.088464230, 2023-11-13T11:01:15.643 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.004681410 2023-11-13T11:01:15.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.643 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2023-11-13T11:01:15.644 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8061, 2023-11-13T11:01:15.644 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 20.079647353, 2023-11-13T11:01:15.644 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002490962 2023-11-13T11:01:15.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.644 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2023-11-13T11:01:15.645 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.645 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.645 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.646 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2023-11-13T11:01:15.646 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.646 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.646 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.647 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2023-11-13T11:01:15.647 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.647 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.647 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.648 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2023-11-13T11:01:15.648 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.648 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.648 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.649 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2023-11-13T11:01:15.649 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2023-11-13T11:01:15.649 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016119678, 2023-11-13T11:01:15.650 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.005373226 2023-11-13T11:01:15.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.650 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2023-11-13T11:01:15.650 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.651 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.651 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.651 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2023-11-13T11:01:15.651 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2023-11-13T11:01:15.652 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.006430661, 2023-11-13T11:01:15.652 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.006430661 2023-11-13T11:01:15.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.652 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2023-11-13T11:01:15.652 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.653 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.653 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.653 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2023-11-13T11:01:15.653 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2023-11-13T11:01:15.654 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.015617211, 2023-11-13T11:01:15.654 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.015617211 2023-11-13T11:01:15.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.654 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2023-11-13T11:01:15.654 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.655 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.655 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.655 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2023-11-13T11:01:15.655 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6, 2023-11-13T11:01:15.656 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.247721739, 2023-11-13T11:01:15.656 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.041286956 2023-11-13T11:01:15.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.656 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2023-11-13T11:01:15.656 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2253, 2023-11-13T11:01:15.657 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 424.334683844, 2023-11-13T11:01:15.657 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.188342070 2023-11-13T11:01:15.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.657 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2023-11-13T11:01:15.657 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.657 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.658 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.658 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2023-11-13T11:01:15.658 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.659 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.659 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.659 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2023-11-13T11:01:15.659 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.659 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.660 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.660 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2023-11-13T11:01:15.660 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2023-11-13T11:01:15.660 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.010406493, 2023-11-13T11:01:15.661 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.010406493 2023-11-13T11:01:15.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.661 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2023-11-13T11:01:15.661 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.661 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.662 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.662 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2023-11-13T11:01:15.662 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2023-11-13T11:01:15.662 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000976048, 2023-11-13T11:01:15.663 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000976048 2023-11-13T11:01:15.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.663 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2023-11-13T11:01:15.663 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.663 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.664 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.664 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2023-11-13T11:01:15.664 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.664 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.665 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.665 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2023-11-13T11:01:15.665 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.665 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.666 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.666 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2023-11-13T11:01:15.666 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.666 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.667 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.667 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2023-11-13T11:01:15.667 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.667 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.668 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.668 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2023-11-13T11:01:15.668 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8000, 2023-11-13T11:01:15.668 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 36.644200838, 2023-11-13T11:01:15.669 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.004580525 2023-11-13T11:01:15.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.669 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2023-11-13T11:01:15.669 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2023-11-13T11:01:15.669 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2023-11-13T11:01:15.670 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.670 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.670 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.670 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:15.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.671 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2023-11-13T11:01:15.671 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2023-11-13T11:01:15.671 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2023-11-13T11:01:15.671 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2023-11-13T11:01:15.672 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2023-11-13T11:01:15.672 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2023-11-13T11:01:15.672 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 12077, 2023-11-13T11:01:15.672 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 12077, 2023-11-13T11:01:15.672 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 222, 2023-11-13T11:01:15.673 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2023-11-13T11:01:15.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.673 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2023-11-13T11:01:15.673 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2023-11-13T11:01:15.673 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2023-11-13T11:01:15.673 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2023-11-13T11:01:15.674 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2023-11-13T11:01:15.674 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2023-11-13T11:01:15.674 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2023-11-13T11:01:15.674 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-13T11:01:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2023-11-13T11:01:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-13T11:01:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2023-11-13T11:01:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2023-11-13T11:01:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2023-11-13T11:01:15.676 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-13T11:01:15.676 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2023-11-13T11:01:15.676 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2023-11-13T11:01:15.676 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2023-11-13T11:01:15.677 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2023-11-13T11:01:15.677 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2023-11-13T11:01:15.677 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-13T11:01:15.677 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2023-11-13T11:01:15.677 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-13T11:01:15.678 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2023-11-13T11:01:15.678 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2023-11-13T11:01:15.678 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2023-11-13T11:01:15.678 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2023-11-13T11:01:15.678 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2023-11-13T11:01:15.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-13T11:01:15.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2023-11-13T11:01:15.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2023-11-13T11:01:15.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2023-11-13T11:01:15.679 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2023-11-13T11:01:15.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2023-11-13T11:01:15.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2023-11-13T11:01:15.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2023-11-13T11:01:15.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2023-11-13T11:01:15.680 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2023-11-13T11:01:15.680 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 252288, 2023-11-13T11:01:15.681 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 89, 2023-11-13T11:01:15.681 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2023-11-13T11:01:15.681 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2023-11-13T11:01:15.681 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2023-11-13T11:01:15.681 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2023-11-13T11:01:15.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2023-11-13T11:01:15.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2023-11-13T11:01:15.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2023-11-13T11:01:15.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2023-11-13T11:01:15.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 6008, 2023-11-13T11:01:15.683 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 88, 2023-11-13T11:01:15.683 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2023-11-13T11:01:15.683 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2023-11-13T11:01:15.683 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2023-11-13T11:01:15.683 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2023-11-13T11:01:15.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 13479868, 2023-11-13T11:01:15.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 661535, 2023-11-13T11:01:15.684 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2023-11-13T11:01:15.684 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2023-11-13T11:01:15.684 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2023-11-13T11:01:15.685 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2023-11-13T11:01:15.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.685 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2023-11-13T11:01:15.685 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2023-11-13T11:01:15.685 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2023-11-13T11:01:15.686 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 12688, 2023-11-13T11:01:15.686 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 79474099, 2023-11-13T11:01:15.686 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 31, 2023-11-13T11:01:15.686 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 12688, 2023-11-13T11:01:15.687 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2023-11-13T11:01:15.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 12688, 2023-11-13T11:01:15.687 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 72.989464887, 2023-11-13T11:01:15.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.005752637 2023-11-13T11:01:15.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.688 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2023-11-13T11:01:15.688 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2023-11-13T11:01:15.688 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 12688, 2023-11-13T11:01:15.688 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 12742 2023-11-13T11:01:15.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.689 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 12688, 2023-11-13T11:01:15.689 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2055, 2023-11-13T11:01:15.689 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 10633, 2023-11-13T11:01:15.689 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2023-11-13T11:01:15.689 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2023-11-13T11:01:15.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 0, 2023-11-13T11:01:15.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 11, 2023-11-13T11:01:15.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 145, 2023-11-13T11:01:15.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 6743, 2023-11-13T11:01:15.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 92, 2023-11-13T11:01:15.691 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2023-11-13T11:01:15.691 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2023-11-13T11:01:15.691 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 1707, 2023-11-13T11:01:15.691 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2023-11-13T11:01:15.691 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 2030, 2023-11-13T11:01:15.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2023-11-13T11:01:15.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2023-11-13T11:01:15.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2023-11-13T11:01:15.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1910, 2023-11-13T11:01:15.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 22, 2023-11-13T11:01:15.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2023-11-13T11:01:15.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2023-11-13T11:01:15.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2023-11-13T11:01:15.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2023-11-13T11:01:15.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2023-11-13T11:01:15.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2023-11-13T11:01:15.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2023-11-13T11:01:15.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2023-11-13T11:01:15.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2023-11-13T11:01:15.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2023-11-13T11:01:15.698 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2023-11-13T11:01:15.698 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2023-11-13T11:01:15.699 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2023-11-13T11:01:15.699 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2023-11-13T11:01:15.699 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2023-11-13T11:01:15.699 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2023-11-13T11:01:15.699 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2023-11-13T11:01:15.700 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2023-11-13T11:01:15.700 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2023-11-13T11:01:15.700 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2023-11-13T11:01:15.700 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2023-11-13T11:01:15.700 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2023-11-13T11:01:15.701 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2023-11-13T11:01:15.701 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2023-11-13T11:01:15.701 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2023-11-13T11:01:15.701 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2023-11-13T11:01:15.702 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 44, 2023-11-13T11:01:15.702 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2023-11-13T11:01:15.702 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 5, 2023-11-13T11:01:15.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 8, 2023-11-13T11:01:15.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 8, 2023-11-13T11:01:15.703 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2023-11-13T11:01:15.703 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2023-11-13T11:01:15.703 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 70, 2023-11-13T11:01:15.703 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 0, 2023-11-13T11:01:15.703 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 12 2023-11-13T11:01:15.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.704 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2023-11-13T11:01:15.704 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 1, 2023-11-13T11:01:15.704 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2023-11-13T11:01:15.704 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2023-11-13T11:01:15.704 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2023-11-13T11:01:15.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.705 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2023-11-13T11:01:15.705 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2023-11-13T11:01:15.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.705 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2023-11-13T11:01:15.706 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2023-11-13T11:01:15.706 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 32, 2023-11-13T11:01:15.706 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2023-11-13T11:01:15.706 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 32, 2023-11-13T11:01:15.706 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 3707, 2023-11-13T11:01:15.707 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2023-11-13T11:01:15.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.707 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-13T11:01:15.707 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2023-11-13T11:01:15.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2023-11-13T11:01:15.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:01:15.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 45413, 2023-11-13T11:01:15.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 11141906, 2023-11-13T11:01:15.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:01:15.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 45413, 2023-11-13T11:01:15.708 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2023-11-13T11:01:15.709 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2023-11-13T11:01:15.709 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 45413, 2023-11-13T11:01:15.709 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 11141906, 2023-11-13T11:01:15.709 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:01:15.709 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.710 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.710 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.710 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:15.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.710 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2023-11-13T11:01:15.711 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2023-11-13T11:01:15.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2023-11-13T11:01:15.711 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:01:15.711 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2023-11-13T11:01:15.711 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2023-11-13T11:01:15.712 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:01:15.712 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2023-11-13T11:01:15.712 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 12688, 2023-11-13T11:01:15.712 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 79862162, 2023-11-13T11:01:15.712 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 12688, 2023-11-13T11:01:15.713 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 79862162, 2023-11-13T11:01:15.713 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:01:15.713 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.713 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.713 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:15.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.714 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2023-11-13T11:01:15.714 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2023-11-13T11:01:15.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2023-11-13T11:01:15.715 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:01:15.715 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2023-11-13T11:01:15.715 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2023-11-13T11:01:15.715 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:01:15.715 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2023-11-13T11:01:15.716 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 12688, 2023-11-13T11:01:15.716 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 12688, 2023-11-13T11:01:15.716 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 12688, 2023-11-13T11:01:15.716 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 12688, 2023-11-13T11:01:15.716 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:01:15.717 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.717 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.717 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.717 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:15.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.718 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2023-11-13T11:01:15.718 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2023-11-13T11:01:15.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2023-11-13T11:01:15.718 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:01:15.718 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 3707, 2023-11-13T11:01:15.719 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 388063, 2023-11-13T11:01:15.719 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:01:15.719 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 3707, 2023-11-13T11:01:15.719 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2023-11-13T11:01:15.719 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2023-11-13T11:01:15.719 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 143, 2023-11-13T11:01:15.720 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 388063, 2023-11-13T11:01:15.720 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:01:15.720 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.720 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.720 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.721 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:15.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:15.721 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55f93add66e0": { 2023-11-13T11:01:15.721 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2023-11-13T11:01:15.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2023-11-13T11:01:15.722 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2023-11-13T11:01:15.722 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 23983, 2023-11-13T11:01:15.722 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 79052366, 2023-11-13T11:01:15.722 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2023-11-13T11:01:15.722 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 23983, 2023-11-13T11:01:15.723 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2023-11-13T11:01:15.723 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2023-11-13T11:01:15.723 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 6600, 2023-11-13T11:01:15.723 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 79052366, 2023-11-13T11:01:15.723 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2023-11-13T11:01:15.724 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2023-11-13T11:01:15.724 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2023-11-13T11:01:15.724 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2023-11-13T11:01:15.724 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:15.724 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:15.725 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:01:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 -- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f62bc06ae70 msgr2=0x7f62bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f62bc06ae70 0x7f62bc06d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f62c4036000 tx=0x7f62c4030000 comp rx=0 tx=0).stop 2023-11-13T11:01:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 -- 172.21.15.16:0/2681319859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc0b2970 msgr2=0x7f62dc159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc0b2970 0x7f62dc159a60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00d5f0 tx=0x7f62cc00dab0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 -- 172.21.15.16:0/2681319859 shutdown_connections 2023-11-13T11:01:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f62bc06ae70 0x7f62bc06d330 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62dc161640 0x7f62dc1ecc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62dc15b5c0 0x7f62dc15c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 --2- 172.21.15.16:0/2681319859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc0b2970 0x7f62dc159a60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 -- 172.21.15.16:0/2681319859 >> 172.21.15.16:0/2681319859 conn(0x7f62dc0b0970 msgr2=0x7f62dc0b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 -- 172.21.15.16:0/2681319859 shutdown_connections 2023-11-13T11:01:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:15.570+0000 7f62db59e640 1 -- 172.21.15.16:0/2681319859 wait complete. 2023-11-13T11:01:15.730 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-13T11:01:15.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:01:15.855 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:01:15.986 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-13T11:01:15.987 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:01:15.988 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:01:16.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:01:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.273+0000 7fa88011d640 1 Processor -- start 2023-11-13T11:01:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.273+0000 7fa88011d640 1 -- start start 2023-11-13T11:01:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.275+0000 7fa88011d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa8780ccc60 0x7fa878071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.275+0000 7fa88011d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa878071980 0x7fa87806fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.275+0000 7fa88011d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8780cd110 con 0x7fa878155410 2023-11-13T11:01:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.275+0000 7fa88011d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa878071e10 con 0x7fa878071980 2023-11-13T11:01:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.275+0000 7fa88011d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87815b500 con 0x7fa8780ccc60 2023-11-13T11:01:16.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.275+0000 7fa87e6ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa878071980 0x7fa87806fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.275+0000 7fa87e6ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa878071980 0x7fa87806fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41886/0 (socket says 172.21.15.16:41886) 2023-11-13T11:01:16.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.275+0000 7fa87e6ba640 1 -- 172.21.15.16:0/3150563218 learned_addr learned my addr 172.21.15.16:0/3150563218 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:16.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.275+0000 7fa87deb9640 1 --1- 172.21.15.16:0/3150563218 >> v1:172.21.15.16:6789/0 conn(0x7fa878155410 0x7fa8780ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36332/0 (socket says 172.21.15.16:36332) 2023-11-13T11:01:16.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.276+0000 7fa87d6b8640 1 --2- 172.21.15.16:0/3150563218 >> v2:172.21.15.184:3300/0 conn(0x7fa8780ccc60 0x7fa878071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.276+0000 7fa87e6ba640 1 -- 172.21.15.16:0/3150563218 >> v2:172.21.15.184:3300/0 conn(0x7fa8780ccc60 msgr2=0x7fa878071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:16.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.276+0000 7fa87e6ba640 1 --2- 172.21.15.16:0/3150563218 >> v2:172.21.15.184:3300/0 conn(0x7fa8780ccc60 0x7fa878071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.276+0000 7fa87e6ba640 1 -- 172.21.15.16:0/3150563218 >> v1:172.21.15.16:6789/0 conn(0x7fa878155410 legacy=0x7fa8780ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.276+0000 7fa87e6ba640 1 -- 172.21.15.16:0/3150563218 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87815b7e0 con 0x7fa878071980 2023-11-13T11:01:16.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.277+0000 7fa87e6ba640 1 --2- 172.21.15.16:0/3150563218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 0x7fa87806fe40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa87806e000 tx=0x7fa87002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d912bb4f57d363 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:16.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.277+0000 7fa87ceb7640 1 -- 172.21.15.16:0/3150563218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87002cb10 con 0x7fa878071980 2023-11-13T11:01:16.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.278+0000 7fa87ceb7640 1 -- 172.21.15.16:0/3150563218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87002cc70 con 0x7fa878071980 2023-11-13T11:01:16.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.278+0000 7fa87ceb7640 1 -- 172.21.15.16:0/3150563218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8700343f0 con 0x7fa878071980 2023-11-13T11:01:16.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.278+0000 7fa88011d640 1 -- 172.21.15.16:0/3150563218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 msgr2=0x7fa87806fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.278+0000 7fa88011d640 1 --2- 172.21.15.16:0/3150563218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 0x7fa87806fe40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa87806e000 tx=0x7fa87002bf70 comp rx=0 tx=0).stop 2023-11-13T11:01:16.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.278+0000 7fa88011d640 1 -- 172.21.15.16:0/3150563218 shutdown_connections 2023-11-13T11:01:16.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.278+0000 7fa88011d640 1 --2- 172.21.15.16:0/3150563218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 0x7fa87806fe40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.278+0000 7fa88011d640 1 --2- 172.21.15.16:0/3150563218 >> v2:172.21.15.184:3300/0 conn(0x7fa8780ccc60 0x7fa878071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.278+0000 7fa88011d640 1 -- 172.21.15.16:0/3150563218 >> 172.21.15.16:0/3150563218 conn(0x7fa87806a190 msgr2=0x7fa87806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.279+0000 7fa88011d640 1 -- 172.21.15.16:0/3150563218 shutdown_connections 2023-11-13T11:01:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.279+0000 7fa88011d640 1 -- 172.21.15.16:0/3150563218 wait complete. 2023-11-13T11:01:16.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.280+0000 7fa88011d640 1 Processor -- start 2023-11-13T11:01:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.281+0000 7fa88011d640 1 -- start start 2023-11-13T11:01:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa88011d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 0x7fa87814a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa88011d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780ccc60 0x7fa878156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa88011d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa878155410 0x7fa8781535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa88011d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878151a00 con 0x7fa878071980 2023-11-13T11:01:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa88011d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87814a550 con 0x7fa878155410 2023-11-13T11:01:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa88011d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa878157180 con 0x7fa8780ccc60 2023-11-13T11:01:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa87deb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 0x7fa87814a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa87deb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 0x7fa87814a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41890/0 (socket says 172.21.15.16:41890) 2023-11-13T11:01:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa87deb9640 1 -- 172.21.15.16:0/4126389968 learned_addr learned my addr 172.21.15.16:0/4126389968 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa87deb9640 1 -- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780ccc60 msgr2=0x7fa878156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa87deb9640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780ccc60 0x7fa878156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa87deb9640 1 -- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa878155410 msgr2=0x7fa8781535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa87deb9640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa878155410 0x7fa8781535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.282+0000 7fa87deb9640 1 -- 172.21.15.16:0/4126389968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870009b30 con 0x7fa878071980 2023-11-13T11:01:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.283+0000 7fa87deb9640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 0x7fa87814a010 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c3e0 tx=0x7fa87400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.283+0000 7fa86e7fc640 1 -- 172.21.15.16:0/4126389968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8740074e0 con 0x7fa878071980 2023-11-13T11:01:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.283+0000 7fa86e7fc640 1 -- 172.21.15.16:0/4126389968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa874007640 con 0x7fa878071980 2023-11-13T11:01:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.283+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa878157460 con 0x7fa878071980 2023-11-13T11:01:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.283+0000 7fa86e7fc640 1 -- 172.21.15.16:0/4126389968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8740049e0 con 0x7fa878071980 2023-11-13T11:01:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.283+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa878157a20 con 0x7fa878071980 2023-11-13T11:01:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.284+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa878067090 con 0x7fa878071980 2023-11-13T11:01:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.285+0000 7fa86e7fc640 1 -- 172.21.15.16:0/4126389968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa874004b40 con 0x7fa878071980 2023-11-13T11:01:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.286+0000 7fa86e7fc640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa85c06ae40 0x7fa85c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.286+0000 7fa86e7fc640 1 -- 172.21.15.16:0/4126389968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fa874089200 con 0x7fa878071980 2023-11-13T11:01:16.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.287+0000 7fa87d6b8640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa85c06ae40 0x7fa85c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.289+0000 7fa87d6b8640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa85c06ae40 0x7fa85c06d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa8780adc00 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:16.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.291+0000 7fa86e7fc640 1 -- 172.21.15.16:0/4126389968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8740540b0 con 0x7fa878071980 2023-11-13T11:01:16.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.485+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa87806a990 con 0x7fa878071980 2023-11-13T11:01:16.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.488+0000 7fa86e7fc640 1 -- 172.21.15.16:0/4126389968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+17026 (secure 0 0 0) 0x7fa8740576d0 con 0x7fa878071980 2023-11-13T11:01:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:16.493 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:00:55.896214+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:57:18.637969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-13T10:57:19.541678+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":43,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.2","mappings":[{"from":4,"to":7}]},{"pgid":"6.14","mappings":[{"from":3,"to":5}]},{"pgid":"6.18","mappings":[{"from":4,"to":2}]},{"pgid":"6.38","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:01:16.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.490+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa85c06ae40 msgr2=0x7fa85c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.490+0000 7fa88011d640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa85c06ae40 0x7fa85c06d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa8780adc00 tx=0x7fa868008040 comp rx=0 tx=0).stop 2023-11-13T11:01:16.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.490+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 msgr2=0x7fa87814a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.490+0000 7fa88011d640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 0x7fa87814a010 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c3e0 tx=0x7fa87400c8a0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.491+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 shutdown_connections 2023-11-13T11:01:16.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.491+0000 7fa88011d640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa85c06ae40 0x7fa85c06d300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.491+0000 7fa88011d640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa878155410 0x7fa8781535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.491+0000 7fa88011d640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8780ccc60 0x7fa878156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.491+0000 7fa88011d640 1 --2- 172.21.15.16:0/4126389968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa878071980 0x7fa87814a010 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.491+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 >> 172.21.15.16:0/4126389968 conn(0x7fa87806a190 msgr2=0x7fa8780b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:16.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.491+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 shutdown_connections 2023-11-13T11:01:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.491+0000 7fa88011d640 1 -- 172.21.15.16:0/4126389968 wait complete. 2023-11-13T11:01:16.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:57:18.637969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:57:19.541678+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 43, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:01:16.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:01:16.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.744+0000 7f9322e1a640 1 Processor -- start 2023-11-13T11:01:16.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.744+0000 7f9322e1a640 1 -- start start 2023-11-13T11:01:16.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.746+0000 7f9322e1a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f931c1622f0 0x7f931c0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.746+0000 7f9322e1a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f931c0b1f10 0x7f931c1469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.746+0000 7f9322e1a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c144660 con 0x7f931c0cf090 2023-11-13T11:01:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.746+0000 7f9322e1a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c162760 con 0x7f931c0b1f10 2023-11-13T11:01:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.746+0000 7f9322e1a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c0cf470 con 0x7f931c1622f0 2023-11-13T11:01:16.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.746+0000 7f93213b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f931c0b1f10 0x7f931c1469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.746+0000 7f93213b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f931c0b1f10 0x7f931c1469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41894/0 (socket says 172.21.15.16:41894) 2023-11-13T11:01:16.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.746+0000 7f93213b7640 1 -- 172.21.15.16:0/2358891441 learned_addr learned my addr 172.21.15.16:0/2358891441 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:16.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.746+0000 7f931bfff640 1 --2- 172.21.15.16:0/2358891441 >> v2:172.21.15.184:3300/0 conn(0x7f931c1622f0 0x7f931c0b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.747+0000 7f93213b7640 1 -- 172.21.15.16:0/2358891441 >> v2:172.21.15.184:3300/0 conn(0x7f931c1622f0 msgr2=0x7f931c0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:16.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.747+0000 7f93213b7640 1 --2- 172.21.15.16:0/2358891441 >> v2:172.21.15.184:3300/0 conn(0x7f931c1622f0 0x7f931c0b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.747+0000 7f93213b7640 1 -- 172.21.15.16:0/2358891441 >> v1:172.21.15.184:6789/0 conn(0x7f931c0cf090 legacy=0x7f931c164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:16.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.747+0000 7f93213b7640 1 -- 172.21.15.16:0/2358891441 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c161e60 con 0x7f931c0b1f10 2023-11-13T11:01:16.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.748+0000 7f93213b7640 1 --2- 172.21.15.16:0/2358891441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 0x7f931c1469a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f931c070ff0 tx=0x7f931402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16eeeab3632a0b90 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:16.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.748+0000 7f931b7fe640 1 -- 172.21.15.16:0/2358891441 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931402c9f0 con 0x7f931c0b1f10 2023-11-13T11:01:16.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.748+0000 7f931b7fe640 1 -- 172.21.15.16:0/2358891441 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931402cb50 con 0x7f931c0b1f10 2023-11-13T11:01:16.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.748+0000 7f931b7fe640 1 -- 172.21.15.16:0/2358891441 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140343f0 con 0x7f931c0b1f10 2023-11-13T11:01:16.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.749+0000 7f9322e1a640 1 -- 172.21.15.16:0/2358891441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 msgr2=0x7f931c1469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.749+0000 7f9322e1a640 1 --2- 172.21.15.16:0/2358891441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 0x7f931c1469a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f931c070ff0 tx=0x7f931402be50 comp rx=0 tx=0).stop 2023-11-13T11:01:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.749+0000 7f9322e1a640 1 -- 172.21.15.16:0/2358891441 shutdown_connections 2023-11-13T11:01:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.749+0000 7f9322e1a640 1 --2- 172.21.15.16:0/2358891441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 0x7f931c1469a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.749+0000 7f9322e1a640 1 --2- 172.21.15.16:0/2358891441 >> v2:172.21.15.184:3300/0 conn(0x7f931c1622f0 0x7f931c0b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.749+0000 7f9322e1a640 1 -- 172.21.15.16:0/2358891441 >> 172.21.15.16:0/2358891441 conn(0x7f931c06a190 msgr2=0x7f931c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.749+0000 7f9322e1a640 1 -- 172.21.15.16:0/2358891441 shutdown_connections 2023-11-13T11:01:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.749+0000 7f9322e1a640 1 -- 172.21.15.16:0/2358891441 wait complete. 2023-11-13T11:01:16.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.751+0000 7f9322e1a640 1 Processor -- start 2023-11-13T11:01:16.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.751+0000 7f9322e1a640 1 -- start start 2023-11-13T11:01:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.752+0000 7f9322e1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 0x7f931c1414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.752+0000 7f9322e1a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c0cf090 0x7f931c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.752+0000 7f9322e1a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c1622f0 0x7f931c0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.752+0000 7f9322e1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0a8a10 con 0x7f931c0b1f10 2023-11-13T11:01:16.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.752+0000 7f9322e1a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c159af0 con 0x7f931c0cf090 2023-11-13T11:01:16.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.752+0000 7f9322e1a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c159dd0 con 0x7f931c1622f0 2023-11-13T11:01:16.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.752+0000 7f9320bb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 0x7f931c1414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f9320bb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 0x7f931c1414e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41908/0 (socket says 172.21.15.16:41908) 2023-11-13T11:01:16.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f9320bb6640 1 -- 172.21.15.16:0/1693721579 learned_addr learned my addr 172.21.15.16:0/1693721579 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f931bfff640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c0cf090 0x7f931c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f93213b7640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c1622f0 0x7f931c0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f9320bb6640 1 -- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c1622f0 msgr2=0x7f931c0a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f9320bb6640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c1622f0 0x7f931c0a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f9320bb6640 1 -- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c0cf090 msgr2=0x7f931c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f9320bb6640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c0cf090 0x7f931c1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f9320bb6640 1 -- 172.21.15.16:0/1693721579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009b30 con 0x7f931c0b1f10 2023-11-13T11:01:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.753+0000 7f9320bb6640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 0x7f931c1414e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f931000c430 tx=0x7f931000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.754+0000 7f93197fa640 1 -- 172.21.15.16:0/1693721579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310011790 con 0x7f931c0b1f10 2023-11-13T11:01:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.754+0000 7f93197fa640 1 -- 172.21.15.16:0/1693721579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93100118f0 con 0x7f931c0b1f10 2023-11-13T11:01:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.754+0000 7f93197fa640 1 -- 172.21.15.16:0/1693721579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310011ce0 con 0x7f931c0b1f10 2023-11-13T11:01:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.754+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c15a0b0 con 0x7f931c0b1f10 2023-11-13T11:01:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.754+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c15a670 con 0x7f931c0b1f10 2023-11-13T11:01:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.755+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c072520 con 0x7f931c0b1f10 2023-11-13T11:01:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.756+0000 7f93197fa640 1 -- 172.21.15.16:0/1693721579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f931001c460 con 0x7f931c0b1f10 2023-11-13T11:01:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.757+0000 7f93197fa640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f930006ae40 0x7f930006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.758+0000 7f93197fa640 1 -- 172.21.15.16:0/1693721579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f9310089800 con 0x7f931c0b1f10 2023-11-13T11:01:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.758+0000 7f931bfff640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f930006ae40 0x7f930006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.759+0000 7f931bfff640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f930006ae40 0x7f930006d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9308006fd0 tx=0x7f9308008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:16.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.762+0000 7f93197fa640 1 -- 172.21.15.16:0/1693721579 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9310054620 con 0x7f931c0b1f10 2023-11-13T11:01:16.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.955+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f931c04e5b0 con 0x7f931c0b1f10 2023-11-13T11:01:16.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.956+0000 7f93197fa640 1 -- 172.21.15.16:0/1693721579 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f931005e030 con 0x7f931c0b1f10 2023-11-13T11:01:16.960 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:01:16.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f930006ae40 msgr2=0x7f930006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f930006ae40 0x7f930006d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9308006fd0 tx=0x7f9308008040 comp rx=0 tx=0).stop 2023-11-13T11:01:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 msgr2=0x7f931c1414e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 0x7f931c1414e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f931000c430 tx=0x7f931000c8f0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 shutdown_connections 2023-11-13T11:01:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f930006ae40 0x7f930006d300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c1622f0 0x7f931c0a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c0cf090 0x7f931c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 --2- 172.21.15.16:0/1693721579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0b1f10 0x7f931c1414e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.958+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 >> 172.21.15.16:0/1693721579 conn(0x7f931c06a190 msgr2=0x7f931c160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.959+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 shutdown_connections 2023-11-13T11:01:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:16.959+0000 7f9322e1a640 1 -- 172.21.15.16:0/1693721579 wait complete. 2023-11-13T11:01:16.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T11:01:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.204+0000 7fea3759e640 1 Processor -- start 2023-11-13T11:01:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.205+0000 7fea3759e640 1 -- start start 2023-11-13T11:01:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.206+0000 7fea3759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea380d1850 0x7fea380d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.206+0000 7fea3759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea380ce0e0 con 0x7fea380d2d00 2023-11-13T11:01:17.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.206+0000 7fea3759e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea380ce3c0 con 0x7fea3815ce70 2023-11-13T11:01:17.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.206+0000 7fea3759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea380ce6a0 con 0x7fea380d1850 2023-11-13T11:01:17.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.207+0000 7fea35d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea380d1850 0x7fea380d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.207+0000 7fea36d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea380d2d00 0x7fea380c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36348/0 (socket says 172.21.15.16:36348) 2023-11-13T11:01:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.207+0000 7fea36d9d640 1 -- 172.21.15.16:0/4231447448 learned_addr learned my addr 172.21.15.16:0/4231447448 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.207+0000 7fea3559a640 1 -- 172.21.15.16:0/4231447448 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080015210 0 0) 0x7fea380ce0e0 con 0x7fea380d2d00 2023-11-13T11:01:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.207+0000 7fea3559a640 1 -- 172.21.15.16:0/4231447448 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea24003520 con 0x7fea380d2d00 2023-11-13T11:01:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.207+0000 7fea35d9b640 1 -- 172.21.15.16:0/4231447448 >> v1:172.21.15.184:6789/0 conn(0x7fea3815ce70 legacy=0x7fea380cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.207+0000 7fea35d9b640 1 -- 172.21.15.16:0/4231447448 >> v1:172.21.15.16:6789/0 conn(0x7fea380d2d00 legacy=0x7fea380c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.207+0000 7fea35d9b640 1 -- 172.21.15.16:0/4231447448 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea380ce980 con 0x7fea380d1850 2023-11-13T11:01:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.209+0000 7fea35d9b640 1 --2- 172.21.15.16:0/4231447448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d1850 0x7fea380d27c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fea200097a0 tx=0x7fea2002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac92fb6d9928745c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.209+0000 7fea3559a640 1 -- 172.21.15.16:0/4231447448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20030610 con 0x7fea380d1850 2023-11-13T11:01:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.209+0000 7fea3559a640 1 -- 172.21.15.16:0/4231447448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea20030770 con 0x7fea380d1850 2023-11-13T11:01:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.209+0000 7fea3559a640 1 -- 172.21.15.16:0/4231447448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20030b00 con 0x7fea380d1850 2023-11-13T11:01:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.209+0000 7fea3759e640 1 -- 172.21.15.16:0/4231447448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d1850 msgr2=0x7fea380d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.209+0000 7fea3759e640 1 --2- 172.21.15.16:0/4231447448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d1850 0x7fea380d27c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fea200097a0 tx=0x7fea2002b960 comp rx=0 tx=0).stop 2023-11-13T11:01:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.209+0000 7fea3759e640 1 -- 172.21.15.16:0/4231447448 shutdown_connections 2023-11-13T11:01:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.209+0000 7fea3759e640 1 --2- 172.21.15.16:0/4231447448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d1850 0x7fea380d27c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.209+0000 7fea3759e640 1 -- 172.21.15.16:0/4231447448 >> 172.21.15.16:0/4231447448 conn(0x7fea3808c580 msgr2=0x7fea3809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.210+0000 7fea3759e640 1 -- 172.21.15.16:0/4231447448 shutdown_connections 2023-11-13T11:01:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.210+0000 7fea3759e640 1 -- 172.21.15.16:0/4231447448 wait complete. 2023-11-13T11:01:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.211+0000 7fea3759e640 1 Processor -- start 2023-11-13T11:01:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.212+0000 7fea3759e640 1 -- start start 2023-11-13T11:01:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea3759e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea380d1850 0x7fea38159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea3759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d2d00 0x7fea3815e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea3759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3815ce70 0x7fea380b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea3759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3815fe20 con 0x7fea380d2d00 2023-11-13T11:01:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea3759e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3815a1d0 con 0x7fea380d1850 2023-11-13T11:01:17.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea3759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3815f210 con 0x7fea3815ce70 2023-11-13T11:01:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d2d00 0x7fea3815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d2d00 0x7fea3815e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41926/0 (socket says 172.21.15.16:41926) 2023-11-13T11:01:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea35d9b640 1 -- 172.21.15.16:0/2689827495 learned_addr learned my addr 172.21.15.16:0/2689827495 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea3659c640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea380d1850 0x7fea38159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea36d9d640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3815ce70 0x7fea380b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea35d9b640 1 -- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3815ce70 msgr2=0x7fea380b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea35d9b640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3815ce70 0x7fea380b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea35d9b640 1 -- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea380d1850 msgr2=0x7fea38159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea35d9b640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea380d1850 0x7fea38159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea35d9b640 1 -- 172.21.15.16:0/2689827495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea20009400 con 0x7fea380d2d00 2023-11-13T11:01:17.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.213+0000 7fea36d9d640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3815ce70 0x7fea380b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:17.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.214+0000 7fea35d9b640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d2d00 0x7fea3815e790 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fea20009890 tx=0x7fea20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:17.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.214+0000 7fea2effd640 1 -- 172.21.15.16:0/2689827495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2003e040 con 0x7fea380d2d00 2023-11-13T11:01:17.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.214+0000 7fea2effd640 1 -- 172.21.15.16:0/2689827495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea20039ce0 con 0x7fea380d2d00 2023-11-13T11:01:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.214+0000 7fea2effd640 1 -- 172.21.15.16:0/2689827495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20043680 con 0x7fea380d2d00 2023-11-13T11:01:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.214+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3815f4f0 con 0x7fea380d2d00 2023-11-13T11:01:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.214+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3815fab0 con 0x7fea380d2d00 2023-11-13T11:01:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.216+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3815ffd0 con 0x7fea380d2d00 2023-11-13T11:01:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.216+0000 7fea2effd640 1 -- 172.21.15.16:0/2689827495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea200423e0 con 0x7fea380d2d00 2023-11-13T11:01:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.217+0000 7fea2effd640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fea1806ae70 0x7fea1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.217+0000 7fea2effd640 1 -- 172.21.15.16:0/2689827495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fea200ab7b0 con 0x7fea380d2d00 2023-11-13T11:01:17.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.218+0000 7fea3659c640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fea1806ae70 0x7fea1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.219+0000 7fea3659c640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fea1806ae70 0x7fea1806d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea280012e0 tx=0x7fea28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.223+0000 7fea2effd640 1 -- 172.21.15.16:0/2689827495 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea3815ffd0 con 0x7fea380d2d00 2023-11-13T11:01:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.417+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea3815ffd0 con 0x7fea380d2d00 2023-11-13T11:01:17.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.418+0000 7fea2effd640 1 -- 172.21.15.16:0/2689827495 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea3815ffd0 con 0x7fea380d2d00 2023-11-13T11:01:17.422 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T11:01:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fea1806ae70 msgr2=0x7fea1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fea1806ae70 0x7fea1806d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea280012e0 tx=0x7fea28009040 comp rx=0 tx=0).stop 2023-11-13T11:01:17.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d2d00 msgr2=0x7fea3815e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d2d00 0x7fea3815e790 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fea20009890 tx=0x7fea20004730 comp rx=0 tx=0).stop 2023-11-13T11:01:17.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 shutdown_connections 2023-11-13T11:01:17.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fea1806ae70 0x7fea1806d330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea3815ce70 0x7fea380b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380d2d00 0x7fea3815e790 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 --2- 172.21.15.16:0/2689827495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea380d1850 0x7fea38159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 >> 172.21.15.16:0/2689827495 conn(0x7fea3808c580 msgr2=0x7fea380ca180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 shutdown_connections 2023-11-13T11:01:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.421+0000 7fea3759e640 1 -- 172.21.15.16:0/2689827495 wait complete. 2023-11-13T11:01:17.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T11:01:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.669+0000 7f2dfa3dc640 1 Processor -- start 2023-11-13T11:01:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.670+0000 7f2dfa3dc640 1 -- start start 2023-11-13T11:01:17.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.671+0000 7f2dfa3dc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2df41589a0 0x7f2df40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.671+0000 7f2dfa3dc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2df40ca730 0x7f2df406a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.671+0000 7f2dfa3dc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df415b590 con 0x7f2df406a420 2023-11-13T11:01:17.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.671+0000 7f2dfa3dc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df406adc0 con 0x7f2df40ca730 2023-11-13T11:01:17.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.671+0000 7f2dfa3dc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df415b870 con 0x7f2df41589a0 2023-11-13T11:01:17.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.672+0000 7f2df8979640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2df40ca730 0x7f2df406a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.672+0000 7f2df37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2df406a420 0x7f2df40ca020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36358/0 (socket says 172.21.15.16:36358) 2023-11-13T11:01:17.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.672+0000 7f2df37fe640 1 -- 172.21.15.16:0/2876682345 learned_addr learned my addr 172.21.15.16:0/2876682345 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:17.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.672+0000 7f2df3fff640 1 --2- 172.21.15.16:0/2876682345 >> v2:172.21.15.184:3300/0 conn(0x7f2df41589a0 0x7f2df40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.673+0000 7f2df8979640 1 -- 172.21.15.16:0/2876682345 >> v2:172.21.15.184:3300/0 conn(0x7f2df41589a0 msgr2=0x7f2df40cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:17.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.673+0000 7f2df8979640 1 --2- 172.21.15.16:0/2876682345 >> v2:172.21.15.184:3300/0 conn(0x7f2df41589a0 0x7f2df40cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.673+0000 7f2df8979640 1 -- 172.21.15.16:0/2876682345 >> v1:172.21.15.16:6789/0 conn(0x7f2df406a420 legacy=0x7f2df40ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.673+0000 7f2df8979640 1 -- 172.21.15.16:0/2876682345 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df415bb50 con 0x7f2df40ca730 2023-11-13T11:01:17.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.674+0000 7f2df8979640 1 --2- 172.21.15.16:0/2876682345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 0x7f2df406a880 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2df4073860 tx=0x7f2dec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a5896a992ea0d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:17.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.674+0000 7f2df2ffd640 1 -- 172.21.15.16:0/2876682345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec031830 con 0x7f2df40ca730 2023-11-13T11:01:17.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.674+0000 7f2df2ffd640 1 -- 172.21.15.16:0/2876682345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec031990 con 0x7f2df40ca730 2023-11-13T11:01:17.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.674+0000 7f2df2ffd640 1 -- 172.21.15.16:0/2876682345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec031d20 con 0x7f2df40ca730 2023-11-13T11:01:17.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.675+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/2876682345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 msgr2=0x7f2df406a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.675+0000 7f2dfa3dc640 1 --2- 172.21.15.16:0/2876682345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 0x7f2df406a880 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2df4073860 tx=0x7f2dec02b8d0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.675+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/2876682345 shutdown_connections 2023-11-13T11:01:17.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.675+0000 7f2dfa3dc640 1 --2- 172.21.15.16:0/2876682345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 0x7f2df406a880 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.675+0000 7f2dfa3dc640 1 --2- 172.21.15.16:0/2876682345 >> v2:172.21.15.184:3300/0 conn(0x7f2df41589a0 0x7f2df40cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.675+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/2876682345 >> 172.21.15.16:0/2876682345 conn(0x7f2df40cf9d0 msgr2=0x7f2df40cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:17.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.675+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/2876682345 shutdown_connections 2023-11-13T11:01:17.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.675+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/2876682345 wait complete. 2023-11-13T11:01:17.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.677+0000 7f2dfa3dc640 1 Processor -- start 2023-11-13T11:01:17.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.677+0000 7f2dfa3dc640 1 -- start start 2023-11-13T11:01:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2dfa3dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2df406a420 0x7f2df41511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2dfa3dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 0x7f2df4151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2dfa3dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2df41589a0 0x7f2df414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2dfa3dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40b56a0 con 0x7f2df40ca730 2023-11-13T11:01:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2dfa3dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df406cc60 con 0x7f2df406a420 2023-11-13T11:01:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2dfa3dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df406cf40 con 0x7f2df41589a0 2023-11-13T11:01:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 0x7f2df4151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 0x7f2df4151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41940/0 (socket says 172.21.15.16:41940) 2023-11-13T11:01:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df37fe640 1 -- 172.21.15.16:0/1840933528 learned_addr learned my addr 172.21.15.16:0/1840933528 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df8979640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2df41589a0 0x7f2df414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df37fe640 1 -- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2df41589a0 msgr2=0x7f2df414dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df37fe640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2df41589a0 0x7f2df414dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df3fff640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2df406a420 0x7f2df41511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df37fe640 1 -- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2df406a420 msgr2=0x7f2df41511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df37fe640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2df406a420 0x7f2df41511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.678+0000 7f2df37fe640 1 -- 172.21.15.16:0/1840933528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec009400 con 0x7f2df40ca730 2023-11-13T11:01:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.679+0000 7f2df37fe640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 0x7f2df4151720 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00c4a0 tx=0x7f2ddc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.679+0000 7f2df0ff9640 1 -- 172.21.15.16:0/1840933528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc005ce0 con 0x7f2df40ca730 2023-11-13T11:01:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.679+0000 7f2df0ff9640 1 -- 172.21.15.16:0/1840933528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc005e40 con 0x7f2df40ca730 2023-11-13T11:01:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.679+0000 7f2df0ff9640 1 -- 172.21.15.16:0/1840933528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc007270 con 0x7f2df40ca730 2023-11-13T11:01:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.679+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df406d220 con 0x7f2df40ca730 2023-11-13T11:01:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.679+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df406d7e0 con 0x7f2df40ca730 2023-11-13T11:01:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.681+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df406d9d0 con 0x7f2df40ca730 2023-11-13T11:01:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.681+0000 7f2df0ff9640 1 -- 172.21.15.16:0/1840933528 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ddc007490 con 0x7f2df40ca730 2023-11-13T11:01:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.682+0000 7f2df0ff9640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.682+0000 7f2df0ff9640 1 -- 172.21.15.16:0/1840933528 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f2ddc088d60 con 0x7f2df40ca730 2023-11-13T11:01:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.683+0000 7f2df3fff640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.684+0000 7f2df3fff640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2df40c5280 tx=0x7f2de400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:17.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.686+0000 7f2df0ff9640 1 -- 172.21.15.16:0/1840933528 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ddc053c30 con 0x7f2df40ca730 2023-11-13T11:01:17.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.881+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2df4072430 con 0x7f2df40ca730 2023-11-13T11:01:17.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.882+0000 7f2df0ff9640 1 -- 172.21.15.16:0/1840933528 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ddc017020 con 0x7f2df40ca730 2023-11-13T11:01:17.886 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T11:01:17.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.884+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 msgr2=0x7f2dd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.884+0000 7f2dfa3dc640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2df40c5280 tx=0x7f2de400b040 comp rx=0 tx=0).stop 2023-11-13T11:01:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.884+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 msgr2=0x7f2df4151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.884+0000 7f2dfa3dc640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 0x7f2df4151720 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00c4a0 tx=0x7f2ddc00c960 comp rx=0 tx=0).stop 2023-11-13T11:01:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.885+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 shutdown_connections 2023-11-13T11:01:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.885+0000 7f2dfa3dc640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2dd406ae40 0x7f2dd406d300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.885+0000 7f2dfa3dc640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2df41589a0 0x7f2df414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.885+0000 7f2dfa3dc640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df40ca730 0x7f2df4151720 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.885+0000 7f2dfa3dc640 1 --2- 172.21.15.16:0/1840933528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2df406a420 0x7f2df41511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.885+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 >> 172.21.15.16:0/1840933528 conn(0x7f2df40cf9d0 msgr2=0x7f2df40c4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:17.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.885+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 shutdown_connections 2023-11-13T11:01:17.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:17.885+0000 7f2dfa3dc640 1 -- 172.21.15.16:0/1840933528 wait complete. 2023-11-13T11:01:17.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T11:01:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.137+0000 7f076961a640 1 Processor -- start 2023-11-13T11:01:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.137+0000 7f076961a640 1 -- start start 2023-11-13T11:01:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f076961a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07640cb190 0x7f076415b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f076961a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f07640716b0 0x7f076406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f076961a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0764158c20 con 0x7f076406a190 2023-11-13T11:01:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f076961a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07640b1830 con 0x7f07640cb190 2023-11-13T11:01:18.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f076961a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076415b780 con 0x7f07640716b0 2023-11-13T11:01:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f0763fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07640cb190 0x7f076415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f0763fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07640cb190 0x7f076415b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41952/0 (socket says 172.21.15.16:41952) 2023-11-13T11:01:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f0763fff640 1 -- 172.21.15.16:0/3281835161 learned_addr learned my addr 172.21.15.16:0/3281835161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f075bfff640 1 --2- 172.21.15.16:0/3281835161 >> v2:172.21.15.184:3300/0 conn(0x7f07640716b0 0x7f076406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f0763fff640 1 -- 172.21.15.16:0/3281835161 >> v2:172.21.15.184:3300/0 conn(0x7f07640716b0 msgr2=0x7f076406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.139+0000 7f0763fff640 1 --2- 172.21.15.16:0/3281835161 >> v2:172.21.15.184:3300/0 conn(0x7f07640716b0 0x7f076406f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.140+0000 7f0763fff640 1 -- 172.21.15.16:0/3281835161 >> v1:172.21.15.184:6789/0 conn(0x7f076406a190 legacy=0x7f07640b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.140+0000 7f0763fff640 1 -- 172.21.15.16:0/3281835161 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076415ba60 con 0x7f07640cb190 2023-11-13T11:01:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.140+0000 7f075bfff640 1 --2- 172.21.15.16:0/3281835161 >> v2:172.21.15.184:3300/0 conn(0x7f07640716b0 0x7f076406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.141+0000 7f0763fff640 1 --2- 172.21.15.16:0/3281835161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640cb190 0x7f076415b190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f07640ae3e0 tx=0x7f075402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891ca547caddb55a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.141+0000 7f07637fe640 1 -- 172.21.15.16:0/3281835161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754031870 con 0x7f07640cb190 2023-11-13T11:01:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.141+0000 7f07637fe640 1 -- 172.21.15.16:0/3281835161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07540319d0 con 0x7f07640cb190 2023-11-13T11:01:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.141+0000 7f07637fe640 1 -- 172.21.15.16:0/3281835161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754031d60 con 0x7f07640cb190 2023-11-13T11:01:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.142+0000 7f076961a640 1 -- 172.21.15.16:0/3281835161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640cb190 msgr2=0x7f076415b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.142+0000 7f076961a640 1 --2- 172.21.15.16:0/3281835161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640cb190 0x7f076415b190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f07640ae3e0 tx=0x7f075402b910 comp rx=0 tx=0).stop 2023-11-13T11:01:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.142+0000 7f076961a640 1 -- 172.21.15.16:0/3281835161 shutdown_connections 2023-11-13T11:01:18.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.142+0000 7f076961a640 1 --2- 172.21.15.16:0/3281835161 >> v2:172.21.15.184:3300/0 conn(0x7f07640716b0 0x7f076406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.142+0000 7f076961a640 1 --2- 172.21.15.16:0/3281835161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640cb190 0x7f076415b190 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.142+0000 7f076961a640 1 -- 172.21.15.16:0/3281835161 >> 172.21.15.16:0/3281835161 conn(0x7f07640b04a0 msgr2=0x7f07640b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.142+0000 7f076961a640 1 -- 172.21.15.16:0/3281835161 shutdown_connections 2023-11-13T11:01:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.142+0000 7f076961a640 1 -- 172.21.15.16:0/3281835161 wait complete. 2023-11-13T11:01:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.144+0000 7f076961a640 1 Processor -- start 2023-11-13T11:01:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.144+0000 7f076961a640 1 -- start start 2023-11-13T11:01:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.145+0000 7f076961a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f076406a190 0x7f0764157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.145+0000 7f076961a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640716b0 0x7f07640c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.145+0000 7f076961a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07640cb190 0x7f076415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.145+0000 7f076961a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0764152fd0 con 0x7f07640716b0 2023-11-13T11:01:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.145+0000 7f076961a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0764152350 con 0x7f07640cb190 2023-11-13T11:01:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.145+0000 7f076961a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0764152630 con 0x7f076406a190 2023-11-13T11:01:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.145+0000 7f075bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640716b0 0x7f07640c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.145+0000 7f075bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640716b0 0x7f07640c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41962/0 (socket says 172.21.15.16:41962) 2023-11-13T11:01:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.145+0000 7f075bfff640 1 -- 172.21.15.16:0/372698299 learned_addr learned my addr 172.21.15.16:0/372698299 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f075bfff640 1 -- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f076406a190 msgr2=0x7f0764157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f075bfff640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f076406a190 0x7f0764157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f075bfff640 1 -- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07640cb190 msgr2=0x7f076415e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f075bfff640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07640cb190 0x7f076415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f075bfff640 1 -- 172.21.15.16:0/372698299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07540094d0 con 0x7f07640716b0 2023-11-13T11:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f075bfff640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640716b0 0x7f07640c9fc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f074c00c7f0 tx=0x7f074c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f07617fa640 1 -- 172.21.15.16:0/372698299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c007a30 con 0x7f07640716b0 2023-11-13T11:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f07617fa640 1 -- 172.21.15.16:0/372698299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c007b90 con 0x7f07640716b0 2023-11-13T11:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f07617fa640 1 -- 172.21.15.16:0/372698299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c00ce40 con 0x7f07640716b0 2023-11-13T11:01:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0764152910 con 0x7f07640716b0 2023-11-13T11:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.146+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0764153ff0 con 0x7f07640716b0 2023-11-13T11:01:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.147+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0764153110 con 0x7f07640716b0 2023-11-13T11:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.148+0000 7f07617fa640 1 -- 172.21.15.16:0/372698299 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f074c0113f0 con 0x7f07640716b0 2023-11-13T11:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.149+0000 7f07617fa640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f074406ae70 0x7f074406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.150+0000 7f07617fa640 1 -- 172.21.15.16:0/372698299 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f074c089850 con 0x7f07640716b0 2023-11-13T11:01:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.150+0000 7f0763fff640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f074406ae70 0x7f074406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.151+0000 7f0763fff640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f074406ae70 0x7f074406d330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0754036000 tx=0x7f0754030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.154+0000 7f07617fa640 1 -- 172.21.15.16:0/372698299 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0764153110 con 0x7f07640716b0 2023-11-13T11:01:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.361+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0764153110 con 0x7f07640716b0 2023-11-13T11:01:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.362+0000 7f07617fa640 1 -- 172.21.15.16:0/372698299 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0764153110 con 0x7f07640716b0 2023-11-13T11:01:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T11:01:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f074406ae70 msgr2=0x7f074406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f074406ae70 0x7f074406d330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0754036000 tx=0x7f0754030000 comp rx=0 tx=0).stop 2023-11-13T11:01:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640716b0 msgr2=0x7f07640c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640716b0 0x7f07640c9fc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f074c00c7f0 tx=0x7f074c00cbf0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 shutdown_connections 2023-11-13T11:01:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f074406ae70 0x7f074406d330 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07640cb190 0x7f076415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07640716b0 0x7f07640c9fc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 --2- 172.21.15.16:0/372698299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f076406a190 0x7f0764157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 >> 172.21.15.16:0/372698299 conn(0x7f07640b04a0 msgr2=0x7f07640b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.365+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 shutdown_connections 2023-11-13T11:01:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:18.366+0000 7f076961a640 1 -- 172.21.15.16:0/372698299 wait complete. 2023-11-13T11:01:18.387 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-13T11:01:18.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:18.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-13T11:01:18.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[102488]: fuse finished with error 0 and tester_r 0 2023-11-13T11:01:18.510 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:01:18.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:18.511 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-13T11:01:18.712 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:01:18.712 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:01:18.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:01:18.773 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:01:18.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:18.774 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-13T11:01:18.826 INFO:teuthology.orchestra.run.smithi184.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-13T11:01:18.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:01:18.828 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:01:18.828 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:01:18.892 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:01:18.892 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:01:18.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:18.893 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-13T11:01:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-13T11:01:18.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:01:18.952 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:01:18.952 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:01:19.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:01:19.011 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:01:19.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:19.012 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-13T11:01:19.065 INFO:teuthology.orchestra.run.smithi184.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-13T11:01:19.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:01:19.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:01:19.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:01:19.124 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:01:19.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.293+0000 7efef9eda640 1 Processor -- start 2023-11-13T11:01:19.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.294+0000 7efef9eda640 1 -- start start 2023-11-13T11:01:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.295+0000 7efef9eda640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efef4155410 0x7efef40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.295+0000 7efef9eda640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efef40ce990 0x7efef40cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.295+0000 7efef9eda640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef415b500 con 0x7efef4071440 2023-11-13T11:01:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.295+0000 7efef9eda640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef415b7e0 con 0x7efef4155410 2023-11-13T11:01:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.295+0000 7efef9eda640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef415bac0 con 0x7efef40ce990 2023-11-13T11:01:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.295+0000 7efef37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efef4155410 0x7efef40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.296+0000 7efef37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efef4155410 0x7efef40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41970/0 (socket says 172.21.15.16:41970) 2023-11-13T11:01:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.296+0000 7efef37fe640 1 -- 172.21.15.16:0/337202 learned_addr learned my addr 172.21.15.16:0/337202 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.296+0000 7efef2ffd640 1 --2- 172.21.15.16:0/337202 >> v2:172.21.15.184:3300/0 conn(0x7efef40ce990 0x7efef40cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.296+0000 7efef37fe640 1 -- 172.21.15.16:0/337202 >> v2:172.21.15.184:3300/0 conn(0x7efef40ce990 msgr2=0x7efef40cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.296+0000 7efef37fe640 1 --2- 172.21.15.16:0/337202 >> v2:172.21.15.184:3300/0 conn(0x7efef40ce990 0x7efef40cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.296+0000 7efef37fe640 1 -- 172.21.15.16:0/337202 >> v1:172.21.15.184:6789/0 conn(0x7efef4071440 legacy=0x7efef406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.296+0000 7efef37fe640 1 -- 172.21.15.16:0/337202 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef415bda0 con 0x7efef4155410 2023-11-13T11:01:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.297+0000 7efef37fe640 1 --2- 172.21.15.16:0/337202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4155410 0x7efef40ccc60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efef406e000 tx=0x7efee402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b858e93acd0142 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.298+0000 7efef27fc640 1 -- 172.21.15.16:0/337202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4031870 con 0x7efef4155410 2023-11-13T11:01:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.298+0000 7efef27fc640 1 -- 172.21.15.16:0/337202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee40319d0 con 0x7efef4155410 2023-11-13T11:01:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.298+0000 7efef27fc640 1 -- 172.21.15.16:0/337202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4031d60 con 0x7efef4155410 2023-11-13T11:01:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.298+0000 7efef9eda640 1 -- 172.21.15.16:0/337202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4155410 msgr2=0x7efef40ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.298+0000 7efef9eda640 1 --2- 172.21.15.16:0/337202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4155410 0x7efef40ccc60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efef406e000 tx=0x7efee402b910 comp rx=0 tx=0).stop 2023-11-13T11:01:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.299+0000 7efef9eda640 1 -- 172.21.15.16:0/337202 shutdown_connections 2023-11-13T11:01:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.299+0000 7efef9eda640 1 --2- 172.21.15.16:0/337202 >> v2:172.21.15.184:3300/0 conn(0x7efef40ce990 0x7efef40cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.299+0000 7efef9eda640 1 --2- 172.21.15.16:0/337202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4155410 0x7efef40ccc60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.299+0000 7efef9eda640 1 -- 172.21.15.16:0/337202 >> 172.21.15.16:0/337202 conn(0x7efef406a190 msgr2=0x7efef406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.299+0000 7efef9eda640 1 -- 172.21.15.16:0/337202 shutdown_connections 2023-11-13T11:01:19.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.299+0000 7efef9eda640 1 -- 172.21.15.16:0/337202 wait complete. 2023-11-13T11:01:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.300+0000 7efef9eda640 1 Processor -- start 2023-11-13T11:01:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.301+0000 7efef9eda640 1 -- start start 2023-11-13T11:01:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef9eda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4071440 0x7efef41497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef9eda640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efef40ce990 0x7efef415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef9eda640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efef4155410 0x7efef40aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef9eda640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef406d710 con 0x7efef4071440 2023-11-13T11:01:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef9eda640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef406cb00 con 0x7efef4155410 2023-11-13T11:01:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef9eda640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef406cde0 con 0x7efef40ce990 2023-11-13T11:01:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4071440 0x7efef41497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4071440 0x7efef41497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41974/0 (socket says 172.21.15.16:41974) 2023-11-13T11:01:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef37fe640 1 -- 172.21.15.16:0/2107511332 learned_addr learned my addr 172.21.15.16:0/2107511332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef37fe640 1 -- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efef40ce990 msgr2=0x7efef415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef37fe640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efef40ce990 0x7efef415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef37fe640 1 -- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efef4155410 msgr2=0x7efef40aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef3fff640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efef4155410 0x7efef40aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef37fe640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efef4155410 0x7efef40aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.302+0000 7efef37fe640 1 -- 172.21.15.16:0/2107511332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee40094d0 con 0x7efef4071440 2023-11-13T11:01:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.303+0000 7efef3fff640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efef4155410 0x7efef40aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.303+0000 7efef37fe640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4071440 0x7efef41497e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efee4000c00 tx=0x7efee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.303+0000 7efeebfff640 1 -- 172.21.15.16:0/2107511332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee403abe0 con 0x7efef4071440 2023-11-13T11:01:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.303+0000 7efeebfff640 1 -- 172.21.15.16:0/2107511332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee403ad40 con 0x7efef4071440 2023-11-13T11:01:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.303+0000 7efeebfff640 1 -- 172.21.15.16:0/2107511332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4044610 con 0x7efef4071440 2023-11-13T11:01:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.303+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef406d0c0 con 0x7efef4071440 2023-11-13T11:01:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.303+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef415d510 con 0x7efef4071440 2023-11-13T11:01:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.305+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef406cf20 con 0x7efef4071440 2023-11-13T11:01:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.305+0000 7efeebfff640 1 -- 172.21.15.16:0/2107511332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efee404d450 con 0x7efef4071440 2023-11-13T11:01:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.307+0000 7efeebfff640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efed406ae40 0x7efed406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.307+0000 7efeebfff640 1 -- 172.21.15.16:0/2107511332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7efee402d040 con 0x7efef4071440 2023-11-13T11:01:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.308+0000 7efef2ffd640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efed406ae40 0x7efed406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.309+0000 7efef2ffd640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efed406ae40 0x7efed406d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efef41f6650 tx=0x7efedc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.311+0000 7efeebfff640 1 -- 172.21.15.16:0/2107511332 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efef406cf20 con 0x7efef4071440 2023-11-13T11:01:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.541+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efef406cf20 con 0x7efef4071440 2023-11-13T11:01:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.542+0000 7efeebfff640 1 -- 172.21.15.16:0/2107511332 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4919 (secure 0 0 0) 0x7efef406cf20 con 0x7efef4071440 2023-11-13T11:01:19.546 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:19.546 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3678615052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3678615052},{"type":"v1","addr":"172.21.15.16:6833","nonce":3678615052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3358463124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3358463124},{"type":"v1","addr":"172.21.15.16:6835","nonce":3358463124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2329286743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2329286743},{"type":"v1","addr":"172.21.15.184:6836","nonce":2329286743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-13T10:57:20.523743+0000","modified":"2023-11-13T10:57:32.253697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6837/3171288994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3171288994},{"type":"v1","addr":"172.21.15.184:6837","nonce":3171288994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-13T11:01:19.547 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 19 2023-11-13T11:01:19.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efed406ae40 msgr2=0x7efed406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:19.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efed406ae40 0x7efed406d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efef41f6650 tx=0x7efedc008040 comp rx=0 tx=0).stop 2023-11-13T11:01:19.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4071440 msgr2=0x7efef41497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:19.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4071440 0x7efef41497e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efee4000c00 tx=0x7efee40046c0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 shutdown_connections 2023-11-13T11:01:19.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efed406ae40 0x7efed406d300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efef4155410 0x7efef40aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efef40ce990 0x7efef415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 --2- 172.21.15.16:0/2107511332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef4071440 0x7efef41497e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.544+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 >> 172.21.15.16:0/2107511332 conn(0x7efef406a190 msgr2=0x7efef40ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:19.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.545+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 shutdown_connections 2023-11-13T11:01:19.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.545+0000 7efef9eda640 1 -- 172.21.15.16:0/2107511332 wait complete. 2023-11-13T11:01:19.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:01:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.791+0000 7efbfdc7c640 1 Processor -- start 2023-11-13T11:01:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.791+0000 7efbfdc7c640 1 -- start start 2023-11-13T11:01:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.792+0000 7efbfdc7c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efbf815d170 0x7efbf80d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.792+0000 7efbfdc7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efbf80d18b0 0x7efbf80b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.792+0000 7efbfdc7c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf80d0aa0 con 0x7efbf80b1d00 2023-11-13T11:01:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.792+0000 7efbfdc7c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf80b0080 con 0x7efbf80d18b0 2023-11-13T11:01:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.792+0000 7efbfdc7c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf80d0fd0 con 0x7efbf815d170 2023-11-13T11:01:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.793+0000 7efbf7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efbf80d18b0 0x7efbf80b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.793+0000 7efbf7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efbf80d18b0 0x7efbf80b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41978/0 (socket says 172.21.15.16:41978) 2023-11-13T11:01:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.793+0000 7efbf7fff640 1 -- 172.21.15.16:0/3651277729 learned_addr learned my addr 172.21.15.16:0/3651277729 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.793+0000 7efbf77fe640 1 --2- 172.21.15.16:0/3651277729 >> v2:172.21.15.184:3300/0 conn(0x7efbf815d170 0x7efbf80d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.793+0000 7efbf7fff640 1 -- 172.21.15.16:0/3651277729 >> v2:172.21.15.184:3300/0 conn(0x7efbf815d170 msgr2=0x7efbf80d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.793+0000 7efbf7fff640 1 --2- 172.21.15.16:0/3651277729 >> v2:172.21.15.184:3300/0 conn(0x7efbf815d170 0x7efbf80d37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.793+0000 7efbf7fff640 1 -- 172.21.15.16:0/3651277729 >> v1:172.21.15.16:6789/0 conn(0x7efbf80b1d00 legacy=0x7efbf80d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:19.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.793+0000 7efbf7fff640 1 -- 172.21.15.16:0/3651277729 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf80d0cf0 con 0x7efbf80d18b0 2023-11-13T11:01:19.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.794+0000 7efbf7fff640 1 --2- 172.21.15.16:0/3651277729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf80d18b0 0x7efbf80b04a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efbf80d2c20 tx=0x7efbf002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653e9fa44fcd868 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.795+0000 7efbf67fc640 1 -- 172.21.15.16:0/3651277729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0030860 con 0x7efbf80d18b0 2023-11-13T11:01:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.795+0000 7efbf67fc640 1 -- 172.21.15.16:0/3651277729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf00309c0 con 0x7efbf80d18b0 2023-11-13T11:01:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.795+0000 7efbf67fc640 1 -- 172.21.15.16:0/3651277729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0030d50 con 0x7efbf80d18b0 2023-11-13T11:01:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.795+0000 7efbfdc7c640 1 -- 172.21.15.16:0/3651277729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf80d18b0 msgr2=0x7efbf80b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.795+0000 7efbfdc7c640 1 --2- 172.21.15.16:0/3651277729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf80d18b0 0x7efbf80b04a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efbf80d2c20 tx=0x7efbf002b9a0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.796+0000 7efbfdc7c640 1 -- 172.21.15.16:0/3651277729 shutdown_connections 2023-11-13T11:01:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.796+0000 7efbfdc7c640 1 --2- 172.21.15.16:0/3651277729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf80d18b0 0x7efbf80b04a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.796+0000 7efbfdc7c640 1 --2- 172.21.15.16:0/3651277729 >> v2:172.21.15.184:3300/0 conn(0x7efbf815d170 0x7efbf80d37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.796+0000 7efbfdc7c640 1 -- 172.21.15.16:0/3651277729 >> 172.21.15.16:0/3651277729 conn(0x7efbf80d2cc0 msgr2=0x7efbf80d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.796+0000 7efbfdc7c640 1 -- 172.21.15.16:0/3651277729 shutdown_connections 2023-11-13T11:01:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.796+0000 7efbfdc7c640 1 -- 172.21.15.16:0/3651277729 wait complete. 2023-11-13T11:01:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.798+0000 7efbfdc7c640 1 Processor -- start 2023-11-13T11:01:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.798+0000 7efbfdc7c640 1 -- start start 2023-11-13T11:01:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbfdc7c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf80b1d00 0x7efbf8159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbfdc7c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf80d18b0 0x7efbf815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbfdc7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf815d170 0x7efbf8160260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbfdc7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80cb240 con 0x7efbf815d170 2023-11-13T11:01:19.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbfdc7c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80b79d0 con 0x7efbf80b1d00 2023-11-13T11:01:19.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbfdc7c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80caf10 con 0x7efbf80d18b0 2023-11-13T11:01:19.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf815d170 0x7efbf8160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf815d170 0x7efbf8160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41986/0 (socket says 172.21.15.16:41986) 2023-11-13T11:01:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbf7fff640 1 -- 172.21.15.16:0/2767120730 learned_addr learned my addr 172.21.15.16:0/2767120730 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.799+0000 7efbf77fe640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf80b1d00 0x7efbf8159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.800+0000 7efbf6ffd640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf80d18b0 0x7efbf815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.800+0000 7efbf7fff640 1 -- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf80d18b0 msgr2=0x7efbf815fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.800+0000 7efbf7fff640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf80d18b0 0x7efbf815fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.800+0000 7efbf7fff640 1 -- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf80b1d00 msgr2=0x7efbf8159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.800+0000 7efbf7fff640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf80b1d00 0x7efbf8159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.800+0000 7efbf7fff640 1 -- 172.21.15.16:0/2767120730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf00094d0 con 0x7efbf815d170 2023-11-13T11:01:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.800+0000 7efbf7fff640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf815d170 0x7efbf8160260 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efbf00090b0 tx=0x7efbf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.800+0000 7efbdffff640 1 -- 172.21.15.16:0/2767120730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0039c90 con 0x7efbf815d170 2023-11-13T11:01:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.800+0000 7efbdffff640 1 -- 172.21.15.16:0/2767120730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf0039df0 con 0x7efbf815d170 2023-11-13T11:01:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.801+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf8159a90 con 0x7efbf815d170 2023-11-13T11:01:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.801+0000 7efbdffff640 1 -- 172.21.15.16:0/2767120730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0043640 con 0x7efbf815d170 2023-11-13T11:01:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.801+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf80b76f0 con 0x7efbf815d170 2023-11-13T11:01:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.802+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf80cb3b0 con 0x7efbf815d170 2023-11-13T11:01:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.803+0000 7efbdffff640 1 -- 172.21.15.16:0/2767120730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efbf00437a0 con 0x7efbf815d170 2023-11-13T11:01:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.804+0000 7efbdffff640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbe406ae70 0x7efbe406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.804+0000 7efbdffff640 1 -- 172.21.15.16:0/2767120730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7efbf003e070 con 0x7efbf815d170 2023-11-13T11:01:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.804+0000 7efbf77fe640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbe406ae70 0x7efbe406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.805+0000 7efbf77fe640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbe406ae70 0x7efbe406d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efbe80099c0 tx=0x7efbe8003990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:19.808+0000 7efbdffff640 1 -- 172.21.15.16:0/2767120730 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efbf0077830 con 0x7efbf815d170 2023-11-13T11:01:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.010+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efbf80677a0 con 0x7efbf815d170 2023-11-13T11:01:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.012+0000 7efbdffff640 1 -- 172.21.15.16:0/2767120730 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+17026 (secure 0 0 0) 0x7efbf007ae50 con 0x7efbf815d170 2023-11-13T11:01:20.016 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:20.017 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:00:55.896214+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:57:18.637969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-13T10:57:19.541678+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":43,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.2","mappings":[{"from":4,"to":7}]},{"pgid":"6.14","mappings":[{"from":3,"to":5}]},{"pgid":"6.18","mappings":[{"from":4,"to":2}]},{"pgid":"6.38","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:01:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbe406ae70 msgr2=0x7efbe406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbe406ae70 0x7efbe406d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efbe80099c0 tx=0x7efbe8003990 comp rx=0 tx=0).stop 2023-11-13T11:01:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf815d170 msgr2=0x7efbf8160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf815d170 0x7efbf8160260 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efbf00090b0 tx=0x7efbf0004730 comp rx=0 tx=0).stop 2023-11-13T11:01:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 shutdown_connections 2023-11-13T11:01:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efbe406ae70 0x7efbe406d330 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf815d170 0x7efbf8160260 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf80d18b0 0x7efbf815fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 --2- 172.21.15.16:0/2767120730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf80b1d00 0x7efbf8159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.014+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 >> 172.21.15.16:0/2767120730 conn(0x7efbf80d2cc0 msgr2=0x7efbf80ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.015+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 shutdown_connections 2023-11-13T11:01:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.015+0000 7efbfdc7c640 1 -- 172.21.15.16:0/2767120730 wait complete. 2023-11-13T11:01:20.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T10:57:18.637969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T10:57:19.541678+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 43, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:01:20.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:01:20.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.265+0000 7f893791d640 1 Processor -- start 2023-11-13T11:01:20.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.266+0000 7f893791d640 1 -- start start 2023-11-13T11:01:20.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.267+0000 7f893791d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f89300cf090 0x7f89301622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.268+0000 7f893791d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8930144660 con 0x7f89300b20e0 2023-11-13T11:01:20.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.268+0000 7f893791d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8930161e60 con 0x7f8930164dd0 2023-11-13T11:01:20.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.268+0000 7f893791d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89300cf470 con 0x7f89300cf090 2023-11-13T11:01:20.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.268+0000 7f8935eba640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f89300b20e0 0x7f89301469a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36380/0 (socket says 172.21.15.16:36380) 2023-11-13T11:01:20.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.268+0000 7f8935eba640 1 -- 172.21.15.16:0/2015288765 learned_addr learned my addr 172.21.15.16:0/2015288765 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:20.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.268+0000 7f89356b9640 1 --2- 172.21.15.16:0/2015288765 >> v2:172.21.15.184:3300/0 conn(0x7f89300cf090 0x7f89301622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.268+0000 7f8927fff640 1 -- 172.21.15.16:0/2015288765 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206295778 0 0) 0x7f8930144660 con 0x7f89300b20e0 2023-11-13T11:01:20.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.269+0000 7f8927fff640 1 -- 172.21.15.16:0/2015288765 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f891c003520 con 0x7f89300b20e0 2023-11-13T11:01:20.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.269+0000 7f89356b9640 1 -- 172.21.15.16:0/2015288765 >> v1:172.21.15.184:6789/0 conn(0x7f8930164dd0 legacy=0x7f89300b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.269+0000 7f89356b9640 1 -- 172.21.15.16:0/2015288765 >> v1:172.21.15.16:6789/0 conn(0x7f89300b20e0 legacy=0x7f89301469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.269+0000 7f8927fff640 1 -- 172.21.15.16:0/2015288765 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2070262122 0 0) 0x7f891c003520 con 0x7f89300b20e0 2023-11-13T11:01:20.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.269+0000 7f89356b9640 1 -- 172.21.15.16:0/2015288765 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8930165870 con 0x7f89300cf090 2023-11-13T11:01:20.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.270+0000 7f89356b9640 1 --2- 172.21.15.16:0/2015288765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89300cf090 0x7f89301622f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f892c009d90 tx=0x7f892c02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41678724e21607bc server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:20.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.271+0000 7f8927fff640 1 -- 172.21.15.16:0/2015288765 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928001670 con 0x7f89300cf090 2023-11-13T11:01:20.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.271+0000 7f8927fff640 1 -- 172.21.15.16:0/2015288765 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c02c8b0 con 0x7f89300cf090 2023-11-13T11:01:20.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.271+0000 7f8927fff640 1 -- 172.21.15.16:0/2015288765 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c02cc20 con 0x7f89300cf090 2023-11-13T11:01:20.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.272+0000 7f893791d640 1 -- 172.21.15.16:0/2015288765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89300cf090 msgr2=0x7f89301622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.272+0000 7f893791d640 1 --2- 172.21.15.16:0/2015288765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89300cf090 0x7f89301622f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f892c009d90 tx=0x7f892c02bd80 comp rx=0 tx=0).stop 2023-11-13T11:01:20.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.272+0000 7f893791d640 1 -- 172.21.15.16:0/2015288765 shutdown_connections 2023-11-13T11:01:20.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.272+0000 7f893791d640 1 --2- 172.21.15.16:0/2015288765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89300cf090 0x7f89301622f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.272+0000 7f893791d640 1 -- 172.21.15.16:0/2015288765 >> 172.21.15.16:0/2015288765 conn(0x7f893006a190 msgr2=0x7f893006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:20.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.272+0000 7f893791d640 1 -- 172.21.15.16:0/2015288765 shutdown_connections 2023-11-13T11:01:20.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.272+0000 7f893791d640 1 -- 172.21.15.16:0/2015288765 wait complete. 2023-11-13T11:01:20.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.274+0000 7f893791d640 1 Processor -- start 2023-11-13T11:01:20.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.274+0000 7f893791d640 1 -- start start 2023-11-13T11:01:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f893791d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89300b20e0 0x7f8930157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f893791d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89300cf090 0x7f89301ebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f893791d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8930164dd0 0x7f89301ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f893791d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89301ec820 con 0x7f89300cf090 2023-11-13T11:01:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f893791d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89301ecb00 con 0x7f8930164dd0 2023-11-13T11:01:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f893791d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89301ecde0 con 0x7f89300b20e0 2023-11-13T11:01:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f8934eb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89300cf090 0x7f89301ebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f8934eb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89300cf090 0x7f89301ebcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42002/0 (socket says 172.21.15.16:42002) 2023-11-13T11:01:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f8934eb8640 1 -- 172.21.15.16:0/557864217 learned_addr learned my addr 172.21.15.16:0/557864217 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.275+0000 7f8935eba640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8930164dd0 0x7f89301ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f89356b9640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89300b20e0 0x7f8930157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f8934eb8640 1 -- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89300b20e0 msgr2=0x7f8930157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f8934eb8640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89300b20e0 0x7f8930157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f8934eb8640 1 -- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8930164dd0 msgr2=0x7f89301ec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f8934eb8640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8930164dd0 0x7f89301ec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f8934eb8640 1 -- 172.21.15.16:0/557864217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c009a70 con 0x7f89300cf090 2023-11-13T11:01:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f89356b9640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89300b20e0 0x7f8930157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f8934eb8640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89300cf090 0x7f89301ebcb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f892000c400 tx=0x7f892000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f8925ffb640 1 -- 172.21.15.16:0/557864217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89200052d0 con 0x7f89300cf090 2023-11-13T11:01:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f8925ffb640 1 -- 172.21.15.16:0/557864217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8920005450 con 0x7f89300cf090 2023-11-13T11:01:20.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f8925ffb640 1 -- 172.21.15.16:0/557864217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920007260 con 0x7f89300cf090 2023-11-13T11:01:20.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89301ed0c0 con 0x7f89300cf090 2023-11-13T11:01:20.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.276+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8930157e80 con 0x7f89300cf090 2023-11-13T11:01:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.278+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89301ed300 con 0x7f89300cf090 2023-11-13T11:01:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.279+0000 7f8925ffb640 1 -- 172.21.15.16:0/557864217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89200073c0 con 0x7f89300cf090 2023-11-13T11:01:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.279+0000 7f8925ffb640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f891406ae40 0x7f891406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.280+0000 7f8925ffb640 1 -- 172.21.15.16:0/557864217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f8920088c40 con 0x7f89300cf090 2023-11-13T11:01:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.280+0000 7f89356b9640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f891406ae40 0x7f891406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.281+0000 7f89356b9640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f891406ae40 0x7f891406d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f892c036000 tx=0x7f892c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:20.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.284+0000 7f8925ffb640 1 -- 172.21.15.16:0/557864217 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8920053af0 con 0x7f89300cf090 2023-11-13T11:01:20.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.477+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8930072520 con 0x7f89300cf090 2023-11-13T11:01:20.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.478+0000 7f8925ffb640 1 -- 172.21.15.16:0/557864217 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8920057110 con 0x7f89300cf090 2023-11-13T11:01:20.482 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:01:20.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f891406ae40 msgr2=0x7f891406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f891406ae40 0x7f891406d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f892c036000 tx=0x7f892c030000 comp rx=0 tx=0).stop 2023-11-13T11:01:20.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89300cf090 msgr2=0x7f89301ebcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89300cf090 0x7f89301ebcb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f892000c400 tx=0x7f892000c8c0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 shutdown_connections 2023-11-13T11:01:20.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f891406ae40 0x7f891406d300 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8930164dd0 0x7f89301ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89300cf090 0x7f89301ebcb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 --2- 172.21.15.16:0/557864217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89300b20e0 0x7f8930157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 >> 172.21.15.16:0/557864217 conn(0x7f893006a190 msgr2=0x7f893015fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 shutdown_connections 2023-11-13T11:01:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.481+0000 7f893791d640 1 -- 172.21.15.16:0/557864217 wait complete. 2023-11-13T11:01:20.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T11:01:20.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.731+0000 7f2467085640 1 Processor -- start 2023-11-13T11:01:20.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.731+0000 7f2467085640 1 -- start start 2023-11-13T11:01:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.733+0000 7f2467085640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f246015b5c0 0x7f2460164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.733+0000 7f2467085640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2460165b40 con 0x7f24600b2b40 2023-11-13T11:01:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.733+0000 7f2467085640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24601652a0 con 0x7f2460161640 2023-11-13T11:01:20.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.733+0000 7f2467085640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2460165580 con 0x7f246015b5c0 2023-11-13T11:01:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.733+0000 7f2466884640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24600b2b40 0x7f24600ad520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36388/0 (socket says 172.21.15.16:36388) 2023-11-13T11:01:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.733+0000 7f2466884640 1 -- 172.21.15.16:0/530895680 learned_addr learned my addr 172.21.15.16:0/530895680 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2466083640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f246015b5c0 0x7f2460164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914414748 0 0) 0x7f2460165b40 con 0x7f24600b2b40 2023-11-13T11:01:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c003520 con 0x7f24600b2b40 2023-11-13T11:01:20.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1147013973 0 0) 0x7f244c003520 con 0x7f24600b2b40 2023-11-13T11:01:20.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f246015b5c0 msgr2=0x7f2460164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:20.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f246015b5c0 0x7f2460164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 >> v1:172.21.15.184:6789/0 conn(0x7f2460161640 legacy=0x7f24600b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2460165860 con 0x7f24600b2b40 2023-11-13T11:01:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f24580019f0 con 0x7f24600b2b40 2023-11-13T11:01:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 >> v1:172.21.15.16:6789/0 conn(0x7f24600b2b40 legacy=0x7f24600ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.734+0000 7f2465081640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f244c0039a0 0x7f244c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2465081640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f246015b5c0 0x7f244c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2465081640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c004d00 0x7f244c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2460165580 con 0x7f244c004d00 2023-11-13T11:01:20.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c00ba60 con 0x7f244c0039a0 2023-11-13T11:01:20.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c00bc20 con 0x7f246015b5c0 2023-11-13T11:01:20.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2466884640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c004d00 0x7f244c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2466083640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f244c0039a0 0x7f244c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2465882640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f246015b5c0 0x7f244c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2466884640 1 -- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f246015b5c0 msgr2=0x7f244c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2466884640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f246015b5c0 0x7f244c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2466884640 1 -- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f244c0039a0 msgr2=0x7f244c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2466884640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f244c0039a0 0x7f244c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.735+0000 7f2466884640 1 -- 172.21.15.16:0/530895680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c003520 con 0x7f244c004d00 2023-11-13T11:01:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.736+0000 7f2466884640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c004d00 0x7f244c005180 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2458002230 tx=0x7f244c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efbb608e2eaf3b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.737+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458034630 con 0x7f244c004d00 2023-11-13T11:01:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.737+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245802f070 con 0x7f244c004d00 2023-11-13T11:01:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.737+0000 7f2465081640 1 -- 172.21.15.16:0/530895680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458034a10 con 0x7f244c004d00 2023-11-13T11:01:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.737+0000 7f2467085640 1 -- 172.21.15.16:0/530895680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c004d00 msgr2=0x7f244c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.737+0000 7f2467085640 1 --2- 172.21.15.16:0/530895680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c004d00 0x7f244c005180 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2458002230 tx=0x7f244c0057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.737+0000 7f2466884640 1 -- 172.21.15.16:0/530895680 reap_dead start 2023-11-13T11:01:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.737+0000 7f2467085640 1 -- 172.21.15.16:0/530895680 shutdown_connections 2023-11-13T11:01:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.737+0000 7f2467085640 1 -- 172.21.15.16:0/530895680 >> 172.21.15.16:0/530895680 conn(0x7f24600b0970 msgr2=0x7f24600b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.738+0000 7f2467085640 1 -- 172.21.15.16:0/530895680 shutdown_connections 2023-11-13T11:01:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.738+0000 7f2467085640 1 -- 172.21.15.16:0/530895680 wait complete. 2023-11-13T11:01:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.739+0000 7f2467085640 1 Processor -- start 2023-11-13T11:01:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.740+0000 7f2467085640 1 -- start start 2023-11-13T11:01:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2467085640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f244c004d00 0x7f2460160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2467085640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f246015dc00 0x7f24601f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2467085640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2460158350 0x7f24601587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2467085640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2460158cf0 con 0x7f2460158350 2023-11-13T11:01:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2467085640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246015f020 con 0x7f246015dc00 2023-11-13T11:01:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2467085640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246015e410 con 0x7f244c004d00 2023-11-13T11:01:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2466884640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2460158350 0x7f24601587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2466884640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2460158350 0x7f24601587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42022/0 (socket says 172.21.15.16:42022) 2023-11-13T11:01:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2466884640 1 -- 172.21.15.16:0/2301104517 learned_addr learned my addr 172.21.15.16:0/2301104517 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2465882640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f246015dc00 0x7f24601f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.741+0000 7f2466083640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f244c004d00 0x7f2460160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2466884640 1 -- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f244c004d00 msgr2=0x7f2460160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2466884640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f244c004d00 0x7f2460160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2466884640 1 -- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f246015dc00 msgr2=0x7f24601f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2466884640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f246015dc00 0x7f24601f9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2466884640 1 -- 172.21.15.16:0/2301104517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246015e6f0 con 0x7f2460158350 2023-11-13T11:01:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2466884640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2460158350 0x7f24601587b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2458002230 tx=0x7f2458001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2456ffd640 1 -- 172.21.15.16:0/2301104517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458034840 con 0x7f2460158350 2023-11-13T11:01:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2456ffd640 1 -- 172.21.15.16:0/2301104517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24580349a0 con 0x7f2460158350 2023-11-13T11:01:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2456ffd640 1 -- 172.21.15.16:0/2301104517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458034d30 con 0x7f2460158350 2023-11-13T11:01:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.742+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246015e9d0 con 0x7f2460158350 2023-11-13T11:01:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.743+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246015e080 con 0x7f2460158350 2023-11-13T11:01:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.744+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246004e4d0 con 0x7f2460158350 2023-11-13T11:01:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.745+0000 7f2456ffd640 1 -- 172.21.15.16:0/2301104517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2458031030 con 0x7f2460158350 2023-11-13T11:01:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.746+0000 7f2456ffd640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f244406ae40 0x7f244406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.746+0000 7f2456ffd640 1 -- 172.21.15.16:0/2301104517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f24580abbb0 con 0x7f2460158350 2023-11-13T11:01:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.746+0000 7f2466083640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f244406ae40 0x7f244406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.747+0000 7f2466083640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f244406ae40 0x7f244406d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f246015ab80 tx=0x7f245c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.750+0000 7f2456ffd640 1 -- 172.21.15.16:0/2301104517 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2458076b90 con 0x7f2460158350 2023-11-13T11:01:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.951+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f246004e4d0 con 0x7f2460158350 2023-11-13T11:01:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.951+0000 7f2456ffd640 1 -- 172.21.15.16:0/2301104517 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7f245807a1b0 con 0x7f2460158350 2023-11-13T11:01:20.956 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T11:01:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.954+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f244406ae40 msgr2=0x7f244406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.954+0000 7f2467085640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f244406ae40 0x7f244406d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f246015ab80 tx=0x7f245c008040 comp rx=0 tx=0).stop 2023-11-13T11:01:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.954+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2460158350 msgr2=0x7f24601587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.954+0000 7f2467085640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2460158350 0x7f24601587b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2458002230 tx=0x7f2458001cb0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.955+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 shutdown_connections 2023-11-13T11:01:20.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.955+0000 7f2467085640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f244406ae40 0x7f244406d300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.955+0000 7f2467085640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2460158350 0x7f24601587b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.955+0000 7f2467085640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f246015dc00 0x7f24601f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.955+0000 7f2467085640 1 --2- 172.21.15.16:0/2301104517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f244c004d00 0x7f2460160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:20.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.955+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 >> 172.21.15.16:0/2301104517 conn(0x7f24600b0970 msgr2=0x7f2460159750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:20.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.955+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 shutdown_connections 2023-11-13T11:01:20.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:20.955+0000 7f2467085640 1 -- 172.21.15.16:0/2301104517 wait complete. 2023-11-13T11:01:20.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T11:01:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.199+0000 7f588c1c9640 1 Processor -- start 2023-11-13T11:01:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.200+0000 7f588c1c9640 1 -- start start 2023-11-13T11:01:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.201+0000 7f588c1c9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58840b30b0 0x7f58840d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.201+0000 7f588c1c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840ceca0 0x7f58840d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.201+0000 7f588c1c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58840b41b0 con 0x7f58840b3490 2023-11-13T11:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.201+0000 7f588c1c9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58840d16d0 con 0x7f58840b30b0 2023-11-13T11:01:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.201+0000 7f588c1c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58840d19b0 con 0x7f58840ceca0 2023-11-13T11:01:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.202+0000 7f588a766640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58840b3490 0x7f58840cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36396/0 (socket says 172.21.15.16:36396) 2023-11-13T11:01:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.202+0000 7f588a766640 1 -- 172.21.15.16:0/2034694162 learned_addr learned my addr 172.21.15.16:0/2034694162 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:21.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.202+0000 7f5889f65640 1 --2- 172.21.15.16:0/2034694162 >> v2:172.21.15.184:3300/0 conn(0x7f58840b30b0 0x7f58840d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.202+0000 7f5889764640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840ceca0 0x7f58840d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.202+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290158449 0 0) 0x7f58840b41b0 con 0x7f58840b3490 2023-11-13T11:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.202+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5870003520 con 0x7f58840b3490 2023-11-13T11:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2146366593 0 0) 0x7f5870003520 con 0x7f58840b3490 2023-11-13T11:01:21.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840ceca0 msgr2=0x7f58840d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840ceca0 0x7f58840d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 >> v2:172.21.15.184:3300/0 conn(0x7f58840b30b0 msgr2=0x7f58840d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 --2- 172.21.15.16:0/2034694162 >> v2:172.21.15.184:3300/0 conn(0x7f58840b30b0 0x7f58840d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840d1c90 con 0x7f58840b3490 2023-11-13T11:01:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f587c001670 con 0x7f58840b3490 2023-11-13T11:01:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 >> v1:172.21.15.16:6789/0 conn(0x7f58840b3490 legacy=0x7f58840cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f5870003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840ceca0 0x7f58700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5870004d00 0x7f5870005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840d16d0 con 0x7f5870003970 2023-11-13T11:01:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840d19b0 con 0x7f5870004d00 2023-11-13T11:01:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587000b980 con 0x7f58840ceca0 2023-11-13T11:01:21.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5889f65640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f5870003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5889764640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840ceca0 0x7f58700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5889f65640 1 -- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840ceca0 msgr2=0x7f58700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:21.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5889f65640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840ceca0 0x7f58700047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f588a766640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5870004d00 0x7f5870005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.203+0000 7f5889f65640 1 -- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5870004d00 msgr2=0x7f5870005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:21.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.204+0000 7f5889f65640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5870004d00 0x7f5870005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.204+0000 7f5889f65640 1 -- 172.21.15.16:0/2034694162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5870003520 con 0x7f5870003970 2023-11-13T11:01:21.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.204+0000 7f588a766640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5870004d00 0x7f5870005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:21.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.205+0000 7f5889f65640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f5870003d90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5880009fc0 tx=0x7f58700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b840c4b869f12b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:21.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.205+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588002cc60 con 0x7f5870003970 2023-11-13T11:01:21.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.205+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588000f040 con 0x7f5870003970 2023-11-13T11:01:21.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.205+0000 7f5888f63640 1 -- 172.21.15.16:0/2034694162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880034440 con 0x7f5870003970 2023-11-13T11:01:21.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.205+0000 7f588c1c9640 1 -- 172.21.15.16:0/2034694162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 msgr2=0x7f5870003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.205+0000 7f588c1c9640 1 --2- 172.21.15.16:0/2034694162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f5870003d90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5880009fc0 tx=0x7f58700057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.205+0000 7f588a766640 1 -- 172.21.15.16:0/2034694162 reap_dead start 2023-11-13T11:01:21.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.206+0000 7f588c1c9640 1 -- 172.21.15.16:0/2034694162 shutdown_connections 2023-11-13T11:01:21.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.206+0000 7f588c1c9640 1 -- 172.21.15.16:0/2034694162 >> 172.21.15.16:0/2034694162 conn(0x7f58840d2d20 msgr2=0x7f58840d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:21.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.206+0000 7f588c1c9640 1 -- 172.21.15.16:0/2034694162 shutdown_connections 2023-11-13T11:01:21.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.206+0000 7f588c1c9640 1 -- 172.21.15.16:0/2034694162 wait complete. 2023-11-13T11:01:21.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.207+0000 7f588c1c9640 1 Processor -- start 2023-11-13T11:01:21.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.208+0000 7f588c1c9640 1 -- start start 2023-11-13T11:01:21.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f588c1c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f588415bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f588c1c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588415c300 0x7f58840a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f588c1c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840c78e0 0x7f58840c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f588c1c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840b5d50 con 0x7f5870003970 2023-11-13T11:01:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f588c1c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840c82a0 con 0x7f588415c300 2023-11-13T11:01:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f588c1c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588415c110 con 0x7f58840c78e0 2023-11-13T11:01:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f5889f65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f588415bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f5889f65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f588415bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42034/0 (socket says 172.21.15.16:42034) 2023-11-13T11:01:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f5889f65640 1 -- 172.21.15.16:0/671585148 learned_addr learned my addr 172.21.15.16:0/671585148 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f5889f65640 1 -- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840c78e0 msgr2=0x7f58840c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f5889f65640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840c78e0 0x7f58840c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f5889f65640 1 -- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588415c300 msgr2=0x7f58840a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f5889f65640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588415c300 0x7f58840a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.209+0000 7f5889f65640 1 -- 172.21.15.16:0/671585148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840b59b0 con 0x7f5870003970 2023-11-13T11:01:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.210+0000 7f5889f65640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f588415bbd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f588002c6d0 tx=0x7f5880009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.210+0000 7f587a7fc640 1 -- 172.21.15.16:0/671585148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880034c50 con 0x7f5870003970 2023-11-13T11:01:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.210+0000 7f587a7fc640 1 -- 172.21.15.16:0/671585148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5880034db0 con 0x7f5870003970 2023-11-13T11:01:21.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.210+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588415db40 con 0x7f5870003970 2023-11-13T11:01:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.210+0000 7f587a7fc640 1 -- 172.21.15.16:0/671585148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880003b90 con 0x7f5870003970 2023-11-13T11:01:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.210+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58840a0960 con 0x7f5870003970 2023-11-13T11:01:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.211+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58840a0cd0 con 0x7f5870003970 2023-11-13T11:01:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.212+0000 7f587a7fc640 1 -- 172.21.15.16:0/671585148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5880040450 con 0x7f5870003970 2023-11-13T11:01:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.213+0000 7f587a7fc640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f586806aec0 0x7f586806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.213+0000 7f587a7fc640 1 -- 172.21.15.16:0/671585148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f58800abf30 con 0x7f5870003970 2023-11-13T11:01:21.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.214+0000 7f5889764640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f586806aec0 0x7f586806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.215+0000 7f5889764640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f586806aec0 0x7f586806d380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f588415c800 tx=0x7f5874008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.218+0000 7f587a7fc640 1 -- 172.21.15.16:0/671585148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5880076e60 con 0x7f5870003970 2023-11-13T11:01:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.410+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f588404e5b0 con 0x7f5870003970 2023-11-13T11:01:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.411+0000 7f587a7fc640 1 -- 172.21.15.16:0/671585148 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7f588007a480 con 0x7f5870003970 2023-11-13T11:01:21.415 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T11:01:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f586806aec0 msgr2=0x7f586806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f586806aec0 0x7f586806d380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f588415c800 tx=0x7f5874008040 comp rx=0 tx=0).stop 2023-11-13T11:01:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 msgr2=0x7f588415bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f588415bbd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f588002c6d0 tx=0x7f5880009750 comp rx=0 tx=0).stop 2023-11-13T11:01:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 shutdown_connections 2023-11-13T11:01:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f586806aec0 0x7f586806d380 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58840c78e0 0x7f58840c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588415c300 0x7f58840a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 --2- 172.21.15.16:0/671585148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5870003970 0x7f588415bbd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.413+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 >> 172.21.15.16:0/671585148 conn(0x7f58840d2d20 msgr2=0x7f588406b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.414+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 shutdown_connections 2023-11-13T11:01:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.414+0000 7f588c1c9640 1 -- 172.21.15.16:0/671585148 wait complete. 2023-11-13T11:01:21.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:01:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.651+0000 7f59ecfbf640 1 Processor -- start 2023-11-13T11:01:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.651+0000 7f59ecfbf640 1 -- start start 2023-11-13T11:01:21.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59ecfbf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59e80cb190 0x7f59e815b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59ecfbf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f59e80716b0 0x7f59e806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59ecfbf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e8158c20 con 0x7f59e806a190 2023-11-13T11:01:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59ecfbf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e80b1830 con 0x7f59e80cb190 2023-11-13T11:01:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59ecfbf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e815b780 con 0x7f59e80716b0 2023-11-13T11:01:21.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59e77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59e80cb190 0x7f59e815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59e77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59e80cb190 0x7f59e815b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42036/0 (socket says 172.21.15.16:42036) 2023-11-13T11:01:21.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59e77fe640 1 -- 172.21.15.16:0/1052891810 learned_addr learned my addr 172.21.15.16:0/1052891810 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:21.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59e7fff640 1 --1- 172.21.15.16:0/1052891810 >> v1:172.21.15.16:6789/0 conn(0x7f59e806a190 0x7f59e80b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36412/0 (socket says 172.21.15.16:36412) 2023-11-13T11:01:21.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.653+0000 7f59e6ffd640 1 --2- 172.21.15.16:0/1052891810 >> v2:172.21.15.184:3300/0 conn(0x7f59e80716b0 0x7f59e806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.654+0000 7f59e77fe640 1 -- 172.21.15.16:0/1052891810 >> v2:172.21.15.184:3300/0 conn(0x7f59e80716b0 msgr2=0x7f59e806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:21.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.654+0000 7f59e77fe640 1 --2- 172.21.15.16:0/1052891810 >> v2:172.21.15.184:3300/0 conn(0x7f59e80716b0 0x7f59e806f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.654+0000 7f59e77fe640 1 -- 172.21.15.16:0/1052891810 >> v1:172.21.15.16:6789/0 conn(0x7f59e806a190 legacy=0x7f59e80b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.654+0000 7f59e67fc640 1 -- 172.21.15.16:0/1052891810 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453533636 0 0) 0x7f59e8158c20 con 0x7f59e806a190 2023-11-13T11:01:21.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.654+0000 7f59e77fe640 1 -- 172.21.15.16:0/1052891810 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e815ba60 con 0x7f59e80cb190 2023-11-13T11:01:21.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.655+0000 7f59e77fe640 1 --2- 172.21.15.16:0/1052891810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 0x7f59e815b190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f59e80ae3e0 tx=0x7f59d802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c138aa1e5971dd5b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:21.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.655+0000 7f59e67fc640 1 -- 172.21.15.16:0/1052891810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8030860 con 0x7f59e80cb190 2023-11-13T11:01:21.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.655+0000 7f59e67fc640 1 -- 172.21.15.16:0/1052891810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d80309c0 con 0x7f59e80cb190 2023-11-13T11:01:21.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.655+0000 7f59e67fc640 1 -- 172.21.15.16:0/1052891810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8030d50 con 0x7f59e80cb190 2023-11-13T11:01:21.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.656+0000 7f59ecfbf640 1 -- 172.21.15.16:0/1052891810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 msgr2=0x7f59e815b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.656+0000 7f59ecfbf640 1 --2- 172.21.15.16:0/1052891810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 0x7f59e815b190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f59e80ae3e0 tx=0x7f59d802b9a0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.656+0000 7f59ecfbf640 1 -- 172.21.15.16:0/1052891810 shutdown_connections 2023-11-13T11:01:21.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.656+0000 7f59ecfbf640 1 --2- 172.21.15.16:0/1052891810 >> v2:172.21.15.184:3300/0 conn(0x7f59e80716b0 0x7f59e806f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.656+0000 7f59ecfbf640 1 --2- 172.21.15.16:0/1052891810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 0x7f59e815b190 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.656+0000 7f59ecfbf640 1 -- 172.21.15.16:0/1052891810 >> 172.21.15.16:0/1052891810 conn(0x7f59e80b04a0 msgr2=0x7f59e80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:21.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.656+0000 7f59ecfbf640 1 -- 172.21.15.16:0/1052891810 shutdown_connections 2023-11-13T11:01:21.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.656+0000 7f59ecfbf640 1 -- 172.21.15.16:0/1052891810 wait complete. 2023-11-13T11:01:21.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.657+0000 7f59ecfbf640 1 Processor -- start 2023-11-13T11:01:21.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.658+0000 7f59ecfbf640 1 -- start start 2023-11-13T11:01:21.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.658+0000 7f59ecfbf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59e806a190 0x7f59e814d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.658+0000 7f59ecfbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59e80716b0 0x7f59e80cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.658+0000 7f59ecfbf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 0x7f59e80ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.658+0000 7f59ecfbf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e80ca200 con 0x7f59e80cb190 2023-11-13T11:01:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.658+0000 7f59ecfbf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e80c8c10 con 0x7f59e80716b0 2023-11-13T11:01:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.658+0000 7f59ecfbf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e80c8eb0 con 0x7f59e806a190 2023-11-13T11:01:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 0x7f59e80ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 0x7f59e80ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42044/0 (socket says 172.21.15.16:42044) 2023-11-13T11:01:21.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e7fff640 1 -- 172.21.15.16:0/3861024792 learned_addr learned my addr 172.21.15.16:0/3861024792 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:21.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e6ffd640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59e80716b0 0x7f59e80cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e77fe640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59e806a190 0x7f59e814d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e7fff640 1 -- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59e806a190 msgr2=0x7f59e814d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e7fff640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59e806a190 0x7f59e814d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e7fff640 1 -- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59e80716b0 msgr2=0x7f59e80cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e7fff640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59e80716b0 0x7f59e80cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e7fff640 1 -- 172.21.15.16:0/3861024792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d80094d0 con 0x7f59e80cb190 2023-11-13T11:01:21.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.659+0000 7f59e6ffd640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59e80716b0 0x7f59e80cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:21.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.660+0000 7f59e7fff640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 0x7f59e80ccfc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f59e000c780 tx=0x7f59e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:21.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.660+0000 7f59cffff640 1 -- 172.21.15.16:0/3861024792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e000ce90 con 0x7f59e80cb190 2023-11-13T11:01:21.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.660+0000 7f59cffff640 1 -- 172.21.15.16:0/3861024792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e000a640 con 0x7f59e80cb190 2023-11-13T11:01:21.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.660+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e80c9190 con 0x7f59e80cb190 2023-11-13T11:01:21.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.660+0000 7f59cffff640 1 -- 172.21.15.16:0/3861024792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e000a9d0 con 0x7f59e80cb190 2023-11-13T11:01:21.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.660+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e815e210 con 0x7f59e80cb190 2023-11-13T11:01:21.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.661+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e80c96b0 con 0x7f59e80cb190 2023-11-13T11:01:21.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.662+0000 7f59cffff640 1 -- 172.21.15.16:0/3861024792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f59e000ab30 con 0x7f59e80cb190 2023-11-13T11:01:21.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.663+0000 7f59cffff640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f59d406aec0 0x7f59d406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:21.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.663+0000 7f59cffff640 1 -- 172.21.15.16:0/3861024792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f59e0089070 con 0x7f59e80cb190 2023-11-13T11:01:21.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.664+0000 7f59e77fe640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f59d406aec0 0x7f59d406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:21.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.665+0000 7f59e77fe640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f59d406aec0 0x7f59d406d380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f59d8036000 tx=0x7f59d803b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:21.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.667+0000 7f59cffff640 1 -- 172.21.15.16:0/3861024792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59e00bbe90 con 0x7f59e80cb190 2023-11-13T11:01:21.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.892+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59e804e4d0 con 0x7f59e80cb190 2023-11-13T11:01:21.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.893+0000 7f59cffff640 1 -- 172.21.15.16:0/3861024792 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f59e0018070 con 0x7f59e80cb190 2023-11-13T11:01:21.897 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:21.897 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3678615052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3678615052},{"type":"v1","addr":"172.21.15.16:6833","nonce":3678615052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3358463124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3358463124},{"type":"v1","addr":"172.21.15.16:6835","nonce":3358463124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2329286743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2329286743},{"type":"v1","addr":"172.21.15.184:6836","nonce":2329286743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-13T10:57:20.523743+0000","modified":"2023-11-13T10:57:32.253697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6837/3171288994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3171288994},{"type":"v1","addr":"172.21.15.184:6837","nonce":3171288994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-13T11:01:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 19 2023-11-13T11:01:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f59d406aec0 msgr2=0x7f59d406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f59d406aec0 0x7f59d406d380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f59d8036000 tx=0x7f59d803b040 comp rx=0 tx=0).stop 2023-11-13T11:01:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 msgr2=0x7f59e80ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 0x7f59e80ccfc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f59e000c780 tx=0x7f59e000cc40 comp rx=0 tx=0).stop 2023-11-13T11:01:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 shutdown_connections 2023-11-13T11:01:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f59d406aec0 0x7f59d406d380 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59e80cb190 0x7f59e80ccfc0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59e80716b0 0x7f59e80cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 --2- 172.21.15.16:0/3861024792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59e806a190 0x7f59e814d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.895+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 >> 172.21.15.16:0/3861024792 conn(0x7f59e80b04a0 msgr2=0x7f59e81574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.896+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 shutdown_connections 2023-11-13T11:01:21.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:21.896+0000 7f59ecfbf640 1 -- 172.21.15.16:0/3861024792 wait complete. 2023-11-13T11:01:21.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:01:22.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.143+0000 7f173f59e640 1 Processor -- start 2023-11-13T11:01:22.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.143+0000 7f173f59e640 1 -- start start 2023-11-13T11:01:22.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.145+0000 7f173f59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f17400cb190 0x7f174015b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.145+0000 7f173f59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1740158c20 con 0x7f17400716b0 2023-11-13T11:01:22.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.145+0000 7f173f59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17400b1830 con 0x7f174006a190 2023-11-13T11:01:22.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.145+0000 7f173f59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174015b780 con 0x7f17400cb190 2023-11-13T11:01:22.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.145+0000 7f1735d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f17400716b0 0x7f174006f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36418/0 (socket says 172.21.15.16:36418) 2023-11-13T11:01:22.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.145+0000 7f1735d9b640 1 -- 172.21.15.16:0/2606049081 learned_addr learned my addr 172.21.15.16:0/2606049081 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:22.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173e59c640 1 --2- 172.21.15.16:0/2606049081 >> v2:172.21.15.184:3300/0 conn(0x7f17400cb190 0x7f174015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127951366 0 0) 0x7f1740158c20 con 0x7f17400716b0 2023-11-13T11:01:22.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f172c003520 con 0x7f17400716b0 2023-11-13T11:01:22.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1818781416 0 0) 0x7f172c003520 con 0x7f17400716b0 2023-11-13T11:01:22.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 >> v2:172.21.15.184:3300/0 conn(0x7f17400cb190 msgr2=0x7f174015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:22.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 --2- 172.21.15.16:0/2606049081 >> v2:172.21.15.184:3300/0 conn(0x7f17400cb190 0x7f174015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 >> v1:172.21.15.184:6789/0 conn(0x7f174006a190 legacy=0x7f17400b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174015ba60 con 0x7f17400716b0 2023-11-13T11:01:22.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f1728001670 con 0x7f17400716b0 2023-11-13T11:01:22.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 >> v1:172.21.15.16:6789/0 conn(0x7f17400716b0 legacy=0x7f174006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f172c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f172c0047c0 0x7f172c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f172c005180 0x7f172c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174015b780 con 0x7f172c003970 2023-11-13T11:01:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c00c0d0 con 0x7f172c0047c0 2023-11-13T11:01:22.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c00c290 con 0x7f172c005180 2023-11-13T11:01:22.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.146+0000 7f173e59c640 1 --2- 172.21.15.16:0/2606049081 >> v2:172.21.15.184:3300/0 conn(0x7f17400cb190 0x7f174015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:01:22.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.147+0000 7f173e59c640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f172c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.147+0000 7f1735d9b640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f172c0047c0 0x7f172c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.147+0000 7f173ed9d640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f172c005180 0x7f172c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.147+0000 7f173e59c640 1 -- 172.21.15.16:0/2606049081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f172c005180 msgr2=0x7f172c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:22.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.147+0000 7f173e59c640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f172c005180 0x7f172c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.147+0000 7f173e59c640 1 -- 172.21.15.16:0/2606049081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f172c0047c0 msgr2=0x7f172c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:22.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.147+0000 7f173e59c640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f172c0047c0 0x7f172c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.147+0000 7f173e59c640 1 -- 172.21.15.16:0/2606049081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c003520 con 0x7f172c003970 2023-11-13T11:01:22.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.147+0000 7f173ed9d640 1 -- 172.21.15.16:0/2606049081 reap_dead start 2023-11-13T11:01:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.148+0000 7f173e59c640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f172c003d90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f17300073e0 tx=0x7f172c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d69d1d326f64176 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.149+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173002cd60 con 0x7f172c003970 2023-11-13T11:01:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.149+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1730034440 con 0x7f172c003970 2023-11-13T11:01:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.149+0000 7f173dd9b640 1 -- 172.21.15.16:0/2606049081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17300347b0 con 0x7f172c003970 2023-11-13T11:01:22.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.149+0000 7f173f59e640 1 -- 172.21.15.16:0/2606049081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 msgr2=0x7f172c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.149+0000 7f173f59e640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f172c003d90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f17300073e0 tx=0x7f172c0057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.149+0000 7f173f59e640 1 -- 172.21.15.16:0/2606049081 shutdown_connections 2023-11-13T11:01:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.149+0000 7f173f59e640 1 --2- 172.21.15.16:0/2606049081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f172c003d90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.149+0000 7f173f59e640 1 -- 172.21.15.16:0/2606049081 >> 172.21.15.16:0/2606049081 conn(0x7f17400b04a0 msgr2=0x7f17400b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:22.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.149+0000 7f173f59e640 1 -- 172.21.15.16:0/2606049081 shutdown_connections 2023-11-13T11:01:22.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.150+0000 7f173f59e640 1 -- 172.21.15.16:0/2606049081 wait complete. 2023-11-13T11:01:22.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.151+0000 7f173f59e640 1 Processor -- start 2023-11-13T11:01:22.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.152+0000 7f173f59e640 1 -- start start 2023-11-13T11:01:22.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.152+0000 7f173f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f17401f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173f59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17400c9000 0x7f1740159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173f59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400a8ff0 0x7f1740159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173f59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400c94c0 con 0x7f172c003970 2023-11-13T11:01:22.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173f59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400c97a0 con 0x7f17400a8ff0 2023-11-13T11:01:22.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173f59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400c9a80 con 0x7f17400c9000 2023-11-13T11:01:22.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f17401f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f17401f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42072/0 (socket says 172.21.15.16:42072) 2023-11-13T11:01:22.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173e59c640 1 -- 172.21.15.16:0/2865319773 learned_addr learned my addr 172.21.15.16:0/2865319773 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:22.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173ed9d640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400a8ff0 0x7f1740159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173e59c640 1 -- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17400c9000 msgr2=0x7f1740159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173e59c640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17400c9000 0x7f1740159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173e59c640 1 -- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400a8ff0 msgr2=0x7f1740159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173e59c640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400a8ff0 0x7f1740159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.153+0000 7f173e59c640 1 -- 172.21.15.16:0/2865319773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400c9d60 con 0x7f172c003970 2023-11-13T11:01:22.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.154+0000 7f173e59c640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f17401f37a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f173002c700 tx=0x7f1730004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:22.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.154+0000 7f17377fe640 1 -- 172.21.15.16:0/2865319773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173003e8d0 con 0x7f172c003970 2023-11-13T11:01:22.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.154+0000 7f17377fe640 1 -- 172.21.15.16:0/2865319773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173003ea30 con 0x7f172c003970 2023-11-13T11:01:22.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.154+0000 7f17377fe640 1 -- 172.21.15.16:0/2865319773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173003ede0 con 0x7f172c003970 2023-11-13T11:01:22.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.154+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17400ca040 con 0x7f172c003970 2023-11-13T11:01:22.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.154+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174015a1c0 con 0x7f172c003970 2023-11-13T11:01:22.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.155+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174004e4d0 con 0x7f172c003970 2023-11-13T11:01:22.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.156+0000 7f17377fe640 1 -- 172.21.15.16:0/2865319773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f173003d3a0 con 0x7f172c003970 2023-11-13T11:01:22.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.158+0000 7f17377fe640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f172006ae40 0x7f172006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.158+0000 7f17377fe640 1 -- 172.21.15.16:0/2865319773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f17300aba90 con 0x7f172c003970 2023-11-13T11:01:22.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.158+0000 7f1735d9b640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f172006ae40 0x7f172006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.160+0000 7f1735d9b640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f172006ae40 0x7f172006d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1728001200 tx=0x7f1728001150 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:22.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.162+0000 7f17377fe640 1 -- 172.21.15.16:0/2865319773 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1730076960 con 0x7f172c003970 2023-11-13T11:01:22.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.365+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17400c8de0 con 0x7f172c003970 2023-11-13T11:01:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.368+0000 7f17377fe640 1 -- 172.21.15.16:0/2865319773 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+17026 (secure 0 0 0) 0x7f1730079f80 con 0x7f172c003970 2023-11-13T11:01:22.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:22.372 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:00:55.896214+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:57:18.637969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-13T10:57:19.541678+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":43,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.2","mappings":[{"from":4,"to":7}]},{"pgid":"6.14","mappings":[{"from":3,"to":5}]},{"pgid":"6.18","mappings":[{"from":4,"to":2}]},{"pgid":"6.38","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:01:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.370+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f172006ae40 msgr2=0x7f172006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.370+0000 7f173f59e640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f172006ae40 0x7f172006d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1728001200 tx=0x7f1728001150 comp rx=0 tx=0).stop 2023-11-13T11:01:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.370+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 msgr2=0x7f17401f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.370+0000 7f173f59e640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f17401f37a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f173002c700 tx=0x7f1730004ab0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.371+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 shutdown_connections 2023-11-13T11:01:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.371+0000 7f173f59e640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f172006ae40 0x7f172006d300 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.371+0000 7f173f59e640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17400a8ff0 0x7f1740159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.371+0000 7f173f59e640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17400c9000 0x7f1740159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.371+0000 7f173f59e640 1 --2- 172.21.15.16:0/2865319773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f172c003970 0x7f17401f37a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.371+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 >> 172.21.15.16:0/2865319773 conn(0x7f17400b04a0 msgr2=0x7f174006f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.371+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 shutdown_connections 2023-11-13T11:01:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.371+0000 7f173f59e640 1 -- 172.21.15.16:0/2865319773 wait complete. 2023-11-13T11:01:22.393 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-13T11:01:22.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:01:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.617+0000 7f414790f640 1 Processor -- start 2023-11-13T11:01:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.617+0000 7f414790f640 1 -- start start 2023-11-13T11:01:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.618+0000 7f414790f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f414015ddd0 0x7f414015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f414790f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f414015a610 0x7f41400b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f414790f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414006db90 con 0x7f414015d9f0 2023-11-13T11:01:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f414790f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414015b070 con 0x7f414015ddd0 2023-11-13T11:01:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f414790f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414015aab0 con 0x7f414015a610 2023-11-13T11:01:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f4144eaa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f414015ddd0 0x7f414015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f4144eaa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f414015ddd0 0x7f414015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42086/0 (socket says 172.21.15.16:42086) 2023-11-13T11:01:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f4144eaa640 1 -- 172.21.15.16:0/131156110 learned_addr learned my addr 172.21.15.16:0/131156110 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:22.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f4144eaa640 1 -- 172.21.15.16:0/131156110 >> v2:172.21.15.184:3300/0 conn(0x7f414015a610 msgr2=0x7f41400b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:22.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f4145eac640 1 --2- 172.21.15.16:0/131156110 >> v2:172.21.15.184:3300/0 conn(0x7f414015a610 0x7f41400b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f4144eaa640 1 --2- 172.21.15.16:0/131156110 >> v2:172.21.15.184:3300/0 conn(0x7f414015a610 0x7f41400b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.619+0000 7f4144eaa640 1 -- 172.21.15.16:0/131156110 >> v1:172.21.15.184:6789/0 conn(0x7f414015d9f0 legacy=0x7f414006d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.620+0000 7f4144eaa640 1 -- 172.21.15.16:0/131156110 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414015ad90 con 0x7f414015ddd0 2023-11-13T11:01:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.620+0000 7f4145eac640 1 --2- 172.21.15.16:0/131156110 >> v2:172.21.15.184:3300/0 conn(0x7f414015a610 0x7f41400b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.621+0000 7f4144eaa640 1 --2- 172.21.15.16:0/131156110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015ddd0 0x7f414015a0d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f41400b3750 tx=0x7f413002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf087456abfd495b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.621+0000 7f4137fff640 1 -- 172.21.15.16:0/131156110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300317e0 con 0x7f414015ddd0 2023-11-13T11:01:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.621+0000 7f4137fff640 1 -- 172.21.15.16:0/131156110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4130031940 con 0x7f414015ddd0 2023-11-13T11:01:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.621+0000 7f4137fff640 1 -- 172.21.15.16:0/131156110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130031cd0 con 0x7f414015ddd0 2023-11-13T11:01:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.622+0000 7f414790f640 1 -- 172.21.15.16:0/131156110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015ddd0 msgr2=0x7f414015a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.622+0000 7f414790f640 1 --2- 172.21.15.16:0/131156110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015ddd0 0x7f414015a0d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f41400b3750 tx=0x7f413002b880 comp rx=0 tx=0).stop 2023-11-13T11:01:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.622+0000 7f414790f640 1 -- 172.21.15.16:0/131156110 shutdown_connections 2023-11-13T11:01:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.622+0000 7f414790f640 1 --2- 172.21.15.16:0/131156110 >> v2:172.21.15.184:3300/0 conn(0x7f414015a610 0x7f41400b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.622+0000 7f414790f640 1 --2- 172.21.15.16:0/131156110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015ddd0 0x7f414015a0d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.622+0000 7f414790f640 1 -- 172.21.15.16:0/131156110 >> 172.21.15.16:0/131156110 conn(0x7f414006a190 msgr2=0x7f414006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.622+0000 7f414790f640 1 -- 172.21.15.16:0/131156110 shutdown_connections 2023-11-13T11:01:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.622+0000 7f414790f640 1 -- 172.21.15.16:0/131156110 wait complete. 2023-11-13T11:01:22.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.624+0000 7f414790f640 1 Processor -- start 2023-11-13T11:01:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.624+0000 7f414790f640 1 -- start start 2023-11-13T11:01:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f414790f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f414015a610 0x7f41401460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f414790f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015d9f0 0x7f41400c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f414790f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f414015ddd0 0x7f41400c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f414790f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400c41d0 con 0x7f414015d9f0 2023-11-13T11:01:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f414790f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400c44b0 con 0x7f414015a610 2023-11-13T11:01:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f414790f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41400c4790 con 0x7f414015ddd0 2023-11-13T11:01:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f4144eaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015d9f0 0x7f41400c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f4144eaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015d9f0 0x7f41400c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42094/0 (socket says 172.21.15.16:42094) 2023-11-13T11:01:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f4144eaa640 1 -- 172.21.15.16:0/4244223565 learned_addr learned my addr 172.21.15.16:0/4244223565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f4144eaa640 1 -- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f414015ddd0 msgr2=0x7f41400c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f4144eaa640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f414015ddd0 0x7f41400c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f4144eaa640 1 -- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f414015a610 msgr2=0x7f41401460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.625+0000 7f41456ab640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f414015a610 0x7f41401460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f4144eaa640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f414015a610 0x7f41401460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f4144eaa640 1 -- 172.21.15.16:0/4244223565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300094d0 con 0x7f414015d9f0 2023-11-13T11:01:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f41456ab640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f414015a610 0x7f41401460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f4144eaa640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015d9f0 0x7f41400c37f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f413002b8c0 tx=0x7f41300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f4135ffb640 1 -- 172.21.15.16:0/4244223565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300329c0 con 0x7f414015d9f0 2023-11-13T11:01:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f4135ffb640 1 -- 172.21.15.16:0/4244223565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4130032b40 con 0x7f414015d9f0 2023-11-13T11:01:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f4135ffb640 1 -- 172.21.15.16:0/4244223565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300453f0 con 0x7f414015d9f0 2023-11-13T11:01:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41400c4a70 con 0x7f414015d9f0 2023-11-13T11:01:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.626+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4140146610 con 0x7f414015d9f0 2023-11-13T11:01:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.628+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41400c4f90 con 0x7f414015d9f0 2023-11-13T11:01:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.628+0000 7f4135ffb640 1 -- 172.21.15.16:0/4244223565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f413004d3f0 con 0x7f414015d9f0 2023-11-13T11:01:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.629+0000 7f4135ffb640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f412406ae40 0x7f412406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.629+0000 7f4135ffb640 1 -- 172.21.15.16:0/4244223565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f41300ab730 con 0x7f414015d9f0 2023-11-13T11:01:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.630+0000 7f41456ab640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f412406ae40 0x7f412406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.631+0000 7f41456ab640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f412406ae40 0x7f412406d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f413c00a830 tx=0x7f413c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.634+0000 7f4135ffb640 1 -- 172.21.15.16:0/4244223565 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41400c4f90 con 0x7f414015d9f0 2023-11-13T11:01:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.868+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41400c4f90 con 0x7f414015d9f0 2023-11-13T11:01:22.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.869+0000 7f4135ffb640 1 -- 172.21.15.16:0/4244223565 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f41400c4f90 con 0x7f414015d9f0 2023-11-13T11:01:22.873 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:22.873 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3678615052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3678615052},{"type":"v1","addr":"172.21.15.16:6833","nonce":3678615052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3358463124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3358463124},{"type":"v1","addr":"172.21.15.16:6835","nonce":3358463124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2329286743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2329286743},{"type":"v1","addr":"172.21.15.184:6836","nonce":2329286743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-13T10:57:20.523743+0000","modified":"2023-11-13T10:57:32.253697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6837/3171288994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3171288994},{"type":"v1","addr":"172.21.15.184:6837","nonce":3171288994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-13T11:01:22.874 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 19 2023-11-13T11:01:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f412406ae40 msgr2=0x7f412406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f412406ae40 0x7f412406d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f413c00a830 tx=0x7f413c008040 comp rx=0 tx=0).stop 2023-11-13T11:01:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015d9f0 msgr2=0x7f41400c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:22.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015d9f0 0x7f41400c37f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f413002b8c0 tx=0x7f41300046c0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 shutdown_connections 2023-11-13T11:01:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f412406ae40 0x7f412406d300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f414015ddd0 0x7f41400c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414015d9f0 0x7f41400c37f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 --2- 172.21.15.16:0/4244223565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f414015a610 0x7f41401460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:22.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 >> 172.21.15.16:0/4244223565 conn(0x7f414006a190 msgr2=0x7f4140153600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:22.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 shutdown_connections 2023-11-13T11:01:22.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:22.872+0000 7f414790f640 1 -- 172.21.15.16:0/4244223565 wait complete. 2023-11-13T11:01:22.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:01:23.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.113+0000 7f765b95a640 1 Processor -- start 2023-11-13T11:01:23.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.113+0000 7f765b95a640 1 -- start start 2023-11-13T11:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.115+0000 7f765b95a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7654155410 0x7f76540ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.115+0000 7f765b95a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76540ce990 0x7f76540cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.115+0000 7f765b95a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765415b500 con 0x7f7654071440 2023-11-13T11:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.115+0000 7f765b95a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765415b7e0 con 0x7f76540ce990 2023-11-13T11:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.115+0000 7f765b95a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765415bac0 con 0x7f7654155410 2023-11-13T11:01:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.115+0000 7f7658ef5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76540ce990 0x7f76540cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.116+0000 7f7658ef5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76540ce990 0x7f76540cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49166/0 (socket says 172.21.15.16:49166) 2023-11-13T11:01:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.116+0000 7f7658ef5640 1 -- 172.21.15.16:0/1926774577 learned_addr learned my addr 172.21.15.16:0/1926774577 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.116+0000 7f76596f6640 1 --2- 172.21.15.16:0/1926774577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7654155410 0x7f76540ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.116+0000 7f7658ef5640 1 -- 172.21.15.16:0/1926774577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7654155410 msgr2=0x7f76540ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.116+0000 7f7658ef5640 1 --2- 172.21.15.16:0/1926774577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7654155410 0x7f76540ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.116+0000 7f7658ef5640 1 -- 172.21.15.16:0/1926774577 >> v1:172.21.15.184:6789/0 conn(0x7f7654071440 legacy=0x7f765406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.116+0000 7f7658ef5640 1 -- 172.21.15.16:0/1926774577 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765415bda0 con 0x7f76540ce990 2023-11-13T11:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.118+0000 7f7658ef5640 1 --2- 172.21.15.16:0/1926774577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76540ce990 0x7f76540cee10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7644009870 tx=0x7f764402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c6acaeffd1127 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.118+0000 7f764bfff640 1 -- 172.21.15.16:0/1926774577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644030840 con 0x7f76540ce990 2023-11-13T11:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.118+0000 7f764bfff640 1 -- 172.21.15.16:0/1926774577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76440309a0 con 0x7f76540ce990 2023-11-13T11:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.118+0000 7f764bfff640 1 -- 172.21.15.16:0/1926774577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644030d30 con 0x7f76540ce990 2023-11-13T11:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.119+0000 7f765b95a640 1 -- 172.21.15.16:0/1926774577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76540ce990 msgr2=0x7f76540cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.119+0000 7f765b95a640 1 --2- 172.21.15.16:0/1926774577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76540ce990 0x7f76540cee10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7644009870 tx=0x7f764402b980 comp rx=0 tx=0).stop 2023-11-13T11:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.119+0000 7f765b95a640 1 -- 172.21.15.16:0/1926774577 shutdown_connections 2023-11-13T11:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.119+0000 7f765b95a640 1 --2- 172.21.15.16:0/1926774577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76540ce990 0x7f76540cee10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.119+0000 7f765b95a640 1 --2- 172.21.15.16:0/1926774577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7654155410 0x7f76540ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.119+0000 7f765b95a640 1 -- 172.21.15.16:0/1926774577 >> 172.21.15.16:0/1926774577 conn(0x7f765406a190 msgr2=0x7f765406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.119+0000 7f765b95a640 1 -- 172.21.15.16:0/1926774577 shutdown_connections 2023-11-13T11:01:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.119+0000 7f765b95a640 1 -- 172.21.15.16:0/1926774577 wait complete. 2023-11-13T11:01:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.121+0000 7f765b95a640 1 Processor -- start 2023-11-13T11:01:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.121+0000 7f765b95a640 1 -- start start 2023-11-13T11:01:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.122+0000 7f765b95a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7654071440 0x7f765414d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.122+0000 7f765b95a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76540ce990 0x7f76541527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.122+0000 7f765b95a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7654155410 0x7f765414a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.122+0000 7f765b95a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765414ecd0 con 0x7f7654155410 2023-11-13T11:01:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.122+0000 7f765b95a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765414ef40 con 0x7f7654071440 2023-11-13T11:01:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.122+0000 7f765b95a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765414f220 con 0x7f76540ce990 2023-11-13T11:01:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.122+0000 7f7659ef7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7654155410 0x7f765414a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.122+0000 7f7659ef7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7654155410 0x7f765414a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49182/0 (socket says 172.21.15.16:49182) 2023-11-13T11:01:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.122+0000 7f7659ef7640 1 -- 172.21.15.16:0/3832115630 learned_addr learned my addr 172.21.15.16:0/3832115630 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f76596f6640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7654071440 0x7f765414d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f7658ef5640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76540ce990 0x7f76541527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f7659ef7640 1 -- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76540ce990 msgr2=0x7f76541527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f7659ef7640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76540ce990 0x7f76541527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f7659ef7640 1 -- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7654071440 msgr2=0x7f765414d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f7659ef7640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7654071440 0x7f765414d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f7659ef7640 1 -- 172.21.15.16:0/3832115630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76440094d0 con 0x7f7654155410 2023-11-13T11:01:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f7659ef7640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7654155410 0x7f765414a9d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f764c001eb0 tx=0x7f764c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f7649ffb640 1 -- 172.21.15.16:0/3832115630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0052d0 con 0x7f7654155410 2023-11-13T11:01:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f7649ffb640 1 -- 172.21.15.16:0/3832115630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c005430 con 0x7f7654155410 2023-11-13T11:01:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.123+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765414f500 con 0x7f7654155410 2023-11-13T11:01:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.124+0000 7f7649ffb640 1 -- 172.21.15.16:0/3832115630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c003810 con 0x7f7654155410 2023-11-13T11:01:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.124+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765414d550 con 0x7f7654155410 2023-11-13T11:01:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.125+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765414d6e0 con 0x7f7654155410 2023-11-13T11:01:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.126+0000 7f7649ffb640 1 -- 172.21.15.16:0/3832115630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f764c003970 con 0x7f7654155410 2023-11-13T11:01:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.127+0000 7f7649ffb640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f763806ae40 0x7f763806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.127+0000 7f7649ffb640 1 -- 172.21.15.16:0/3832115630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f764c088dd0 con 0x7f7654155410 2023-11-13T11:01:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.128+0000 7f76596f6640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f763806ae40 0x7f763806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.129+0000 7f76596f6640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f763806ae40 0x7f763806d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f76540aca70 tx=0x7f76500039b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.131+0000 7f7649ffb640 1 -- 172.21.15.16:0/3832115630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f764c053c90 con 0x7f7654155410 2023-11-13T11:01:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.364+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7654067090 con 0x7f7654155410 2023-11-13T11:01:23.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.365+0000 7f7649ffb640 1 -- 172.21.15.16:0/3832115630 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f764c017090 con 0x7f7654155410 2023-11-13T11:01:23.369 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:23.370 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3678615052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3678615052},{"type":"v1","addr":"172.21.15.16:6833","nonce":3678615052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3358463124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3358463124},{"type":"v1","addr":"172.21.15.16:6835","nonce":3358463124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2329286743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2329286743},{"type":"v1","addr":"172.21.15.184:6836","nonce":2329286743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-13T10:57:20.523743+0000","modified":"2023-11-13T10:57:32.253697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4990":{"gid":4990,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6837/3171288994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3171288994},{"type":"v1","addr":"172.21.15.184:6837","nonce":3171288994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-13T11:01:23.370 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 19 2023-11-13T11:01:23.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f763806ae40 msgr2=0x7f763806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f763806ae40 0x7f763806d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f76540aca70 tx=0x7f76500039b0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7654155410 msgr2=0x7f765414a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7654155410 0x7f765414a9d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f764c001eb0 tx=0x7f764c00c890 comp rx=0 tx=0).stop 2023-11-13T11:01:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 shutdown_connections 2023-11-13T11:01:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f763806ae40 0x7f763806d300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7654155410 0x7f765414a9d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76540ce990 0x7f76541527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 --2- 172.21.15.16:0/3832115630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7654071440 0x7f765414d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 >> 172.21.15.16:0/3832115630 conn(0x7f765406a190 msgr2=0x7f7654153cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 shutdown_connections 2023-11-13T11:01:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.368+0000 7f765b95a640 1 -- 172.21.15.16:0/3832115630 wait complete. 2023-11-13T11:01:23.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:01:23.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.600+0000 7f128ccec640 1 Processor -- start 2023-11-13T11:01:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.601+0000 7f128ccec640 1 -- start start 2023-11-13T11:01:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.602+0000 7f128ccec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f128815b190 0x7f128806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.602+0000 7f128ccec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128806a190 0x7f128806a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.602+0000 7f128ccec640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1288158c20 con 0x7f12880cb190 2023-11-13T11:01:23.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.602+0000 7f128ccec640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128806ab40 con 0x7f128815b190 2023-11-13T11:01:23.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.602+0000 7f128ccec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12880b1830 con 0x7f128806a190 2023-11-13T11:01:23.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.603+0000 7f1287fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128806a190 0x7f128806a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.603+0000 7f127ffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f128815b190 0x7f128806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.603+0000 7f1287fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128806a190 0x7f128806a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:37958/0 (socket says 172.21.15.16:37958) 2023-11-13T11:01:23.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.603+0000 7f1287fff640 1 -- 172.21.15.16:0/4269925483 learned_addr learned my addr 172.21.15.16:0/4269925483 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:23.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.604+0000 7f1287fff640 1 -- 172.21.15.16:0/4269925483 >> v2:172.21.15.184:3300/0 conn(0x7f128815b190 msgr2=0x7f128806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:23.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.604+0000 7f1287fff640 1 --2- 172.21.15.16:0/4269925483 >> v2:172.21.15.184:3300/0 conn(0x7f128815b190 0x7f128806f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.604+0000 7f1287fff640 1 -- 172.21.15.16:0/4269925483 >> v1:172.21.15.184:6789/0 conn(0x7f12880cb190 legacy=0x7f12880716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.604+0000 7f1287fff640 1 -- 172.21.15.16:0/4269925483 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128815b780 con 0x7f128806a190 2023-11-13T11:01:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.605+0000 7f1287fff640 1 --2- 172.21.15.16:0/4269925483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128806a190 0x7f128806a600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1280009e50 tx=0x7f128002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89d3695d06667c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.606+0000 7f1286ffd640 1 -- 172.21.15.16:0/4269925483 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128002ca70 con 0x7f128806a190 2023-11-13T11:01:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.606+0000 7f1286ffd640 1 -- 172.21.15.16:0/4269925483 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128002cbd0 con 0x7f128806a190 2023-11-13T11:01:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.606+0000 7f1286ffd640 1 -- 172.21.15.16:0/4269925483 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800343f0 con 0x7f128806a190 2023-11-13T11:01:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.607+0000 7f128ccec640 1 -- 172.21.15.16:0/4269925483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128806a190 msgr2=0x7f128806a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.607+0000 7f128ccec640 1 --2- 172.21.15.16:0/4269925483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128806a190 0x7f128806a600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1280009e50 tx=0x7f128002bed0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.607+0000 7f128ccec640 1 -- 172.21.15.16:0/4269925483 shutdown_connections 2023-11-13T11:01:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.607+0000 7f128ccec640 1 --2- 172.21.15.16:0/4269925483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128806a190 0x7f128806a600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.607+0000 7f128ccec640 1 --2- 172.21.15.16:0/4269925483 >> v2:172.21.15.184:3300/0 conn(0x7f128815b190 0x7f128806f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.607+0000 7f128ccec640 1 -- 172.21.15.16:0/4269925483 >> 172.21.15.16:0/4269925483 conn(0x7f12880b04a0 msgr2=0x7f12880b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.607+0000 7f128ccec640 1 -- 172.21.15.16:0/4269925483 shutdown_connections 2023-11-13T11:01:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.607+0000 7f128ccec640 1 -- 172.21.15.16:0/4269925483 wait complete. 2023-11-13T11:01:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.609+0000 7f128ccec640 1 Processor -- start 2023-11-13T11:01:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.609+0000 7f128ccec640 1 -- start start 2023-11-13T11:01:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.610+0000 7f128ccec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128806a190 0x7f12881f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.610+0000 7f128ccec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880cb190 0x7f1288156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.610+0000 7f128ccec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128815b190 0x7f128806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.610+0000 7f128ccec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1288153bb0 con 0x7f128806a190 2023-11-13T11:01:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.610+0000 7f128ccec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1288154540 con 0x7f12880cb190 2023-11-13T11:01:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.610+0000 7f128ccec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1288153e00 con 0x7f128815b190 2023-11-13T11:01:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.610+0000 7f12877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128806a190 0x7f12881f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.610+0000 7f12877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128806a190 0x7f12881f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49188/0 (socket says 172.21.15.16:49188) 2023-11-13T11:01:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.610+0000 7f12877fe640 1 -- 172.21.15.16:0/4288348847 learned_addr learned my addr 172.21.15.16:0/4288348847 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f127ffff640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880cb190 0x7f1288156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f1287fff640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128815b190 0x7f128806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f12877fe640 1 -- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128815b190 msgr2=0x7f128806c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f12877fe640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128815b190 0x7f128806c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f12877fe640 1 -- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880cb190 msgr2=0x7f1288156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f12877fe640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880cb190 0x7f1288156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f12877fe640 1 -- 172.21.15.16:0/4288348847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1280009b30 con 0x7f128806a190 2023-11-13T11:01:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f12877fe640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128806a190 0x7f12881f33d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f127800c430 tx=0x7f127800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f1284ff9640 1 -- 172.21.15.16:0/4288348847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12780052d0 con 0x7f128806a190 2023-11-13T11:01:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f1284ff9640 1 -- 172.21.15.16:0/4288348847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1278005430 con 0x7f128806a190 2023-11-13T11:01:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f1284ff9640 1 -- 172.21.15.16:0/4288348847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278003850 con 0x7f128806a190 2023-11-13T11:01:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.611+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12881540e0 con 0x7f128806a190 2023-11-13T11:01:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.612+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1288154a70 con 0x7f128806a190 2023-11-13T11:01:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.613+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1288154c90 con 0x7f128806a190 2023-11-13T11:01:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.613+0000 7f1284ff9640 1 -- 172.21.15.16:0/4288348847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f12780039b0 con 0x7f128806a190 2023-11-13T11:01:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.615+0000 7f1284ff9640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f126806ae40 0x7f126806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.615+0000 7f1284ff9640 1 -- 172.21.15.16:0/4288348847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f1278088e50 con 0x7f128806a190 2023-11-13T11:01:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.615+0000 7f127ffff640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f126806ae40 0x7f126806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.617+0000 7f127ffff640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f126806ae40 0x7f126806d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f128815ada0 tx=0x7f1270008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:23.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.619+0000 7f1284ff9640 1 -- 172.21.15.16:0/4288348847 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1278053c70 con 0x7f128806a190 2023-11-13T11:01:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.815+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f128804e4d0 con 0x7f128806a190 2023-11-13T11:01:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.818+0000 7f1284ff9640 1 -- 172.21.15.16:0/4288348847 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+17026 (secure 0 0 0) 0x7f1278017090 con 0x7f128806a190 2023-11-13T11:01:23.823 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:00:55.896214+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-13T10:57:18.637969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-13T10:57:19.541678+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":43,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.2","mappings":[{"from":4,"to":7}]},{"pgid":"6.14","mappings":[{"from":3,"to":5}]},{"pgid":"6.18","mappings":[{"from":4,"to":2}]},{"pgid":"6.38","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:01:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f126806ae40 msgr2=0x7f126806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f126806ae40 0x7f126806d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f128815ada0 tx=0x7f1270008040 comp rx=0 tx=0).stop 2023-11-13T11:01:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128806a190 msgr2=0x7f12881f33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128806a190 0x7f12881f33d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f127800c430 tx=0x7f127800c8f0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 shutdown_connections 2023-11-13T11:01:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f126806ae40 0x7f126806d300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128815b190 0x7f128806c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880cb190 0x7f1288156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 --2- 172.21.15.16:0/4288348847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128806a190 0x7f12881f33d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 >> 172.21.15.16:0/4288348847 conn(0x7f12880b04a0 msgr2=0x7f128815efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 shutdown_connections 2023-11-13T11:01:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:23.821+0000 7f128ccec640 1 -- 172.21.15.16:0/4288348847 wait complete. 2023-11-13T11:01:23.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-13T11:01:24.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.068+0000 7f3a3d0ec640 1 Processor -- start 2023-11-13T11:01:24.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.069+0000 7f3a3d0ec640 1 -- start start 2023-11-13T11:01:24.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.070+0000 7f3a3d0ec640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a3815cf20 0x7f3a3806cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.070+0000 7f3a3d0ec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3a3806bb90 0x7f3a380ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.070+0000 7f3a3d0ec640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a380b2900 con 0x7f3a380b2d90 2023-11-13T11:01:24.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.070+0000 7f3a3d0ec640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3815d300 con 0x7f3a3815cf20 2023-11-13T11:01:24.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.070+0000 7f3a3d0ec640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a380b1fd0 con 0x7f3a3806bb90 2023-11-13T11:01:24.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.071+0000 7f3a36d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a3815cf20 0x7f3a3806cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.071+0000 7f3a3759e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3a380b2d90 0x7f3a380ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49738/0 (socket says 172.21.15.16:49738) 2023-11-13T11:01:24.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.071+0000 7f3a3759e640 1 -- 172.21.15.16:0/2828524471 learned_addr learned my addr 172.21.15.16:0/2828524471 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:24.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.071+0000 7f3a3659c640 1 --2- 172.21.15.16:0/2828524471 >> v2:172.21.15.184:3300/0 conn(0x7f3a3806bb90 0x7f3a380ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.071+0000 7f3a36d9d640 1 -- 172.21.15.16:0/2828524471 >> v2:172.21.15.184:3300/0 conn(0x7f3a3806bb90 msgr2=0x7f3a380ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:24.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.071+0000 7f3a36d9d640 1 --2- 172.21.15.16:0/2828524471 >> v2:172.21.15.184:3300/0 conn(0x7f3a3806bb90 0x7f3a380ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.071+0000 7f3a36d9d640 1 -- 172.21.15.16:0/2828524471 >> v1:172.21.15.16:6789/0 conn(0x7f3a380b2d90 legacy=0x7f3a380ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.071+0000 7f3a35d9b640 1 -- 172.21.15.16:0/2828524471 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233187081 0 0) 0x7f3a380b2900 con 0x7f3a380b2d90 2023-11-13T11:01:24.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.071+0000 7f3a36d9d640 1 -- 172.21.15.16:0/2828524471 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380b22b0 con 0x7f3a3815cf20 2023-11-13T11:01:24.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.073+0000 7f3a36d9d640 1 --2- 172.21.15.16:0/2828524471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815cf20 0x7f3a3806cab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3a380cadd0 tx=0x7f3a2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f6a79c5f144b73 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:24.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.073+0000 7f3a35d9b640 1 -- 172.21.15.16:0/2828524471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2802cc80 con 0x7f3a3815cf20 2023-11-13T11:01:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.073+0000 7f3a35d9b640 1 -- 172.21.15.16:0/2828524471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2802cde0 con 0x7f3a3815cf20 2023-11-13T11:01:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.073+0000 7f3a35d9b640 1 -- 172.21.15.16:0/2828524471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28034660 con 0x7f3a3815cf20 2023-11-13T11:01:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.074+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/2828524471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815cf20 msgr2=0x7f3a3806cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.074+0000 7f3a3d0ec640 1 --2- 172.21.15.16:0/2828524471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815cf20 0x7f3a3806cab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3a380cadd0 tx=0x7f3a2802bed0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.074+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/2828524471 shutdown_connections 2023-11-13T11:01:24.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.074+0000 7f3a3d0ec640 1 --2- 172.21.15.16:0/2828524471 >> v2:172.21.15.184:3300/0 conn(0x7f3a3806bb90 0x7f3a380ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.074+0000 7f3a3d0ec640 1 --2- 172.21.15.16:0/2828524471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815cf20 0x7f3a3806cab0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.074+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/2828524471 >> 172.21.15.16:0/2828524471 conn(0x7f3a3806a190 msgr2=0x7f3a3806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:24.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.074+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/2828524471 shutdown_connections 2023-11-13T11:01:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.074+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/2828524471 wait complete. 2023-11-13T11:01:24.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.076+0000 7f3a3d0ec640 1 Processor -- start 2023-11-13T11:01:24.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.076+0000 7f3a3d0ec640 1 -- start start 2023-11-13T11:01:24.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.077+0000 7f3a3d0ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3806bb90 0x7f3a380c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.077+0000 7f3a3d0ec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a380b2d90 0x7f3a38152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.077+0000 7f3a3d0ec640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a3815cf20 0x7f3a380c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.077+0000 7f3a3d0ec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a38152f50 con 0x7f3a3806bb90 2023-11-13T11:01:24.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.077+0000 7f3a3d0ec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a38153c10 con 0x7f3a380b2d90 2023-11-13T11:01:24.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.077+0000 7f3a3d0ec640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380c4cb0 con 0x7f3a3815cf20 2023-11-13T11:01:24.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.077+0000 7f3a36d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3806bb90 0x7f3a380c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.077+0000 7f3a36d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3806bb90 0x7f3a380c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49198/0 (socket says 172.21.15.16:49198) 2023-11-13T11:01:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.077+0000 7f3a36d9d640 1 -- 172.21.15.16:0/1088661987 learned_addr learned my addr 172.21.15.16:0/1088661987 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.078+0000 7f3a3659c640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a380b2d90 0x7f3a38152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.078+0000 7f3a3759e640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a3815cf20 0x7f3a380c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.078+0000 7f3a36d9d640 1 -- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a3815cf20 msgr2=0x7f3a380c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.078+0000 7f3a36d9d640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a3815cf20 0x7f3a380c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.078+0000 7f3a36d9d640 1 -- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a380b2d90 msgr2=0x7f3a38152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.078+0000 7f3a36d9d640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a380b2d90 0x7f3a38152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.078+0000 7f3a36d9d640 1 -- 172.21.15.16:0/1088661987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a28009a70 con 0x7f3a3806bb90 2023-11-13T11:01:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.078+0000 7f3a36d9d640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3806bb90 0x7f3a380c46d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3a28037040 tx=0x7f3a28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:24.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.078+0000 7f3a2f7fe640 1 -- 172.21.15.16:0/1088661987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28040650 con 0x7f3a3806bb90 2023-11-13T11:01:24.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.079+0000 7f3a2f7fe640 1 -- 172.21.15.16:0/1088661987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a280407f0 con 0x7f3a3806bb90 2023-11-13T11:01:24.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.079+0000 7f3a2f7fe640 1 -- 172.21.15.16:0/1088661987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28040ba0 con 0x7f3a3806bb90 2023-11-13T11:01:24.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.079+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a380c4f90 con 0x7f3a3806bb90 2023-11-13T11:01:24.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.079+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a380c5550 con 0x7f3a3806bb90 2023-11-13T11:01:24.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.080+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a380c56e0 con 0x7f3a3806bb90 2023-11-13T11:01:24.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.081+0000 7f3a2f7fe640 1 -- 172.21.15.16:0/1088661987 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a28040d00 con 0x7f3a3806bb90 2023-11-13T11:01:24.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.082+0000 7f3a2f7fe640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a1806ae40 0x7f3a1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.082+0000 7f3a2f7fe640 1 -- 172.21.15.16:0/1088661987 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f3a280ac660 con 0x7f3a3806bb90 2023-11-13T11:01:24.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.083+0000 7f3a3659c640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a1806ae40 0x7f3a1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.084+0000 7f3a3659c640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a1806ae40 0x7f3a1806d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3a381597d0 tx=0x7f3a20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:24.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.086+0000 7f3a2f7fe640 1 -- 172.21.15.16:0/1088661987 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a28077630 con 0x7f3a3806bb90 2023-11-13T11:01:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.324+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3a38067090 con 0x7f3a3806bb90 2023-11-13T11:01:24.329 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-13T11:01:24.325+0000 7fe1550d2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-13T11:01:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.328+0000 7f3a2f7fe640 1 -- 172.21.15.16:0/1088661987 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3a2807ac50 con 0x7f3a3806bb90 2023-11-13T11:01:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-13T11:01:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.330+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a1806ae40 msgr2=0x7f3a1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.330+0000 7f3a3d0ec640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a1806ae40 0x7f3a1806d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3a381597d0 tx=0x7f3a20008040 comp rx=0 tx=0).stop 2023-11-13T11:01:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.330+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3806bb90 msgr2=0x7f3a380c46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.330+0000 7f3a3d0ec640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3806bb90 0x7f3a380c46d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3a28037040 tx=0x7f3a28004a40 comp rx=0 tx=0).stop 2023-11-13T11:01:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.330+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 shutdown_connections 2023-11-13T11:01:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.330+0000 7f3a3d0ec640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a1806ae40 0x7f3a1806d300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.331+0000 7f3a3d0ec640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a3815cf20 0x7f3a380c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.331+0000 7f3a3d0ec640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a380b2d90 0x7f3a38152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.331+0000 7f3a3d0ec640 1 --2- 172.21.15.16:0/1088661987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3806bb90 0x7f3a380c46d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.331+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 >> 172.21.15.16:0/1088661987 conn(0x7f3a3806a190 msgr2=0x7f3a380ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.331+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 shutdown_connections 2023-11-13T11:01:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.332+0000 7f3a3d0ec640 1 -- 172.21.15.16:0/1088661987 wait complete. 2023-11-13T11:01:24.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-13T11:01:24.464 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-11-13T11:01:24.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.585+0000 7fca77d7c640 1 Processor -- start 2023-11-13T11:01:24.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.585+0000 7fca77d7c640 1 -- start start 2023-11-13T11:01:24.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.587+0000 7fca77d7c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca700b3b90 0x7fca700b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.587+0000 7fca77d7c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7006c3b0 0x7fca70070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.587+0000 7fca77d7c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca700b2e00 con 0x7fca7015cf90 2023-11-13T11:01:24.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.587+0000 7fca77d7c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca700b21f0 con 0x7fca700b3b90 2023-11-13T11:01:24.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.587+0000 7fca77d7c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca700b24d0 con 0x7fca7006c3b0 2023-11-13T11:01:24.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.588+0000 7fca75317640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca700b3b90 0x7fca700b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.588+0000 7fca76319640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7006c3b0 0x7fca70070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.588+0000 7fca75317640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fca700b3b90 0x7fca700b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45680/0 (socket says 172.21.15.16:45680) 2023-11-13T11:01:24.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.588+0000 7fca76319640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7006c3b0 0x7fca70070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:37974/0 (socket says 172.21.15.16:37974) 2023-11-13T11:01:24.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.588+0000 7fca75317640 1 -- 172.21.15.16:0/3403336837 learned_addr learned my addr 172.21.15.16:0/3403336837 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:24.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.589+0000 7fca75317640 1 -- 172.21.15.16:0/3403336837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7006c3b0 msgr2=0x7fca70070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.589+0000 7fca75317640 1 --2- 172.21.15.16:0/3403336837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7006c3b0 0x7fca70070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.589+0000 7fca75317640 1 -- 172.21.15.16:0/3403336837 >> v1:172.21.15.184:6789/0 conn(0x7fca7015cf90 legacy=0x7fca700b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.589+0000 7fca75317640 1 -- 172.21.15.16:0/3403336837 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700b27b0 con 0x7fca700b3b90 2023-11-13T11:01:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.590+0000 7fca75317640 1 --2- 172.21.15.16:0/3403336837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 0x7fca700b1c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fca60009870 tx=0x7fca6002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbd7fba3f2370afd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.590+0000 7fca74b16640 1 -- 172.21.15.16:0/3403336837 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60031640 con 0x7fca700b3b90 2023-11-13T11:01:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.590+0000 7fca74b16640 1 -- 172.21.15.16:0/3403336837 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca600317a0 con 0x7fca700b3b90 2023-11-13T11:01:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.590+0000 7fca74b16640 1 -- 172.21.15.16:0/3403336837 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60031b30 con 0x7fca700b3b90 2023-11-13T11:01:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.591+0000 7fca77d7c640 1 -- 172.21.15.16:0/3403336837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 msgr2=0x7fca700b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.591+0000 7fca77d7c640 1 --2- 172.21.15.16:0/3403336837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 0x7fca700b1c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fca60009870 tx=0x7fca6002b8f0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.591+0000 7fca77d7c640 1 -- 172.21.15.16:0/3403336837 shutdown_connections 2023-11-13T11:01:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.591+0000 7fca77d7c640 1 --2- 172.21.15.16:0/3403336837 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7006c3b0 0x7fca70070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.591+0000 7fca77d7c640 1 --2- 172.21.15.16:0/3403336837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 0x7fca700b1c10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.591+0000 7fca77d7c640 1 -- 172.21.15.16:0/3403336837 >> 172.21.15.16:0/3403336837 conn(0x7fca7006a190 msgr2=0x7fca7006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.591+0000 7fca77d7c640 1 -- 172.21.15.16:0/3403336837 shutdown_connections 2023-11-13T11:01:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.591+0000 7fca77d7c640 1 -- 172.21.15.16:0/3403336837 wait complete. 2023-11-13T11:01:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.593+0000 7fca77d7c640 1 Processor -- start 2023-11-13T11:01:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.593+0000 7fca77d7c640 1 -- start start 2023-11-13T11:01:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.594+0000 7fca77d7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7006c3b0 0x7fca7015e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.594+0000 7fca77d7c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 0x7fca700cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.594+0000 7fca77d7c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7015cf90 0x7fca701e5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.594+0000 7fca77d7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca701e7990 con 0x7fca7006c3b0 2023-11-13T11:01:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.594+0000 7fca77d7c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7015ee30 con 0x7fca700b3b90 2023-11-13T11:01:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.594+0000 7fca77d7c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7015f110 con 0x7fca7015cf90 2023-11-13T11:01:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.594+0000 7fca75b18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7006c3b0 0x7fca7015e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75b18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7006c3b0 0x7fca7015e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49210/0 (socket says 172.21.15.16:49210) 2023-11-13T11:01:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75b18640 1 -- 172.21.15.16:0/4279493217 learned_addr learned my addr 172.21.15.16:0/4279493217 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:24.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75317640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 0x7fca700cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca76319640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7015cf90 0x7fca701e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75b18640 1 -- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7015cf90 msgr2=0x7fca701e5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75b18640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7015cf90 0x7fca701e5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75b18640 1 -- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 msgr2=0x7fca700cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75b18640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 0x7fca700cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75b18640 1 -- 172.21.15.16:0/4279493217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca600094d0 con 0x7fca7006c3b0 2023-11-13T11:01:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75317640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 0x7fca700cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca75b18640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7006c3b0 0x7fca7015e730 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00c400 tx=0x7fca6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.595+0000 7fca667fc640 1 -- 172.21.15.16:0/4279493217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0052d0 con 0x7fca7006c3b0 2023-11-13T11:01:24.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.596+0000 7fca667fc640 1 -- 172.21.15.16:0/4279493217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c005430 con 0x7fca7006c3b0 2023-11-13T11:01:24.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.596+0000 7fca667fc640 1 -- 172.21.15.16:0/4279493217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c003830 con 0x7fca7006c3b0 2023-11-13T11:01:24.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.596+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7015f3f0 con 0x7fca7006c3b0 2023-11-13T11:01:24.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.596+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca701e5ea0 con 0x7fca7006c3b0 2023-11-13T11:01:24.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.597+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca701e6090 con 0x7fca7006c3b0 2023-11-13T11:01:24.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.598+0000 7fca667fc640 1 -- 172.21.15.16:0/4279493217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca6c003990 con 0x7fca7006c3b0 2023-11-13T11:01:24.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.599+0000 7fca667fc640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca5406ae40 0x7fca5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:24.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.600+0000 7fca667fc640 1 -- 172.21.15.16:0/4279493217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fca6c088dc0 con 0x7fca7006c3b0 2023-11-13T11:01:24.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.600+0000 7fca75317640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca5406ae40 0x7fca5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:24.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.602+0000 7fca75317640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca5406ae40 0x7fca5406d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fca60036000 tx=0x7fca6003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:24.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.604+0000 7fca667fc640 1 -- 172.21.15.16:0/4279493217 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca6c053c40 con 0x7fca7006c3b0 2023-11-13T11:01:24.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:24.839+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fca701e77f0 con 0x7fca7006c3b0 2023-11-13T11:01:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.331+0000 7fca667fc640 1 -- 172.21.15.16:0/4279493217 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fca701e77f0 con 0x7fca7006c3b0 2023-11-13T11:01:25.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca5406ae40 msgr2=0x7fca5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:25.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca5406ae40 0x7fca5406d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fca60036000 tx=0x7fca6003c040 comp rx=0 tx=0).stop 2023-11-13T11:01:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7006c3b0 msgr2=0x7fca7015e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7006c3b0 0x7fca7015e730 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00c400 tx=0x7fca6c00c8c0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 shutdown_connections 2023-11-13T11:01:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fca5406ae40 0x7fca5406d300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fca7015cf90 0x7fca701e5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca700b3b90 0x7fca700cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 --2- 172.21.15.16:0/4279493217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7006c3b0 0x7fca7015e730 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 >> 172.21.15.16:0/4279493217 conn(0x7fca7006a190 msgr2=0x7fca700ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.337+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 shutdown_connections 2023-11-13T11:01:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.338+0000 7fca77d7c640 1 -- 172.21.15.16:0/4279493217 wait complete. 2023-11-13T11:01:25.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-13T11:01:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.597+0000 7fd131085640 1 Processor -- start 2023-11-13T11:01:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.597+0000 7fd131085640 1 -- start start 2023-11-13T11:01:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.598+0000 7fd131085640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd12c1608c0 0x7fd12c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:25.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.598+0000 7fd131085640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c15c7a0 0x7fd12c15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.598+0000 7fd131085640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c0d1b80 con 0x7fd12c160ca0 2023-11-13T11:01:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.598+0000 7fd131085640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0cea50 con 0x7fd12c1608c0 2023-11-13T11:01:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.598+0000 7fd131085640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0d1000 con 0x7fd12c15c7a0 2023-11-13T11:01:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.598+0000 7fd12bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd12c1608c0 0x7fd12c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.599+0000 7fd12bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd12c1608c0 0x7fd12c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49226/0 (socket says 172.21.15.16:49226) 2023-11-13T11:01:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.599+0000 7fd12bfff640 1 -- 172.21.15.16:0/4197629655 learned_addr learned my addr 172.21.15.16:0/4197629655 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.599+0000 7fd1237fe640 1 --1- 172.21.15.16:0/4197629655 >> v1:172.21.15.16:6789/0 conn(0x7fd12c160ca0 0x7fd12c0d0720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49746/0 (socket says 172.21.15.16:49746) 2023-11-13T11:01:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.599+0000 7fd12bfff640 1 -- 172.21.15.16:0/4197629655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c15c7a0 msgr2=0x7fd12c15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.599+0000 7fd130884640 1 --2- 172.21.15.16:0/4197629655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c15c7a0 0x7fd12c15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.599+0000 7fd12bfff640 1 --2- 172.21.15.16:0/4197629655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c15c7a0 0x7fd12c15cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.599+0000 7fd12bfff640 1 -- 172.21.15.16:0/4197629655 >> v1:172.21.15.16:6789/0 conn(0x7fd12c160ca0 legacy=0x7fd12c0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.599+0000 7fd12bfff640 1 -- 172.21.15.16:0/4197629655 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c0d12e0 con 0x7fd12c1608c0 2023-11-13T11:01:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.599+0000 7fd12b7fe640 1 -- 172.21.15.16:0/4197629655 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161673395 0 0) 0x7fd114001110 con 0x7fd12c160ca0 2023-11-13T11:01:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.600+0000 7fd12bfff640 1 --2- 172.21.15.16:0/4197629655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 0x7fd12c0ce510 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009e50 tx=0x7fd11c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e04b9f0388e423 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.600+0000 7fd12b7fe640 1 -- 172.21.15.16:0/4197629655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c02cb00 con 0x7fd12c1608c0 2023-11-13T11:01:25.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd12b7fe640 1 -- 172.21.15.16:0/4197629655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c02cc60 con 0x7fd12c1608c0 2023-11-13T11:01:25.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd12b7fe640 1 -- 172.21.15.16:0/4197629655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0343f0 con 0x7fd12c1608c0 2023-11-13T11:01:25.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd131085640 1 -- 172.21.15.16:0/4197629655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 msgr2=0x7fd12c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:25.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd131085640 1 --2- 172.21.15.16:0/4197629655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 0x7fd12c0ce510 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009e50 tx=0x7fd11c02bf60 comp rx=0 tx=0).stop 2023-11-13T11:01:25.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd131085640 1 -- 172.21.15.16:0/4197629655 shutdown_connections 2023-11-13T11:01:25.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd131085640 1 --2- 172.21.15.16:0/4197629655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c15c7a0 0x7fd12c15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd131085640 1 --2- 172.21.15.16:0/4197629655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 0x7fd12c0ce510 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd131085640 1 -- 172.21.15.16:0/4197629655 >> 172.21.15.16:0/4197629655 conn(0x7fd12c0d2b50 msgr2=0x7fd12c0d2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:25.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd131085640 1 -- 172.21.15.16:0/4197629655 shutdown_connections 2023-11-13T11:01:25.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.601+0000 7fd131085640 1 -- 172.21.15.16:0/4197629655 wait complete. 2023-11-13T11:01:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.603+0000 7fd131085640 1 Processor -- start 2023-11-13T11:01:25.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.603+0000 7fd131085640 1 -- start start 2023-11-13T11:01:25.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.604+0000 7fd131085640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd12c15c7a0 0x7fd12c158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:25.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.604+0000 7fd131085640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 0x7fd12c0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.604+0000 7fd131085640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c160ca0 0x7fd12c0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.604+0000 7fd131085640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0ca6f0 con 0x7fd12c1608c0 2023-11-13T11:01:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.604+0000 7fd131085640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0cc8c0 con 0x7fd12c15c7a0 2023-11-13T11:01:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.604+0000 7fd131085640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0c9dc0 con 0x7fd12c160ca0 2023-11-13T11:01:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd1237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 0x7fd12c0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd1237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 0x7fd12c0cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49232/0 (socket says 172.21.15.16:49232) 2023-11-13T11:01:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd1237fe640 1 -- 172.21.15.16:0/4078672489 learned_addr learned my addr 172.21.15.16:0/4078672489 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd130884640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c160ca0 0x7fd12c0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd12bfff640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd12c15c7a0 0x7fd12c158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd1237fe640 1 -- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c160ca0 msgr2=0x7fd12c0cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd1237fe640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c160ca0 0x7fd12c0cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd1237fe640 1 -- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd12c15c7a0 msgr2=0x7fd12c158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd1237fe640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd12c15c7a0 0x7fd12c158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd1237fe640 1 -- 172.21.15.16:0/4078672489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c009b30 con 0x7fd12c1608c0 2023-11-13T11:01:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd12bfff640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd12c15c7a0 0x7fd12c158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.605+0000 7fd130884640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c160ca0 0x7fd12c0cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.606+0000 7fd1237fe640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 0x7fd12c0cbe40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd11400c780 tx=0x7fd11400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.606+0000 7fd1297fa640 1 -- 172.21.15.16:0/4078672489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1140072d0 con 0x7fd12c1608c0 2023-11-13T11:01:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.606+0000 7fd1297fa640 1 -- 172.21.15.16:0/4078672489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd114007450 con 0x7fd12c1608c0 2023-11-13T11:01:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.606+0000 7fd1297fa640 1 -- 172.21.15.16:0/4078672489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1140047e0 con 0x7fd12c1608c0 2023-11-13T11:01:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.606+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c0ca0a0 con 0x7fd12c1608c0 2023-11-13T11:01:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.606+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c159470 con 0x7fd12c1608c0 2023-11-13T11:01:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.607+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c0675b0 con 0x7fd12c1608c0 2023-11-13T11:01:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.609+0000 7fd1297fa640 1 -- 172.21.15.16:0/4078672489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd114004960 con 0x7fd12c1608c0 2023-11-13T11:01:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.610+0000 7fd1297fa640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd10c06aec0 0x7fd10c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.610+0000 7fd1297fa640 1 -- 172.21.15.16:0/4078672489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6260+0+0 (secure 0 0 0) 0x7fd114088d50 con 0x7fd12c1608c0 2023-11-13T11:01:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.611+0000 7fd12bfff640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd10c06aec0 0x7fd10c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.612+0000 7fd12bfff640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd10c06aec0 0x7fd10c06d380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd11c035000 tx=0x7fd11c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.615+0000 7fd1297fa640 1 -- 172.21.15.16:0/4078672489 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd114053bb0 con 0x7fd12c1608c0 2023-11-13T11:01:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:25.808+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd12c0d1b80 con 0x7fd12c1608c0 2023-11-13T11:01:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.335+0000 7fd1297fa640 1 -- 172.21.15.16:0/4078672489 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd12c0d1b80 con 0x7fd12c1608c0 2023-11-13T11:01:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-13T11:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.341+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd10c06aec0 msgr2=0x7fd10c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.341+0000 7fd131085640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd10c06aec0 0x7fd10c06d380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd11c035000 tx=0x7fd11c030000 comp rx=0 tx=0).stop 2023-11-13T11:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.341+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 msgr2=0x7fd12c0cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.341+0000 7fd131085640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 0x7fd12c0cbe40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd11400c780 tx=0x7fd11400cc40 comp rx=0 tx=0).stop 2023-11-13T11:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.342+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 shutdown_connections 2023-11-13T11:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.342+0000 7fd131085640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd10c06aec0 0x7fd10c06d380 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.342+0000 7fd131085640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd12c160ca0 0x7fd12c0cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.342+0000 7fd131085640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd12c1608c0 0x7fd12c0cbe40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.342+0000 7fd131085640 1 --2- 172.21.15.16:0/4078672489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd12c15c7a0 0x7fd12c158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.342+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 >> 172.21.15.16:0/4078672489 conn(0x7fd12c0d2b50 msgr2=0x7fd12c0d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.342+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 shutdown_connections 2023-11-13T11:01:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.342+0000 7fd131085640 1 -- 172.21.15.16:0/4078672489 wait complete. 2023-11-13T11:01:26.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-13T11:01:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.615+0000 7f871dcaf640 1 Processor -- start 2023-11-13T11:01:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.615+0000 7f871dcaf640 1 -- start start 2023-11-13T11:01:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.616+0000 7f871dcaf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87180adbc0 0x7f87180ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.616+0000 7f871dcaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f871815d680 0x7f87180cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.616+0000 7f871dcaf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8718159ec0 con 0x7f8718159ae0 2023-11-13T11:01:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.616+0000 7f871dcaf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871815dc20 con 0x7f87180adbc0 2023-11-13T11:01:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.616+0000 7f871dcaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871815df00 con 0x7f871815d680 2023-11-13T11:01:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.617+0000 7f87177fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8718159ae0 0x7f87180cbb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49750/0 (socket says 172.21.15.16:49750) 2023-11-13T11:01:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.617+0000 7f87177fe640 1 -- 172.21.15.16:0/4169997144 learned_addr learned my addr 172.21.15.16:0/4169997144 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.617+0000 7f8717fff640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f871815d680 0x7f87180cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.617+0000 7f870ffff640 1 --2- 172.21.15.16:0/4169997144 >> v2:172.21.15.184:3300/0 conn(0x7f87180adbc0 0x7f87180ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.617+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159936162 0 0) 0x7f8718159ec0 con 0x7f8718159ae0 2023-11-13T11:01:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.617+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8704003520 con 0x7f8718159ae0 2023-11-13T11:01:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782744485 0 0) 0x7f8704003520 con 0x7f8718159ae0 2023-11-13T11:01:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f871815d680 msgr2=0x7f87180cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f871815d680 0x7f87180cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 >> v2:172.21.15.184:3300/0 conn(0x7f87180adbc0 msgr2=0x7f87180ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 --2- 172.21.15.16:0/4169997144 >> v2:172.21.15.184:3300/0 conn(0x7f87180adbc0 0x7f87180ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871815e1e0 con 0x7f8718159ae0 2023-11-13T11:01:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f8708001670 con 0x7f8718159ae0 2023-11-13T11:01:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 >> v1:172.21.15.16:6789/0 conn(0x7f8718159ae0 legacy=0x7f87180cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f871815d680 0x7f8704003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8704003ef0 0x7f87040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8704004d00 0x7f8704005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871815dc20 con 0x7f8704004d00 2023-11-13T11:01:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871815df00 con 0x7f8704003ef0 2023-11-13T11:01:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870400db90 con 0x7f871815d680 2023-11-13T11:01:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f8717fff640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8704004d00 0x7f8704005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f870ffff640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8704003ef0 0x7f87040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.618+0000 7f87177fe640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f871815d680 0x7f8704003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.619+0000 7f8717fff640 1 -- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f871815d680 msgr2=0x7f8704003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.619+0000 7f8717fff640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f871815d680 0x7f8704003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.619+0000 7f8717fff640 1 -- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8704003ef0 msgr2=0x7f87040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.619+0000 7f8717fff640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8704003ef0 0x7f87040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.619+0000 7f8717fff640 1 -- 172.21.15.16:0/4169997144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8704003520 con 0x7f8704004d00 2023-11-13T11:01:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.619+0000 7f870ffff640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8704003ef0 0x7f87040047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.619+0000 7f87177fe640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f871815d680 0x7f8704003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.620+0000 7f8717fff640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8704004d00 0x7f8704005180 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8710009fc0 tx=0x7f87040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf9b3211cd8ad01 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.620+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871002cc00 con 0x7f8704004d00 2023-11-13T11:01:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.620+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871000f040 con 0x7f8704004d00 2023-11-13T11:01:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.620+0000 7f8716ffd640 1 -- 172.21.15.16:0/4169997144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710034440 con 0x7f8704004d00 2023-11-13T11:01:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.620+0000 7f871dcaf640 1 -- 172.21.15.16:0/4169997144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8704004d00 msgr2=0x7f8704005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.620+0000 7f871dcaf640 1 --2- 172.21.15.16:0/4169997144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8704004d00 0x7f8704005180 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8710009fc0 tx=0x7f87040057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.620+0000 7f8717fff640 1 -- 172.21.15.16:0/4169997144 reap_dead start 2023-11-13T11:01:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.621+0000 7f871dcaf640 1 -- 172.21.15.16:0/4169997144 shutdown_connections 2023-11-13T11:01:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.621+0000 7f871dcaf640 1 -- 172.21.15.16:0/4169997144 >> 172.21.15.16:0/4169997144 conn(0x7f871806a190 msgr2=0x7f871806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.621+0000 7f871dcaf640 1 -- 172.21.15.16:0/4169997144 shutdown_connections 2023-11-13T11:01:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.621+0000 7f871dcaf640 1 -- 172.21.15.16:0/4169997144 wait complete. 2023-11-13T11:01:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.622+0000 7f871dcaf640 1 Processor -- start 2023-11-13T11:01:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.623+0000 7f871dcaf640 1 -- start start 2023-11-13T11:01:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f871dcaf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8704004d00 0x7f87180ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f871dcaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8718155ee0 0x7f87181f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f871dcaf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87181f1fb0 0x7f87181ee480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f871dcaf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87181577f0 con 0x7f8718155ee0 2023-11-13T11:01:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f871dcaf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87181f2430 con 0x7f87181f1fb0 2023-11-13T11:01:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f871dcaf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8718156360 con 0x7f8704004d00 2023-11-13T11:01:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f870ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8718155ee0 0x7f87181f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f870ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8718155ee0 0x7f87181f1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49254/0 (socket says 172.21.15.16:49254) 2023-11-13T11:01:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f870ffff640 1 -- 172.21.15.16:0/596296955 learned_addr learned my addr 172.21.15.16:0/596296955 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f87177fe640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8704004d00 0x7f87180ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f8717fff640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87181f1fb0 0x7f87181ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f870ffff640 1 -- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8704004d00 msgr2=0x7f87180ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f870ffff640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8704004d00 0x7f87180ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f870ffff640 1 -- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87181f1fb0 msgr2=0x7f87181ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f870ffff640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87181f1fb0 0x7f87181ee480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f870ffff640 1 -- 172.21.15.16:0/596296955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8718158960 con 0x7f8718155ee0 2023-11-13T11:01:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.624+0000 7f87177fe640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8704004d00 0x7f87180ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.625+0000 7f870ffff640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8718155ee0 0x7f87181f1a70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f87000109d0 tx=0x7f8700010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.625+0000 7f8714ff9640 1 -- 172.21.15.16:0/596296955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000ebc0 con 0x7f8718155ee0 2023-11-13T11:01:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.625+0000 7f8714ff9640 1 -- 172.21.15.16:0/596296955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000ed20 con 0x7f8718155ee0 2023-11-13T11:01:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.625+0000 7f8714ff9640 1 -- 172.21.15.16:0/596296955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870001c620 con 0x7f8718155ee0 2023-11-13T11:01:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.625+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8718158c40 con 0x7f8718155ee0 2023-11-13T11:01:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.625+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8718157410 con 0x7f8718155ee0 2023-11-13T11:01:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.626+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8718067090 con 0x7f8718155ee0 2023-11-13T11:01:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.627+0000 7f8714ff9640 1 -- 172.21.15.16:0/596296955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f870001c780 con 0x7f8718155ee0 2023-11-13T11:01:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.628+0000 7f8714ff9640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f86f806ae70 0x7f86f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.629+0000 7f8714ff9640 1 -- 172.21.15.16:0/596296955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5822+0+0 (secure 0 0 0) 0x7f8700089bc0 con 0x7f8718155ee0 2023-11-13T11:01:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.629+0000 7f87177fe640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f86f806ae70 0x7f86f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.630+0000 7f87177fe640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f86f806ae70 0x7f86f806d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f87181ec070 tx=0x7f8708009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.633+0000 7f8714ff9640 1 -- 172.21.15.16:0/596296955 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8700054d40 con 0x7f8718155ee0 2023-11-13T11:01:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:26.831+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87180cc0c0 con 0x7f8718155ee0 2023-11-13T11:01:27.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.338+0000 7f8714ff9640 1 -- 172.21.15.16:0/596296955 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8700058360 con 0x7f8718155ee0 2023-11-13T11:01:27.342 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-13T11:01:27.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.342+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f86f806ae70 msgr2=0x7f86f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:27.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.342+0000 7f871dcaf640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f86f806ae70 0x7f86f806d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f87181ec070 tx=0x7f8708009040 comp rx=0 tx=0).stop 2023-11-13T11:01:27.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.342+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8718155ee0 msgr2=0x7f87181f1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:27.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.342+0000 7f871dcaf640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8718155ee0 0x7f87181f1a70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f87000109d0 tx=0x7f8700010e90 comp rx=0 tx=0).stop 2023-11-13T11:01:27.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.344+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 shutdown_connections 2023-11-13T11:01:27.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.344+0000 7f871dcaf640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f86f806ae70 0x7f86f806d330 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.344+0000 7f871dcaf640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87181f1fb0 0x7f87181ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.344+0000 7f871dcaf640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8718155ee0 0x7f87181f1a70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.344+0000 7f871dcaf640 1 --2- 172.21.15.16:0/596296955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8704004d00 0x7f87180ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.344+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 >> 172.21.15.16:0/596296955 conn(0x7f871806a190 msgr2=0x7f87180ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:27.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.344+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 shutdown_connections 2023-11-13T11:01:27.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.345+0000 7f871dcaf640 1 -- 172.21.15.16:0/596296955 wait complete. 2023-11-13T11:01:27.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-13T11:01:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.594+0000 7f20e359e640 1 Processor -- start 2023-11-13T11:01:27.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.595+0000 7f20e359e640 1 -- start start 2023-11-13T11:01:27.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.596+0000 7f20e359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e40d1850 0x7f20e40d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:27.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.596+0000 7f20e359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f20e40d2d00 0x7f20e40c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.596+0000 7f20e359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e40ce0e0 con 0x7f20e415ce70 2023-11-13T11:01:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.596+0000 7f20e359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e40ce3c0 con 0x7f20e40d2d00 2023-11-13T11:01:27.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.596+0000 7f20e359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40ce6a0 con 0x7f20e40d1850 2023-11-13T11:01:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.597+0000 7f20e1d9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e40d1850 0x7f20e40d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.597+0000 7f20e2d9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f20e40d2d00 0x7f20e40c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.597+0000 7f20e1d9b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e40d1850 0x7f20e40d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:38016/0 (socket says 172.21.15.16:38016) 2023-11-13T11:01:27.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.597+0000 7f20e1d9b640 1 -- 172.21.15.16:0/894721678 learned_addr learned my addr 172.21.15.16:0/894721678 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:27.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.598+0000 7f20e1d9b640 1 -- 172.21.15.16:0/894721678 >> v2:172.21.15.184:3300/0 conn(0x7f20e40d2d00 msgr2=0x7f20e40c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:27.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.598+0000 7f20e1d9b640 1 --2- 172.21.15.16:0/894721678 >> v2:172.21.15.184:3300/0 conn(0x7f20e40d2d00 0x7f20e40c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.598+0000 7f20e1d9b640 1 -- 172.21.15.16:0/894721678 >> v1:172.21.15.184:6789/0 conn(0x7f20e415ce70 legacy=0x7f20e40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:27.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.598+0000 7f20e1d9b640 1 -- 172.21.15.16:0/894721678 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40ce980 con 0x7f20e40d1850 2023-11-13T11:01:27.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.600+0000 7f20e1d9b640 1 --2- 172.21.15.16:0/894721678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e40d1850 0x7f20e40d27c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009870 tx=0x7f20cc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7fd45d3e9f508e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:27.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.600+0000 7f20e159a640 1 -- 172.21.15.16:0/894721678 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc030840 con 0x7f20e40d1850 2023-11-13T11:01:27.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.600+0000 7f20e159a640 1 -- 172.21.15.16:0/894721678 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc0309a0 con 0x7f20e40d1850 2023-11-13T11:01:27.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.600+0000 7f20e159a640 1 -- 172.21.15.16:0/894721678 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc030d30 con 0x7f20e40d1850 2023-11-13T11:01:27.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.601+0000 7f20e359e640 1 -- 172.21.15.16:0/894721678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e40d1850 msgr2=0x7f20e40d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:27.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.601+0000 7f20e359e640 1 --2- 172.21.15.16:0/894721678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e40d1850 0x7f20e40d27c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009870 tx=0x7f20cc02b980 comp rx=0 tx=0).stop 2023-11-13T11:01:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.601+0000 7f20e359e640 1 -- 172.21.15.16:0/894721678 shutdown_connections 2023-11-13T11:01:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.601+0000 7f20e359e640 1 --2- 172.21.15.16:0/894721678 >> v2:172.21.15.184:3300/0 conn(0x7f20e40d2d00 0x7f20e40c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.601+0000 7f20e359e640 1 --2- 172.21.15.16:0/894721678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e40d1850 0x7f20e40d27c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.601+0000 7f20e359e640 1 -- 172.21.15.16:0/894721678 >> 172.21.15.16:0/894721678 conn(0x7f20e408c580 msgr2=0x7f20e409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.601+0000 7f20e359e640 1 -- 172.21.15.16:0/894721678 shutdown_connections 2023-11-13T11:01:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.602+0000 7f20e359e640 1 -- 172.21.15.16:0/894721678 wait complete. 2023-11-13T11:01:27.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.604+0000 7f20e359e640 1 Processor -- start 2023-11-13T11:01:27.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.604+0000 7f20e359e640 1 -- start start 2023-11-13T11:01:27.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.605+0000 7f20e359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40d1850 0x7f20e40b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:27.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.605+0000 7f20e359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20e40d2d00 0x7f20e41eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:27.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.605+0000 7f20e359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e415ce70 0x7f20e41f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:27.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.605+0000 7f20e359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e41f38f0 con 0x7f20e40d1850 2023-11-13T11:01:27.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.605+0000 7f20e359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40b6590 con 0x7f20e40d2d00 2023-11-13T11:01:27.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.605+0000 7f20e359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40c76a0 con 0x7f20e415ce70 2023-11-13T11:01:27.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.605+0000 7f20e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40d1850 0x7f20e40b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:27.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.605+0000 7f20e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40d1850 0x7f20e40b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49264/0 (socket says 172.21.15.16:49264) 2023-11-13T11:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.605+0000 7f20e259c640 1 -- 172.21.15.16:0/4251207754 learned_addr learned my addr 172.21.15.16:0/4251207754 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e1d9b640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20e40d2d00 0x7f20e41eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e2d9d640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e415ce70 0x7f20e41f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:27.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e259c640 1 -- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e415ce70 msgr2=0x7f20e41f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:27.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e259c640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e415ce70 0x7f20e41f33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e259c640 1 -- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20e40d2d00 msgr2=0x7f20e41eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:27.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e259c640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20e40d2d00 0x7f20e41eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e259c640 1 -- 172.21.15.16:0/4251207754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc0094d0 con 0x7f20e40d1850 2023-11-13T11:01:27.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e1d9b640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20e40d2d00 0x7f20e41eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:27.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e259c640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40d1850 0x7f20e40b5fb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c730 tx=0x7f20d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:27.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20daffd640 1 -- 172.21.15.16:0/4251207754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4006d50 con 0x7f20e40d1850 2023-11-13T11:01:27.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20daffd640 1 -- 172.21.15.16:0/4251207754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d400ce40 con 0x7f20e40d1850 2023-11-13T11:01:27.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e40c7980 con 0x7f20e40d1850 2023-11-13T11:01:27.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20daffd640 1 -- 172.21.15.16:0/4251207754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4003bb0 con 0x7f20e40d1850 2023-11-13T11:01:27.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.606+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e40c7f40 con 0x7f20e40d1850 2023-11-13T11:01:27.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.607+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e4160630 con 0x7f20e40d1850 2023-11-13T11:01:27.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.609+0000 7f20daffd640 1 -- 172.21.15.16:0/4251207754 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20d4003d10 con 0x7f20e40d1850 2023-11-13T11:01:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.610+0000 7f20daffd640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20c406ae40 0x7f20c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.610+0000 7f20daffd640 1 -- 172.21.15.16:0/4251207754 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f20d4088fa0 con 0x7f20e40d1850 2023-11-13T11:01:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.610+0000 7f20e1d9b640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20c406ae40 0x7f20c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.612+0000 7f20e1d9b640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20c406ae40 0x7f20c406d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f20cc036000 tx=0x7f20cc03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.614+0000 7f20daffd640 1 -- 172.21.15.16:0/4251207754 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20d40bbe90 con 0x7f20e40d1850 2023-11-13T11:01:27.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.810+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f20e404e4d0 con 0x7f20e40d1850 2023-11-13T11:01:27.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.816+0000 7f20daffd640 1 -- 172.21.15.16:0/4251207754 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f20d4057d30 con 0x7f20e40d1850 2023-11-13T11:01:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20c406ae40 msgr2=0x7f20c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20c406ae40 0x7f20c406d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f20cc036000 tx=0x7f20cc03b040 comp rx=0 tx=0).stop 2023-11-13T11:01:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40d1850 msgr2=0x7f20e40b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40d1850 0x7f20e40b5fb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c730 tx=0x7f20d400cbf0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 shutdown_connections 2023-11-13T11:01:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f20c406ae40 0x7f20c406d300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20e415ce70 0x7f20e41f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20e40d2d00 0x7f20e41eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 --2- 172.21.15.16:0/4251207754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40d1850 0x7f20e40b5fb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 >> 172.21.15.16:0/4251207754 conn(0x7f20e408c580 msgr2=0x7f20e40b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 shutdown_connections 2023-11-13T11:01:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:27.819+0000 7f20e359e640 1 -- 172.21.15.16:0/4251207754 wait complete. 2023-11-13T11:01:27.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2023-11-13T11:01:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.069+0000 7f69f359e640 1 Processor -- start 2023-11-13T11:01:28.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.069+0000 7f69f359e640 1 -- start start 2023-11-13T11:01:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.070+0000 7f69f359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f40cb9f0 0x7f69f406c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.071+0000 7f69f359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f69f406c780 0x7f69f40ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.071+0000 7f69f359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f40ce4c0 con 0x7f69f409ccb0 2023-11-13T11:01:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.071+0000 7f69f359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f40cd990 con 0x7f69f406c780 2023-11-13T11:01:28.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.071+0000 7f69f359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40cece0 con 0x7f69f40cb9f0 2023-11-13T11:01:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.071+0000 7f69f2d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f69f409ccb0 0x7f69f40cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49760/0 (socket says 172.21.15.16:49760) 2023-11-13T11:01:28.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.071+0000 7f69f2d9d640 1 -- 172.21.15.16:0/520498383 learned_addr learned my addr 172.21.15.16:0/520498383 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.071+0000 7f69ebfff640 1 --2- 172.21.15.16:0/520498383 >> v2:172.21.15.184:3300/0 conn(0x7f69f406c780 0x7f69f40ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.071+0000 7f69f259c640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f40cb9f0 0x7f69f406c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:28.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.071+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155273031 0 0) 0x7f69f40ce4c0 con 0x7f69f409ccb0 2023-11-13T11:01:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e0003520 con 0x7f69f409ccb0 2023-11-13T11:01:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2674666206 0 0) 0x7f69e0003520 con 0x7f69f409ccb0 2023-11-13T11:01:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f40cb9f0 msgr2=0x7f69f406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f40cb9f0 0x7f69f406c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 >> v2:172.21.15.184:3300/0 conn(0x7f69f406c780 msgr2=0x7f69f40ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:28.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 --2- 172.21.15.16:0/520498383 >> v2:172.21.15.184:3300/0 conn(0x7f69f406c780 0x7f69f40ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f415d220 con 0x7f69f409ccb0 2023-11-13T11:01:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f69ec0019f0 con 0x7f69f409ccb0 2023-11-13T11:01:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 >> v1:172.21.15.16:6789/0 conn(0x7f69f409ccb0 legacy=0x7f69f40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69e0003970 0x7f69e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:28.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f40cb9f0 0x7f69e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:28.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:28.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40cece0 con 0x7f69e0004d00 2023-11-13T11:01:28.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e000b980 con 0x7f69e0003970 2023-11-13T11:01:28.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.072+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e000baf0 con 0x7f69f40cb9f0 2023-11-13T11:01:28.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.073+0000 7f69f2d9d640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:28.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.073+0000 7f69f259c640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69e0003970 0x7f69e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:28.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.073+0000 7f69ebfff640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f40cb9f0 0x7f69e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:28.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.073+0000 7f69f2d9d640 1 -- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f40cb9f0 msgr2=0x7f69e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:28.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.073+0000 7f69f2d9d640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f40cb9f0 0x7f69e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:28.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.073+0000 7f69f2d9d640 1 -- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69e0003970 msgr2=0x7f69e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:28.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.073+0000 7f69f2d9d640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69e0003970 0x7f69e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:28.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.073+0000 7f69f2d9d640 1 -- 172.21.15.16:0/520498383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e0003520 con 0x7f69e0004d00 2023-11-13T11:01:28.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.074+0000 7f69f2d9d640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69e0005180 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00a760 tx=0x7f69e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd115a13a81f66f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:28.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.074+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec034630 con 0x7f69e0004d00 2023-11-13T11:01:28.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.074+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec02f070 con 0x7f69e0004d00 2023-11-13T11:01:28.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.074+0000 7f69f1d9b640 1 -- 172.21.15.16:0/520498383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec0349f0 con 0x7f69e0004d00 2023-11-13T11:01:28.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.075+0000 7f69f359e640 1 -- 172.21.15.16:0/520498383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 msgr2=0x7f69e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:28.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.075+0000 7f69f359e640 1 --2- 172.21.15.16:0/520498383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69e0005180 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00a760 tx=0x7f69e00057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:28.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.075+0000 7f69f2d9d640 1 -- 172.21.15.16:0/520498383 reap_dead start 2023-11-13T11:01:28.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.075+0000 7f69f359e640 1 -- 172.21.15.16:0/520498383 shutdown_connections 2023-11-13T11:01:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.075+0000 7f69f359e640 1 -- 172.21.15.16:0/520498383 >> 172.21.15.16:0/520498383 conn(0x7f69f40b0cf0 msgr2=0x7f69f40b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.075+0000 7f69f359e640 1 -- 172.21.15.16:0/520498383 shutdown_connections 2023-11-13T11:01:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.075+0000 7f69f359e640 1 -- 172.21.15.16:0/520498383 wait complete. 2023-11-13T11:01:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.077+0000 7f69f359e640 1 Processor -- start 2023-11-13T11:01:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.077+0000 7f69f359e640 1 -- start start 2023-11-13T11:01:28.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.078+0000 7f69f359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69f414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.078+0000 7f69f359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f415de00 0x7f69f41f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.078+0000 7f69f359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f415e260 0x7f69f41f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.078+0000 7f69f359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f415ab10 con 0x7f69e0004d00 2023-11-13T11:01:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.078+0000 7f69f359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f4071700 con 0x7f69f415de00 2023-11-13T11:01:28.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.078+0000 7f69f359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40719e0 con 0x7f69f415e260 2023-11-13T11:01:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.078+0000 7f69f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69f414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.078+0000 7f69f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69f414fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49272/0 (socket says 172.21.15.16:49272) 2023-11-13T11:01:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.078+0000 7f69f259c640 1 -- 172.21.15.16:0/117314838 learned_addr learned my addr 172.21.15.16:0/117314838 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69ebfff640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f415de00 0x7f69f41f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69f2d9d640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f415e260 0x7f69f41f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:28.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69f259c640 1 -- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f415e260 msgr2=0x7f69f41f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69f259c640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f415e260 0x7f69f41f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69f259c640 1 -- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f415de00 msgr2=0x7f69f41f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69f259c640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f415de00 0x7f69f41f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69f259c640 1 -- 172.21.15.16:0/117314838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f415a450 con 0x7f69e0004d00 2023-11-13T11:01:28.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69ebfff640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f415de00 0x7f69f41f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:28.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69f259c640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69f414fc90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f69e4011540 tx=0x7f69e4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:28.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69eaffd640 1 -- 172.21.15.16:0/117314838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4015930 con 0x7f69e0004d00 2023-11-13T11:01:28.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69eaffd640 1 -- 172.21.15.16:0/117314838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e4015a90 con 0x7f69e0004d00 2023-11-13T11:01:28.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.079+0000 7f69eaffd640 1 -- 172.21.15.16:0/117314838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4015e20 con 0x7f69e0004d00 2023-11-13T11:01:28.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.080+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f415a730 con 0x7f69e0004d00 2023-11-13T11:01:28.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.080+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f40701e0 con 0x7f69e0004d00 2023-11-13T11:01:28.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.081+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f415a8c0 con 0x7f69e0004d00 2023-11-13T11:01:28.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.082+0000 7f69eaffd640 1 -- 172.21.15.16:0/117314838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69e400f460 con 0x7f69e0004d00 2023-11-13T11:01:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.083+0000 7f69eaffd640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f69d406ae70 0x7f69d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.083+0000 7f69eaffd640 1 -- 172.21.15.16:0/117314838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f69e4089bf0 con 0x7f69e0004d00 2023-11-13T11:01:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.084+0000 7f69ebfff640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f69d406ae70 0x7f69d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.085+0000 7f69ebfff640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f69d406ae70 0x7f69d406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f69f406cb60 tx=0x7f69dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:28.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.088+0000 7f69eaffd640 1 -- 172.21.15.16:0/117314838 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69e4054e60 con 0x7f69e0004d00 2023-11-13T11:01:28.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:28.360+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f69f4071b50 con 0x7f69e0004d00 2023-11-13T11:01:29.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.344+0000 7f69eaffd640 1 -- 172.21.15.16:0/117314838 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v268) v1 ==== 148+0+0 (secure 0 0 0) 0x7f69e4058480 con 0x7f69e0004d00 2023-11-13T11:01:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f69d406ae70 msgr2=0x7f69d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f69d406ae70 0x7f69d406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f69f406cb60 tx=0x7f69dc008040 comp rx=0 tx=0).stop 2023-11-13T11:01:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 msgr2=0x7f69f414fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69f414fc90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f69e4011540 tx=0x7f69e4011a00 comp rx=0 tx=0).stop 2023-11-13T11:01:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 shutdown_connections 2023-11-13T11:01:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f69d406ae70 0x7f69d406d330 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69f415e260 0x7f69f41f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69f415de00 0x7f69f41f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 --2- 172.21.15.16:0/117314838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0004d00 0x7f69f414fc90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.349+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 >> 172.21.15.16:0/117314838 conn(0x7f69f40b0cf0 msgr2=0x7f69f40b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.350+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 shutdown_connections 2023-11-13T11:01:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.350+0000 7f69f359e640 1 -- 172.21.15.16:0/117314838 wait complete. 2023-11-13T11:01:29.375 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2023-11-13T11:01:29.376 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-13T11:01:29.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:01:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.606+0000 7f477db2c640 1 Processor -- start 2023-11-13T11:01:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.606+0000 7f477db2c640 1 -- start start 2023-11-13T11:01:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.608+0000 7f477db2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47780ccc60 0x7f4778071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.608+0000 7f477db2c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47780cd110 con 0x7f4778071980 2023-11-13T11:01:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.608+0000 7f477db2c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477815b500 con 0x7f4778155410 2023-11-13T11:01:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.608+0000 7f477db2c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4778071e10 con 0x7f47780ccc60 2023-11-13T11:01:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.608+0000 7f4776ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47780ccc60 0x7f4778071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.608+0000 7f4776ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47780ccc60 0x7f4778071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49282/0 (socket says 172.21.15.16:49282) 2023-11-13T11:01:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.608+0000 7f4776ffd640 1 -- 172.21.15.16:0/4288531444 learned_addr learned my addr 172.21.15.16:0/4288531444 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.609+0000 7f4776ffd640 1 -- 172.21.15.16:0/4288531444 >> v1:172.21.15.184:6789/0 conn(0x7f4778155410 legacy=0x7f47780ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.609+0000 7f4776ffd640 1 -- 172.21.15.16:0/4288531444 >> v1:172.21.15.16:6789/0 conn(0x7f4778071980 legacy=0x7f477806fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.609+0000 7f4776ffd640 1 -- 172.21.15.16:0/4288531444 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477815b7e0 con 0x7f47780ccc60 2023-11-13T11:01:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.610+0000 7f4776ffd640 1 --2- 172.21.15.16:0/4288531444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780ccc60 0x7f4778071440 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f476c009870 tx=0x7f476c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270d058df39bf067 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.610+0000 7f47767fc640 1 -- 172.21.15.16:0/4288531444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c030860 con 0x7f47780ccc60 2023-11-13T11:01:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.610+0000 7f47767fc640 1 -- 172.21.15.16:0/4288531444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c0309c0 con 0x7f47780ccc60 2023-11-13T11:01:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.610+0000 7f47767fc640 1 -- 172.21.15.16:0/4288531444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c030d50 con 0x7f47780ccc60 2023-11-13T11:01:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.611+0000 7f477db2c640 1 -- 172.21.15.16:0/4288531444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780ccc60 msgr2=0x7f4778071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.611+0000 7f477db2c640 1 --2- 172.21.15.16:0/4288531444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780ccc60 0x7f4778071440 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f476c009870 tx=0x7f476c02b9a0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.611+0000 7f477db2c640 1 -- 172.21.15.16:0/4288531444 shutdown_connections 2023-11-13T11:01:29.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.611+0000 7f477db2c640 1 --2- 172.21.15.16:0/4288531444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47780ccc60 0x7f4778071440 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.611+0000 7f477db2c640 1 -- 172.21.15.16:0/4288531444 >> 172.21.15.16:0/4288531444 conn(0x7f477806a190 msgr2=0x7f477806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:29.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.611+0000 7f477db2c640 1 -- 172.21.15.16:0/4288531444 shutdown_connections 2023-11-13T11:01:29.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.611+0000 7f477db2c640 1 -- 172.21.15.16:0/4288531444 wait complete. 2023-11-13T11:01:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.613+0000 7f477db2c640 1 Processor -- start 2023-11-13T11:01:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.613+0000 7f477db2c640 1 -- start start 2023-11-13T11:01:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.614+0000 7f477db2c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778071980 0x7f477814c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.614+0000 7f477db2c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47780ccc60 0x7f4778151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.614+0000 7f477db2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4778155410 0x7f4778151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.614+0000 7f477db2c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4778157b80 con 0x7f4778155410 2023-11-13T11:01:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.614+0000 7f477db2c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4778156c20 con 0x7f4778071980 2023-11-13T11:01:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.614+0000 7f477db2c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4778156f00 con 0x7f47780ccc60 2023-11-13T11:01:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4777fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4778155410 0x7f4778151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4777fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4778155410 0x7f4778151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49296/0 (socket says 172.21.15.16:49296) 2023-11-13T11:01:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4777fff640 1 -- 172.21.15.16:0/2476201686 learned_addr learned my addr 172.21.15.16:0/2476201686 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:29.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f47777fe640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778071980 0x7f477814c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:29.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4776ffd640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47780ccc60 0x7f4778151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:29.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4777fff640 1 -- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47780ccc60 msgr2=0x7f4778151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:29.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4777fff640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47780ccc60 0x7f4778151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4777fff640 1 -- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778071980 msgr2=0x7f477814c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:29.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4777fff640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778071980 0x7f477814c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4777fff640 1 -- 172.21.15.16:0/2476201686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0094d0 con 0x7f4778155410 2023-11-13T11:01:29.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.615+0000 7f4777fff640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4778155410 0x7f4778151540 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f477000c7a0 tx=0x7f477000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:29.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.616+0000 7f475ffff640 1 -- 172.21.15.16:0/2476201686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770003b70 con 0x7f4778155410 2023-11-13T11:01:29.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.616+0000 7f475ffff640 1 -- 172.21.15.16:0/2476201686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770003cf0 con 0x7f4778155410 2023-11-13T11:01:29.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.616+0000 7f475ffff640 1 -- 172.21.15.16:0/2476201686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000aa90 con 0x7f4778155410 2023-11-13T11:01:29.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.616+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47781571e0 con 0x7f4778155410 2023-11-13T11:01:29.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.616+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47781577a0 con 0x7f4778155410 2023-11-13T11:01:29.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.617+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4778067090 con 0x7f4778155410 2023-11-13T11:01:29.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.618+0000 7f475ffff640 1 -- 172.21.15.16:0/2476201686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f477000abf0 con 0x7f4778155410 2023-11-13T11:01:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.619+0000 7f475ffff640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f476406ae40 0x7f476406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.619+0000 7f475ffff640 1 -- 172.21.15.16:0/2476201686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4770089350 con 0x7f4778155410 2023-11-13T11:01:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.620+0000 7f47777fe640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f476406ae40 0x7f476406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:29.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.621+0000 7f47777fe640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f476406ae40 0x7f476406d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4768006fd0 tx=0x7f4768008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.623+0000 7f475ffff640 1 -- 172.21.15.16:0/2476201686 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4770054520 con 0x7f4778155410 2023-11-13T11:01:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.816+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47780ae1d0 con 0x7f4778155410 2023-11-13T11:01:29.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.818+0000 7f475ffff640 1 -- 172.21.15.16:0/2476201686 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13205 (secure 0 0 0) 0x7f4770003e60 con 0x7f4778155410 2023-11-13T11:01:29.823 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:29.823 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:01:27.335233+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6837/3171288994":"2023-11-14T11:01:24.325039+0000","172.21.15.184:6835/3171288994":"2023-11-14T11:01:24.325039+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:01:29.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f476406ae40 msgr2=0x7f476406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:29.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f476406ae40 0x7f476406d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4768006fd0 tx=0x7f4768008040 comp rx=0 tx=0).stop 2023-11-13T11:01:29.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4778155410 msgr2=0x7f4778151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:29.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4778155410 0x7f4778151540 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f477000c7a0 tx=0x7f477000cc60 comp rx=0 tx=0).stop 2023-11-13T11:01:29.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 shutdown_connections 2023-11-13T11:01:29.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f476406ae40 0x7f476406d300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4778155410 0x7f4778151540 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47780ccc60 0x7f4778151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 --2- 172.21.15.16:0/2476201686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778071980 0x7f477814c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:29.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 >> 172.21.15.16:0/2476201686 conn(0x7f477806a190 msgr2=0x7f477815f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:29.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 shutdown_connections 2023-11-13T11:01:29.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:29.821+0000 7f477db2c640 1 -- 172.21.15.16:0/2476201686 wait complete. 2023-11-13T11:01:29.846 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:01:29.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:01:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.065+0000 7fe8ded2e640 1 Processor -- start 2023-11-13T11:01:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.066+0000 7fe8ded2e640 1 -- start start 2023-11-13T11:01:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.067+0000 7fe8ded2e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe8d815b8e0 0x7fe8d806c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.067+0000 7fe8ded2e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8d80ce260 0x7fe8d80cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.067+0000 7fe8ded2e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d806b860 con 0x7fe8d815bcc0 2023-11-13T11:01:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.067+0000 7fe8ded2e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d806bb40 con 0x7fe8d80ce260 2023-11-13T11:01:30.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.067+0000 7fe8ded2e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d806be20 con 0x7fe8d815b8e0 2023-11-13T11:01:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.067+0000 7fe8de52d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8d80ce260 0x7fe8d80cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.067+0000 7fe8dd52b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe8d815bcc0 0x7fe8d80cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49776/0 (socket says 172.21.15.16:49776) 2023-11-13T11:01:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.067+0000 7fe8dd52b640 1 -- 172.21.15.16:0/502937559 learned_addr learned my addr 172.21.15.16:0/502937559 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.068+0000 7fe8ddd2c640 1 --2- 172.21.15.16:0/502937559 >> v2:172.21.15.184:3300/0 conn(0x7fe8d815b8e0 0x7fe8d806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.068+0000 7fe8de52d640 1 -- 172.21.15.16:0/502937559 >> v2:172.21.15.184:3300/0 conn(0x7fe8d815b8e0 msgr2=0x7fe8d806c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.068+0000 7fe8de52d640 1 --2- 172.21.15.16:0/502937559 >> v2:172.21.15.184:3300/0 conn(0x7fe8d815b8e0 0x7fe8d806c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.068+0000 7fe8de52d640 1 -- 172.21.15.16:0/502937559 >> v1:172.21.15.16:6789/0 conn(0x7fe8d815bcc0 legacy=0x7fe8d80cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.068+0000 7fe8de52d640 1 -- 172.21.15.16:0/502937559 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d806c100 con 0x7fe8d80ce260 2023-11-13T11:01:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.069+0000 7fe8de52d640 1 --2- 172.21.15.16:0/502937559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d80ce260 0x7fe8d80cbae0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8071d70 tx=0x7fe8d002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9c36cbb535adc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.069+0000 7fe8dcd2a640 1 -- 172.21.15.16:0/502937559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d002cc70 con 0x7fe8d80ce260 2023-11-13T11:01:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.069+0000 7fe8dcd2a640 1 -- 172.21.15.16:0/502937559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d002cdd0 con 0x7fe8d80ce260 2023-11-13T11:01:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.069+0000 7fe8dcd2a640 1 -- 172.21.15.16:0/502937559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0034660 con 0x7fe8d80ce260 2023-11-13T11:01:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.070+0000 7fe8ded2e640 1 -- 172.21.15.16:0/502937559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d80ce260 msgr2=0x7fe8d80cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.070+0000 7fe8ded2e640 1 --2- 172.21.15.16:0/502937559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d80ce260 0x7fe8d80cbae0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8071d70 tx=0x7fe8d002bec0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.070+0000 7fe8ded2e640 1 -- 172.21.15.16:0/502937559 shutdown_connections 2023-11-13T11:01:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.070+0000 7fe8ded2e640 1 --2- 172.21.15.16:0/502937559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d80ce260 0x7fe8d80cbae0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.070+0000 7fe8ded2e640 1 --2- 172.21.15.16:0/502937559 >> v2:172.21.15.184:3300/0 conn(0x7fe8d815b8e0 0x7fe8d806c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.070+0000 7fe8ded2e640 1 -- 172.21.15.16:0/502937559 >> 172.21.15.16:0/502937559 conn(0x7fe8d80b0970 msgr2=0x7fe8d80b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.070+0000 7fe8ded2e640 1 -- 172.21.15.16:0/502937559 shutdown_connections 2023-11-13T11:01:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.070+0000 7fe8ded2e640 1 -- 172.21.15.16:0/502937559 wait complete. 2023-11-13T11:01:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.072+0000 7fe8ded2e640 1 Processor -- start 2023-11-13T11:01:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.072+0000 7fe8ded2e640 1 -- start start 2023-11-13T11:01:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.073+0000 7fe8ded2e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8d80ce260 0x7fe8d80713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.073+0000 7fe8ded2e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8d815b8e0 0x7fe8d8149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.073+0000 7fe8ded2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8153d00 0x7fe8d80b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.073+0000 7fe8ded2e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8071910 con 0x7fe8d8153d00 2023-11-13T11:01:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.073+0000 7fe8ded2e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8155230 con 0x7fe8d815b8e0 2023-11-13T11:01:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.073+0000 7fe8ded2e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8154190 con 0x7fe8d80ce260 2023-11-13T11:01:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.073+0000 7fe8de52d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8153d00 0x7fe8d80b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8de52d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8153d00 0x7fe8d80b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49316/0 (socket says 172.21.15.16:49316) 2023-11-13T11:01:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8de52d640 1 -- 172.21.15.16:0/3819399743 learned_addr learned my addr 172.21.15.16:0/3819399743 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8ddd2c640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8d80ce260 0x7fe8d80713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8dd52b640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8d815b8e0 0x7fe8d8149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8de52d640 1 -- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8d80ce260 msgr2=0x7fe8d80713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8de52d640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8d80ce260 0x7fe8d80713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8de52d640 1 -- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8d815b8e0 msgr2=0x7fe8d8149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8de52d640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8d815b8e0 0x7fe8d8149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8de52d640 1 -- 172.21.15.16:0/3819399743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d0009a70 con 0x7fe8d8153d00 2023-11-13T11:01:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8dd52b640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8d815b8e0 0x7fe8d8149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.074+0000 7fe8de52d640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8153d00 0x7fe8d80b2c40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009e80 tx=0x7fe8d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.075+0000 7fe8ce7fc640 1 -- 172.21.15.16:0/3819399743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0040650 con 0x7fe8d8153d00 2023-11-13T11:01:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.075+0000 7fe8ce7fc640 1 -- 172.21.15.16:0/3819399743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d00407d0 con 0x7fe8d8153d00 2023-11-13T11:01:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.075+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d8154470 con 0x7fe8d8153d00 2023-11-13T11:01:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.075+0000 7fe8ce7fc640 1 -- 172.21.15.16:0/3819399743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0040b80 con 0x7fe8d8153d00 2023-11-13T11:01:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.075+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d8154a30 con 0x7fe8d8153d00 2023-11-13T11:01:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.076+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d804e4d0 con 0x7fe8d8153d00 2023-11-13T11:01:30.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.077+0000 7fe8ce7fc640 1 -- 172.21.15.16:0/3819399743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe8d0040ce0 con 0x7fe8d8153d00 2023-11-13T11:01:30.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.078+0000 7fe8ce7fc640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8bc06ae70 0x7fe8bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.078+0000 7fe8ce7fc640 1 -- 172.21.15.16:0/3819399743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe8d00ac2c0 con 0x7fe8d8153d00 2023-11-13T11:01:30.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.078+0000 7fe8ddd2c640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8bc06ae70 0x7fe8bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.080+0000 7fe8ddd2c640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8bc06ae70 0x7fe8bc06d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4006fd0 tx=0x7fe8d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:30.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.082+0000 7fe8ce7fc640 1 -- 172.21.15.16:0/3819399743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8d0077490 con 0x7fe8d8153d00 2023-11-13T11:01:30.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.276+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8d80b3c10 con 0x7fe8d8153d00 2023-11-13T11:01:30.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.276+0000 7fe8ce7fc640 1 -- 172.21.15.16:0/3819399743 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8d007aab0 con 0x7fe8d8153d00 2023-11-13T11:01:30.281 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:01:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8bc06ae70 msgr2=0x7fe8bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8bc06ae70 0x7fe8bc06d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4006fd0 tx=0x7fe8d4008040 comp rx=0 tx=0).stop 2023-11-13T11:01:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8153d00 msgr2=0x7fe8d80b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8153d00 0x7fe8d80b2c40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009e80 tx=0x7fe8d0004a40 comp rx=0 tx=0).stop 2023-11-13T11:01:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 shutdown_connections 2023-11-13T11:01:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe8bc06ae70 0x7fe8bc06d330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8153d00 0x7fe8d80b2c40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8d815b8e0 0x7fe8d8149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 --2- 172.21.15.16:0/3819399743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8d80ce260 0x7fe8d80713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 >> 172.21.15.16:0/3819399743 conn(0x7fe8d80b0970 msgr2=0x7fe8d80b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 shutdown_connections 2023-11-13T11:01:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.280+0000 7fe8ded2e640 1 -- 172.21.15.16:0/3819399743 wait complete. 2023-11-13T11:01:30.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2023-11-13T11:01:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.515+0000 7f4835ebb640 1 Processor -- start 2023-11-13T11:01:30.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.516+0000 7f4835ebb640 1 -- start start 2023-11-13T11:01:30.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.517+0000 7f4835ebb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4830070cf0 0x7f48300b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.517+0000 7f4835ebb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483006fc50 con 0x7f483006a8a0 2023-11-13T11:01:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.518+0000 7f4835ebb640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483006ff30 con 0x7f483015cc90 2023-11-13T11:01:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.518+0000 7f4835ebb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4830070210 con 0x7f4830070cf0 2023-11-13T11:01:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.518+0000 7f482ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f483006a8a0 0x7f48300b2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49784/0 (socket says 172.21.15.16:49784) 2023-11-13T11:01:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.518+0000 7f482ffff640 1 -- 172.21.15.16:0/3973419231 learned_addr learned my addr 172.21.15.16:0/3973419231 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.518+0000 7f48356ba640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4830070cf0 0x7f48300b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.518+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025964423 0 0) 0x7f483006fc50 con 0x7f483006a8a0 2023-11-13T11:01:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.518+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4820003520 con 0x7f483006a8a0 2023-11-13T11:01:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2630526907 0 0) 0x7f4820003520 con 0x7f483006a8a0 2023-11-13T11:01:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4830070cf0 msgr2=0x7f48300b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4830070cf0 0x7f48300b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 >> v1:172.21.15.184:6789/0 conn(0x7f483015cc90 legacy=0x7f483006a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48300704f0 con 0x7f483006a8a0 2023-11-13T11:01:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f481c001670 con 0x7f483006a8a0 2023-11-13T11:01:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 >> v1:172.21.15.16:6789/0 conn(0x7f483006a8a0 legacy=0x7f48300b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48200039a0 0x7f4820003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48200047c0 0x7f4820004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4830070cf0 0x7f4820005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4830070210 con 0x7f48200047c0 2023-11-13T11:01:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482000ba30 con 0x7f48200039a0 2023-11-13T11:01:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482000bba0 con 0x7f4830070cf0 2023-11-13T11:01:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.519+0000 7f482ffff640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48200047c0 0x7f4820004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.520+0000 7f48356ba640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4830070cf0 0x7f4820005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.520+0000 7f4834eb9640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48200039a0 0x7f4820003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.520+0000 7f482ffff640 1 -- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4830070cf0 msgr2=0x7f4820005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.520+0000 7f482ffff640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4830070cf0 0x7f4820005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.520+0000 7f482ffff640 1 -- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48200039a0 msgr2=0x7f4820003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.520+0000 7f482ffff640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48200039a0 0x7f4820003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.520+0000 7f482ffff640 1 -- 172.21.15.16:0/3973419231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4820003520 con 0x7f48200047c0 2023-11-13T11:01:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.520+0000 7f4834eb9640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48200039a0 0x7f4820003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.521+0000 7f482ffff640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48200047c0 0x7f4820004c40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f481c00c160 tx=0x7f48200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bbd79c13a3eca5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.521+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c005ad0 con 0x7f48200047c0 2023-11-13T11:01:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.521+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c005c30 con 0x7f48200047c0 2023-11-13T11:01:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.521+0000 7f482f7fe640 1 -- 172.21.15.16:0/3973419231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c02c9f0 con 0x7f48200047c0 2023-11-13T11:01:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.522+0000 7f4835ebb640 1 -- 172.21.15.16:0/3973419231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48200047c0 msgr2=0x7f4820004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.522+0000 7f4835ebb640 1 --2- 172.21.15.16:0/3973419231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48200047c0 0x7f4820004c40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f481c00c160 tx=0x7f48200057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.522+0000 7f48356ba640 1 -- 172.21.15.16:0/3973419231 reap_dead start 2023-11-13T11:01:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.522+0000 7f4835ebb640 1 -- 172.21.15.16:0/3973419231 shutdown_connections 2023-11-13T11:01:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.522+0000 7f4835ebb640 1 -- 172.21.15.16:0/3973419231 >> 172.21.15.16:0/3973419231 conn(0x7f48300b04a0 msgr2=0x7f48300b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.522+0000 7f4835ebb640 1 -- 172.21.15.16:0/3973419231 shutdown_connections 2023-11-13T11:01:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.522+0000 7f4835ebb640 1 -- 172.21.15.16:0/3973419231 wait complete. 2023-11-13T11:01:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.524+0000 7f4835ebb640 1 Processor -- start 2023-11-13T11:01:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.524+0000 7f4835ebb640 1 -- start start 2023-11-13T11:01:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.525+0000 7f4835ebb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48200047c0 0x7f4830145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.525+0000 7f4835ebb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4830158ba0 0x7f48300c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.525+0000 7f4835ebb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48300c8320 0x7f48300c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.525+0000 7f4835ebb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48300c8d10 con 0x7f48300c8320 2023-11-13T11:01:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.525+0000 7f4835ebb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48300c4a00 con 0x7f4830158ba0 2023-11-13T11:01:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.525+0000 7f4835ebb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48300c4ce0 con 0x7f48200047c0 2023-11-13T11:01:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.525+0000 7f48356ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48300c8320 0x7f48300c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.525+0000 7f48356ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48300c8320 0x7f48300c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49330/0 (socket says 172.21.15.16:49330) 2023-11-13T11:01:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.525+0000 7f48356ba640 1 -- 172.21.15.16:0/2123604050 learned_addr learned my addr 172.21.15.16:0/2123604050 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.526+0000 7f482ffff640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4830158ba0 0x7f48300c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.526+0000 7f4834eb9640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48200047c0 0x7f4830145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.526+0000 7f48356ba640 1 -- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48200047c0 msgr2=0x7f4830145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.526+0000 7f48356ba640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48200047c0 0x7f4830145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.526+0000 7f48356ba640 1 -- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4830158ba0 msgr2=0x7f48300c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.526+0000 7f48356ba640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4830158ba0 0x7f48300c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.526+0000 7f48356ba640 1 -- 172.21.15.16:0/2123604050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48300c4fc0 con 0x7f48300c8320 2023-11-13T11:01:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.526+0000 7f48356ba640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48300c8320 0x7f48300c87d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f48300caa10 tx=0x7f48280115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.526+0000 7f482d7fa640 1 -- 172.21.15.16:0/2123604050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482800e070 con 0x7f48300c8320 2023-11-13T11:01:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.527+0000 7f482d7fa640 1 -- 172.21.15.16:0/2123604050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4828011e10 con 0x7f48300c8320 2023-11-13T11:01:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.527+0000 7f482d7fa640 1 -- 172.21.15.16:0/2123604050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4828003990 con 0x7f48300c8320 2023-11-13T11:01:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.527+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4830145ab0 con 0x7f48300c8320 2023-11-13T11:01:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.527+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4830145650 con 0x7f48300c8320 2023-11-13T11:01:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.528+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48300ada30 con 0x7f48300c8320 2023-11-13T11:01:30.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.529+0000 7f482d7fa640 1 -- 172.21.15.16:0/2123604050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4828003af0 con 0x7f48300c8320 2023-11-13T11:01:30.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.530+0000 7f482d7fa640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f481406ae70 0x7f481406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.530+0000 7f482d7fa640 1 -- 172.21.15.16:0/2123604050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4828089c40 con 0x7f48300c8320 2023-11-13T11:01:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.530+0000 7f4834eb9640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f481406ae70 0x7f481406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.532+0000 7f4834eb9640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f481406ae70 0x7f481406d330 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4824001860 tx=0x7f4824009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.534+0000 7f482d7fa640 1 -- 172.21.15.16:0/2123604050 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4828054e10 con 0x7f48300c8320 2023-11-13T11:01:30.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:30.795+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f4830145fd0 con 0x7f48300c8320 2023-11-13T11:01:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.358+0000 7f482d7fa640 1 -- 172.21.15.16:0/2123604050 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v270) v1 ==== 146+0+0 (secure 0 0 0) 0x7f4830145fd0 con 0x7f48300c8320 2023-11-13T11:01:31.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f481406ae70 msgr2=0x7f481406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:31.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f481406ae70 0x7f481406d330 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4824001860 tx=0x7f4824009040 comp rx=0 tx=0).stop 2023-11-13T11:01:31.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48300c8320 msgr2=0x7f48300c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:31.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48300c8320 0x7f48300c87d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f48300caa10 tx=0x7f48280115a0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 shutdown_connections 2023-11-13T11:01:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f481406ae70 0x7f481406d330 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48300c8320 0x7f48300c87d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4830158ba0 0x7f48300c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 --2- 172.21.15.16:0/2123604050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48200047c0 0x7f4830145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.361+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 >> 172.21.15.16:0/2123604050 conn(0x7f48300b04a0 msgr2=0x7f48300b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.362+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 shutdown_connections 2023-11-13T11:01:31.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.362+0000 7f4835ebb640 1 -- 172.21.15.16:0/2123604050 wait complete. 2023-11-13T11:01:31.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-13T11:01:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.621+0000 7fd785439640 1 Processor -- start 2023-11-13T11:01:31.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.622+0000 7fd785439640 1 -- start start 2023-11-13T11:01:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.623+0000 7fd785439640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd780159620 0x7fd7800cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.623+0000 7fd785439640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd7800b3920 0x7fd78006cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.624+0000 7fd785439640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7800ade20 con 0x7fd780159240 2023-11-13T11:01:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.624+0000 7fd785439640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd780159e90 con 0x7fd7800b3920 2023-11-13T11:01:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.624+0000 7fd785439640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800ae0c0 con 0x7fd780159620 2023-11-13T11:01:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.624+0000 7fd784c38640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd7800b3920 0x7fd78006cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.624+0000 7fd784c38640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd7800b3920 0x7fd78006cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:45832/0 (socket says 172.21.15.16:45832) 2023-11-13T11:01:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.624+0000 7fd784c38640 1 -- 172.21.15.16:0/3859697518 learned_addr learned my addr 172.21.15.16:0/3859697518 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.624+0000 7fd77f7fe640 1 --2- 172.21.15.16:0/3859697518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd780159620 0x7fd7800cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.625+0000 7fd784c38640 1 -- 172.21.15.16:0/3859697518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd780159620 msgr2=0x7fd7800cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.625+0000 7fd784c38640 1 --2- 172.21.15.16:0/3859697518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd780159620 0x7fd7800cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.625+0000 7fd784c38640 1 -- 172.21.15.16:0/3859697518 >> v1:172.21.15.184:6789/0 conn(0x7fd780159240 legacy=0x7fd78015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.625+0000 7fd784c38640 1 -- 172.21.15.16:0/3859697518 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780159ab0 con 0x7fd7800b3920 2023-11-13T11:01:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.626+0000 7fd784c38640 1 --2- 172.21.15.16:0/3859697518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7800b3920 0x7fd78006cc10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd77800d150 tx=0x7fd77802e0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c5e09c8d2331b0a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:31.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.627+0000 7fd77effd640 1 -- 172.21.15.16:0/3859697518 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77802ea80 con 0x7fd7800b3920 2023-11-13T11:01:31.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.627+0000 7fd77effd640 1 -- 172.21.15.16:0/3859697518 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77802ebe0 con 0x7fd7800b3920 2023-11-13T11:01:31.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.627+0000 7fd77effd640 1 -- 172.21.15.16:0/3859697518 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd778009600 con 0x7fd7800b3920 2023-11-13T11:01:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.628+0000 7fd785439640 1 -- 172.21.15.16:0/3859697518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7800b3920 msgr2=0x7fd78006cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.628+0000 7fd785439640 1 --2- 172.21.15.16:0/3859697518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7800b3920 0x7fd78006cc10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd77800d150 tx=0x7fd77802e0e0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.628+0000 7fd785439640 1 -- 172.21.15.16:0/3859697518 shutdown_connections 2023-11-13T11:01:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.628+0000 7fd785439640 1 --2- 172.21.15.16:0/3859697518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7800b3920 0x7fd78006cc10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.628+0000 7fd785439640 1 --2- 172.21.15.16:0/3859697518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd780159620 0x7fd7800cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.628+0000 7fd785439640 1 -- 172.21.15.16:0/3859697518 >> 172.21.15.16:0/3859697518 conn(0x7fd7800ce810 msgr2=0x7fd7800cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.628+0000 7fd785439640 1 -- 172.21.15.16:0/3859697518 shutdown_connections 2023-11-13T11:01:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.628+0000 7fd785439640 1 -- 172.21.15.16:0/3859697518 wait complete. 2023-11-13T11:01:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.630+0000 7fd785439640 1 Processor -- start 2023-11-13T11:01:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.631+0000 7fd785439640 1 -- start start 2023-11-13T11:01:31.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.632+0000 7fd785439640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7800b3920 0x7fd78014c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:31.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.632+0000 7fd785439640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780159240 0x7fd780155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:31.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.632+0000 7fd785439640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd78014d410 0x7fd78015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:31.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.632+0000 7fd785439640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78014d280 con 0x7fd780159240 2023-11-13T11:01:31.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.632+0000 7fd785439640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd780154990 con 0x7fd78014d410 2023-11-13T11:01:31.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.632+0000 7fd785439640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd780154c70 con 0x7fd7800b3920 2023-11-13T11:01:31.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.632+0000 7fd77f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780159240 0x7fd780155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:31.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780159240 0x7fd780155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49332/0 (socket says 172.21.15.16:49332) 2023-11-13T11:01:31.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77f7fe640 1 -- 172.21.15.16:0/183149322 learned_addr learned my addr 172.21.15.16:0/183149322 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:31.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd784c38640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd78014d410 0x7fd78015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:31.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77ffff640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7800b3920 0x7fd78014c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:31.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77f7fe640 1 -- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7800b3920 msgr2=0x7fd78014c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:31.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77f7fe640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7800b3920 0x7fd78014c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77f7fe640 1 -- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd78014d410 msgr2=0x7fd78015bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:31.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77f7fe640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd78014d410 0x7fd78015bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77f7fe640 1 -- 172.21.15.16:0/183149322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77800bd40 con 0x7fd780159240 2023-11-13T11:01:31.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd784c38640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd78014d410 0x7fd78015bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:31.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77ffff640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7800b3920 0x7fd78014c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:31.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77f7fe640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780159240 0x7fd780155750 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd77400e9b0 tx=0x7fd77400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:31.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.633+0000 7fd77cff9640 1 -- 172.21.15.16:0/183149322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774009670 con 0x7fd780159240 2023-11-13T11:01:31.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.634+0000 7fd77cff9640 1 -- 172.21.15.16:0/183149322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77400c9a0 con 0x7fd780159240 2023-11-13T11:01:31.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.634+0000 7fd77cff9640 1 -- 172.21.15.16:0/183149322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77400cd50 con 0x7fd780159240 2023-11-13T11:01:31.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.634+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd780154f50 con 0x7fd780159240 2023-11-13T11:01:31.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.634+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78014cbe0 con 0x7fd780159240 2023-11-13T11:01:31.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.635+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd780071c00 con 0x7fd780159240 2023-11-13T11:01:31.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.636+0000 7fd77cff9640 1 -- 172.21.15.16:0/183149322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd774013470 con 0x7fd780159240 2023-11-13T11:01:31.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.637+0000 7fd77cff9640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd76c06ae40 0x7fd76c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:31.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.637+0000 7fd77cff9640 1 -- 172.21.15.16:0/183149322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd7740897c0 con 0x7fd780159240 2023-11-13T11:01:31.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.638+0000 7fd77ffff640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd76c06ae40 0x7fd76c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:31.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.639+0000 7fd77ffff640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd76c06ae40 0x7fd76c06d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd7700015e0 tx=0x7fd770009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:31.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.641+0000 7fd77cff9640 1 -- 172.21.15.16:0/183149322 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd774054910 con 0x7fd780159240 2023-11-13T11:01:31.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.834+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd78004e4d0 con 0x7fd780159240 2023-11-13T11:01:31.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.839+0000 7fd77cff9640 1 -- 172.21.15.16:0/183149322 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd774057f30 con 0x7fd780159240 2023-11-13T11:01:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd76c06ae40 msgr2=0x7fd76c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd76c06ae40 0x7fd76c06d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd7700015e0 tx=0x7fd770009040 comp rx=0 tx=0).stop 2023-11-13T11:01:31.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780159240 msgr2=0x7fd780155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780159240 0x7fd780155750 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd77400e9b0 tx=0x7fd77400ee70 comp rx=0 tx=0).stop 2023-11-13T11:01:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 shutdown_connections 2023-11-13T11:01:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd76c06ae40 0x7fd76c06d300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd78014d410 0x7fd78015bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780159240 0x7fd780155750 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 --2- 172.21.15.16:0/183149322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7800b3920 0x7fd78014c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 >> 172.21.15.16:0/183149322 conn(0x7fd7800ce810 msgr2=0x7fd78015a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 shutdown_connections 2023-11-13T11:01:31.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:31.842+0000 7fd785439640 1 -- 172.21.15.16:0/183149322 wait complete. 2023-11-13T11:01:31.864 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-13T11:01:31.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:01:31.999 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:01:32.123 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-13T11:01:32.123 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:01:32.124 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:01:32.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:01:32.187 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:01:32.188 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:01:32.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:01:32.249 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:01:32.249 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:01:32.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:01:32.313 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:01:32.314 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:01:32.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:01:32.374 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-13T11:01:32.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:01:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.607+0000 7feeaffff640 1 Processor -- start 2023-11-13T11:01:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.607+0000 7feeaffff640 1 -- start start 2023-11-13T11:01:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.609+0000 7feeaffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feeb00cb190 0x7feeb015b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.609+0000 7feeaffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feeb006a190 0x7feeb006a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.609+0000 7feeaffff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb0158c20 con 0x7feeb00716b0 2023-11-13T11:01:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.609+0000 7feeaffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb006ab40 con 0x7feeb006a190 2023-11-13T11:01:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.609+0000 7feeaffff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb00b1830 con 0x7feeb00cb190 2023-11-13T11:01:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.610+0000 7feeaf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feeb006a190 0x7feeb006a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.610+0000 7feeaf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feeb006a190 0x7feeb006a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49338/0 (socket says 172.21.15.16:49338) 2023-11-13T11:01:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.610+0000 7feeaf7fe640 1 -- 172.21.15.16:0/3585685941 learned_addr learned my addr 172.21.15.16:0/3585685941 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.610+0000 7feeaeffd640 1 --2- 172.21.15.16:0/3585685941 >> v2:172.21.15.184:3300/0 conn(0x7feeb00cb190 0x7feeb015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.610+0000 7feeaf7fe640 1 -- 172.21.15.16:0/3585685941 >> v2:172.21.15.184:3300/0 conn(0x7feeb00cb190 msgr2=0x7feeb015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.610+0000 7feeaf7fe640 1 --2- 172.21.15.16:0/3585685941 >> v2:172.21.15.184:3300/0 conn(0x7feeb00cb190 0x7feeb015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.610+0000 7feeaf7fe640 1 -- 172.21.15.16:0/3585685941 >> v1:172.21.15.184:6789/0 conn(0x7feeb00716b0 legacy=0x7feeb006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.610+0000 7feeaf7fe640 1 -- 172.21.15.16:0/3585685941 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb015b780 con 0x7feeb006a190 2023-11-13T11:01:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.610+0000 7feeaeffd640 1 --2- 172.21.15.16:0/3585685941 >> v2:172.21.15.184:3300/0 conn(0x7feeb00cb190 0x7feeb015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.611+0000 7feeaf7fe640 1 --2- 172.21.15.16:0/3585685941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb006a190 0x7feeb006a600 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7feeb00b1560 tx=0x7feea802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8406b6c51979f6ea server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeadffb640 1 -- 172.21.15.16:0/3585685941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea802ca80 con 0x7feeb006a190 2023-11-13T11:01:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeadffb640 1 -- 172.21.15.16:0/3585685941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea802cbe0 con 0x7feeb006a190 2023-11-13T11:01:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeadffb640 1 -- 172.21.15.16:0/3585685941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80343f0 con 0x7feeb006a190 2023-11-13T11:01:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeaffff640 1 -- 172.21.15.16:0/3585685941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb006a190 msgr2=0x7feeb006a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeaffff640 1 --2- 172.21.15.16:0/3585685941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb006a190 0x7feeb006a600 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7feeb00b1560 tx=0x7feea802bee0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeaffff640 1 -- 172.21.15.16:0/3585685941 shutdown_connections 2023-11-13T11:01:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeaffff640 1 --2- 172.21.15.16:0/3585685941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb006a190 0x7feeb006a600 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeaffff640 1 --2- 172.21.15.16:0/3585685941 >> v2:172.21.15.184:3300/0 conn(0x7feeb00cb190 0x7feeb015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeaffff640 1 -- 172.21.15.16:0/3585685941 >> 172.21.15.16:0/3585685941 conn(0x7feeb00b04a0 msgr2=0x7feeb00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeaffff640 1 -- 172.21.15.16:0/3585685941 shutdown_connections 2023-11-13T11:01:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.612+0000 7feeaffff640 1 -- 172.21.15.16:0/3585685941 wait complete. 2023-11-13T11:01:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.614+0000 7feeaffff640 1 Processor -- start 2023-11-13T11:01:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.614+0000 7feeaffff640 1 -- start start 2023-11-13T11:01:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.615+0000 7feeaffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb006a190 0x7feeb0156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.615+0000 7feeaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb00716b0 0x7feeb006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:32.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.615+0000 7feeaffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb00cb190 0x7feeb006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:32.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.615+0000 7feeaffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb006c820 con 0x7feeb00716b0 2023-11-13T11:01:32.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.615+0000 7feeaffff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb006cb00 con 0x7feeb006a190 2023-11-13T11:01:32.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.615+0000 7feeaffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb006cde0 con 0x7feeb00cb190 2023-11-13T11:01:32.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.616+0000 7feeae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb00716b0 0x7feeb006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:32.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.616+0000 7feeae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb00716b0 0x7feeb006bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49348/0 (socket says 172.21.15.16:49348) 2023-11-13T11:01:32.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.616+0000 7feeae7fc640 1 -- 172.21.15.16:0/345812292 learned_addr learned my addr 172.21.15.16:0/345812292 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:32.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.616+0000 7feeae7fc640 1 -- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb00cb190 msgr2=0x7feeb006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:32.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.616+0000 7feeae7fc640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb00cb190 0x7feeb006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.616+0000 7feeae7fc640 1 -- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb006a190 msgr2=0x7feeb0156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.616+0000 7feeae7fc640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb006a190 0x7feeb0156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.616+0000 7feeae7fc640 1 -- 172.21.15.16:0/345812292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8009b30 con 0x7feeb00716b0 2023-11-13T11:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.616+0000 7feeae7fc640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb00716b0 0x7feeb006bcb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7feea400c410 tx=0x7feea400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.617+0000 7fee977fe640 1 -- 172.21.15.16:0/345812292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea40035f0 con 0x7feeb00716b0 2023-11-13T11:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.617+0000 7fee977fe640 1 -- 172.21.15.16:0/345812292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea4003750 con 0x7feeb00716b0 2023-11-13T11:01:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.617+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb006d0c0 con 0x7feeb00716b0 2023-11-13T11:01:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.617+0000 7fee977fe640 1 -- 172.21.15.16:0/345812292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4003b40 con 0x7feeb00716b0 2023-11-13T11:01:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.617+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb00a8780 con 0x7feeb00716b0 2023-11-13T11:01:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.618+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb006d5e0 con 0x7feeb00716b0 2023-11-13T11:01:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.619+0000 7fee977fe640 1 -- 172.21.15.16:0/345812292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feea4003ca0 con 0x7feeb00716b0 2023-11-13T11:01:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.620+0000 7fee977fe640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fee9c06ae40 0x7fee9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.620+0000 7fee977fe640 1 -- 172.21.15.16:0/345812292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7feea4088b70 con 0x7feeb00716b0 2023-11-13T11:01:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.621+0000 7feeaeffd640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fee9c06ae40 0x7fee9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.622+0000 7feeaeffd640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fee9c06ae40 0x7fee9c06d300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feea0006fd0 tx=0x7feea0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.624+0000 7fee977fe640 1 -- 172.21.15.16:0/345812292 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feeb006d5e0 con 0x7feeb00716b0 2023-11-13T11:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.855+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feeb006d5e0 con 0x7feeb00716b0 2023-11-13T11:01:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.856+0000 7fee977fe640 1 -- 172.21.15.16:0/345812292 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+3705 (secure 0 0 0) 0x7feeb006d5e0 con 0x7feeb00716b0 2023-11-13T11:01:32.860 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:32.860 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3678615052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3678615052},{"type":"v1","addr":"172.21.15.16:6833","nonce":3678615052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3358463124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3358463124},{"type":"v1","addr":"172.21.15.16:6835","nonce":3358463124}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/2329286743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2329286743},{"type":"v1","addr":"172.21.15.184:6836","nonce":2329286743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3568899041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":3568899041},{"type":"v1","addr":"172.21.15.184:6837","nonce":3568899041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[]} 2023-11-13T11:01:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2023-11-13T11:01:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fee9c06ae40 msgr2=0x7fee9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fee9c06ae40 0x7fee9c06d300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feea0006fd0 tx=0x7feea0008040 comp rx=0 tx=0).stop 2023-11-13T11:01:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb00716b0 msgr2=0x7feeb006bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb00716b0 0x7feeb006bcb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7feea400c410 tx=0x7feea400c8d0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 shutdown_connections 2023-11-13T11:01:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fee9c06ae40 0x7fee9c06d300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb00cb190 0x7feeb006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb00716b0 0x7feeb006bcb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 --2- 172.21.15.16:0/345812292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb006a190 0x7feeb0156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 >> 172.21.15.16:0/345812292 conn(0x7feeb00b04a0 msgr2=0x7feeb01583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 shutdown_connections 2023-11-13T11:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:32.859+0000 7feeaffff640 1 -- 172.21.15.16:0/345812292 wait complete. 2023-11-13T11:01:32.884 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-13T11:01:32.884 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-13T11:01:32.885 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-13T11:01:32.885 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:01:32.885 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-13T11:01:32.886 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-13T11:01:32.886 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-13T11:01:32.886 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:01:32.887 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-13T11:01:32.887 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-13T11:01:32.887 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-13T11:01:32.887 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:01:32.888 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-13T11:01:32.888 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-13T11:01:32.888 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-13T11:01:32.889 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:01:32.902 INFO:tasks.ceph.mds.c:Stopped 2023-11-13T11:01:32.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-13T11:01:32.904 INFO:tasks.ceph.mds.a:Stopped 2023-11-13T11:01:32.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-13T11:01:32.912 INFO:tasks.ceph.mds.b:Stopped 2023-11-13T11:01:32.912 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-13T11:01:32.913 INFO:tasks.ceph.mds.d:Stopped 2023-11-13T11:01:32.913 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-13T11:01:32.953 INFO:tasks.ceph.mds.c:Started 2023-11-13T11:01:32.954 DEBUG:teuthology.parallel:result is None 2023-11-13T11:01:32.961 INFO:tasks.ceph.mds.b:Started 2023-11-13T11:01:32.962 INFO:tasks.ceph.mds.a:Started 2023-11-13T11:01:32.962 DEBUG:teuthology.parallel:result is None 2023-11-13T11:01:32.962 DEBUG:teuthology.parallel:result is None 2023-11-13T11:01:32.969 INFO:tasks.ceph.mds.d:Started 2023-11-13T11:01:32.970 DEBUG:teuthology.parallel:result is None 2023-11-13T11:01:32.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-13T11:01:33.119 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-13T11:01:33.124 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-11-13T11:01:33.125 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-13T11:01:33.128 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-11-13T11:01:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.163+0000 7f9d95333640 1 Processor -- start 2023-11-13T11:01:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.163+0000 7f9d95333640 1 -- start start 2023-11-13T11:01:33.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.165+0000 7f9d95333640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9d900cecc0 0x7f9d9006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:33.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.165+0000 7f9d95333640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d9006bb90 0x7f9d90070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:33.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.165+0000 7f9d95333640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900b2900 con 0x7f9d9015cf20 2023-11-13T11:01:33.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.165+0000 7f9d95333640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9015d300 con 0x7f9d9006bb90 2023-11-13T11:01:33.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.165+0000 7f9d95333640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900b1fd0 con 0x7f9d900cecc0 2023-11-13T11:01:33.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.165+0000 7f9d8f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d9006bb90 0x7f9d90070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.166+0000 7f9d8f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d9006bb90 0x7f9d90070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43106/0 (socket says 172.21.15.16:43106) 2023-11-13T11:01:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.166+0000 7f9d8f7fe640 1 -- 172.21.15.16:0/3215111963 learned_addr learned my addr 172.21.15.16:0/3215111963 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.166+0000 7f9d87fff640 1 --2- 172.21.15.16:0/3215111963 >> v2:172.21.15.184:3300/0 conn(0x7f9d900cecc0 0x7f9d9006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.166+0000 7f9d8f7fe640 1 -- 172.21.15.16:0/3215111963 >> v2:172.21.15.184:3300/0 conn(0x7f9d900cecc0 msgr2=0x7f9d9006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.166+0000 7f9d8f7fe640 1 --2- 172.21.15.16:0/3215111963 >> v2:172.21.15.184:3300/0 conn(0x7f9d900cecc0 0x7f9d9006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.166+0000 7f9d8f7fe640 1 -- 172.21.15.16:0/3215111963 >> v1:172.21.15.184:6789/0 conn(0x7f9d9015cf20 legacy=0x7f9d900ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.166+0000 7f9d8f7fe640 1 -- 172.21.15.16:0/3215111963 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900b22b0 con 0x7f9d9006bb90 2023-11-13T11:01:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.167+0000 7f9d8f7fe640 1 --2- 172.21.15.16:0/3215111963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9006bb90 0x7f9d90070330 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9d900cadd0 tx=0x7f9d8802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6306ee952dc5ea server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.167+0000 7f9d8e7fc640 1 -- 172.21.15.16:0/3215111963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8802c8d0 con 0x7f9d9006bb90 2023-11-13T11:01:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d8e7fc640 1 -- 172.21.15.16:0/3215111963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8802ca30 con 0x7f9d9006bb90 2023-11-13T11:01:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d8e7fc640 1 -- 172.21.15.16:0/3215111963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8802cdc0 con 0x7f9d9006bb90 2023-11-13T11:01:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d95333640 1 -- 172.21.15.16:0/3215111963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9006bb90 msgr2=0x7f9d90070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d95333640 1 --2- 172.21.15.16:0/3215111963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9006bb90 0x7f9d90070330 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9d900cadd0 tx=0x7f9d8802bdb0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d95333640 1 -- 172.21.15.16:0/3215111963 shutdown_connections 2023-11-13T11:01:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d95333640 1 --2- 172.21.15.16:0/3215111963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9006bb90 0x7f9d90070330 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d95333640 1 --2- 172.21.15.16:0/3215111963 >> v2:172.21.15.184:3300/0 conn(0x7f9d900cecc0 0x7f9d9006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d95333640 1 -- 172.21.15.16:0/3215111963 >> 172.21.15.16:0/3215111963 conn(0x7f9d9006a190 msgr2=0x7f9d9006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:33.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d95333640 1 -- 172.21.15.16:0/3215111963 shutdown_connections 2023-11-13T11:01:33.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.168+0000 7f9d95333640 1 -- 172.21.15.16:0/3215111963 wait complete. 2023-11-13T11:01:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.170+0000 7f9d95333640 1 Processor -- start 2023-11-13T11:01:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.170+0000 7f9d95333640 1 -- start start 2023-11-13T11:01:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.171+0000 7f9d95333640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9006bb90 0x7f9d9015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.171+0000 7f9d95333640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d900cecc0 0x7f9d9014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.171+0000 7f9d95333640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9015cf20 0x7f9d90152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.171+0000 7f9d95333640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90154b40 con 0x7f9d900cecc0 2023-11-13T11:01:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.171+0000 7f9d95333640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90160240 con 0x7f9d9006bb90 2023-11-13T11:01:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.171+0000 7f9d95333640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9015e420 con 0x7f9d9015cf20 2023-11-13T11:01:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.171+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d900cecc0 0x7f9d9014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.171+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d900cecc0 0x7f9d9014f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43116/0 (socket says 172.21.15.16:43116) 2023-11-13T11:01:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.171+0000 7f9d87fff640 1 -- 172.21.15.16:0/3545233882 learned_addr learned my addr 172.21.15.16:0/3545233882 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.172+0000 7f9d8f7fe640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9015cf20 0x7f9d90152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.172+0000 7f9d8effd640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9006bb90 0x7f9d9015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.172+0000 7f9d87fff640 1 -- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9015cf20 msgr2=0x7f9d90152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.172+0000 7f9d87fff640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9015cf20 0x7f9d90152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.172+0000 7f9d87fff640 1 -- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9006bb90 msgr2=0x7f9d9015fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.172+0000 7f9d87fff640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9006bb90 0x7f9d9015fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.172+0000 7f9d87fff640 1 -- 172.21.15.16:0/3545233882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009b30 con 0x7f9d900cecc0 2023-11-13T11:01:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.172+0000 7f9d8effd640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9006bb90 0x7f9d9015fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:01:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.172+0000 7f9d87fff640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d900cecc0 0x7f9d9014f5b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800e9b0 tx=0x7f9d7800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.173+0000 7f9d877fe640 1 -- 172.21.15.16:0/3545233882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78009670 con 0x7f9d900cecc0 2023-11-13T11:01:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.173+0000 7f9d877fe640 1 -- 172.21.15.16:0/3545233882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7800c9a0 con 0x7f9d900cecc0 2023-11-13T11:01:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.173+0000 7f9d877fe640 1 -- 172.21.15.16:0/3545233882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800cd50 con 0x7f9d900cecc0 2023-11-13T11:01:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.173+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9015e700 con 0x7f9d900cecc0 2023-11-13T11:01:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.173+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9015ecc0 con 0x7f9d900cecc0 2023-11-13T11:01:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.174+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d90154d40 con 0x7f9d900cecc0 2023-11-13T11:01:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.175+0000 7f9d877fe640 1 -- 172.21.15.16:0/3545233882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d78013470 con 0x7f9d900cecc0 2023-11-13T11:01:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.177+0000 7f9d877fe640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9d7006ae40 0x7f9d7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.177+0000 7f9d877fe640 1 -- 172.21.15.16:0/3545233882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f9d78089560 con 0x7f9d900cecc0 2023-11-13T11:01:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.178+0000 7f9d8effd640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9d7006ae40 0x7f9d7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.179+0000 7f9d8effd640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9d7006ae40 0x7f9d7006d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d80007fd0 tx=0x7f9d80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.181+0000 7f9d877fe640 1 -- 172.21.15.16:0/3545233882 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d90154d40 con 0x7f9d900cecc0 2023-11-13T11:01:33.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.387+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9d90154d40 con 0x7f9d900cecc0 2023-11-13T11:01:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.866+0000 7f9d877fe640 1 -- 172.21.15.16:0/3545233882 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9d90154d40 con 0x7f9d900cecc0 2023-11-13T11:01:33.870 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-11-13T11:01:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.870+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9d7006ae40 msgr2=0x7f9d7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.870+0000 7f9d95333640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9d7006ae40 0x7f9d7006d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d80007fd0 tx=0x7f9d80009040 comp rx=0 tx=0).stop 2023-11-13T11:01:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.870+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d900cecc0 msgr2=0x7f9d9014f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.870+0000 7f9d95333640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d900cecc0 0x7f9d9014f5b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800e9b0 tx=0x7f9d7800ee70 comp rx=0 tx=0).stop 2023-11-13T11:01:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.871+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 shutdown_connections 2023-11-13T11:01:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.871+0000 7f9d95333640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9d7006ae40 0x7f9d7006d300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.871+0000 7f9d95333640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d9015cf20 0x7f9d90152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.871+0000 7f9d95333640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d900cecc0 0x7f9d9014f5b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.871+0000 7f9d95333640 1 --2- 172.21.15.16:0/3545233882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d9006bb90 0x7f9d9015fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:33.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.871+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 >> 172.21.15.16:0/3545233882 conn(0x7f9d9006a190 msgr2=0x7f9d9006a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:33.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.871+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 shutdown_connections 2023-11-13T11:01:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:33.871+0000 7f9d95333640 1 -- 172.21.15.16:0/3545233882 wait complete. 2023-11-13T11:01:33.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-13T11:01:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.106+0000 7f3f37475640 1 Processor -- start 2023-11-13T11:01:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.106+0000 7f3f37475640 1 -- start start 2023-11-13T11:01:34.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.108+0000 7f3f37475640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30155080 0x7f3f30070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.108+0000 7f3f37475640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f30071680 0x7f3f300ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.108+0000 7f3f37475640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3006c070 con 0x7f3f3015e5d0 2023-11-13T11:01:34.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.108+0000 7f3f37475640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f300709c0 con 0x7f3f30071680 2023-11-13T11:01:34.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.108+0000 7f3f37475640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f30155460 con 0x7f3f30155080 2023-11-13T11:01:34.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.109+0000 7f3f36473640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30155080 0x7f3f30070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.109+0000 7f3f35c72640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3f30071680 0x7f3f300ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.109+0000 7f3f36c74640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f3f3015e5d0 0x7f3f3006c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:54536/0 (socket says 172.21.15.16:54536) 2023-11-13T11:01:34.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.109+0000 7f3f36c74640 1 -- 172.21.15.16:0/2564168259 learned_addr learned my addr 172.21.15.16:0/2564168259 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:34.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.109+0000 7f3f35471640 1 -- 172.21.15.16:0/2564168259 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254639606 0 0) 0x7f3f3006c070 con 0x7f3f3015e5d0 2023-11-13T11:01:34.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.110+0000 7f3f35471640 1 -- 172.21.15.16:0/2564168259 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f1c003520 con 0x7f3f3015e5d0 2023-11-13T11:01:34.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.110+0000 7f3f35c72640 1 -- 172.21.15.16:0/2564168259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30155080 msgr2=0x7f3f30070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:34.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.110+0000 7f3f35c72640 1 --2- 172.21.15.16:0/2564168259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30155080 0x7f3f30070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.110+0000 7f3f35c72640 1 -- 172.21.15.16:0/2564168259 >> v1:172.21.15.184:6789/0 conn(0x7f3f3015e5d0 legacy=0x7f3f3006c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.110+0000 7f3f35c72640 1 -- 172.21.15.16:0/2564168259 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3006b740 con 0x7f3f30071680 2023-11-13T11:01:34.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.111+0000 7f3f35c72640 1 --2- 172.21.15.16:0/2564168259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f30071680 0x7f3f300ccab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3f200097a0 tx=0x7f3f2002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea445830b0fda9a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:34.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.111+0000 7f3f35471640 1 -- 172.21.15.16:0/2564168259 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2002d040 con 0x7f3f30071680 2023-11-13T11:01:34.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f35471640 1 -- 172.21.15.16:0/2564168259 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f20031810 con 0x7f3f30071680 2023-11-13T11:01:34.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f35471640 1 -- 172.21.15.16:0/2564168259 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20031ba0 con 0x7f3f30071680 2023-11-13T11:01:34.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f37475640 1 -- 172.21.15.16:0/2564168259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f30071680 msgr2=0x7f3f300ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f37475640 1 --2- 172.21.15.16:0/2564168259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f30071680 0x7f3f300ccab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3f200097a0 tx=0x7f3f2002b8b0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f37475640 1 -- 172.21.15.16:0/2564168259 shutdown_connections 2023-11-13T11:01:34.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f37475640 1 --2- 172.21.15.16:0/2564168259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f30071680 0x7f3f300ccab0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f37475640 1 --2- 172.21.15.16:0/2564168259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30155080 0x7f3f30070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f37475640 1 -- 172.21.15.16:0/2564168259 >> 172.21.15.16:0/2564168259 conn(0x7f3f300b0580 msgr2=0x7f3f300b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:34.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f37475640 1 -- 172.21.15.16:0/2564168259 shutdown_connections 2023-11-13T11:01:34.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.112+0000 7f3f37475640 1 -- 172.21.15.16:0/2564168259 wait complete. 2023-11-13T11:01:34.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.114+0000 7f3f37475640 1 Processor -- start 2023-11-13T11:01:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.114+0000 7f3f37475640 1 -- start start 2023-11-13T11:01:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.115+0000 7f3f37475640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30071680 0x7f3f30141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.115+0000 7f3f37475640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f30155080 0x7f3f3014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.115+0000 7f3f37475640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3015e5d0 0x7f3f300b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.115+0000 7f3f37475640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f30150760 con 0x7f3f3015e5d0 2023-11-13T11:01:34.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.115+0000 7f3f37475640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f30150a40 con 0x7f3f30155080 2023-11-13T11:01:34.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.115+0000 7f3f37475640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f30150d20 con 0x7f3f30071680 2023-11-13T11:01:34.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36c74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3015e5d0 0x7f3f300b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36c74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3015e5d0 0x7f3f300b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43118/0 (socket says 172.21.15.16:43118) 2023-11-13T11:01:34.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36c74640 1 -- 172.21.15.16:0/364841542 learned_addr learned my addr 172.21.15.16:0/364841542 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:34.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f35c72640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f30155080 0x7f3f3014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36473640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30071680 0x7f3f30141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36c74640 1 -- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30071680 msgr2=0x7f3f30141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36c74640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30071680 0x7f3f30141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36c74640 1 -- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f30155080 msgr2=0x7f3f3014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36c74640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f30155080 0x7f3f3014f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36c74640 1 -- 172.21.15.16:0/364841542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f20009400 con 0x7f3f3015e5d0 2023-11-13T11:01:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.116+0000 7f3f36c74640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3015e5d0 0x7f3f300b2b00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800caa0 tx=0x7f3f2800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.117+0000 7f3f26ffd640 1 -- 172.21.15.16:0/364841542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2800ac00 con 0x7f3f3015e5d0 2023-11-13T11:01:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.117+0000 7f3f26ffd640 1 -- 172.21.15.16:0/364841542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2800ad60 con 0x7f3f3015e5d0 2023-11-13T11:01:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.117+0000 7f3f26ffd640 1 -- 172.21.15.16:0/364841542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28012620 con 0x7f3f3015e5d0 2023-11-13T11:01:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.117+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f30151000 con 0x7f3f3015e5d0 2023-11-13T11:01:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.117+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3014f610 con 0x7f3f3015e5d0 2023-11-13T11:01:34.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.118+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f30151520 con 0x7f3f3015e5d0 2023-11-13T11:01:34.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.119+0000 7f3f26ffd640 1 -- 172.21.15.16:0/364841542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f28007070 con 0x7f3f3015e5d0 2023-11-13T11:01:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.120+0000 7f3f26ffd640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f1406ae40 0x7f3f1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.120+0000 7f3f26ffd640 1 -- 172.21.15.16:0/364841542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3f280893f0 con 0x7f3f3015e5d0 2023-11-13T11:01:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.121+0000 7f3f36473640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f1406ae40 0x7f3f1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.122+0000 7f3f36473640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f1406ae40 0x7f3f1406d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009a10 tx=0x7f3f2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:34.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.124+0000 7f3f26ffd640 1 -- 172.21.15.16:0/364841542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f30151520 con 0x7f3f3015e5d0 2023-11-13T11:01:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.376+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3f30151520 con 0x7f3f3015e5d0 2023-11-13T11:01:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.377+0000 7f3f26ffd640 1 -- 172.21.15.16:0/364841542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3f30151520 con 0x7f3f3015e5d0 2023-11-13T11:01:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:34.382 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:01:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-11-13T11:01:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-11-13T11:01:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBcAFJlFf7FKBAAzl2CzFIHt1vexIHicDcMOQ==", 2023-11-13T11:01:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:01:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:01:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:01:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-11-13T11:01:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJlxksXERAApz8MD99vmb9+TVqjK4yoHQ==", 2023-11-13T11:01:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:01:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:01:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:01:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-11-13T11:01:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJl3f8tABAAn77Y5rtLX6KXfDt6O64jHw==", 2023-11-13T11:01:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:01:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:01:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:01:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-11-13T11:01:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJliC8UHxAAp4RpLcBhknvkfcD6+u0cWg==", 2023-11-13T11:01:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:01:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:01:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:01:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-11-13T11:01:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBgAFJl6OfsFRAA/pxN6JyHj0g/ZTvQTTBvHg==", 2023-11-13T11:01:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:01:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:01:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-11-13T11:01:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBjAFJlCcthIRAAD9rCdKqLeqILvx+RcVu3aA==", 2023-11-13T11:01:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:01:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:01:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-11-13T11:01:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBmAFJldeAFLRAA6BDzhw1Sml5AWptU6G7MEA==", 2023-11-13T11:01:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:01:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:01:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-11-13T11:01:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBqAFJlhz82DBAAmRhdLjHKXpAyNNqUkXk5qw==", 2023-11-13T11:01:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:01:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:01:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-11-13T11:01:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBxAFJlgXLmHxAAHpM0gY+C/Ionsp0sG9KoKQ==", 2023-11-13T11:01:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:01:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:01:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-11-13T11:01:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQB0AFJlfsdPKxAA5Ba0gTAcUVQ6PueL9Ul6qQ==", 2023-11-13T11:01:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:01:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:01:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-11-13T11:01:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQB4AFJlD6j5CBAAXuiAkwIbpfUSJ1j1j3QP4w==", 2023-11-13T11:01:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:01:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:01:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-11-13T11:01:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQB7AFJlkX9TIhAAPjuDlChUGXjej7ot5x6vDg==", 2023-11-13T11:01:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:01:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:01:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-11-13T11:01:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJl9p66LRAAYRThccLplmNpsr+W647dOA==", 2023-11-13T11:01:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:01:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-11-13T11:01:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-13T11:01:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-11-13T11:01:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBeAFJlkymOCRAATnUAZgjg1er9vu82gnrxtw==", 2023-11-13T11:01:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:01:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T11:01:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T11:01:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T11:01:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-11-13T11:01:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJlrk4zORAA7tCfwi0UlPhDcL1GZ6UIIg==", 2023-11-13T11:01:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:01:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T11:01:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T11:01:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T11:01:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-11-13T11:01:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBeAFJlrh6DFBAAtrGSamisl/Jexs9ewdiAkQ==", 2023-11-13T11:01:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:01:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T11:01:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T11:01:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T11:01:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-11-13T11:01:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBbAFJlhaZyGxAANZe05VzLnzM3KiFjOH93JA==", 2023-11-13T11:01:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-13T11:01:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:01:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:01:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-11-13T11:01:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBcAFJlvFoIHBAAIXc5UQJIKTtHmeVznGZPSA==", 2023-11-13T11:01:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-13T11:01:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-13T11:01:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-11-13T11:01:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBcAFJluuvTERAAn6zxosZEaeBkgENpZDAHqg==", 2023-11-13T11:01:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:01:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-13T11:01:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-13T11:01:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:01:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-11-13T11:01:34.418 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:01:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f1406ae40 msgr2=0x7f3f1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f1406ae40 0x7f3f1406d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009a10 tx=0x7f3f2c008040 comp rx=0 tx=0).stop 2023-11-13T11:01:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3015e5d0 msgr2=0x7f3f300b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3015e5d0 0x7f3f300b2b00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800caa0 tx=0x7f3f2800cf60 comp rx=0 tx=0).stop 2023-11-13T11:01:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 shutdown_connections 2023-11-13T11:01:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3f1406ae40 0x7f3f1406d300 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3015e5d0 0x7f3f300b2b00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f30155080 0x7f3f3014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 --2- 172.21.15.16:0/364841542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3f30071680 0x7f3f30141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 >> 172.21.15.16:0/364841542 conn(0x7f3f300b0580 msgr2=0x7f3f300cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 shutdown_connections 2023-11-13T11:01:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.380+0000 7f3f37475640 1 -- 172.21.15.16:0/364841542 wait complete. 2023-11-13T11:01:34.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:01:34.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.644+0000 7fcb73fff640 1 Processor -- start 2023-11-13T11:01:34.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.644+0000 7fcb73fff640 1 -- start start 2023-11-13T11:01:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.646+0000 7fcb73fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcb740cb190 0x7fcb7415b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.646+0000 7fcb73fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcb740716b0 0x7fcb7406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.646+0000 7fcb73fff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb74158c20 con 0x7fcb7406a190 2023-11-13T11:01:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.646+0000 7fcb73fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb740b1830 con 0x7fcb740716b0 2023-11-13T11:01:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.646+0000 7fcb73fff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7415b780 con 0x7fcb740cb190 2023-11-13T11:01:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.646+0000 7fcb727fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcb740716b0 0x7fcb7406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.646+0000 7fcb727fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcb740716b0 0x7fcb7406f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43126/0 (socket says 172.21.15.16:43126) 2023-11-13T11:01:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.646+0000 7fcb727fc640 1 -- 172.21.15.16:0/3343953518 learned_addr learned my addr 172.21.15.16:0/3343953518 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.647+0000 7fcb72ffd640 1 --2- 172.21.15.16:0/3343953518 >> v2:172.21.15.184:3300/0 conn(0x7fcb740cb190 0x7fcb7415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.647+0000 7fcb727fc640 1 -- 172.21.15.16:0/3343953518 >> v2:172.21.15.184:3300/0 conn(0x7fcb740cb190 msgr2=0x7fcb7415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.647+0000 7fcb727fc640 1 --2- 172.21.15.16:0/3343953518 >> v2:172.21.15.184:3300/0 conn(0x7fcb740cb190 0x7fcb7415b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.647+0000 7fcb727fc640 1 -- 172.21.15.16:0/3343953518 >> v1:172.21.15.184:6789/0 conn(0x7fcb7406a190 legacy=0x7fcb740b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.647+0000 7fcb727fc640 1 -- 172.21.15.16:0/3343953518 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7415ba60 con 0x7fcb740716b0 2023-11-13T11:01:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.648+0000 7fcb727fc640 1 --2- 172.21.15.16:0/3343953518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb740716b0 0x7fcb7406f970 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcb740b1560 tx=0x7fcb6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da87370c5c59855 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.648+0000 7fcb71ffb640 1 -- 172.21.15.16:0/3343953518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68031870 con 0x7fcb740716b0 2023-11-13T11:01:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.648+0000 7fcb71ffb640 1 -- 172.21.15.16:0/3343953518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb680319d0 con 0x7fcb740716b0 2023-11-13T11:01:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.648+0000 7fcb71ffb640 1 -- 172.21.15.16:0/3343953518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68031d60 con 0x7fcb740716b0 2023-11-13T11:01:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.649+0000 7fcb73fff640 1 -- 172.21.15.16:0/3343953518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb740716b0 msgr2=0x7fcb7406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.649+0000 7fcb73fff640 1 --2- 172.21.15.16:0/3343953518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb740716b0 0x7fcb7406f970 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcb740b1560 tx=0x7fcb6802b910 comp rx=0 tx=0).stop 2023-11-13T11:01:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.649+0000 7fcb73fff640 1 -- 172.21.15.16:0/3343953518 shutdown_connections 2023-11-13T11:01:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.649+0000 7fcb73fff640 1 --2- 172.21.15.16:0/3343953518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb740716b0 0x7fcb7406f970 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.649+0000 7fcb73fff640 1 --2- 172.21.15.16:0/3343953518 >> v2:172.21.15.184:3300/0 conn(0x7fcb740cb190 0x7fcb7415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.649+0000 7fcb73fff640 1 -- 172.21.15.16:0/3343953518 >> 172.21.15.16:0/3343953518 conn(0x7fcb740b04a0 msgr2=0x7fcb740b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:34.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.649+0000 7fcb73fff640 1 -- 172.21.15.16:0/3343953518 shutdown_connections 2023-11-13T11:01:34.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.649+0000 7fcb73fff640 1 -- 172.21.15.16:0/3343953518 wait complete. 2023-11-13T11:01:34.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.651+0000 7fcb73fff640 1 Processor -- start 2023-11-13T11:01:34.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.651+0000 7fcb73fff640 1 -- start start 2023-11-13T11:01:34.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.652+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb7406a190 0x7fcb741f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.652+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb740716b0 0x7fcb741428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.652+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb740cb190 0x7fcb741448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.652+0000 7fcb73fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7406c530 con 0x7fcb7406a190 2023-11-13T11:01:34.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.652+0000 7fcb73fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb74144e00 con 0x7fcb740cb190 2023-11-13T11:01:34.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.652+0000 7fcb73fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb741450d0 con 0x7fcb740716b0 2023-11-13T11:01:34.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.652+0000 7fcb72ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb7406a190 0x7fcb741f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.652+0000 7fcb72ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb7406a190 0x7fcb741f67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43142/0 (socket says 172.21.15.16:43142) 2023-11-13T11:01:34.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.652+0000 7fcb72ffd640 1 -- 172.21.15.16:0/3766750761 learned_addr learned my addr 172.21.15.16:0/3766750761 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:34.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb72ffd640 1 -- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb740716b0 msgr2=0x7fcb741428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb727fc640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb740716b0 0x7fcb741428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb737fe640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb740cb190 0x7fcb741448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb72ffd640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb740716b0 0x7fcb741428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb72ffd640 1 -- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb740cb190 msgr2=0x7fcb741448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb72ffd640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb740cb190 0x7fcb741448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb72ffd640 1 -- 172.21.15.16:0/3766750761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680094d0 con 0x7fcb7406a190 2023-11-13T11:01:34.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb737fe640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb740cb190 0x7fcb741448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:34.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb72ffd640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb7406a190 0x7fcb741f67b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400d800 tx=0x7fcb6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:34.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb5b7fe640 1 -- 172.21.15.16:0/3766750761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400b6c0 con 0x7fcb7406a190 2023-11-13T11:01:34.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb5b7fe640 1 -- 172.21.15.16:0/3766750761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6400b820 con 0x7fcb7406a190 2023-11-13T11:01:34.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb5b7fe640 1 -- 172.21.15.16:0/3766750761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400bbb0 con 0x7fcb7406a190 2023-11-13T11:01:34.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb741453b0 con 0x7fcb7406a190 2023-11-13T11:01:34.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.653+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7406c780 con 0x7fcb7406a190 2023-11-13T11:01:34.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.655+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7404e4d0 con 0x7fcb7406a190 2023-11-13T11:01:34.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.655+0000 7fcb5b7fe640 1 -- 172.21.15.16:0/3766750761 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb6400bd10 con 0x7fcb7406a190 2023-11-13T11:01:34.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.656+0000 7fcb5b7fe640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcb6006ae40 0x7fcb6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:34.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.656+0000 7fcb5b7fe640 1 -- 172.21.15.16:0/3766750761 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fcb64089910 con 0x7fcb7406a190 2023-11-13T11:01:34.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.657+0000 7fcb727fc640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcb6006ae40 0x7fcb6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:34.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.658+0000 7fcb727fc640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcb6006ae40 0x7fcb6006d300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcb68036000 tx=0x7fcb6803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:34.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.661+0000 7fcb5b7fe640 1 -- 172.21.15.16:0/3766750761 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb64054c60 con 0x7fcb7406a190 2023-11-13T11:01:34.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.856+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb740b3100 con 0x7fcb7406a190 2023-11-13T11:01:34.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.858+0000 7fcb5b7fe640 1 -- 172.21.15.16:0/3766750761 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13074 (secure 0 0 0) 0x7fcb64058280 con 0x7fcb7406a190 2023-11-13T11:01:34.863 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:34.863 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:01:33.878859+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:01:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.861+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcb6006ae40 msgr2=0x7fcb6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.861+0000 7fcb73fff640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcb6006ae40 0x7fcb6006d300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcb68036000 tx=0x7fcb6803c040 comp rx=0 tx=0).stop 2023-11-13T11:01:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.861+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb7406a190 msgr2=0x7fcb741f67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.861+0000 7fcb73fff640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb7406a190 0x7fcb741f67b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400d800 tx=0x7fcb6400dcc0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.862+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 shutdown_connections 2023-11-13T11:01:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.862+0000 7fcb73fff640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcb6006ae40 0x7fcb6006d300 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.862+0000 7fcb73fff640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb740cb190 0x7fcb741448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.862+0000 7fcb73fff640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb740716b0 0x7fcb741428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.862+0000 7fcb73fff640 1 --2- 172.21.15.16:0/3766750761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb7406a190 0x7fcb741f67b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.862+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 >> 172.21.15.16:0/3766750761 conn(0x7fcb740b04a0 msgr2=0x7fcb74070120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.862+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 shutdown_connections 2023-11-13T11:01:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:34.862+0000 7fcb73fff640 1 -- 172.21.15.16:0/3766750761 wait complete. 2023-11-13T11:01:34.886 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:01:34.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:01:35.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.107+0000 7fe4ef59e640 1 Processor -- start 2023-11-13T11:01:35.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.107+0000 7fe4ef59e640 1 -- start start 2023-11-13T11:01:35.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.108+0000 7fe4ef59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe4f015ce70 0x7fe4f00cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.109+0000 7fe4ef59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00ce0e0 con 0x7fe4f00d1850 2023-11-13T11:01:35.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.109+0000 7fe4ef59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00ce3c0 con 0x7fe4f00cdbb0 2023-11-13T11:01:35.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.109+0000 7fe4ef59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00ce6a0 con 0x7fe4f015ce70 2023-11-13T11:01:35.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.109+0000 7fe4ee59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe4f015ce70 0x7fe4f00cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.109+0000 7fe4eed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe4f00d1850 0x7fe4f00c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56448/0 (socket says 172.21.15.16:56448) 2023-11-13T11:01:35.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.109+0000 7fe4eed9d640 1 -- 172.21.15.16:0/1310732181 learned_addr learned my addr 172.21.15.16:0/1310732181 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:35.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.109+0000 7fe4ee59c640 1 -- 172.21.15.16:0/1310732181 >> v1:172.21.15.184:6789/0 conn(0x7fe4f00cdbb0 legacy=0x7fe4f00d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:35.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.110+0000 7fe4ee59c640 1 -- 172.21.15.16:0/1310732181 >> v1:172.21.15.16:6789/0 conn(0x7fe4f00d1850 legacy=0x7fe4f00c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.110+0000 7fe4ee59c640 1 -- 172.21.15.16:0/1310732181 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00ce980 con 0x7fe4f015ce70 2023-11-13T11:01:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.110+0000 7fe4ed59a640 1 -- 172.21.15.16:0/1310732181 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700246084 0 0) 0x7fe4f00ce0e0 con 0x7fe4f00d1850 2023-11-13T11:01:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.111+0000 7fe4ee59c640 1 --2- 172.21.15.16:0/1310732181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 0x7fe4f00cd670 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00097a0 tx=0x7fe4e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce54f9462022445e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.111+0000 7fe4ed59a640 1 -- 172.21.15.16:0/1310732181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0031830 con 0x7fe4f015ce70 2023-11-13T11:01:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.111+0000 7fe4ed59a640 1 -- 172.21.15.16:0/1310732181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0031990 con 0x7fe4f015ce70 2023-11-13T11:01:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.111+0000 7fe4ed59a640 1 -- 172.21.15.16:0/1310732181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0031d20 con 0x7fe4f015ce70 2023-11-13T11:01:35.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.112+0000 7fe4ef59e640 1 -- 172.21.15.16:0/1310732181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 msgr2=0x7fe4f00cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.112+0000 7fe4ef59e640 1 --2- 172.21.15.16:0/1310732181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 0x7fe4f00cd670 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00097a0 tx=0x7fe4e002b8d0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.112+0000 7fe4ef59e640 1 -- 172.21.15.16:0/1310732181 shutdown_connections 2023-11-13T11:01:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.112+0000 7fe4ef59e640 1 --2- 172.21.15.16:0/1310732181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 0x7fe4f00cd670 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.112+0000 7fe4ef59e640 1 -- 172.21.15.16:0/1310732181 >> 172.21.15.16:0/1310732181 conn(0x7fe4f008c580 msgr2=0x7fe4f009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.112+0000 7fe4ef59e640 1 -- 172.21.15.16:0/1310732181 shutdown_connections 2023-11-13T11:01:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.112+0000 7fe4ef59e640 1 -- 172.21.15.16:0/1310732181 wait complete. 2023-11-13T11:01:35.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.114+0000 7fe4ef59e640 1 Processor -- start 2023-11-13T11:01:35.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.114+0000 7fe4ef59e640 1 -- start start 2023-11-13T11:01:35.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.115+0000 7fe4ef59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4f00cdbb0 0x7fe4f0151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.115+0000 7fe4ef59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4f00d1850 0x7fe4f006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.115+0000 7fe4ef59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 0x7fe4f006bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.115+0000 7fe4ef59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b8000 con 0x7fe4f015ce70 2023-11-13T11:01:35.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.115+0000 7fe4ef59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b78a0 con 0x7fe4f00cdbb0 2023-11-13T11:01:35.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.115+0000 7fe4ef59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b6870 con 0x7fe4f00d1850 2023-11-13T11:01:35.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 0x7fe4f006bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 0x7fe4f006bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43146/0 (socket says 172.21.15.16:43146) 2023-11-13T11:01:35.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4eed9d640 1 -- 172.21.15.16:0/863871855 learned_addr learned my addr 172.21.15.16:0/863871855 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:35.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4ee59c640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4f00cdbb0 0x7fe4f0151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4edd9b640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4f00d1850 0x7fe4f006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4eed9d640 1 -- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4f00d1850 msgr2=0x7fe4f006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4eed9d640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4f00d1850 0x7fe4f006ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4eed9d640 1 -- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4f00cdbb0 msgr2=0x7fe4f0151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4eed9d640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4f00cdbb0 0x7fe4f0151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.116+0000 7fe4eed9d640 1 -- 172.21.15.16:0/863871855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e0009400 con 0x7fe4f015ce70 2023-11-13T11:01:35.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.117+0000 7fe4eed9d640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 0x7fe4f006bf40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c7e0 tx=0x7fe4e800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:35.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.117+0000 7fe4e6ffd640 1 -- 172.21.15.16:0/863871855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800a640 con 0x7fe4f015ce70 2023-11-13T11:01:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.117+0000 7fe4e6ffd640 1 -- 172.21.15.16:0/863871855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e800a7a0 con 0x7fe4f015ce70 2023-11-13T11:01:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.117+0000 7fe4e6ffd640 1 -- 172.21.15.16:0/863871855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800ab30 con 0x7fe4f015ce70 2023-11-13T11:01:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.117+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f00b6ae0 con 0x7fe4f015ce70 2023-11-13T11:01:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.117+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f00b7380 con 0x7fe4f015ce70 2023-11-13T11:01:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.118+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f004e4d0 con 0x7fe4f015ce70 2023-11-13T11:01:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.119+0000 7fe4e6ffd640 1 -- 172.21.15.16:0/863871855 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe4e800ac90 con 0x7fe4f015ce70 2023-11-13T11:01:35.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.120+0000 7fe4e6ffd640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe4d006ae40 0x7fe4d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.120+0000 7fe4e6ffd640 1 -- 172.21.15.16:0/863871855 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe4e8088890 con 0x7fe4f015ce70 2023-11-13T11:01:35.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.121+0000 7fe4ee59c640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe4d006ae40 0x7fe4d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.122+0000 7fe4ee59c640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe4d006ae40 0x7fe4d006d300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0036000 tx=0x7fe4e003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:35.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.124+0000 7fe4e6ffd640 1 -- 172.21.15.16:0/863871855 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4e8053b40 con 0x7fe4f015ce70 2023-11-13T11:01:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.310+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4f015b580 con 0x7fe4f015ce70 2023-11-13T11:01:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.311+0000 7fe4e6ffd640 1 -- 172.21.15.16:0/863871855 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4e8018020 con 0x7fe4f015ce70 2023-11-13T11:01:35.315 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:01:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe4d006ae40 msgr2=0x7fe4d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe4d006ae40 0x7fe4d006d300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0036000 tx=0x7fe4e003c040 comp rx=0 tx=0).stop 2023-11-13T11:01:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 msgr2=0x7fe4f006bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 0x7fe4f006bf40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c7e0 tx=0x7fe4e800cca0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 shutdown_connections 2023-11-13T11:01:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe4d006ae40 0x7fe4d006d300 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f015ce70 0x7fe4f006bf40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4f00d1850 0x7fe4f006ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 --2- 172.21.15.16:0/863871855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4f00cdbb0 0x7fe4f0151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 >> 172.21.15.16:0/863871855 conn(0x7fe4f008c580 msgr2=0x7fe4f01600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 shutdown_connections 2023-11-13T11:01:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.314+0000 7fe4ef59e640 1 -- 172.21.15.16:0/863871855 wait complete. 2023-11-13T11:01:35.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-13T11:01:35.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.558+0000 7f0156040640 1 Processor -- start 2023-11-13T11:01:35.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.558+0000 7f0156040640 1 -- start start 2023-11-13T11:01:35.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.559+0000 7f0156040640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015015ad50 0x7f01500cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.559+0000 7f0156040640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01500cced0 0x7f01500cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.559+0000 7f0156040640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f015006e690 0x7f015006eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.559+0000 7f0156040640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01500ce410 con 0x7f01500cced0 2023-11-13T11:01:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.559+0000 7f0156040640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01500cdea0 con 0x7f015006e690 2023-11-13T11:01:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.559+0000 7f0156040640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01500ce7a0 con 0x7f015015ad50 2023-11-13T11:01:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.560+0000 7f015483d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01500cced0 0x7f01500cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.560+0000 7f015483d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01500cced0 0x7f01500cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43162/0 (socket says 172.21.15.16:43162) 2023-11-13T11:01:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.560+0000 7f015483d640 1 -- 172.21.15.16:0/1951990776 learned_addr learned my addr 172.21.15.16:0/1951990776 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.560+0000 7f015503e640 1 --2- 172.21.15.16:0/1951990776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015015ad50 0x7f01500cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.560+0000 7f015583f640 1 --2- 172.21.15.16:0/1951990776 >> v2:172.21.15.184:3300/0 conn(0x7f015006e690 0x7f015006eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.561+0000 7f015483d640 1 -- 172.21.15.16:0/1951990776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015015ad50 msgr2=0x7f01500cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.561+0000 7f015483d640 1 --2- 172.21.15.16:0/1951990776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015015ad50 0x7f01500cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.561+0000 7f015483d640 1 -- 172.21.15.16:0/1951990776 >> v2:172.21.15.184:3300/0 conn(0x7f015006e690 msgr2=0x7f015006eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.561+0000 7f015483d640 1 --2- 172.21.15.16:0/1951990776 >> v2:172.21.15.184:3300/0 conn(0x7f015006e690 0x7f015006eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.561+0000 7f015483d640 1 -- 172.21.15.16:0/1951990776 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015006fcc0 con 0x7f01500cced0 2023-11-13T11:01:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.562+0000 7f015483d640 1 --2- 172.21.15.16:0/1951990776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01500cced0 0x7f01500cd960 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f01500ced00 tx=0x7f014002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0701578fb677c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.562+0000 7f0147fff640 1 -- 172.21.15.16:0/1951990776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014002ca70 con 0x7f01500cced0 2023-11-13T11:01:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.562+0000 7f0147fff640 1 -- 172.21.15.16:0/1951990776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014002cbd0 con 0x7f01500cced0 2023-11-13T11:01:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.562+0000 7f0147fff640 1 -- 172.21.15.16:0/1951990776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01400343f0 con 0x7f01500cced0 2023-11-13T11:01:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.563+0000 7f0156040640 1 -- 172.21.15.16:0/1951990776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01500cced0 msgr2=0x7f01500cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.563+0000 7f0156040640 1 --2- 172.21.15.16:0/1951990776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01500cced0 0x7f01500cd960 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f01500ced00 tx=0x7f014002bed0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.563+0000 7f0156040640 1 -- 172.21.15.16:0/1951990776 shutdown_connections 2023-11-13T11:01:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.563+0000 7f0156040640 1 --2- 172.21.15.16:0/1951990776 >> v2:172.21.15.184:3300/0 conn(0x7f015006e690 0x7f015006eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.563+0000 7f0156040640 1 --2- 172.21.15.16:0/1951990776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01500cced0 0x7f01500cd960 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.563+0000 7f0156040640 1 --2- 172.21.15.16:0/1951990776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015015ad50 0x7f01500cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.563+0000 7f0156040640 1 -- 172.21.15.16:0/1951990776 >> 172.21.15.16:0/1951990776 conn(0x7f01500b2700 msgr2=0x7f01500b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.563+0000 7f0156040640 1 -- 172.21.15.16:0/1951990776 shutdown_connections 2023-11-13T11:01:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.563+0000 7f0156040640 1 -- 172.21.15.16:0/1951990776 wait complete. 2023-11-13T11:01:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.565+0000 7f0156040640 1 Processor -- start 2023-11-13T11:01:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.565+0000 7f0156040640 1 -- start start 2023-11-13T11:01:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.566+0000 7f0156040640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015006e690 0x7f01501555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.566+0000 7f0156040640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01500cced0 0x7f01500aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.566+0000 7f0156040640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015ad50 0x7f01500aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.566+0000 7f0156040640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015014b7a0 con 0x7f015015ad50 2023-11-13T11:01:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.566+0000 7f0156040640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500b4690 con 0x7f01500cced0 2023-11-13T11:01:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.566+0000 7f0156040640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01500b4970 con 0x7f015006e690 2023-11-13T11:01:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.566+0000 7f015583f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015ad50 0x7f01500aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.566+0000 7f015583f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015ad50 0x7f01500aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43178/0 (socket says 172.21.15.16:43178) 2023-11-13T11:01:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.566+0000 7f015583f640 1 -- 172.21.15.16:0/4197035536 learned_addr learned my addr 172.21.15.16:0/4197035536 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f015483d640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01500cced0 0x7f01500aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f015503e640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015006e690 0x7f01501555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f015583f640 1 -- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015006e690 msgr2=0x7f01501555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f015583f640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015006e690 0x7f01501555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f015583f640 1 -- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01500cced0 msgr2=0x7f01500aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f015583f640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01500cced0 0x7f01500aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f015583f640 1 -- 172.21.15.16:0/4197035536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140009b30 con 0x7f015015ad50 2023-11-13T11:01:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f015483d640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01500cced0 0x7f01500aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f015583f640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015ad50 0x7f01500aefe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f014800d5d0 tx=0x7f014800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f0145ffb640 1 -- 172.21.15.16:0/4197035536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148014620 con 0x7f015015ad50 2023-11-13T11:01:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f0145ffb640 1 -- 172.21.15.16:0/4197035536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01480147a0 con 0x7f015015ad50 2023-11-13T11:01:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f0145ffb640 1 -- 172.21.15.16:0/4197035536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148014b50 con 0x7f015015ad50 2023-11-13T11:01:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.567+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01500b4c50 con 0x7f015015ad50 2023-11-13T11:01:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.568+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01500b5210 con 0x7f015015ad50 2023-11-13T11:01:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.569+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01500d0560 con 0x7f015015ad50 2023-11-13T11:01:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.570+0000 7f0145ffb640 1 -- 172.21.15.16:0/4197035536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0148014cb0 con 0x7f015015ad50 2023-11-13T11:01:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.571+0000 7f0145ffb640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f013406ae40 0x7f013406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.571+0000 7f0145ffb640 1 -- 172.21.15.16:0/4197035536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0148089820 con 0x7f015015ad50 2023-11-13T11:01:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.571+0000 7f015503e640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f013406ae40 0x7f013406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.573+0000 7f015503e640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f013406ae40 0x7f013406d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f01501f6020 tx=0x7f014c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.575+0000 7f0145ffb640 1 -- 172.21.15.16:0/4197035536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0148054a40 con 0x7f015015ad50 2023-11-13T11:01:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.774+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f01500b5400 con 0x7f015015ad50 2023-11-13T11:01:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.775+0000 7f0145ffb640 1 -- 172.21.15.16:0/4197035536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0148018090 con 0x7f015015ad50 2023-11-13T11:01:35.780 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:35.780 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-13T11:01:35.780 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:01:35.781 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-13T11:01:35.781 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-13T11:01:35.781 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:35.781 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-13T11:01:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.777+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f013406ae40 msgr2=0x7f013406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f013406ae40 0x7f013406d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f01501f6020 tx=0x7f014c008040 comp rx=0 tx=0).stop 2023-11-13T11:01:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015ad50 msgr2=0x7f01500aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015ad50 0x7f01500aefe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f014800d5d0 tx=0x7f014800da90 comp rx=0 tx=0).stop 2023-11-13T11:01:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 shutdown_connections 2023-11-13T11:01:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f013406ae40 0x7f013406d300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f015015ad50 0x7f01500aefe0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01500cced0 0x7f01500aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 --2- 172.21.15.16:0/4197035536 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f015006e690 0x7f01501555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 >> 172.21.15.16:0/4197035536 conn(0x7f01500b2700 msgr2=0x7f015006eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 shutdown_connections 2023-11-13T11:01:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:35.778+0000 7f0156040640 1 -- 172.21.15.16:0/4197035536 wait complete. 2023-11-13T11:01:35.802 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-13T11:01:35.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-13T11:01:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.028+0000 7effb759e640 1 Processor -- start 2023-11-13T11:01:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.028+0000 7effb759e640 1 -- start start 2023-11-13T11:01:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.030+0000 7effb759e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7effb80cc2f0 0x7effb80b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.030+0000 7effb759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 0x7effb80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.030+0000 7effb759e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb80ce5c0 con 0x7effb81442a0 2023-11-13T11:01:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.030+0000 7effb759e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb815bfb0 con 0x7effb80cc2f0 2023-11-13T11:01:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.030+0000 7effb759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb80b2930 con 0x7effb80cdbf0 2023-11-13T11:01:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.031+0000 7effaffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7effb80cc2f0 0x7effb80b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.031+0000 7effaffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7effb80cc2f0 0x7effb80b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54178/0 (socket says 172.21.15.16:54178) 2023-11-13T11:01:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.031+0000 7effb659c640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7effb81442a0 0x7effb80cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:54558/0 (socket says 172.21.15.16:54558) 2023-11-13T11:01:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.031+0000 7effaffff640 1 -- 172.21.15.16:0/1672017074 learned_addr learned my addr 172.21.15.16:0/1672017074 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.031+0000 7effb6d9d640 1 --2- 172.21.15.16:0/1672017074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 0x7effb80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.032+0000 7effaffff640 1 -- 172.21.15.16:0/1672017074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 msgr2=0x7effb80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.032+0000 7effaffff640 1 --2- 172.21.15.16:0/1672017074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 0x7effb80ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.032+0000 7effaffff640 1 -- 172.21.15.16:0/1672017074 >> v1:172.21.15.184:6789/0 conn(0x7effb81442a0 legacy=0x7effb80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.032+0000 7effaffff640 1 -- 172.21.15.16:0/1672017074 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb80b2c10 con 0x7effb80cc2f0 2023-11-13T11:01:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.032+0000 7effb5d9b640 1 -- 172.21.15.16:0/1672017074 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633603909 0 0) 0x7effb80ce5c0 con 0x7effb81442a0 2023-11-13T11:01:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.034+0000 7effaffff640 1 --2- 172.21.15.16:0/1672017074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effb80cc2f0 0x7effb80b39d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7effa0009870 tx=0x7effa002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1a381296486885 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.034+0000 7effb5d9b640 1 -- 172.21.15.16:0/1672017074 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0030840 con 0x7effb80cc2f0 2023-11-13T11:01:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.034+0000 7effb5d9b640 1 -- 172.21.15.16:0/1672017074 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa00309a0 con 0x7effb80cc2f0 2023-11-13T11:01:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.034+0000 7effb5d9b640 1 -- 172.21.15.16:0/1672017074 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0030d30 con 0x7effb80cc2f0 2023-11-13T11:01:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.035+0000 7effb759e640 1 -- 172.21.15.16:0/1672017074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effb80cc2f0 msgr2=0x7effb80b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.035+0000 7effb759e640 1 --2- 172.21.15.16:0/1672017074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effb80cc2f0 0x7effb80b39d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7effa0009870 tx=0x7effa002b980 comp rx=0 tx=0).stop 2023-11-13T11:01:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.035+0000 7effb759e640 1 -- 172.21.15.16:0/1672017074 shutdown_connections 2023-11-13T11:01:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.035+0000 7effb759e640 1 --2- 172.21.15.16:0/1672017074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 0x7effb80ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.035+0000 7effb759e640 1 --2- 172.21.15.16:0/1672017074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effb80cc2f0 0x7effb80b39d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.035+0000 7effb759e640 1 -- 172.21.15.16:0/1672017074 >> 172.21.15.16:0/1672017074 conn(0x7effb80b0970 msgr2=0x7effb80b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.035+0000 7effb759e640 1 -- 172.21.15.16:0/1672017074 shutdown_connections 2023-11-13T11:01:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.036+0000 7effb759e640 1 -- 172.21.15.16:0/1672017074 wait complete. 2023-11-13T11:01:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.037+0000 7effb759e640 1 Processor -- start 2023-11-13T11:01:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.038+0000 7effb759e640 1 -- start start 2023-11-13T11:01:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb759e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effb80cc2f0 0x7effb81404d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 0x7effb80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81442a0 0x7effb8070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb80c5330 con 0x7effb81442a0 2023-11-13T11:01:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb759e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb80c46b0 con 0x7effb80cc2f0 2023-11-13T11:01:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb80c4990 con 0x7effb80cdbf0 2023-11-13T11:01:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81442a0 0x7effb8070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:36.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81442a0 0x7effb8070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43182/0 (socket says 172.21.15.16:43182) 2023-11-13T11:01:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb6d9d640 1 -- 172.21.15.16:0/1296453429 learned_addr learned my addr 172.21.15.16:0/1296453429 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effb659c640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effb80cc2f0 0x7effb81404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.039+0000 7effaffff640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 0x7effb80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effb6d9d640 1 -- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 msgr2=0x7effb80709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:36.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effb6d9d640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 0x7effb80709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effb6d9d640 1 -- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effb80cc2f0 msgr2=0x7effb81404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effb6d9d640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effb80cc2f0 0x7effb81404d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effb6d9d640 1 -- 172.21.15.16:0/1296453429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00094d0 con 0x7effb81442a0 2023-11-13T11:01:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effb6d9d640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81442a0 0x7effb8070f20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7effb000efc0 tx=0x7effb000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effaeffd640 1 -- 172.21.15.16:0/1296453429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0014480 con 0x7effb81442a0 2023-11-13T11:01:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effaeffd640 1 -- 172.21.15.16:0/1296453429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb000f040 con 0x7effb81442a0 2023-11-13T11:01:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb80c4c70 con 0x7effb81442a0 2023-11-13T11:01:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.040+0000 7effaeffd640 1 -- 172.21.15.16:0/1296453429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0014820 con 0x7effb81442a0 2023-11-13T11:01:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.041+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb8140a10 con 0x7effb81442a0 2023-11-13T11:01:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.042+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb80c5470 con 0x7effb81442a0 2023-11-13T11:01:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.043+0000 7effaeffd640 1 -- 172.21.15.16:0/1296453429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effb0014980 con 0x7effb81442a0 2023-11-13T11:01:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.044+0000 7effaeffd640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7eff9806ae40 0x7eff9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.044+0000 7effaeffd640 1 -- 172.21.15.16:0/1296453429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7effb0089fe0 con 0x7effb81442a0 2023-11-13T11:01:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.045+0000 7effb659c640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7eff9806ae40 0x7eff9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.046+0000 7effb659c640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7eff9806ae40 0x7eff9806d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7effa8000f60 tx=0x7effa8007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.049+0000 7effaeffd640 1 -- 172.21.15.16:0/1296453429 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7effb80c5470 con 0x7effb81442a0 2023-11-13T11:01:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.245+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7effb80c5470 con 0x7effb81442a0 2023-11-13T11:01:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.927+0000 7effaeffd640 1 -- 172.21.15.16:0/1296453429 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7effb80c5470 con 0x7effb81442a0 2023-11-13T11:01:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-13T11:01:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7eff9806ae40 msgr2=0x7eff9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7eff9806ae40 0x7eff9806d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7effa8000f60 tx=0x7effa8007c00 comp rx=0 tx=0).stop 2023-11-13T11:01:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81442a0 msgr2=0x7effb8070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81442a0 0x7effb8070f20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7effb000efc0 tx=0x7effb000c490 comp rx=0 tx=0).stop 2023-11-13T11:01:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 shutdown_connections 2023-11-13T11:01:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7eff9806ae40 0x7eff9806d300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb81442a0 0x7effb8070f20 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effb80cdbf0 0x7effb80709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 --2- 172.21.15.16:0/1296453429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effb80cc2f0 0x7effb81404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 >> 172.21.15.16:0/1296453429 conn(0x7effb80b0970 msgr2=0x7effb80ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 shutdown_connections 2023-11-13T11:01:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:36.932+0000 7effb759e640 1 -- 172.21.15.16:0/1296453429 wait complete. 2023-11-13T11:01:36.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-13T11:01:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.191+0000 7fe81b527640 1 Processor -- start 2023-11-13T11:01:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.191+0000 7fe81b527640 1 -- start start 2023-11-13T11:01:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.193+0000 7fe81b527640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8140cf0f0 0x7fe8140b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.193+0000 7fe81b527640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140b3350 0x7fe814159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.193+0000 7fe81b527640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81415dff0 con 0x7fe8140ade10 2023-11-13T11:01:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.193+0000 7fe81b527640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8140b3b40 con 0x7fe8140cf0f0 2023-11-13T11:01:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.193+0000 7fe81b527640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81415d6c0 con 0x7fe8140b3350 2023-11-13T11:01:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.193+0000 7fe81a525640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe8140ade10 0x7fe8140ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56456/0 (socket says 172.21.15.16:56456) 2023-11-13T11:01:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe81a525640 1 -- 172.21.15.16:0/2087547508 learned_addr learned my addr 172.21.15.16:0/2087547508 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:37.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe819d24640 1 --2- 172.21.15.16:0/2087547508 >> v2:172.21.15.16:3300/0 conn(0x7fe8140cf0f0 0x7fe8140b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:37.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe819523640 1 -- 172.21.15.16:0/2087547508 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449463923 0 0) 0x7fe81415dff0 con 0x7fe8140ade10 2023-11-13T11:01:37.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe819523640 1 -- 172.21.15.16:0/2087547508 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe800003520 con 0x7fe8140ade10 2023-11-13T11:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe81ad26640 1 --2- 172.21.15.16:0/2087547508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140b3350 0x7fe814159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe819d24640 1 -- 172.21.15.16:0/2087547508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140b3350 msgr2=0x7fe814159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe819d24640 1 --2- 172.21.15.16:0/2087547508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140b3350 0x7fe814159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe819d24640 1 -- 172.21.15.16:0/2087547508 >> v1:172.21.15.16:6789/0 conn(0x7fe8140ade10 legacy=0x7fe8140ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe819d24640 1 -- 172.21.15.16:0/2087547508 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81415d9a0 con 0x7fe8140cf0f0 2023-11-13T11:01:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.194+0000 7fe81ad26640 1 --2- 172.21.15.16:0/2087547508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140b3350 0x7fe814159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.196+0000 7fe819d24640 1 --2- 172.21.15.16:0/2087547508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140cf0f0 0x7fe8140b2e10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe8040097a0 tx=0x7fe80402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ac019e1e8cfa96 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.196+0000 7fe819523640 1 -- 172.21.15.16:0/2087547508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804031780 con 0x7fe8140cf0f0 2023-11-13T11:01:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.196+0000 7fe819523640 1 -- 172.21.15.16:0/2087547508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8040318e0 con 0x7fe8140cf0f0 2023-11-13T11:01:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.196+0000 7fe819523640 1 -- 172.21.15.16:0/2087547508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804031c70 con 0x7fe8140cf0f0 2023-11-13T11:01:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.196+0000 7fe81b527640 1 -- 172.21.15.16:0/2087547508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140cf0f0 msgr2=0x7fe8140b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.196+0000 7fe81b527640 1 --2- 172.21.15.16:0/2087547508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140cf0f0 0x7fe8140b2e10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe8040097a0 tx=0x7fe80402b820 comp rx=0 tx=0).stop 2023-11-13T11:01:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.196+0000 7fe81b527640 1 -- 172.21.15.16:0/2087547508 shutdown_connections 2023-11-13T11:01:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.197+0000 7fe81b527640 1 --2- 172.21.15.16:0/2087547508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140b3350 0x7fe814159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.197+0000 7fe81b527640 1 --2- 172.21.15.16:0/2087547508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140cf0f0 0x7fe8140b2e10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.197+0000 7fe81b527640 1 -- 172.21.15.16:0/2087547508 >> 172.21.15.16:0/2087547508 conn(0x7fe81406a370 msgr2=0x7fe81406a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.197+0000 7fe81b527640 1 -- 172.21.15.16:0/2087547508 shutdown_connections 2023-11-13T11:01:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.197+0000 7fe81b527640 1 -- 172.21.15.16:0/2087547508 wait complete. 2023-11-13T11:01:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.198+0000 7fe81b527640 1 Processor -- start 2023-11-13T11:01:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.199+0000 7fe81b527640 1 -- start start 2023-11-13T11:01:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.199+0000 7fe81b527640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8140ade10 0x7fe8140cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe81b527640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140b3350 0x7fe814141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe81b527640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140cf0f0 0x7fe81415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe81b527640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8140cc7b0 con 0x7fe8140b3350 2023-11-13T11:01:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe81b527640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8140cd5d0 con 0x7fe8140ade10 2023-11-13T11:01:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe81b527640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8140cca50 con 0x7fe8140cf0f0 2023-11-13T11:01:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe819d24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140b3350 0x7fe814141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe819d24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140b3350 0x7fe814141dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43204/0 (socket says 172.21.15.16:43204) 2023-11-13T11:01:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe819d24640 1 -- 172.21.15.16:0/1768937012 learned_addr learned my addr 172.21.15.16:0/1768937012 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe81a525640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8140ade10 0x7fe8140cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe81ad26640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140cf0f0 0x7fe81415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe819d24640 1 -- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140cf0f0 msgr2=0x7fe81415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe819d24640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140cf0f0 0x7fe81415a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe819d24640 1 -- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8140ade10 msgr2=0x7fe8140cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe819d24640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8140ade10 0x7fe8140cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe819d24640 1 -- 172.21.15.16:0/1768937012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe804009400 con 0x7fe8140b3350 2023-11-13T11:01:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.200+0000 7fe81ad26640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140cf0f0 0x7fe81415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.201+0000 7fe819d24640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140b3350 0x7fe814141dc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe804034040 tx=0x7fe8040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.201+0000 7fe80affd640 1 -- 172.21.15.16:0/1768937012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804032820 con 0x7fe8140b3350 2023-11-13T11:01:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.201+0000 7fe80affd640 1 -- 172.21.15.16:0/1768937012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe804032980 con 0x7fe8140b3350 2023-11-13T11:01:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.201+0000 7fe80affd640 1 -- 172.21.15.16:0/1768937012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804032d30 con 0x7fe8140b3350 2023-11-13T11:01:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.201+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8140ccd30 con 0x7fe8140b3350 2023-11-13T11:01:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.201+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8140cd2f0 con 0x7fe8140b3350 2023-11-13T11:01:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.203+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8141f09e0 con 0x7fe8140b3350 2023-11-13T11:01:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.204+0000 7fe80affd640 1 -- 172.21.15.16:0/1768937012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe80404e440 con 0x7fe8140b3350 2023-11-13T11:01:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.205+0000 7fe80affd640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe7f806ae40 0x7fe7f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.205+0000 7fe80affd640 1 -- 172.21.15.16:0/1768937012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fe8040ab6f0 con 0x7fe8140b3350 2023-11-13T11:01:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.205+0000 7fe81a525640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe7f806ae40 0x7fe7f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.207+0000 7fe81a525640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe7f806ae40 0x7fe7f806d300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe8100011e0 tx=0x7fe810009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.210+0000 7fe80affd640 1 -- 172.21.15.16:0/1768937012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8141f09e0 con 0x7fe8140b3350 2023-11-13T11:01:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.403+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe8141f09e0 con 0x7fe8140b3350 2023-11-13T11:01:37.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.944+0000 7fe80affd640 1 -- 172.21.15.16:0/1768937012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe8141f09e0 con 0x7fe8140b3350 2023-11-13T11:01:37.950 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-13T11:01:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.948+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe7f806ae40 msgr2=0x7fe7f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.948+0000 7fe81b527640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe7f806ae40 0x7fe7f806d300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe8100011e0 tx=0x7fe810009040 comp rx=0 tx=0).stop 2023-11-13T11:01:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.948+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140b3350 msgr2=0x7fe814141dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.948+0000 7fe81b527640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140b3350 0x7fe814141dc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe804034040 tx=0x7fe8040046c0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.949+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 shutdown_connections 2023-11-13T11:01:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.949+0000 7fe81b527640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe7f806ae40 0x7fe7f806d300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.949+0000 7fe81b527640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8140cf0f0 0x7fe81415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.949+0000 7fe81b527640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8140b3350 0x7fe814141dc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.949+0000 7fe81b527640 1 --2- 172.21.15.16:0/1768937012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8140ade10 0x7fe8140cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.949+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 >> 172.21.15.16:0/1768937012 conn(0x7fe81406a370 msgr2=0x7fe814157380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.949+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 shutdown_connections 2023-11-13T11:01:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:37.949+0000 7fe81b527640 1 -- 172.21.15.16:0/1768937012 wait complete. 2023-11-13T11:01:37.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-13T11:01:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.179+0000 7f0deeae0640 1 Processor -- start 2023-11-13T11:01:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.179+0000 7f0deeae0640 1 -- start start 2023-11-13T11:01:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.181+0000 7f0deeae0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0de815cf90 0x7f0de80b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.181+0000 7f0deeae0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0de806c3b0 0x7f0de80b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.181+0000 7f0deeae0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0de80b39c0 0x7f0de80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.181+0000 7f0deeae0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de80b3e50 con 0x7f0de806c3b0 2023-11-13T11:01:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.181+0000 7f0deeae0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de80b2e00 con 0x7f0de80b39c0 2023-11-13T11:01:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.181+0000 7f0deeae0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80b21f0 con 0x7f0de815cf90 2023-11-13T11:01:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.181+0000 7f0ddffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0de806c3b0 0x7f0de80b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.181+0000 7f0ddffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0de806c3b0 0x7f0de80b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43210/0 (socket says 172.21.15.16:43210) 2023-11-13T11:01:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.181+0000 7f0ddffff640 1 -- 172.21.15.16:0/1248118710 learned_addr learned my addr 172.21.15.16:0/1248118710 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.182+0000 7f0ddffff640 1 -- 172.21.15.16:0/1248118710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0de815cf90 msgr2=0x7f0de80b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.182+0000 7f0ddffff640 1 --2- 172.21.15.16:0/1248118710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0de815cf90 0x7f0de80b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.182+0000 7f0ded07d640 1 --2- 172.21.15.16:0/1248118710 >> v2:172.21.15.184:3300/0 conn(0x7f0de80b39c0 0x7f0de80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.182+0000 7f0ddffff640 1 -- 172.21.15.16:0/1248118710 >> v2:172.21.15.184:3300/0 conn(0x7f0de80b39c0 msgr2=0x7f0de80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.182+0000 7f0ddffff640 1 --2- 172.21.15.16:0/1248118710 >> v2:172.21.15.184:3300/0 conn(0x7f0de80b39c0 0x7f0de80ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.182+0000 7f0ddffff640 1 -- 172.21.15.16:0/1248118710 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de80b24d0 con 0x7f0de806c3b0 2023-11-13T11:01:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.182+0000 7f0ded07d640 1 --2- 172.21.15.16:0/1248118710 >> v2:172.21.15.184:3300/0 conn(0x7f0de80b39c0 0x7f0de80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.183+0000 7f0ddffff640 1 --2- 172.21.15.16:0/1248118710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de806c3b0 0x7f0de80b3480 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0de806dbb0 tx=0x7f0dd802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8506c8b6ecd8d13 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.183+0000 7f0ddf7fe640 1 -- 172.21.15.16:0/1248118710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8031860 con 0x7f0de806c3b0 2023-11-13T11:01:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.183+0000 7f0ddf7fe640 1 -- 172.21.15.16:0/1248118710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd80319c0 con 0x7f0de806c3b0 2023-11-13T11:01:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.183+0000 7f0ddf7fe640 1 -- 172.21.15.16:0/1248118710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8031d50 con 0x7f0de806c3b0 2023-11-13T11:01:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.184+0000 7f0deeae0640 1 -- 172.21.15.16:0/1248118710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de806c3b0 msgr2=0x7f0de80b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.184+0000 7f0deeae0640 1 --2- 172.21.15.16:0/1248118710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de806c3b0 0x7f0de80b3480 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0de806dbb0 tx=0x7f0dd802b900 comp rx=0 tx=0).stop 2023-11-13T11:01:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.184+0000 7f0deeae0640 1 -- 172.21.15.16:0/1248118710 shutdown_connections 2023-11-13T11:01:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.184+0000 7f0deeae0640 1 --2- 172.21.15.16:0/1248118710 >> v2:172.21.15.184:3300/0 conn(0x7f0de80b39c0 0x7f0de80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.184+0000 7f0deeae0640 1 --2- 172.21.15.16:0/1248118710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de806c3b0 0x7f0de80b3480 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.184+0000 7f0deeae0640 1 --2- 172.21.15.16:0/1248118710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0de815cf90 0x7f0de80b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.184+0000 7f0deeae0640 1 -- 172.21.15.16:0/1248118710 >> 172.21.15.16:0/1248118710 conn(0x7f0de806a190 msgr2=0x7f0de806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.184+0000 7f0deeae0640 1 -- 172.21.15.16:0/1248118710 shutdown_connections 2023-11-13T11:01:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.185+0000 7f0deeae0640 1 -- 172.21.15.16:0/1248118710 wait complete. 2023-11-13T11:01:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.187+0000 7f0deeae0640 1 Processor -- start 2023-11-13T11:01:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.187+0000 7f0deeae0640 1 -- start start 2023-11-13T11:01:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.188+0000 7f0deeae0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0de806c3b0 0x7f0de8149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.188+0000 7f0deeae0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de80b39c0 0x7f0de80cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.188+0000 7f0deeae0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de815cf90 0x7f0de81559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.188+0000 7f0deeae0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80cd2c0 con 0x7f0de80b39c0 2023-11-13T11:01:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.188+0000 7f0deeae0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80cd750 con 0x7f0de815cf90 2023-11-13T11:01:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.188+0000 7f0deeae0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80cc6b0 con 0x7f0de806c3b0 2023-11-13T11:01:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.188+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de80b39c0 0x7f0de80cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:38.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.188+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de80b39c0 0x7f0de80cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43224/0 (socket says 172.21.15.16:43224) 2023-11-13T11:01:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.188+0000 7f0ddffff640 1 -- 172.21.15.16:0/3068162996 learned_addr learned my addr 172.21.15.16:0/3068162996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.189+0000 7f0ded07d640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de815cf90 0x7f0de81559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.189+0000 7f0dec87c640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0de806c3b0 0x7f0de8149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.189+0000 7f0ddffff640 1 -- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0de806c3b0 msgr2=0x7f0de8149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:38.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.189+0000 7f0ddffff640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0de806c3b0 0x7f0de8149320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.189+0000 7f0ddffff640 1 -- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de815cf90 msgr2=0x7f0de81559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.189+0000 7f0ddffff640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de815cf90 0x7f0de81559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.189+0000 7f0ddffff640 1 -- 172.21.15.16:0/3068162996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd80094d0 con 0x7f0de80b39c0 2023-11-13T11:01:38.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.189+0000 7f0ded07d640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de815cf90 0x7f0de81559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.189+0000 7f0ddffff640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de80b39c0 0x7f0de80cbd60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8000c00 tx=0x7f0dd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.190+0000 7f0ddd7fa640 1 -- 172.21.15.16:0/3068162996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8031830 con 0x7f0de80b39c0 2023-11-13T11:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.190+0000 7f0ddd7fa640 1 -- 172.21.15.16:0/3068162996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd80443f0 con 0x7f0de80b39c0 2023-11-13T11:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.190+0000 7f0ddd7fa640 1 -- 172.21.15.16:0/3068162996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80447a0 con 0x7f0de80b39c0 2023-11-13T11:01:38.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.190+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de80cc990 con 0x7f0de80b39c0 2023-11-13T11:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.190+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de80ccf50 con 0x7f0de80b39c0 2023-11-13T11:01:38.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.191+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de8072520 con 0x7f0de80b39c0 2023-11-13T11:01:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.194+0000 7f0ddd7fa640 1 -- 172.21.15.16:0/3068162996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0dd804d450 con 0x7f0de80b39c0 2023-11-13T11:01:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.194+0000 7f0ddd7fa640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0dd406ae40 0x7f0dd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.195+0000 7f0ddd7fa640 1 -- 172.21.15.16:0/3068162996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f0dd80ac250 con 0x7f0de80b39c0 2023-11-13T11:01:38.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.195+0000 7f0dec87c640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0dd406ae40 0x7f0dd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.196+0000 7f0dec87c640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0dd406ae40 0x7f0dd406d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0de4007dc0 tx=0x7f0de40072f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.198+0000 7f0ddd7fa640 1 -- 172.21.15.16:0/3068162996 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0dd8077300 con 0x7f0de80b39c0 2023-11-13T11:01:38.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.426+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0de80adea0 con 0x7f0de80b39c0 2023-11-13T11:01:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.948+0000 7f0ddd7fa640 1 -- 172.21.15.16:0/3068162996 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v24) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0dd807a920 con 0x7f0de80b39c0 2023-11-13T11:01:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 7 and data pool 8 2023-11-13T11:01:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.953+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0dd406ae40 msgr2=0x7f0dd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.953+0000 7f0deeae0640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0dd406ae40 0x7f0dd406d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0de4007dc0 tx=0x7f0de40072f0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.953+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de80b39c0 msgr2=0x7f0de80cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.953+0000 7f0deeae0640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de80b39c0 0x7f0de80cbd60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8000c00 tx=0x7f0dd80046c0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.954+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 shutdown_connections 2023-11-13T11:01:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.954+0000 7f0deeae0640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0dd406ae40 0x7f0dd406d300 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.954+0000 7f0deeae0640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de815cf90 0x7f0de81559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.954+0000 7f0deeae0640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de80b39c0 0x7f0de80cbd60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.954+0000 7f0deeae0640 1 --2- 172.21.15.16:0/3068162996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0de806c3b0 0x7f0de8149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.954+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 >> 172.21.15.16:0/3068162996 conn(0x7f0de806a190 msgr2=0x7f0de80ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.956+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 shutdown_connections 2023-11-13T11:01:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:38.956+0000 7f0deeae0640 1 -- 172.21.15.16:0/3068162996 wait complete. 2023-11-13T11:01:39.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:01:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.183+0000 7efeb4887640 1 Processor -- start 2023-11-13T11:01:39.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.183+0000 7efeb4887640 1 -- start start 2023-11-13T11:01:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.185+0000 7efeb4887640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb00ccc60 0x7efeb0071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.185+0000 7efeb4887640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb00cd0e0 con 0x7efeb0071980 2023-11-13T11:01:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.185+0000 7efeb4887640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb015b500 con 0x7efeb0155410 2023-11-13T11:01:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.185+0000 7efeb4887640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0071e10 con 0x7efeb00ccc60 2023-11-13T11:01:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.185+0000 7efeaed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efeb0071980 0x7efeb006fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56460/0 (socket says 172.21.15.16:56460) 2023-11-13T11:01:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.185+0000 7efeaed9d640 1 -- 172.21.15.16:0/4238159530 learned_addr learned my addr 172.21.15.16:0/4238159530 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.185+0000 7efeadd9b640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb00ccc60 0x7efeb0071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184680175 0 0) 0x7efeb00cd0e0 con 0x7efeb0071980 2023-11-13T11:01:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe98003520 con 0x7efeb0071980 2023-11-13T11:01:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 519877852 0 0) 0x7efe98003520 con 0x7efeb0071980 2023-11-13T11:01:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb00ccc60 msgr2=0x7efeb0071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb00ccc60 0x7efeb0071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 >> v1:172.21.15.184:6789/0 conn(0x7efeb0155410 legacy=0x7efeb00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb015b7e0 con 0x7efeb0071980 2023-11-13T11:01:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7efea80019f0 con 0x7efeb0071980 2023-11-13T11:01:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 >> v1:172.21.15.16:6789/0 conn(0x7efeb0071980 legacy=0x7efeb006fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb00ccc60 0x7efe980039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe98003f20 0x7efe980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efe98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0071e10 con 0x7efe98004d00 2023-11-13T11:01:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9800ba60 con 0x7efe98003f20 2023-11-13T11:01:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.186+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9800bc20 con 0x7efeb00ccc60 2023-11-13T11:01:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.187+0000 7efeaed9d640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efe98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.187+0000 7efeadd9b640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe98003f20 0x7efe980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.187+0000 7efeae59c640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb00ccc60 0x7efe980039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.187+0000 7efeaed9d640 1 -- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb00ccc60 msgr2=0x7efe980039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.187+0000 7efeaed9d640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb00ccc60 0x7efe980039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.187+0000 7efeaed9d640 1 -- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe98003f20 msgr2=0x7efe980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.187+0000 7efeaed9d640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe98003f20 0x7efe980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.187+0000 7efeaed9d640 1 -- 172.21.15.16:0/4238159530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe98003520 con 0x7efe98004d00 2023-11-13T11:01:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.187+0000 7efeae59c640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb00ccc60 0x7efe980039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.188+0000 7efeaed9d640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efe98005180 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efea800a830 tx=0x7efe980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac17c96beafa5bf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80072d0 con 0x7efe98004d00 2023-11-13T11:01:39.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea802f070 con 0x7efe98004d00 2023-11-13T11:01:39.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efead59a640 1 -- 172.21.15.16:0/4238159530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80076f0 con 0x7efe98004d00 2023-11-13T11:01:39.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efeb4887640 1 -- 172.21.15.16:0/4238159530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 msgr2=0x7efe98005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efeb4887640 1 --2- 172.21.15.16:0/4238159530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efe98005180 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efea800a830 tx=0x7efe980057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efeaed9d640 1 -- 172.21.15.16:0/4238159530 reap_dead start 2023-11-13T11:01:39.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efeb4887640 1 -- 172.21.15.16:0/4238159530 shutdown_connections 2023-11-13T11:01:39.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efeb4887640 1 -- 172.21.15.16:0/4238159530 >> 172.21.15.16:0/4238159530 conn(0x7efeb006a190 msgr2=0x7efeb006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:39.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efeb4887640 1 -- 172.21.15.16:0/4238159530 shutdown_connections 2023-11-13T11:01:39.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.189+0000 7efeb4887640 1 -- 172.21.15.16:0/4238159530 wait complete. 2023-11-13T11:01:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.191+0000 7efeb4887640 1 Processor -- start 2023-11-13T11:01:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.191+0000 7efeb4887640 1 -- start start 2023-11-13T11:01:39.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.192+0000 7efeb4887640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efeb0152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.192+0000 7efeb4887640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb0149130 0x7efeb013f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.192+0000 7efeb4887640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efeb0149660 0x7efeb00701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.192+0000 7efeb4887640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0157280 con 0x7efe98004d00 2023-11-13T11:01:39.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.192+0000 7efeb4887640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0151590 con 0x7efeb0149660 2023-11-13T11:01:39.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.192+0000 7efeb4887640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0151870 con 0x7efeb0149130 2023-11-13T11:01:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.192+0000 7efeae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efeb0152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efeb0152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43248/0 (socket says 172.21.15.16:43248) 2023-11-13T11:01:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeae59c640 1 -- 172.21.15.16:0/2340575589 learned_addr learned my addr 172.21.15.16:0/2340575589 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeadd9b640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb0149130 0x7efeb013f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeaed9d640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efeb0149660 0x7efeb00701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeae59c640 1 -- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb0149130 msgr2=0x7efeb013f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeae59c640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb0149130 0x7efeb013f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeae59c640 1 -- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efeb0149660 msgr2=0x7efeb00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeae59c640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efeb0149660 0x7efeb00701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeae59c640 1 -- 172.21.15.16:0/2340575589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0151b50 con 0x7efe98004d00 2023-11-13T11:01:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeadd9b640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb0149130 0x7efeb013f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.193+0000 7efeae59c640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efeb0152910 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7efea000d7a0 tx=0x7efea000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.194+0000 7efe9effd640 1 -- 172.21.15.16:0/2340575589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0017070 con 0x7efe98004d00 2023-11-13T11:01:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.194+0000 7efe9effd640 1 -- 172.21.15.16:0/2340575589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0005070 con 0x7efe98004d00 2023-11-13T11:01:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.194+0000 7efe9effd640 1 -- 172.21.15.16:0/2340575589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0005420 con 0x7efe98004d00 2023-11-13T11:01:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.194+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb0151e30 con 0x7efe98004d00 2023-11-13T11:01:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.194+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb01523f0 con 0x7efe98004d00 2023-11-13T11:01:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.195+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb00677a0 con 0x7efe98004d00 2023-11-13T11:01:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.196+0000 7efe9effd640 1 -- 172.21.15.16:0/2340575589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efea0007030 con 0x7efe98004d00 2023-11-13T11:01:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.197+0000 7efe9effd640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe9006ae40 0x7efe9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.197+0000 7efe9effd640 1 -- 172.21.15.16:0/2340575589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efea0089080 con 0x7efe98004d00 2023-11-13T11:01:39.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.197+0000 7efeadd9b640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe9006ae40 0x7efe9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.199+0000 7efeadd9b640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe9006ae40 0x7efe9006d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efea4009a00 tx=0x7efea4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:39.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.201+0000 7efe9effd640 1 -- 172.21.15.16:0/2340575589 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efea00bbe90 con 0x7efe98004d00 2023-11-13T11:01:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.393+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efeb0067090 con 0x7efe98004d00 2023-11-13T11:01:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.395+0000 7efe9effd640 1 -- 172.21.15.16:0/2340575589 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16739 (secure 0 0 0) 0x7efea0053fe0 con 0x7efe98004d00 2023-11-13T11:01:39.400 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":54,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:01:38.428523+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:01:36.305098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-13T11:01:37.461852+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:01:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe9006ae40 msgr2=0x7efe9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe9006ae40 0x7efe9006d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efea4009a00 tx=0x7efea4008040 comp rx=0 tx=0).stop 2023-11-13T11:01:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 msgr2=0x7efeb0152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efeb0152910 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7efea000d7a0 tx=0x7efea000dc60 comp rx=0 tx=0).stop 2023-11-13T11:01:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 shutdown_connections 2023-11-13T11:01:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efe9006ae40 0x7efe9006d300 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efeb0149660 0x7efeb00701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efeb0149130 0x7efeb013f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 --2- 172.21.15.16:0/2340575589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe98004d00 0x7efeb0152910 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 >> 172.21.15.16:0/2340575589 conn(0x7efeb006a190 msgr2=0x7efeb01534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 shutdown_connections 2023-11-13T11:01:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.398+0000 7efeb4887640 1 -- 172.21.15.16:0/2340575589 wait complete. 2023-11-13T11:01:39.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:01:39.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.629+0000 7efcccc27640 1 Processor -- start 2023-11-13T11:01:39.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.629+0000 7efcccc27640 1 -- start start 2023-11-13T11:01:39.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.631+0000 7efcccc27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efcc806f970 0x7efcc8162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.631+0000 7efcccc27640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efcc806a190 0x7efcc806a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.631+0000 7efcccc27640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc80b1840 con 0x7efcc8161060 2023-11-13T11:01:39.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.631+0000 7efcccc27640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc806ab60 con 0x7efcc806f970 2023-11-13T11:01:39.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.631+0000 7efcccc27640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc8071600 con 0x7efcc806a190 2023-11-13T11:01:39.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.631+0000 7efcc77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efcc8161060 0x7efcc81621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56476/0 (socket says 172.21.15.16:56476) 2023-11-13T11:01:39.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.631+0000 7efcc77fe640 1 -- 172.21.15.16:0/2219951409 learned_addr learned my addr 172.21.15.16:0/2219951409 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:39.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.631+0000 7efcc6ffd640 1 --2- 172.21.15.16:0/2219951409 >> v2:172.21.15.16:3300/0 conn(0x7efcc806f970 0x7efcc8162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.632+0000 7efcc7fff640 1 --2- 172.21.15.16:0/2219951409 >> v2:172.21.15.184:3300/0 conn(0x7efcc806a190 0x7efcc806a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.632+0000 7efcc6ffd640 1 -- 172.21.15.16:0/2219951409 >> v2:172.21.15.184:3300/0 conn(0x7efcc806a190 msgr2=0x7efcc806a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.632+0000 7efcc6ffd640 1 --2- 172.21.15.16:0/2219951409 >> v2:172.21.15.184:3300/0 conn(0x7efcc806a190 0x7efcc806a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.632+0000 7efcc6ffd640 1 -- 172.21.15.16:0/2219951409 >> v1:172.21.15.16:6789/0 conn(0x7efcc8161060 legacy=0x7efcc81621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.632+0000 7efcc6ffd640 1 -- 172.21.15.16:0/2219951409 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc80718e0 con 0x7efcc806f970 2023-11-13T11:01:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.632+0000 7efcc67fc640 1 -- 172.21.15.16:0/2219951409 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857658871 0 0) 0x7efcc80b1840 con 0x7efcc8161060 2023-11-13T11:01:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.633+0000 7efcc6ffd640 1 --2- 172.21.15.16:0/2219951409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806f970 0x7efcc8162900 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7efcc80cde10 tx=0x7efcb002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a04c7ca97d9c7f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.633+0000 7efcc67fc640 1 -- 172.21.15.16:0/2219951409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0031830 con 0x7efcc806f970 2023-11-13T11:01:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.633+0000 7efcc67fc640 1 -- 172.21.15.16:0/2219951409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb0031990 con 0x7efcc806f970 2023-11-13T11:01:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.633+0000 7efcc67fc640 1 -- 172.21.15.16:0/2219951409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0031d20 con 0x7efcc806f970 2023-11-13T11:01:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.634+0000 7efcccc27640 1 -- 172.21.15.16:0/2219951409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806f970 msgr2=0x7efcc8162900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.634+0000 7efcccc27640 1 --2- 172.21.15.16:0/2219951409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806f970 0x7efcc8162900 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7efcc80cde10 tx=0x7efcb002b8d0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.634+0000 7efcccc27640 1 -- 172.21.15.16:0/2219951409 shutdown_connections 2023-11-13T11:01:39.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.634+0000 7efcccc27640 1 --2- 172.21.15.16:0/2219951409 >> v2:172.21.15.184:3300/0 conn(0x7efcc806a190 0x7efcc806a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.634+0000 7efcccc27640 1 --2- 172.21.15.16:0/2219951409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806f970 0x7efcc8162900 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.634+0000 7efcccc27640 1 -- 172.21.15.16:0/2219951409 >> 172.21.15.16:0/2219951409 conn(0x7efcc80b04a0 msgr2=0x7efcc80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.634+0000 7efcccc27640 1 -- 172.21.15.16:0/2219951409 shutdown_connections 2023-11-13T11:01:39.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.634+0000 7efcccc27640 1 -- 172.21.15.16:0/2219951409 wait complete. 2023-11-13T11:01:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.636+0000 7efcccc27640 1 Processor -- start 2023-11-13T11:01:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.636+0000 7efcccc27640 1 -- start start 2023-11-13T11:01:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.637+0000 7efcccc27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806a190 0x7efcc815d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.637+0000 7efcccc27640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc806f970 0x7efcc80a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.637+0000 7efcccc27640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc8161060 0x7efcc80a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.637+0000 7efcccc27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80a84d0 con 0x7efcc806a190 2023-11-13T11:01:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.637+0000 7efcccc27640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80a9530 con 0x7efcc8161060 2023-11-13T11:01:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.637+0000 7efcccc27640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80a8770 con 0x7efcc806f970 2023-11-13T11:01:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.637+0000 7efcc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806a190 0x7efcc815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.637+0000 7efcc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806a190 0x7efcc815d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43266/0 (socket says 172.21.15.16:43266) 2023-11-13T11:01:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.637+0000 7efcc77fe640 1 -- 172.21.15.16:0/833109892 learned_addr learned my addr 172.21.15.16:0/833109892 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcc6ffd640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc806f970 0x7efcc80a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcc77fe640 1 -- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc806f970 msgr2=0x7efcc80a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcc77fe640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc806f970 0x7efcc80a99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcc7fff640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc8161060 0x7efcc80a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcc77fe640 1 -- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc8161060 msgr2=0x7efcc80a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcc77fe640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc8161060 0x7efcc80a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcc77fe640 1 -- 172.21.15.16:0/833109892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb0009400 con 0x7efcc806a190 2023-11-13T11:01:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcc7fff640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc8161060 0x7efcc80a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:39.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcc77fe640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806a190 0x7efcc815d270 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c4d0 tx=0x7efcb800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.638+0000 7efcbffff640 1 -- 172.21.15.16:0/833109892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb80074e0 con 0x7efcc806a190 2023-11-13T11:01:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.639+0000 7efcbffff640 1 -- 172.21.15.16:0/833109892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb8007660 con 0x7efcc806a190 2023-11-13T11:01:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.639+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc80a8a50 con 0x7efcc806a190 2023-11-13T11:01:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.639+0000 7efcbffff640 1 -- 172.21.15.16:0/833109892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8004a20 con 0x7efcc806a190 2023-11-13T11:01:39.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.639+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc80a9010 con 0x7efcc806a190 2023-11-13T11:01:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.640+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc804e4d0 con 0x7efcc806a190 2023-11-13T11:01:39.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.641+0000 7efcbffff640 1 -- 172.21.15.16:0/833109892 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcb8004b80 con 0x7efcc806a190 2023-11-13T11:01:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.642+0000 7efcbffff640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efca806ae40 0x7efca806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.643+0000 7efcbffff640 1 -- 172.21.15.16:0/833109892 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efcb8088c90 con 0x7efcc806a190 2023-11-13T11:01:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.643+0000 7efcc6ffd640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efca806ae40 0x7efca806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:39.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.645+0000 7efcc6ffd640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efca806ae40 0x7efca806d300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efcb0036000 tx=0x7efcb003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:39.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.647+0000 7efcbffff640 1 -- 172.21.15.16:0/833109892 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efcb8053bf0 con 0x7efcc806a190 2023-11-13T11:01:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.844+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcc80a9790 con 0x7efcc806a190 2023-11-13T11:01:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.847+0000 7efcbffff640 1 -- 172.21.15.16:0/833109892 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16739 (secure 0 0 0) 0x7efcb8018020 con 0x7efcc806a190 2023-11-13T11:01:39.853 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:39.853 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":54,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:01:38.428523+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:01:36.305098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-13T11:01:37.461852+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:01:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efca806ae40 msgr2=0x7efca806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efca806ae40 0x7efca806d300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efcb0036000 tx=0x7efcb003c040 comp rx=0 tx=0).stop 2023-11-13T11:01:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806a190 msgr2=0x7efcc815d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806a190 0x7efcc815d270 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c4d0 tx=0x7efcb800c990 comp rx=0 tx=0).stop 2023-11-13T11:01:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 shutdown_connections 2023-11-13T11:01:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efca806ae40 0x7efca806d300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc8161060 0x7efcc80a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc806f970 0x7efcc80a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 --2- 172.21.15.16:0/833109892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc806a190 0x7efcc815d270 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 >> 172.21.15.16:0/833109892 conn(0x7efcc80b04a0 msgr2=0x7efcc8163d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 shutdown_connections 2023-11-13T11:01:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:39.850+0000 7efcccc27640 1 -- 172.21.15.16:0/833109892 wait complete. 2023-11-13T11:01:39.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-13T11:01:40.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.095+0000 7f13a5fdc640 1 Processor -- start 2023-11-13T11:01:40.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.095+0000 7f13a5fdc640 1 -- start start 2023-11-13T11:01:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.096+0000 7f13a5fdc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 0x7f13a006b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.096+0000 7f13a5fdc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13a015d250 0x7f13a00cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.096+0000 7f13a5fdc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a00cf950 con 0x7f13a00cfde0 2023-11-13T11:01:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.096+0000 7f13a5fdc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a00cf020 con 0x7f13a015d250 2023-11-13T11:01:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.096+0000 7f13a5fdc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00cf300 con 0x7f13a015ce70 2023-11-13T11:01:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.097+0000 7f139effd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13a015d250 0x7f13a00cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.097+0000 7f139f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 0x7f13a006b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.097+0000 7f139effd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13a015d250 0x7f13a00cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:54262/0 (socket says 172.21.15.16:54262) 2023-11-13T11:01:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.097+0000 7f139effd640 1 -- 172.21.15.16:0/687044372 learned_addr learned my addr 172.21.15.16:0/687044372 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.098+0000 7f139effd640 1 -- 172.21.15.16:0/687044372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 msgr2=0x7f13a006b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.098+0000 7f139effd640 1 --2- 172.21.15.16:0/687044372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 0x7f13a006b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.098+0000 7f139effd640 1 -- 172.21.15.16:0/687044372 >> v1:172.21.15.184:6789/0 conn(0x7f13a00cfde0 legacy=0x7f13a00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:40.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.098+0000 7f139effd640 1 -- 172.21.15.16:0/687044372 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a00cf5e0 con 0x7f13a015d250 2023-11-13T11:01:40.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.098+0000 7f139f7fe640 1 --2- 172.21.15.16:0/687044372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 0x7f13a006b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T11:01:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.099+0000 7f139effd640 1 --2- 172.21.15.16:0/687044372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015d250 0x7f13a00cc2d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1388009870 tx=0x7f138802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=799b031ff96a0c3b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.100+0000 7f139e7fc640 1 -- 172.21.15.16:0/687044372 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388030840 con 0x7f13a015d250 2023-11-13T11:01:40.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.100+0000 7f139e7fc640 1 -- 172.21.15.16:0/687044372 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13880309a0 con 0x7f13a015d250 2023-11-13T11:01:40.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.100+0000 7f139e7fc640 1 -- 172.21.15.16:0/687044372 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388030d30 con 0x7f13a015d250 2023-11-13T11:01:40.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.101+0000 7f13a5fdc640 1 -- 172.21.15.16:0/687044372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015d250 msgr2=0x7f13a00cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:40.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.101+0000 7f13a5fdc640 1 --2- 172.21.15.16:0/687044372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015d250 0x7f13a00cc2d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1388009870 tx=0x7f138802b980 comp rx=0 tx=0).stop 2023-11-13T11:01:40.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.101+0000 7f13a5fdc640 1 -- 172.21.15.16:0/687044372 shutdown_connections 2023-11-13T11:01:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.101+0000 7f13a5fdc640 1 --2- 172.21.15.16:0/687044372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015d250 0x7f13a00cc2d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.101+0000 7f13a5fdc640 1 --2- 172.21.15.16:0/687044372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 0x7f13a006b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.101+0000 7f13a5fdc640 1 -- 172.21.15.16:0/687044372 >> 172.21.15.16:0/687044372 conn(0x7f13a00b2690 msgr2=0x7f13a00b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.101+0000 7f13a5fdc640 1 -- 172.21.15.16:0/687044372 shutdown_connections 2023-11-13T11:01:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.101+0000 7f13a5fdc640 1 -- 172.21.15.16:0/687044372 wait complete. 2023-11-13T11:01:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.103+0000 7f13a5fdc640 1 Processor -- start 2023-11-13T11:01:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.103+0000 7f13a5fdc640 1 -- start start 2023-11-13T11:01:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.104+0000 7f13a5fdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a00cfde0 0x7f13a014c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.104+0000 7f13a5fdc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 0x7f13a00cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.104+0000 7f13a5fdc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015a540 0x7f13a00cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.104+0000 7f13a5fdc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a015dff0 con 0x7f13a00cfde0 2023-11-13T11:01:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.104+0000 7f13a5fdc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a015a010 con 0x7f13a015a540 2023-11-13T11:01:40.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.104+0000 7f13a5fdc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a015a260 con 0x7f13a015ce70 2023-11-13T11:01:40.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.104+0000 7f139f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a00cfde0 0x7f13a014c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:40.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.104+0000 7f139f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a00cfde0 0x7f13a014c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43282/0 (socket says 172.21.15.16:43282) 2023-11-13T11:01:40.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.104+0000 7f139f7fe640 1 -- 172.21.15.16:0/2720639241 learned_addr learned my addr 172.21.15.16:0/2720639241 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f139f7fe640 1 -- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 msgr2=0x7f13a00cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f139f7fe640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 0x7f13a00cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f139f7fe640 1 -- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015a540 msgr2=0x7f13a00cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f139ffff640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015a540 0x7f13a00cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f139f7fe640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015a540 0x7f13a00cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f139f7fe640 1 -- 172.21.15.16:0/2720639241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1390009a70 con 0x7f13a00cfde0 2023-11-13T11:01:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f139ffff640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015a540 0x7f13a00cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f139f7fe640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a00cfde0 0x7f13a014c090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f139000efc0 tx=0x7f139000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f1397fff640 1 -- 172.21.15.16:0/2720639241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390014680 con 0x7f13a00cfde0 2023-11-13T11:01:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f1397fff640 1 -- 172.21.15.16:0/2720639241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139000f040 con 0x7f13a00cfde0 2023-11-13T11:01:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.105+0000 7f1397fff640 1 -- 172.21.15.16:0/2720639241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390014a00 con 0x7f13a00cfde0 2023-11-13T11:01:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.106+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13880094d0 con 0x7f13a00cfde0 2023-11-13T11:01:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.106+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a015d930 con 0x7f13a00cfde0 2023-11-13T11:01:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.107+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a004e4d0 con 0x7f13a00cfde0 2023-11-13T11:01:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.108+0000 7f1397fff640 1 -- 172.21.15.16:0/2720639241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1390014b60 con 0x7f13a00cfde0 2023-11-13T11:01:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.109+0000 7f1397fff640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f138006ae40 0x7f138006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.109+0000 7f1397fff640 1 -- 172.21.15.16:0/2720639241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f139008a7b0 con 0x7f13a00cfde0 2023-11-13T11:01:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.109+0000 7f139effd640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f138006ae40 0x7f138006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.110+0000 7f139effd640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f138006ae40 0x7f138006d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1388036000 tx=0x7f138803b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.113+0000 7f1397fff640 1 -- 172.21.15.16:0/2720639241 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1390055680 con 0x7f13a00cfde0 2023-11-13T11:01:40.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.339+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f13a0067b40 con 0x7f13a00cfde0 2023-11-13T11:01:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.988+0000 7f1397fff640 1 -- 172.21.15.16:0/2720639241 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1390058ca0 con 0x7f13a00cfde0 2023-11-13T11:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f138006ae40 msgr2=0x7f138006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f138006ae40 0x7f138006d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1388036000 tx=0x7f138803b040 comp rx=0 tx=0).stop 2023-11-13T11:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a00cfde0 msgr2=0x7f13a014c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a00cfde0 0x7f13a014c090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f139000efc0 tx=0x7f139000c490 comp rx=0 tx=0).stop 2023-11-13T11:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 shutdown_connections 2023-11-13T11:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f138006ae40 0x7f138006d300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13a015a540 0x7f13a00cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13a015ce70 0x7f13a00cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 --2- 172.21.15.16:0/2720639241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a00cfde0 0x7f13a014c090 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 >> 172.21.15.16:0/2720639241 conn(0x7f13a00b2690 msgr2=0x7f13a006ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 shutdown_connections 2023-11-13T11:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:40.992+0000 7f13a5fdc640 1 -- 172.21.15.16:0/2720639241 wait complete. 2023-11-13T11:01:41.018 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-13T11:01:41.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-13T11:01:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.228+0000 7f32f74c3640 1 Processor -- start 2023-11-13T11:01:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.229+0000 7f32f74c3640 1 -- start start 2023-11-13T11:01:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.230+0000 7f32f74c3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32f015b8e0 0x7f32f006c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.230+0000 7f32f74c3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f006b860 con 0x7f32f015bcc0 2023-11-13T11:01:41.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.231+0000 7f32f74c3640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f006bb40 con 0x7f32f00ce260 2023-11-13T11:01:41.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.231+0000 7f32f74c3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f006be20 con 0x7f32f015b8e0 2023-11-13T11:01:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.231+0000 7f32f5cc0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32f015bcc0 0x7f32f00cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56488/0 (socket says 172.21.15.16:56488) 2023-11-13T11:01:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.231+0000 7f32f5cc0640 1 -- 172.21.15.16:0/1048082693 learned_addr learned my addr 172.21.15.16:0/1048082693 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.231+0000 7f32f64c1640 1 --2- 172.21.15.16:0/1048082693 >> v2:172.21.15.16:3300/0 conn(0x7f32f015b8e0 0x7f32f006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.231+0000 7f32f64c1640 1 -- 172.21.15.16:0/1048082693 >> v1:172.21.15.184:6789/0 conn(0x7f32f00ce260 legacy=0x7f32f00cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:41.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.231+0000 7f32f64c1640 1 -- 172.21.15.16:0/1048082693 >> v1:172.21.15.16:6789/0 conn(0x7f32f015bcc0 legacy=0x7f32f00cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:41.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.232+0000 7f32f64c1640 1 -- 172.21.15.16:0/1048082693 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f006c100 con 0x7f32f015b8e0 2023-11-13T11:01:41.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.232+0000 7f32f54bf640 1 -- 172.21.15.16:0/1048082693 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727265562 0 0) 0x7f32f006b860 con 0x7f32f015bcc0 2023-11-13T11:01:41.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.233+0000 7f32f64c1640 1 --2- 172.21.15.16:0/1048082693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f015b8e0 0x7f32f006c900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009d90 tx=0x7f32ec02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e53020f9e22e52 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:41.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.233+0000 7f32f54bf640 1 -- 172.21.15.16:0/1048082693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec02cc60 con 0x7f32f015b8e0 2023-11-13T11:01:41.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.233+0000 7f32f54bf640 1 -- 172.21.15.16:0/1048082693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec02cdc0 con 0x7f32f015b8e0 2023-11-13T11:01:41.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.233+0000 7f32f54bf640 1 -- 172.21.15.16:0/1048082693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec034660 con 0x7f32f015b8e0 2023-11-13T11:01:41.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.233+0000 7f32f74c3640 1 -- 172.21.15.16:0/1048082693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f015b8e0 msgr2=0x7f32f006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:41.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.233+0000 7f32f74c3640 1 --2- 172.21.15.16:0/1048082693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f015b8e0 0x7f32f006c900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009d90 tx=0x7f32ec02beb0 comp rx=0 tx=0).stop 2023-11-13T11:01:41.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.234+0000 7f32f74c3640 1 -- 172.21.15.16:0/1048082693 shutdown_connections 2023-11-13T11:01:41.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.234+0000 7f32f74c3640 1 --2- 172.21.15.16:0/1048082693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f015b8e0 0x7f32f006c900 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:41.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.234+0000 7f32f74c3640 1 -- 172.21.15.16:0/1048082693 >> 172.21.15.16:0/1048082693 conn(0x7f32f00b0970 msgr2=0x7f32f00b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:41.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.234+0000 7f32f74c3640 1 -- 172.21.15.16:0/1048082693 shutdown_connections 2023-11-13T11:01:41.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.234+0000 7f32f74c3640 1 -- 172.21.15.16:0/1048082693 wait complete. 2023-11-13T11:01:41.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.235+0000 7f32f74c3640 1 Processor -- start 2023-11-13T11:01:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.236+0000 7f32f74c3640 1 -- start start 2023-11-13T11:01:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f74c3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32f00ce260 0x7f32f0141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f74c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 0x7f32f0070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f74c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f006f940 0x7f32f0152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:41.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f74c3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f006fe00 con 0x7f32f006f940 2023-11-13T11:01:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f74c3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f00700e0 con 0x7f32f015b8e0 2023-11-13T11:01:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f74c3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f00703c0 con 0x7f32f00ce260 2023-11-13T11:01:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f6cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f006f940 0x7f32f0152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f6cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f006f940 0x7f32f0152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43308/0 (socket says 172.21.15.16:43308) 2023-11-13T11:01:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f6cc2640 1 -- 172.21.15.16:0/757693626 learned_addr learned my addr 172.21.15.16:0/757693626 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f64c1640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32f00ce260 0x7f32f0141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f5cc0640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 0x7f32f0070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f6cc2640 1 -- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32f00ce260 msgr2=0x7f32f0141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f6cc2640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32f00ce260 0x7f32f0141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f6cc2640 1 -- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 msgr2=0x7f32f0070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.237+0000 7f32f6cc2640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 0x7f32f0070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.238+0000 7f32f6cc2640 1 -- 172.21.15.16:0/757693626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec009a70 con 0x7f32f006f940 2023-11-13T11:01:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.238+0000 7f32f6cc2640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f006f940 0x7f32f0152fa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f32e800c3e0 tx=0x7f32e800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.238+0000 7f32e6ffd640 1 -- 172.21.15.16:0/757693626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8011790 con 0x7f32f006f940 2023-11-13T11:01:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.238+0000 7f32e6ffd640 1 -- 172.21.15.16:0/757693626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e80118f0 con 0x7f32f006f940 2023-11-13T11:01:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.238+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f00706a0 con 0x7f32f006f940 2023-11-13T11:01:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.238+0000 7f32e6ffd640 1 -- 172.21.15.16:0/757693626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8011ce0 con 0x7f32f006f940 2023-11-13T11:01:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.238+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f01424e0 con 0x7f32f006f940 2023-11-13T11:01:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.240+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f0142670 con 0x7f32f006f940 2023-11-13T11:01:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.241+0000 7f32e6ffd640 1 -- 172.21.15.16:0/757693626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32e801c460 con 0x7f32f006f940 2023-11-13T11:01:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.242+0000 7f32e6ffd640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32d406ae40 0x7f32d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.243+0000 7f32e6ffd640 1 -- 172.21.15.16:0/757693626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f32e80896a0 con 0x7f32f006f940 2023-11-13T11:01:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.243+0000 7f32f64c1640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32d406ae40 0x7f32d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.244+0000 7f32f64c1640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32d406ae40 0x7f32d406d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32ec036000 tx=0x7f32ec030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.247+0000 7f32e6ffd640 1 -- 172.21.15.16:0/757693626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32f0142670 con 0x7f32f006f940 2023-11-13T11:01:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:41.482+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f32f0070bc0 con 0x7f32f006f940 2023-11-13T11:01:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.002+0000 7f32e6ffd640 1 -- 172.21.15.16:0/757693626 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f32f0070bc0 con 0x7f32f006f940 2023-11-13T11:01:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32d406ae40 msgr2=0x7f32d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32d406ae40 0x7f32d406d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32ec036000 tx=0x7f32ec030000 comp rx=0 tx=0).stop 2023-11-13T11:01:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f006f940 msgr2=0x7f32f0152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f006f940 0x7f32f0152fa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f32e800c3e0 tx=0x7f32e800c8a0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 shutdown_connections 2023-11-13T11:01:42.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32d406ae40 0x7f32d406d300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f006f940 0x7f32f0152fa0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 0x7f32f0070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 --2- 172.21.15.16:0/757693626 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32f00ce260 0x7f32f0141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.006+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 >> 172.21.15.16:0/757693626 conn(0x7f32f00b0970 msgr2=0x7f32f00cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.007+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 shutdown_connections 2023-11-13T11:01:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.007+0000 7f32f74c3640 1 -- 172.21.15.16:0/757693626 wait complete. 2023-11-13T11:01:42.034 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-13T11:01:42.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:01:42.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.260+0000 7efc37047640 1 Processor -- start 2023-11-13T11:01:42.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.261+0000 7efc37047640 1 -- start start 2023-11-13T11:01:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.262+0000 7efc37047640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc300b30e0 0x7efc300d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.262+0000 7efc37047640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300b3cf0 con 0x7efc300d2ae0 2023-11-13T11:01:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.262+0000 7efc37047640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300d3480 con 0x7efc3015cfc0 2023-11-13T11:01:42.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.262+0000 7efc37047640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300b3550 con 0x7efc300b30e0 2023-11-13T11:01:42.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.262+0000 7efc35844640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efc300d2ae0 0x7efc300b29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56490/0 (socket says 172.21.15.16:56490) 2023-11-13T11:01:42.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.262+0000 7efc35844640 1 -- 172.21.15.16:0/434383941 learned_addr learned my addr 172.21.15.16:0/434383941 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.262+0000 7efc36846640 1 --2- 172.21.15.16:0/434383941 >> v2:172.21.15.184:3300/0 conn(0x7efc300b30e0 0x7efc300d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443985044 0 0) 0x7efc300b3cf0 con 0x7efc300d2ae0 2023-11-13T11:01:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc1c003520 con 0x7efc300d2ae0 2023-11-13T11:01:42.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2978103339 0 0) 0x7efc1c003520 con 0x7efc300d2ae0 2023-11-13T11:01:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 >> v2:172.21.15.184:3300/0 conn(0x7efc300b30e0 msgr2=0x7efc300d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 --2- 172.21.15.16:0/434383941 >> v2:172.21.15.184:3300/0 conn(0x7efc300b30e0 0x7efc300d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 >> v1:172.21.15.184:6789/0 conn(0x7efc3015cfc0 legacy=0x7efc300cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300cde80 con 0x7efc300d2ae0 2023-11-13T11:01:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7efc20001670 con 0x7efc300d2ae0 2023-11-13T11:01:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 >> v1:172.21.15.16:6789/0 conn(0x7efc300d2ae0 legacy=0x7efc300b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc1c0047c0 0x7efc1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc1c0051d0 0x7efc1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300b3550 con 0x7efc1c0039c0 2023-11-13T11:01:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c00c120 con 0x7efc1c0047c0 2023-11-13T11:01:42.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.263+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c00c2e0 con 0x7efc1c0051d0 2023-11-13T11:01:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.264+0000 7efc36045640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.264+0000 7efc35844640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc1c0047c0 0x7efc1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.264+0000 7efc36045640 1 -- 172.21.15.16:0/434383941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc1c0051d0 msgr2=0x7efc1c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.264+0000 7efc36045640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc1c0051d0 0x7efc1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.264+0000 7efc36045640 1 -- 172.21.15.16:0/434383941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc1c0047c0 msgr2=0x7efc1c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.264+0000 7efc36045640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc1c0047c0 0x7efc1c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.264+0000 7efc36045640 1 -- 172.21.15.16:0/434383941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c003520 con 0x7efc1c0039c0 2023-11-13T11:01:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.264+0000 7efc36846640 1 -- 172.21.15.16:0/434383941 reap_dead start 2023-11-13T11:01:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.265+0000 7efc36045640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc1c003de0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efc2c001720 tx=0x7efc1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2416c5f41619a44 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.266+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0052d0 con 0x7efc1c0039c0 2023-11-13T11:01:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.266+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c02e050 con 0x7efc1c0039c0 2023-11-13T11:01:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.266+0000 7efc35043640 1 -- 172.21.15.16:0/434383941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c003830 con 0x7efc1c0039c0 2023-11-13T11:01:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.266+0000 7efc37047640 1 -- 172.21.15.16:0/434383941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 msgr2=0x7efc1c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.266+0000 7efc37047640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc1c003de0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efc2c001720 tx=0x7efc1c0057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.266+0000 7efc37047640 1 -- 172.21.15.16:0/434383941 shutdown_connections 2023-11-13T11:01:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.266+0000 7efc37047640 1 --2- 172.21.15.16:0/434383941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc1c003de0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.266+0000 7efc37047640 1 -- 172.21.15.16:0/434383941 >> 172.21.15.16:0/434383941 conn(0x7efc3008c570 msgr2=0x7efc3009be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.266+0000 7efc37047640 1 -- 172.21.15.16:0/434383941 shutdown_connections 2023-11-13T11:01:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.267+0000 7efc37047640 1 -- 172.21.15.16:0/434383941 wait complete. 2023-11-13T11:01:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.268+0000 7efc37047640 1 Processor -- start 2023-11-13T11:01:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.268+0000 7efc37047640 1 -- start start 2023-11-13T11:01:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.269+0000 7efc37047640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc300c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.269+0000 7efc37047640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc300c92a0 0x7efc3015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.269+0000 7efc37047640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc301601a0 0x7efc301f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.269+0000 7efc37047640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc30159290 con 0x7efc1c0039c0 2023-11-13T11:01:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.269+0000 7efc37047640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3015d5d0 con 0x7efc300c92a0 2023-11-13T11:01:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.269+0000 7efc37047640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3015d8b0 con 0x7efc301601a0 2023-11-13T11:01:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc300c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc300c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43330/0 (socket says 172.21.15.16:43330) 2023-11-13T11:01:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36045640 1 -- 172.21.15.16:0/4021773265 learned_addr learned my addr 172.21.15.16:0/4021773265 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc35844640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc300c92a0 0x7efc3015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36846640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc301601a0 0x7efc301f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36045640 1 -- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc301601a0 msgr2=0x7efc301f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36045640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc301601a0 0x7efc301f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36045640 1 -- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc300c92a0 msgr2=0x7efc3015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36045640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc300c92a0 0x7efc3015fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36045640 1 -- 172.21.15.16:0/4021773265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300b6870 con 0x7efc1c0039c0 2023-11-13T11:01:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.270+0000 7efc36846640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc301601a0 0x7efc301f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.271+0000 7efc36045640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc300c8d60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005040 tx=0x7efc2c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.271+0000 7efc26ffd640 1 -- 172.21.15.16:0/4021773265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c007640 con 0x7efc1c0039c0 2023-11-13T11:01:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.271+0000 7efc26ffd640 1 -- 172.21.15.16:0/4021773265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c0069e0 con 0x7efc1c0039c0 2023-11-13T11:01:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.271+0000 7efc26ffd640 1 -- 172.21.15.16:0/4021773265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c006d70 con 0x7efc1c0039c0 2023-11-13T11:01:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.271+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc300b6b50 con 0x7efc1c0039c0 2023-11-13T11:01:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.271+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3015db90 con 0x7efc1c0039c0 2023-11-13T11:01:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.272+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc300d2f40 con 0x7efc1c0039c0 2023-11-13T11:01:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.273+0000 7efc26ffd640 1 -- 172.21.15.16:0/4021773265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc2c0035f0 con 0x7efc1c0039c0 2023-11-13T11:01:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.274+0000 7efc26ffd640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc1406ae70 0x7efc1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.274+0000 7efc26ffd640 1 -- 172.21.15.16:0/4021773265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efc2c0ab4d0 con 0x7efc1c0039c0 2023-11-13T11:01:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.275+0000 7efc35844640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc1406ae70 0x7efc1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.276+0000 7efc35844640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc1406ae70 0x7efc1406d330 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efc300d1eb0 tx=0x7efc20009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.278+0000 7efc26ffd640 1 -- 172.21.15.16:0/4021773265 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc2c076310 con 0x7efc1c0039c0 2023-11-13T11:01:42.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.517+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc30066310 con 0x7efc1c0039c0 2023-11-13T11:01:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.518+0000 7efc26ffd640 1 -- 172.21.15.16:0/4021773265 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4919 (secure 0 0 0) 0x7efc2c041090 con 0x7efc1c0039c0 2023-11-13T11:01:42.524 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:42.524 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:01:41.484670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:01:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2023-11-13T11:01:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc1406ae70 msgr2=0x7efc1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc1406ae70 0x7efc1406d330 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efc300d1eb0 tx=0x7efc20009040 comp rx=0 tx=0).stop 2023-11-13T11:01:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 msgr2=0x7efc300c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc300c8d60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005040 tx=0x7efc2c001120 comp rx=0 tx=0).stop 2023-11-13T11:01:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 shutdown_connections 2023-11-13T11:01:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efc1406ae70 0x7efc1406d330 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc301601a0 0x7efc301f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc300c92a0 0x7efc3015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 --2- 172.21.15.16:0/4021773265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc1c0039c0 0x7efc300c8d60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 >> 172.21.15.16:0/4021773265 conn(0x7efc3008c570 msgr2=0x7efc300d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 shutdown_connections 2023-11-13T11:01:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.522+0000 7efc37047640 1 -- 172.21.15.16:0/4021773265 wait complete. 2023-11-13T11:01:42.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:01:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.766+0000 7f50cffff640 1 Processor -- start 2023-11-13T11:01:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.766+0000 7f50cffff640 1 -- start start 2023-11-13T11:01:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.767+0000 7f50cffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f50d006a8a0 0x7f50d00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50cffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d006fc50 con 0x7f50d015cc90 2023-11-13T11:01:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50cffff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d006ff30 con 0x7f50d0070cf0 2023-11-13T11:01:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50cffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d0070210 con 0x7f50d006a8a0 2023-11-13T11:01:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50c7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f50d006a8a0 0x7f50d00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50ceffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f50d015cc90 0x7f50d006a190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56494/0 (socket says 172.21.15.16:56494) 2023-11-13T11:01:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50ceffd640 1 -- 172.21.15.16:0/3193859976 learned_addr learned my addr 172.21.15.16:0/3193859976 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50c7fff640 1 -- 172.21.15.16:0/3193859976 >> v1:172.21.15.184:6789/0 conn(0x7f50d0070cf0 legacy=0x7f50d00b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50c7fff640 1 -- 172.21.15.16:0/3193859976 >> v1:172.21.15.16:6789/0 conn(0x7f50d015cc90 legacy=0x7f50d006a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50c7fff640 1 -- 172.21.15.16:0/3193859976 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00704f0 con 0x7f50d006a8a0 2023-11-13T11:01:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.768+0000 7f50ce7fc640 1 -- 172.21.15.16:0/3193859976 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240660763 0 0) 0x7f50d006fc50 con 0x7f50d015cc90 2023-11-13T11:01:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.770+0000 7f50c7fff640 1 --2- 172.21.15.16:0/3193859976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 0x7f50d00b2110 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f50b80097a0 tx=0x7f50b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99366bc822e52a29 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.770+0000 7f50ce7fc640 1 -- 172.21.15.16:0/3193859976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8031830 con 0x7f50d006a8a0 2023-11-13T11:01:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.770+0000 7f50ce7fc640 1 -- 172.21.15.16:0/3193859976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b8031990 con 0x7f50d006a8a0 2023-11-13T11:01:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.770+0000 7f50ce7fc640 1 -- 172.21.15.16:0/3193859976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8031d20 con 0x7f50d006a8a0 2023-11-13T11:01:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.770+0000 7f50cffff640 1 -- 172.21.15.16:0/3193859976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 msgr2=0x7f50d00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.770+0000 7f50cffff640 1 --2- 172.21.15.16:0/3193859976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 0x7f50d00b2110 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f50b80097a0 tx=0x7f50b802b8d0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.771+0000 7f50cffff640 1 -- 172.21.15.16:0/3193859976 shutdown_connections 2023-11-13T11:01:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.771+0000 7f50cffff640 1 --2- 172.21.15.16:0/3193859976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 0x7f50d00b2110 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.771+0000 7f50cffff640 1 -- 172.21.15.16:0/3193859976 >> 172.21.15.16:0/3193859976 conn(0x7f50d00b04a0 msgr2=0x7f50d00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.771+0000 7f50cffff640 1 -- 172.21.15.16:0/3193859976 shutdown_connections 2023-11-13T11:01:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.771+0000 7f50cffff640 1 -- 172.21.15.16:0/3193859976 wait complete. 2023-11-13T11:01:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.773+0000 7f50cffff640 1 Processor -- start 2023-11-13T11:01:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.773+0000 7f50cffff640 1 -- start start 2023-11-13T11:01:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 0x7f50d0142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50cffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50d0070cf0 0x7f50d013ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50cffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50d015cc90 0x7f50d0141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50cffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d013fd50 con 0x7f50d006a8a0 2023-11-13T11:01:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50cffff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d01413d0 con 0x7f50d0070cf0 2023-11-13T11:01:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50cffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d0140520 con 0x7f50d015cc90 2023-11-13T11:01:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 0x7f50d0142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 0x7f50d0142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43346/0 (socket says 172.21.15.16:43346) 2023-11-13T11:01:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50ceffd640 1 -- 172.21.15.16:0/4031301967 learned_addr learned my addr 172.21.15.16:0/4031301967 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.774+0000 7f50c7fff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50d0070cf0 0x7f50d013ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50cf7fe640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50d015cc90 0x7f50d0141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50ceffd640 1 -- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50d015cc90 msgr2=0x7f50d0141860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50ceffd640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50d015cc90 0x7f50d0141860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50ceffd640 1 -- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50d0070cf0 msgr2=0x7f50d013ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50ceffd640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50d0070cf0 0x7f50d013ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50ceffd640 1 -- 172.21.15.16:0/4031301967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8009400 con 0x7f50d006a8a0 2023-11-13T11:01:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50c7fff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50d0070cf0 0x7f50d013ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50cf7fe640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50d015cc90 0x7f50d0141860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50ceffd640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 0x7f50d0142c10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c430 tx=0x7f50c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50c77fe640 1 -- 172.21.15.16:0/4031301967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c00074e0 con 0x7f50d006a8a0 2023-11-13T11:01:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50c77fe640 1 -- 172.21.15.16:0/4031301967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0007660 con 0x7f50d006a8a0 2023-11-13T11:01:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50c77fe640 1 -- 172.21.15.16:0/4031301967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0004a20 con 0x7f50d006a8a0 2023-11-13T11:01:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.775+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d01407c0 con 0x7f50d006a8a0 2023-11-13T11:01:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.776+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d0140d80 con 0x7f50d006a8a0 2023-11-13T11:01:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.777+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d004e4d0 con 0x7f50d006a8a0 2023-11-13T11:01:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.778+0000 7f50c77fe640 1 -- 172.21.15.16:0/4031301967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50c0004b80 con 0x7f50d006a8a0 2023-11-13T11:01:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.779+0000 7f50c77fe640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50b006ae40 0x7f50b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.779+0000 7f50c77fe640 1 -- 172.21.15.16:0/4031301967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f50c0089710 con 0x7f50d006a8a0 2023-11-13T11:01:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.780+0000 7f50c7fff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50b006ae40 0x7f50b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.781+0000 7f50c7fff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50b006ae40 0x7f50b006d300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f50b8036000 tx=0x7f50b8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.784+0000 7f50c77fe640 1 -- 172.21.15.16:0/4031301967 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50c0054550 con 0x7f50d006a8a0 2023-11-13T11:01:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.978+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50d00713c0 con 0x7f50d006a8a0 2023-11-13T11:01:42.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.981+0000 7f50c77fe640 1 -- 172.21.15.16:0/4031301967 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16726 (secure 0 0 0) 0x7f50c0057b70 con 0x7f50d006a8a0 2023-11-13T11:01:42.987 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:42.987 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":55,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:01:39.970723+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:01:36.305098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-13T11:01:37.461852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:01:42.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50b006ae40 msgr2=0x7f50b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50b006ae40 0x7f50b006d300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f50b8036000 tx=0x7f50b8008030 comp rx=0 tx=0).stop 2023-11-13T11:01:42.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 msgr2=0x7f50d0142c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:42.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 0x7f50d0142c10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c430 tx=0x7f50c000c8f0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 shutdown_connections 2023-11-13T11:01:42.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f50b006ae40 0x7f50b006d300 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f50d015cc90 0x7f50d0141860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50d0070cf0 0x7f50d013ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 --2- 172.21.15.16:0/4031301967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d006a8a0 0x7f50d0142c10 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 >> 172.21.15.16:0/4031301967 conn(0x7f50d00b04a0 msgr2=0x7f50d015a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 shutdown_connections 2023-11-13T11:01:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:42.984+0000 7f50cffff640 1 -- 172.21.15.16:0/4031301967 wait complete. 2023-11-13T11:01:43.011 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-13T11:01:43.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:01:43.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.237+0000 7f27ce10f640 1 Processor -- start 2023-11-13T11:01:43.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.237+0000 7f27ce10f640 1 -- start start 2023-11-13T11:01:43.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.238+0000 7f27ce10f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f27c815d0f0 0x7f27c80d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.238+0000 7f27ce10f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c80d3c70 con 0x7f27c80d0ac0 2023-11-13T11:01:43.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.238+0000 7f27ce10f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c80c69d0 con 0x7f27c80cfe20 2023-11-13T11:01:43.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.238+0000 7f27ce10f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c80c6cb0 con 0x7f27c815d0f0 2023-11-13T11:01:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.239+0000 7f27c7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f27c80d0ac0 0x7f27c80b4060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59938/0 (socket says 172.21.15.16:59938) 2023-11-13T11:01:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.239+0000 7f27c7fff640 1 -- 172.21.15.16:0/1633492491 learned_addr learned my addr 172.21.15.16:0/1633492491 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.239+0000 7f27c77fe640 1 --2- 172.21.15.16:0/1633492491 >> v2:172.21.15.184:3300/0 conn(0x7f27c815d0f0 0x7f27c80d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.239+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438506603 0 0) 0x7f27c0001110 con 0x7f27c80d0ac0 2023-11-13T11:01:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.239+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b0003520 con 0x7f27c80d0ac0 2023-11-13T11:01:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1713355424 0 0) 0x7f27b0003520 con 0x7f27c80d0ac0 2023-11-13T11:01:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 >> v2:172.21.15.184:3300/0 conn(0x7f27c815d0f0 msgr2=0x7f27c80d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 --2- 172.21.15.16:0/1633492491 >> v2:172.21.15.184:3300/0 conn(0x7f27c815d0f0 0x7f27c80d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 >> v1:172.21.15.184:6789/0 conn(0x7f27c80cfe20 legacy=0x7f27c80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80c6f90 con 0x7f27c80d0ac0 2023-11-13T11:01:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f27c0001800 con 0x7f27c80d0ac0 2023-11-13T11:01:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 >> v1:172.21.15.16:6789/0 conn(0x7f27c80d0ac0 legacy=0x7f27c80b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b0003970 0x7f27b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27b00047c0 0x7f27b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b0005180 0x7f27b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c80c6cb0 con 0x7f27b00047c0 2023-11-13T11:01:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b000c030 con 0x7f27b0005180 2023-11-13T11:01:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b000c1f0 con 0x7f27b0003970 2023-11-13T11:01:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c6ffd640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27b00047c0 0x7f27b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c7fff640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b0005180 0x7f27b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.240+0000 7f27c77fe640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b0003970 0x7f27b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.241+0000 7f27c6ffd640 1 -- 172.21.15.16:0/1633492491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b0003970 msgr2=0x7f27b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.241+0000 7f27c6ffd640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b0003970 0x7f27b0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.241+0000 7f27c6ffd640 1 -- 172.21.15.16:0/1633492491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b0005180 msgr2=0x7f27b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.241+0000 7f27c6ffd640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b0005180 0x7f27b000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.241+0000 7f27c6ffd640 1 -- 172.21.15.16:0/1633492491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0003520 con 0x7f27b00047c0 2023-11-13T11:01:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.241+0000 7f27c7fff640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b0005180 0x7f27b000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.241+0000 7f27c7fff640 1 -- 172.21.15.16:0/1633492491 reap_dead start 2023-11-13T11:01:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.242+0000 7f27c6ffd640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27b00047c0 0x7f27b0004c40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f27bc001e20 tx=0x7f27b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256739b4a25eb557 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.242+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc003b70 con 0x7f27b00047c0 2023-11-13T11:01:43.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.242+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc003cd0 con 0x7f27b00047c0 2023-11-13T11:01:43.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.242+0000 7f27c67fc640 1 -- 172.21.15.16:0/1633492491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc005310 con 0x7f27b00047c0 2023-11-13T11:01:43.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.243+0000 7f27ce10f640 1 -- 172.21.15.16:0/1633492491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27b00047c0 msgr2=0x7f27b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.243+0000 7f27ce10f640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27b00047c0 0x7f27b0004c40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f27bc001e20 tx=0x7f27b00057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.243+0000 7f27ce10f640 1 -- 172.21.15.16:0/1633492491 shutdown_connections 2023-11-13T11:01:43.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.243+0000 7f27ce10f640 1 --2- 172.21.15.16:0/1633492491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27b00047c0 0x7f27b0004c40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.243+0000 7f27ce10f640 1 -- 172.21.15.16:0/1633492491 >> 172.21.15.16:0/1633492491 conn(0x7f27c80d2ce0 msgr2=0x7f27c80d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:43.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.243+0000 7f27ce10f640 1 -- 172.21.15.16:0/1633492491 shutdown_connections 2023-11-13T11:01:43.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.243+0000 7f27ce10f640 1 -- 172.21.15.16:0/1633492491 wait complete. 2023-11-13T11:01:43.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.245+0000 7f27ce10f640 1 Processor -- start 2023-11-13T11:01:43.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.245+0000 7f27ce10f640 1 -- start start 2023-11-13T11:01:43.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27ce10f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b00047c0 0x7f27c81f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27ce10f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27c8155f60 0x7f27c81f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27ce10f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27c81f37a0 0x7f27c80b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27ce10f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81f42a0 con 0x7f27c81f37a0 2023-11-13T11:01:43.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27ce10f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81f7720 con 0x7f27b00047c0 2023-11-13T11:01:43.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27ce10f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81f3ce0 con 0x7f27c8155f60 2023-11-13T11:01:43.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27c81f37a0 0x7f27c80b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27c81f37a0 0x7f27c80b7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41896/0 (socket says 172.21.15.16:41896) 2023-11-13T11:01:43.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27c7fff640 1 -- 172.21.15.16:0/926833510 learned_addr learned my addr 172.21.15.16:0/926833510 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:43.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27c77fe640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b00047c0 0x7f27c81f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.246+0000 7f27c6ffd640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27c8155f60 0x7f27c81f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27c7fff640 1 -- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27c8155f60 msgr2=0x7f27c81f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27c7fff640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27c8155f60 0x7f27c81f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27c7fff640 1 -- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b00047c0 msgr2=0x7f27c81f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27c7fff640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b00047c0 0x7f27c81f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27c7fff640 1 -- 172.21.15.16:0/926833510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c81f3fc0 con 0x7f27c81f37a0 2023-11-13T11:01:43.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27c7fff640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27c81f37a0 0x7f27c80b7f10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f27c0000c00 tx=0x7f27c000d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27affff640 1 -- 172.21.15.16:0/926833510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0007440 con 0x7f27c81f37a0 2023-11-13T11:01:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27affff640 1 -- 172.21.15.16:0/926833510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c00075a0 con 0x7f27c81f37a0 2023-11-13T11:01:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c81f69e0 con 0x7f27c81f37a0 2023-11-13T11:01:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.247+0000 7f27affff640 1 -- 172.21.15.16:0/926833510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0004a00 con 0x7f27c81f37a0 2023-11-13T11:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.248+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c81563e0 con 0x7f27c81f37a0 2023-11-13T11:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.249+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c81f6840 con 0x7f27c81f37a0 2023-11-13T11:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.250+0000 7f27affff640 1 -- 172.21.15.16:0/926833510 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27c0004b60 con 0x7f27c81f37a0 2023-11-13T11:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.251+0000 7f27affff640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f27b406ae40 0x7f27b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.251+0000 7f27affff640 1 -- 172.21.15.16:0/926833510 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f27c0088dd0 con 0x7f27c81f37a0 2023-11-13T11:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.251+0000 7f27c77fe640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f27b406ae40 0x7f27b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.252+0000 7f27c77fe640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f27b406ae40 0x7f27b406d300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009a90 tx=0x7f27b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.256+0000 7f27affff640 1 -- 172.21.15.16:0/926833510 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27c81f6840 con 0x7f27c81f37a0 2023-11-13T11:01:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.446+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27c815d4d0 con 0x7f27b406ae40 2023-11-13T11:01:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.473+0000 7f27affff640 1 -- 172.21.15.16:0/926833510 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286421 (secure 0 0 0) 0x7f27b80129b0 con 0x7f27b406ae40 2023-11-13T11:01:43.482 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:01:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.478+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f27b406ae40 msgr2=0x7f27b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.478+0000 7f27ce10f640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f27b406ae40 0x7f27b406d300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009a90 tx=0x7f27b8008040 comp rx=0 tx=0).stop 2023-11-13T11:01:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.478+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27c81f37a0 msgr2=0x7f27c80b7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.478+0000 7f27ce10f640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27c81f37a0 0x7f27c80b7f10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f27c0000c00 tx=0x7f27c000d950 comp rx=0 tx=0).stop 2023-11-13T11:01:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.479+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 shutdown_connections 2023-11-13T11:01:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.479+0000 7f27ce10f640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f27b406ae40 0x7f27b406d300 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.479+0000 7f27ce10f640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27c81f37a0 0x7f27c80b7f10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.479+0000 7f27ce10f640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27c8155f60 0x7f27c81f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.479+0000 7f27ce10f640 1 --2- 172.21.15.16:0/926833510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b00047c0 0x7f27c81f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.479+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 >> 172.21.15.16:0/926833510 conn(0x7f27c80d2ce0 msgr2=0x7f27c80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.479+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 shutdown_connections 2023-11-13T11:01:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.479+0000 7f27ce10f640 1 -- 172.21.15.16:0/926833510 wait complete. 2023-11-13T11:01:43.502 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-11-13T11:01:42.117958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":184,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":359544,"kb_used_data":6028,"kb_used_omap":12,"kb_used_meta":353395,"kb_avail":754615176,"statfs":{"total":773094113280,"available":772725940224,"internally_reserved":0,"allocated":6172672,"data_stored":3364718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":361877041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.187203"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997544+0000","last_change":"2023-11-13T11:01:38.993640+0000","last_active":"2023-11-13T11:01:38.997544+0000","last_peered":"2023-11-13T11:01:38.997544+0000","last_clean":"2023-11-13T11:01:38.997544+0000","last_became_active":"2023-11-13T11:01:38.991479+0000","last_became_peered":"2023-11-13T11:01:38.991479+0000","last_unstale":"2023-11-13T11:01:38.997544+0000","last_undegraded":"2023-11-13T11:01:38.997544+0000","last_fullsized":"2023-11-13T11:01:38.997544+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977509+0000","last_change":"2023-11-13T11:01:38.975999+0000","last_active":"2023-11-13T11:01:38.977509+0000","last_peered":"2023-11-13T11:01:38.977509+0000","last_clean":"2023-11-13T11:01:38.977509+0000","last_became_active":"2023-11-13T11:01:38.975726+0000","last_became_peered":"2023-11-13T11:01:38.975726+0000","last_unstale":"2023-11-13T11:01:38.977509+0000","last_undegraded":"2023-11-13T11:01:38.977509+0000","last_fullsized":"2023-11-13T11:01:38.977509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:33:09.247389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991945+0000","last_change":"2023-11-13T11:01:38.984201+0000","last_active":"2023-11-13T11:01:38.991945+0000","last_peered":"2023-11-13T11:01:38.991945+0000","last_clean":"2023-11-13T11:01:38.991945+0000","last_became_active":"2023-11-13T11:01:38.983716+0000","last_became_peered":"2023-11-13T11:01:38.983716+0000","last_unstale":"2023-11-13T11:01:38.991945+0000","last_undegraded":"2023-11-13T11:01:38.991945+0000","last_fullsized":"2023-11-13T11:01:38.991945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:37:29.570297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980437+0000","last_change":"2023-11-13T11:01:38.975163+0000","last_active":"2023-11-13T11:01:38.980437+0000","last_peered":"2023-11-13T11:01:38.980437+0000","last_clean":"2023-11-13T11:01:38.980437+0000","last_became_active":"2023-11-13T11:01:38.974878+0000","last_became_peered":"2023-11-13T11:01:38.974878+0000","last_unstale":"2023-11-13T11:01:38.980437+0000","last_undegraded":"2023-11-13T11:01:38.980437+0000","last_fullsized":"2023-11-13T11:01:38.980437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.970996+0000","last_change":"2023-11-13T11:01:38.968974+0000","last_active":"2023-11-13T11:01:38.970996+0000","last_peered":"2023-11-13T11:01:38.970996+0000","last_clean":"2023-11-13T11:01:38.970996+0000","last_became_active":"2023-11-13T11:01:38.968747+0000","last_became_peered":"2023-11-13T11:01:38.968747+0000","last_unstale":"2023-11-13T11:01:38.970996+0000","last_undegraded":"2023-11-13T11:01:38.970996+0000","last_fullsized":"2023-11-13T11:01:38.970996+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:14:08.138073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.961941+0000","last_change":"2023-11-13T11:01:37.961941+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.961941+0000","last_undegraded":"2023-11-13T11:01:37.961941+0000","last_fullsized":"2023-11-13T11:01:37.961941+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.006474+0000","last_change":"2023-11-13T11:01:39.003582+0000","last_active":"2023-11-13T11:01:39.006474+0000","last_peered":"2023-11-13T11:01:39.006474+0000","last_clean":"2023-11-13T11:01:39.006474+0000","last_became_active":"2023-11-13T11:01:39.003324+0000","last_became_peered":"2023-11-13T11:01:39.003324+0000","last_unstale":"2023-11-13T11:01:39.006474+0000","last_undegraded":"2023-11-13T11:01:39.006474+0000","last_fullsized":"2023-11-13T11:01:39.006474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.962816+0000","last_change":"2023-11-13T11:01:37.962816+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.962816+0000","last_undegraded":"2023-11-13T11:01:37.962816+0000","last_fullsized":"2023-11-13T11:01:37.962816+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.003277+0000","last_change":"2023-11-13T11:01:38.997117+0000","last_active":"2023-11-13T11:01:39.003277+0000","last_peered":"2023-11-13T11:01:39.003277+0000","last_clean":"2023-11-13T11:01:39.003277+0000","last_became_active":"2023-11-13T11:01:38.996801+0000","last_became_peered":"2023-11-13T11:01:38.996801+0000","last_unstale":"2023-11-13T11:01:39.003277+0000","last_undegraded":"2023-11-13T11:01:39.003277+0000","last_fullsized":"2023-11-13T11:01:39.003277+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:03:35.448965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990548+0000","last_change":"2023-11-13T11:01:38.987650+0000","last_active":"2023-11-13T11:01:38.990548+0000","last_peered":"2023-11-13T11:01:38.990548+0000","last_clean":"2023-11-13T11:01:38.990548+0000","last_became_active":"2023-11-13T11:01:38.987270+0000","last_became_peered":"2023-11-13T11:01:38.987270+0000","last_unstale":"2023-11-13T11:01:38.990548+0000","last_undegraded":"2023-11-13T11:01:38.990548+0000","last_fullsized":"2023-11-13T11:01:38.990548+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:25.371717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992670+0000","last_change":"2023-11-13T11:01:38.984512+0000","last_active":"2023-11-13T11:01:38.992670+0000","last_peered":"2023-11-13T11:01:38.992670+0000","last_clean":"2023-11-13T11:01:38.992670+0000","last_became_active":"2023-11-13T11:01:38.984243+0000","last_became_peered":"2023-11-13T11:01:38.984243+0000","last_unstale":"2023-11-13T11:01:38.992670+0000","last_undegraded":"2023-11-13T11:01:38.992670+0000","last_fullsized":"2023-11-13T11:01:38.992670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.963379+0000","last_change":"2023-11-13T11:01:37.963379+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.963379+0000","last_undegraded":"2023-11-13T11:01:37.963379+0000","last_fullsized":"2023-11-13T11:01:37.963379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.962841+0000","last_change":"2023-11-13T11:01:37.962841+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.962841+0000","last_undegraded":"2023-11-13T11:01:37.962841+0000","last_fullsized":"2023-11-13T11:01:37.962841+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.994357+0000","last_change":"2023-11-13T11:01:38.992189+0000","last_active":"2023-11-13T11:01:38.994357+0000","last_peered":"2023-11-13T11:01:38.994357+0000","last_clean":"2023-11-13T11:01:38.994357+0000","last_became_active":"2023-11-13T11:01:38.991290+0000","last_became_peered":"2023-11-13T11:01:38.991290+0000","last_unstale":"2023-11-13T11:01:38.994357+0000","last_undegraded":"2023-11-13T11:01:38.994357+0000","last_fullsized":"2023-11-13T11:01:38.994357+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:25.371717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.969075+0000","last_change":"2023-11-13T11:01:38.966119+0000","last_active":"2023-11-13T11:01:38.969075+0000","last_peered":"2023-11-13T11:01:38.969075+0000","last_clean":"2023-11-13T11:01:38.969075+0000","last_became_active":"2023-11-13T11:01:38.965812+0000","last_became_peered":"2023-11-13T11:01:38.965812+0000","last_unstale":"2023-11-13T11:01:38.969075+0000","last_undegraded":"2023-11-13T11:01:38.969075+0000","last_fullsized":"2023-11-13T11:01:38.969075+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.962280+0000","last_change":"2023-11-13T11:01:37.962280+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.962280+0000","last_undegraded":"2023-11-13T11:01:37.962280+0000","last_fullsized":"2023-11-13T11:01:37.962280+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997524+0000","last_change":"2023-11-13T11:01:38.993324+0000","last_active":"2023-11-13T11:01:38.997524+0000","last_peered":"2023-11-13T11:01:38.997524+0000","last_clean":"2023-11-13T11:01:38.997524+0000","last_became_active":"2023-11-13T11:01:38.991072+0000","last_became_peered":"2023-11-13T11:01:38.991072+0000","last_unstale":"2023-11-13T11:01:38.997524+0000","last_undegraded":"2023-11-13T11:01:38.997524+0000","last_fullsized":"2023-11-13T11:01:38.997524+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980403+0000","last_change":"2023-11-13T11:01:38.975150+0000","last_active":"2023-11-13T11:01:38.980403+0000","last_peered":"2023-11-13T11:01:38.980403+0000","last_clean":"2023-11-13T11:01:38.980403+0000","last_became_active":"2023-11-13T11:01:38.974837+0000","last_became_peered":"2023-11-13T11:01:38.974837+0000","last_unstale":"2023-11-13T11:01:38.980403+0000","last_undegraded":"2023-11-13T11:01:38.980403+0000","last_fullsized":"2023-11-13T11:01:38.980403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:19:00.701068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991920+0000","last_change":"2023-11-13T11:01:38.984024+0000","last_active":"2023-11-13T11:01:38.991920+0000","last_peered":"2023-11-13T11:01:38.991920+0000","last_clean":"2023-11-13T11:01:38.991920+0000","last_became_active":"2023-11-13T11:01:38.983338+0000","last_became_peered":"2023-11-13T11:01:38.983338+0000","last_unstale":"2023-11-13T11:01:38.991920+0000","last_undegraded":"2023-11-13T11:01:38.991920+0000","last_fullsized":"2023-11-13T11:01:38.991920+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:06:44.792510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973436+0000","last_change":"2023-11-13T11:01:38.969241+0000","last_active":"2023-11-13T11:01:38.973436+0000","last_peered":"2023-11-13T11:01:38.973436+0000","last_clean":"2023-11-13T11:01:38.973436+0000","last_became_active":"2023-11-13T11:01:38.968942+0000","last_became_peered":"2023-11-13T11:01:38.968942+0000","last_unstale":"2023-11-13T11:01:38.973436+0000","last_undegraded":"2023-11-13T11:01:38.973436+0000","last_fullsized":"2023-11-13T11:01:38.973436+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:19:00.701068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.972503+0000","last_change":"2023-11-13T11:01:38.966987+0000","last_active":"2023-11-13T11:01:38.972503+0000","last_peered":"2023-11-13T11:01:38.972503+0000","last_clean":"2023-11-13T11:01:38.972503+0000","last_became_active":"2023-11-13T11:01:38.966723+0000","last_became_peered":"2023-11-13T11:01:38.966723+0000","last_unstale":"2023-11-13T11:01:38.972503+0000","last_undegraded":"2023-11-13T11:01:38.972503+0000","last_fullsized":"2023-11-13T11:01:38.972503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:33:09.247389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993826+0000","last_change":"2023-11-13T11:01:38.989021+0000","last_active":"2023-11-13T11:01:38.993826+0000","last_peered":"2023-11-13T11:01:38.993826+0000","last_clean":"2023-11-13T11:01:38.993826+0000","last_became_active":"2023-11-13T11:01:38.988715+0000","last_became_peered":"2023-11-13T11:01:38.988715+0000","last_unstale":"2023-11-13T11:01:38.993826+0000","last_undegraded":"2023-11-13T11:01:38.993826+0000","last_fullsized":"2023-11-13T11:01:38.993826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:12:15.067759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993091+0000","last_change":"2023-11-13T11:01:38.986587+0000","last_active":"2023-11-13T11:01:38.993091+0000","last_peered":"2023-11-13T11:01:38.993091+0000","last_clean":"2023-11-13T11:01:38.993091+0000","last_became_active":"2023-11-13T11:01:38.986301+0000","last_became_peered":"2023-11-13T11:01:38.986301+0000","last_unstale":"2023-11-13T11:01:38.993091+0000","last_undegraded":"2023-11-13T11:01:38.993091+0000","last_fullsized":"2023-11-13T11:01:38.993091+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:39.111281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.995951+0000","last_change":"2023-11-13T11:01:38.992108+0000","last_active":"2023-11-13T11:01:38.995951+0000","last_peered":"2023-11-13T11:01:38.995951+0000","last_clean":"2023-11-13T11:01:38.995951+0000","last_became_active":"2023-11-13T11:01:38.988156+0000","last_became_peered":"2023-11-13T11:01:38.988156+0000","last_unstale":"2023-11-13T11:01:38.995951+0000","last_undegraded":"2023-11-13T11:01:38.995951+0000","last_fullsized":"2023-11-13T11:01:38.995951+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:02.526493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000637+0000","last_change":"2023-11-13T11:01:38.996834+0000","last_active":"2023-11-13T11:01:39.000637+0000","last_peered":"2023-11-13T11:01:39.000637+0000","last_clean":"2023-11-13T11:01:39.000637+0000","last_became_active":"2023-11-13T11:01:38.996522+0000","last_became_peered":"2023-11-13T11:01:38.996522+0000","last_unstale":"2023-11-13T11:01:39.000637+0000","last_undegraded":"2023-11-13T11:01:39.000637+0000","last_fullsized":"2023-11-13T11:01:39.000637+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:01.750338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005129+0000","last_change":"2023-11-13T11:01:39.003837+0000","last_active":"2023-11-13T11:01:39.005129+0000","last_peered":"2023-11-13T11:01:39.005129+0000","last_clean":"2023-11-13T11:01:39.005129+0000","last_became_active":"2023-11-13T11:01:39.000372+0000","last_became_peered":"2023-11-13T11:01:39.000372+0000","last_unstale":"2023-11-13T11:01:39.005129+0000","last_undegraded":"2023-11-13T11:01:39.005129+0000","last_fullsized":"2023-11-13T11:01:39.005129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:29:27.396844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.964512+0000","last_change":"2023-11-13T11:01:37.964512+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.964512+0000","last_undegraded":"2023-11-13T11:01:37.964512+0000","last_fullsized":"2023-11-13T11:01:37.964512+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.001088+0000","last_change":"2023-11-13T11:01:39.000662+0000","last_active":"2023-11-13T11:01:39.001088+0000","last_peered":"2023-11-13T11:01:39.001088+0000","last_clean":"2023-11-13T11:01:39.001088+0000","last_became_active":"2023-11-13T11:01:39.000191+0000","last_became_peered":"2023-11-13T11:01:39.000191+0000","last_unstale":"2023-11-13T11:01:39.001088+0000","last_undegraded":"2023-11-13T11:01:39.001088+0000","last_fullsized":"2023-11-13T11:01:39.001088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980942+0000","last_change":"2023-11-13T11:01:38.969203+0000","last_active":"2023-11-13T11:01:38.980942+0000","last_peered":"2023-11-13T11:01:38.980942+0000","last_clean":"2023-11-13T11:01:38.980942+0000","last_became_active":"2023-11-13T11:01:38.968896+0000","last_became_peered":"2023-11-13T11:01:38.968896+0000","last_unstale":"2023-11-13T11:01:38.980942+0000","last_undegraded":"2023-11-13T11:01:38.980942+0000","last_fullsized":"2023-11-13T11:01:38.980942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991585+0000","last_change":"2023-11-13T11:01:38.989974+0000","last_active":"2023-11-13T11:01:38.991585+0000","last_peered":"2023-11-13T11:01:38.991585+0000","last_clean":"2023-11-13T11:01:38.991585+0000","last_became_active":"2023-11-13T11:01:38.989665+0000","last_became_peered":"2023-11-13T11:01:38.989665+0000","last_unstale":"2023-11-13T11:01:38.991585+0000","last_undegraded":"2023-11-13T11:01:38.991585+0000","last_fullsized":"2023-11-13T11:01:38.991585+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:18:26.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000755+0000","last_change":"2023-11-13T11:01:38.995148+0000","last_active":"2023-11-13T11:01:39.000755+0000","last_peered":"2023-11-13T11:01:39.000755+0000","last_clean":"2023-11-13T11:01:39.000755+0000","last_became_active":"2023-11-13T11:01:38.990007+0000","last_became_peered":"2023-11-13T11:01:38.990007+0000","last_unstale":"2023-11-13T11:01:39.000755+0000","last_undegraded":"2023-11-13T11:01:39.000755+0000","last_fullsized":"2023-11-13T11:01:39.000755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:18:26.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.988066+0000","last_change":"2023-11-13T11:01:38.984675+0000","last_active":"2023-11-13T11:01:38.988066+0000","last_peered":"2023-11-13T11:01:38.988066+0000","last_clean":"2023-11-13T11:01:38.988066+0000","last_became_active":"2023-11-13T11:01:38.984073+0000","last_became_peered":"2023-11-13T11:01:38.984073+0000","last_unstale":"2023-11-13T11:01:38.988066+0000","last_undegraded":"2023-11-13T11:01:38.988066+0000","last_fullsized":"2023-11-13T11:01:38.988066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:24:13.781188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979482+0000","last_change":"2023-11-13T11:01:38.973375+0000","last_active":"2023-11-13T11:01:38.979482+0000","last_peered":"2023-11-13T11:01:38.979482+0000","last_clean":"2023-11-13T11:01:38.979482+0000","last_became_active":"2023-11-13T11:01:38.970971+0000","last_became_peered":"2023-11-13T11:01:38.970971+0000","last_unstale":"2023-11-13T11:01:38.979482+0000","last_undegraded":"2023-11-13T11:01:38.979482+0000","last_fullsized":"2023-11-13T11:01:38.979482+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:03:35.448965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.999535+0000","last_change":"2023-11-13T11:01:37.976416+0000","last_active":"2023-11-13T11:01:38.999535+0000","last_peered":"2023-11-13T11:01:38.999535+0000","last_clean":"2023-11-13T11:01:38.999535+0000","last_became_active":"2023-11-13T11:01:37.976080+0000","last_became_peered":"2023-11-13T11:01:37.976080+0000","last_unstale":"2023-11-13T11:01:38.999535+0000","last_undegraded":"2023-11-13T11:01:38.999535+0000","last_fullsized":"2023-11-13T11:01:38.999535+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979796+0000","last_change":"2023-11-13T11:01:38.977746+0000","last_active":"2023-11-13T11:01:38.979796+0000","last_peered":"2023-11-13T11:01:38.979796+0000","last_clean":"2023-11-13T11:01:38.979796+0000","last_became_active":"2023-11-13T11:01:38.977421+0000","last_became_peered":"2023-11-13T11:01:38.977421+0000","last_unstale":"2023-11-13T11:01:38.979796+0000","last_undegraded":"2023-11-13T11:01:38.979796+0000","last_fullsized":"2023-11-13T11:01:38.979796+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:29:27.396844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.985691+0000","last_change":"2023-11-13T11:01:37.984920+0000","last_active":"2023-11-13T11:01:37.985691+0000","last_peered":"2023-11-13T11:01:37.985691+0000","last_clean":"2023-11-13T11:01:37.985691+0000","last_became_active":"2023-11-13T11:01:37.984119+0000","last_became_peered":"2023-11-13T11:01:37.984119+0000","last_unstale":"2023-11-13T11:01:37.985691+0000","last_undegraded":"2023-11-13T11:01:37.985691+0000","last_fullsized":"2023-11-13T11:01:37.985691+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:20:09.176965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977904+0000","last_change":"2023-11-13T11:01:38.971332+0000","last_active":"2023-11-13T11:01:38.977904+0000","last_peered":"2023-11-13T11:01:38.977904+0000","last_clean":"2023-11-13T11:01:38.977904+0000","last_became_active":"2023-11-13T11:01:38.969969+0000","last_became_peered":"2023-11-13T11:01:38.969969+0000","last_unstale":"2023-11-13T11:01:38.977904+0000","last_undegraded":"2023-11-13T11:01:38.977904+0000","last_fullsized":"2023-11-13T11:01:38.977904+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:09:56.697516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000885+0000","last_change":"2023-11-13T11:01:37.970093+0000","last_active":"2023-11-13T11:01:39.000885+0000","last_peered":"2023-11-13T11:01:39.000885+0000","last_clean":"2023-11-13T11:01:39.000885+0000","last_became_active":"2023-11-13T11:01:37.969770+0000","last_became_peered":"2023-11-13T11:01:37.969770+0000","last_unstale":"2023-11-13T11:01:39.000885+0000","last_undegraded":"2023-11-13T11:01:39.000885+0000","last_fullsized":"2023-11-13T11:01:39.000885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:33.254128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990202+0000","last_change":"2023-11-13T11:01:38.980529+0000","last_active":"2023-11-13T11:01:38.990202+0000","last_peered":"2023-11-13T11:01:38.990202+0000","last_clean":"2023-11-13T11:01:38.990202+0000","last_became_active":"2023-11-13T11:01:38.980227+0000","last_became_peered":"2023-11-13T11:01:38.980227+0000","last_unstale":"2023-11-13T11:01:38.990202+0000","last_undegraded":"2023-11-13T11:01:38.990202+0000","last_fullsized":"2023-11-13T11:01:38.990202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:12.303326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.994946+0000","last_change":"2023-11-13T11:01:37.982793+0000","last_active":"2023-11-13T11:01:38.994946+0000","last_peered":"2023-11-13T11:01:38.994946+0000","last_clean":"2023-11-13T11:01:38.994946+0000","last_became_active":"2023-11-13T11:01:37.982111+0000","last_became_peered":"2023-11-13T11:01:37.982111+0000","last_unstale":"2023-11-13T11:01:38.994946+0000","last_undegraded":"2023-11-13T11:01:38.994946+0000","last_fullsized":"2023-11-13T11:01:38.994946+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980711+0000","last_change":"2023-11-13T11:01:38.976171+0000","last_active":"2023-11-13T11:01:38.980711+0000","last_peered":"2023-11-13T11:01:38.980711+0000","last_clean":"2023-11-13T11:01:38.980711+0000","last_became_active":"2023-11-13T11:01:38.975895+0000","last_became_peered":"2023-11-13T11:01:38.975895+0000","last_unstale":"2023-11-13T11:01:38.980711+0000","last_undegraded":"2023-11-13T11:01:38.980711+0000","last_fullsized":"2023-11-13T11:01:38.980711+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.002584+0000","last_change":"2023-11-13T11:01:37.981492+0000","last_active":"2023-11-13T11:01:39.002584+0000","last_peered":"2023-11-13T11:01:39.002584+0000","last_clean":"2023-11-13T11:01:39.002584+0000","last_became_active":"2023-11-13T11:01:37.981149+0000","last_became_peered":"2023-11-13T11:01:37.981149+0000","last_unstale":"2023-11-13T11:01:39.002584+0000","last_undegraded":"2023-11-13T11:01:39.002584+0000","last_fullsized":"2023-11-13T11:01:39.002584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979333+0000","last_change":"2023-11-13T11:01:38.972149+0000","last_active":"2023-11-13T11:01:38.979333+0000","last_peered":"2023-11-13T11:01:38.979333+0000","last_clean":"2023-11-13T11:01:38.979333+0000","last_became_active":"2023-11-13T11:01:38.971860+0000","last_became_peered":"2023-11-13T11:01:38.971860+0000","last_unstale":"2023-11-13T11:01:38.979333+0000","last_undegraded":"2023-11-13T11:01:38.979333+0000","last_fullsized":"2023-11-13T11:01:38.979333+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:44:30.617363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986684+0000","last_change":"2023-11-13T11:01:37.968032+0000","last_active":"2023-11-13T11:01:38.986684+0000","last_peered":"2023-11-13T11:01:38.986684+0000","last_clean":"2023-11-13T11:01:38.986684+0000","last_became_active":"2023-11-13T11:01:37.967709+0000","last_became_peered":"2023-11-13T11:01:37.967709+0000","last_unstale":"2023-11-13T11:01:38.986684+0000","last_undegraded":"2023-11-13T11:01:38.986684+0000","last_fullsized":"2023-11-13T11:01:38.986684+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:44:32.588239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.003718+0000","last_change":"2023-11-13T11:01:39.001367+0000","last_active":"2023-11-13T11:01:39.003718+0000","last_peered":"2023-11-13T11:01:39.003718+0000","last_clean":"2023-11-13T11:01:39.003718+0000","last_became_active":"2023-11-13T11:01:39.001092+0000","last_became_peered":"2023-11-13T11:01:39.001092+0000","last_unstale":"2023-11-13T11:01:39.003718+0000","last_undegraded":"2023-11-13T11:01:39.003718+0000","last_fullsized":"2023-11-13T11:01:39.003718+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.962390+0000","last_change":"2023-11-13T11:01:37.974253+0000","last_active":"2023-11-13T11:01:38.962390+0000","last_peered":"2023-11-13T11:01:38.962390+0000","last_clean":"2023-11-13T11:01:38.962390+0000","last_became_active":"2023-11-13T11:01:37.973388+0000","last_became_peered":"2023-11-13T11:01:37.973388+0000","last_unstale":"2023-11-13T11:01:38.962390+0000","last_undegraded":"2023-11-13T11:01:38.962390+0000","last_fullsized":"2023-11-13T11:01:38.962390+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997619+0000","last_change":"2023-11-13T11:01:38.996067+0000","last_active":"2023-11-13T11:01:38.997619+0000","last_peered":"2023-11-13T11:01:38.997619+0000","last_clean":"2023-11-13T11:01:38.997619+0000","last_became_active":"2023-11-13T11:01:38.995583+0000","last_became_peered":"2023-11-13T11:01:38.995583+0000","last_unstale":"2023-11-13T11:01:38.997619+0000","last_undegraded":"2023-11-13T11:01:38.997619+0000","last_fullsized":"2023-11-13T11:01:38.997619+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973630+0000","last_change":"2023-11-13T11:01:37.980576+0000","last_active":"2023-11-13T11:01:38.973630+0000","last_peered":"2023-11-13T11:01:38.973630+0000","last_clean":"2023-11-13T11:01:38.973630+0000","last_became_active":"2023-11-13T11:01:37.980253+0000","last_became_peered":"2023-11-13T11:01:37.980253+0000","last_unstale":"2023-11-13T11:01:38.973630+0000","last_undegraded":"2023-11-13T11:01:38.973630+0000","last_fullsized":"2023-11-13T11:01:38.973630+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.982849+0000","last_change":"2023-11-13T11:01:38.978906+0000","last_active":"2023-11-13T11:01:38.982849+0000","last_peered":"2023-11-13T11:01:38.982849+0000","last_clean":"2023-11-13T11:01:38.982849+0000","last_became_active":"2023-11-13T11:01:38.978590+0000","last_became_peered":"2023-11-13T11:01:38.978590+0000","last_unstale":"2023-11-13T11:01:38.982849+0000","last_undegraded":"2023-11-13T11:01:38.982849+0000","last_fullsized":"2023-11-13T11:01:38.982849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.957725+0000","last_change":"2023-11-13T11:01:37.967093+0000","last_active":"2023-11-13T11:01:38.957725+0000","last_peered":"2023-11-13T11:01:38.957725+0000","last_clean":"2023-11-13T11:01:38.957725+0000","last_became_active":"2023-11-13T11:01:37.966819+0000","last_became_peered":"2023-11-13T11:01:37.966819+0000","last_unstale":"2023-11-13T11:01:38.957725+0000","last_undegraded":"2023-11-13T11:01:38.957725+0000","last_fullsized":"2023-11-13T11:01:38.957725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980445+0000","last_change":"2023-11-13T11:01:38.978007+0000","last_active":"2023-11-13T11:01:38.980445+0000","last_peered":"2023-11-13T11:01:38.980445+0000","last_clean":"2023-11-13T11:01:38.980445+0000","last_became_active":"2023-11-13T11:01:38.977730+0000","last_became_peered":"2023-11-13T11:01:38.977730+0000","last_unstale":"2023-11-13T11:01:38.980445+0000","last_undegraded":"2023-11-13T11:01:38.980445+0000","last_fullsized":"2023-11-13T11:01:38.980445+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.957148+0000","last_change":"2023-11-13T11:01:37.974872+0000","last_active":"2023-11-13T11:01:38.957148+0000","last_peered":"2023-11-13T11:01:38.957148+0000","last_clean":"2023-11-13T11:01:38.957148+0000","last_became_active":"2023-11-13T11:01:37.973788+0000","last_became_peered":"2023-11-13T11:01:37.973788+0000","last_unstale":"2023-11-13T11:01:38.957148+0000","last_undegraded":"2023-11-13T11:01:38.957148+0000","last_fullsized":"2023-11-13T11:01:38.957148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:11.290032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992368+0000","last_change":"2023-11-13T11:01:38.982930+0000","last_active":"2023-11-13T11:01:38.992368+0000","last_peered":"2023-11-13T11:01:38.992368+0000","last_clean":"2023-11-13T11:01:38.992368+0000","last_became_active":"2023-11-13T11:01:38.982617+0000","last_became_peered":"2023-11-13T11:01:38.982617+0000","last_unstale":"2023-11-13T11:01:38.992368+0000","last_undegraded":"2023-11-13T11:01:38.992368+0000","last_fullsized":"2023-11-13T11:01:38.992368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:35.756266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997924+0000","last_change":"2023-11-13T11:01:37.976846+0000","last_active":"2023-11-13T11:01:38.997924+0000","last_peered":"2023-11-13T11:01:38.997924+0000","last_clean":"2023-11-13T11:01:38.997924+0000","last_became_active":"2023-11-13T11:01:37.976378+0000","last_became_peered":"2023-11-13T11:01:37.976378+0000","last_unstale":"2023-11-13T11:01:38.997924+0000","last_undegraded":"2023-11-13T11:01:38.997924+0000","last_fullsized":"2023-11-13T11:01:38.997924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.985125+0000","last_change":"2023-11-13T11:01:38.983595+0000","last_active":"2023-11-13T11:01:38.985125+0000","last_peered":"2023-11-13T11:01:38.985125+0000","last_clean":"2023-11-13T11:01:38.985125+0000","last_became_active":"2023-11-13T11:01:38.978958+0000","last_became_peered":"2023-11-13T11:01:38.978958+0000","last_unstale":"2023-11-13T11:01:38.985125+0000","last_undegraded":"2023-11-13T11:01:38.985125+0000","last_fullsized":"2023-11-13T11:01:38.985125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:52:25.507954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.008452+0000","last_change":"2023-11-13T11:01:37.983799+0000","last_active":"2023-11-13T11:01:39.008452+0000","last_peered":"2023-11-13T11:01:39.008452+0000","last_clean":"2023-11-13T11:01:39.008452+0000","last_became_active":"2023-11-13T11:01:37.983048+0000","last_became_peered":"2023-11-13T11:01:37.983048+0000","last_unstale":"2023-11-13T11:01:39.008452+0000","last_undegraded":"2023-11-13T11:01:39.008452+0000","last_fullsized":"2023-11-13T11:01:39.008452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:43.793217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979764+0000","last_change":"2023-11-13T11:01:38.972065+0000","last_active":"2023-11-13T11:01:38.979764+0000","last_peered":"2023-11-13T11:01:38.979764+0000","last_clean":"2023-11-13T11:01:38.979764+0000","last_became_active":"2023-11-13T11:01:38.971764+0000","last_became_peered":"2023-11-13T11:01:38.971764+0000","last_unstale":"2023-11-13T11:01:38.979764+0000","last_undegraded":"2023-11-13T11:01:38.979764+0000","last_fullsized":"2023-11-13T11:01:38.979764+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:27:51.217444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.979536+0000","last_change":"2023-11-13T11:01:37.976954+0000","last_active":"2023-11-13T11:01:37.979536+0000","last_peered":"2023-11-13T11:01:37.979536+0000","last_clean":"2023-11-13T11:01:37.979536+0000","last_became_active":"2023-11-13T11:01:37.976659+0000","last_became_peered":"2023-11-13T11:01:37.976659+0000","last_unstale":"2023-11-13T11:01:37.979536+0000","last_undegraded":"2023-11-13T11:01:37.979536+0000","last_fullsized":"2023-11-13T11:01:37.979536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:11:52.113188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000253+0000","last_change":"2023-11-13T11:01:38.995557+0000","last_active":"2023-11-13T11:01:39.000253+0000","last_peered":"2023-11-13T11:01:39.000253+0000","last_clean":"2023-11-13T11:01:39.000253+0000","last_became_active":"2023-11-13T11:01:38.995224+0000","last_became_peered":"2023-11-13T11:01:38.995224+0000","last_unstale":"2023-11-13T11:01:39.000253+0000","last_undegraded":"2023-11-13T11:01:39.000253+0000","last_fullsized":"2023-11-13T11:01:39.000253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.959098+0000","last_change":"2023-11-13T11:01:37.970890+0000","last_active":"2023-11-13T11:01:38.959098+0000","last_peered":"2023-11-13T11:01:38.959098+0000","last_clean":"2023-11-13T11:01:38.959098+0000","last_became_active":"2023-11-13T11:01:37.970585+0000","last_became_peered":"2023-11-13T11:01:37.970585+0000","last_unstale":"2023-11-13T11:01:38.959098+0000","last_undegraded":"2023-11-13T11:01:38.959098+0000","last_fullsized":"2023-11-13T11:01:38.959098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:24.358423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.964731+0000","last_change":"2023-11-13T11:01:37.964731+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.964731+0000","last_undegraded":"2023-11-13T11:01:37.964731+0000","last_fullsized":"2023-11-13T11:01:37.964731+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.961699+0000","last_change":"2023-11-13T11:01:37.966418+0000","last_active":"2023-11-13T11:01:38.961699+0000","last_peered":"2023-11-13T11:01:38.961699+0000","last_clean":"2023-11-13T11:01:38.961699+0000","last_became_active":"2023-11-13T11:01:37.966105+0000","last_became_peered":"2023-11-13T11:01:37.966105+0000","last_unstale":"2023-11-13T11:01:38.961699+0000","last_undegraded":"2023-11-13T11:01:38.961699+0000","last_fullsized":"2023-11-13T11:01:38.961699+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:44:09.646957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991081+0000","last_change":"2023-11-13T11:01:38.985149+0000","last_active":"2023-11-13T11:01:38.991081+0000","last_peered":"2023-11-13T11:01:38.991081+0000","last_clean":"2023-11-13T11:01:38.991081+0000","last_became_active":"2023-11-13T11:01:38.984828+0000","last_became_peered":"2023-11-13T11:01:38.984828+0000","last_unstale":"2023-11-13T11:01:38.991081+0000","last_undegraded":"2023-11-13T11:01:38.991081+0000","last_fullsized":"2023-11-13T11:01:38.991081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:27:51.217444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.985403+0000","last_change":"2023-11-13T11:01:37.983844+0000","last_active":"2023-11-13T11:01:37.985403+0000","last_peered":"2023-11-13T11:01:37.985403+0000","last_clean":"2023-11-13T11:01:37.985403+0000","last_became_active":"2023-11-13T11:01:37.982971+0000","last_became_peered":"2023-11-13T11:01:37.982971+0000","last_unstale":"2023-11-13T11:01:37.985403+0000","last_undegraded":"2023-11-13T11:01:37.985403+0000","last_fullsized":"2023-11-13T11:01:37.985403+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:06:49.184941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990249+0000","last_change":"2023-11-13T11:01:38.984132+0000","last_active":"2023-11-13T11:01:38.990249+0000","last_peered":"2023-11-13T11:01:38.990249+0000","last_clean":"2023-11-13T11:01:38.990249+0000","last_became_active":"2023-11-13T11:01:38.981507+0000","last_became_peered":"2023-11-13T11:01:38.981507+0000","last_unstale":"2023-11-13T11:01:38.990249+0000","last_undegraded":"2023-11-13T11:01:38.990249+0000","last_fullsized":"2023-11-13T11:01:38.990249+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.986923+0000","last_change":"2023-11-13T11:01:37.986267+0000","last_active":"2023-11-13T11:01:37.986923+0000","last_peered":"2023-11-13T11:01:37.986923+0000","last_clean":"2023-11-13T11:01:37.986923+0000","last_became_active":"2023-11-13T11:01:37.985787+0000","last_became_peered":"2023-11-13T11:01:37.985787+0000","last_unstale":"2023-11-13T11:01:37.986923+0000","last_undegraded":"2023-11-13T11:01:37.986923+0000","last_fullsized":"2023-11-13T11:01:37.986923+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:51:14.827858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992067+0000","last_change":"2023-11-13T11:01:38.984798+0000","last_active":"2023-11-13T11:01:38.992067+0000","last_peered":"2023-11-13T11:01:38.992067+0000","last_clean":"2023-11-13T11:01:38.992067+0000","last_became_active":"2023-11-13T11:01:38.984518+0000","last_became_peered":"2023-11-13T11:01:38.984518+0000","last_unstale":"2023-11-13T11:01:38.992067+0000","last_undegraded":"2023-11-13T11:01:38.992067+0000","last_fullsized":"2023-11-13T11:01:38.992067+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.979645+0000","last_change":"2023-11-13T11:01:37.977771+0000","last_active":"2023-11-13T11:01:37.979645+0000","last_peered":"2023-11-13T11:01:37.979645+0000","last_clean":"2023-11-13T11:01:37.979645+0000","last_became_active":"2023-11-13T11:01:37.977506+0000","last_became_peered":"2023-11-13T11:01:37.977506+0000","last_unstale":"2023-11-13T11:01:37.979645+0000","last_undegraded":"2023-11-13T11:01:37.979645+0000","last_fullsized":"2023-11-13T11:01:37.979645+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:48:24.890322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.995489+0000","last_change":"2023-11-13T11:01:38.994873+0000","last_active":"2023-11-13T11:01:38.995489+0000","last_peered":"2023-11-13T11:01:38.995489+0000","last_clean":"2023-11-13T11:01:38.995489+0000","last_became_active":"2023-11-13T11:01:38.994490+0000","last_became_peered":"2023-11-13T11:01:38.994490+0000","last_unstale":"2023-11-13T11:01:38.995489+0000","last_undegraded":"2023-11-13T11:01:38.995489+0000","last_fullsized":"2023-11-13T11:01:38.995489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:46:41.523607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997386+0000","last_change":"2023-11-13T11:01:37.982308+0000","last_active":"2023-11-13T11:01:38.997386+0000","last_peered":"2023-11-13T11:01:38.997386+0000","last_clean":"2023-11-13T11:01:38.997386+0000","last_became_active":"2023-11-13T11:01:37.981963+0000","last_became_peered":"2023-11-13T11:01:37.981963+0000","last_unstale":"2023-11-13T11:01:38.997386+0000","last_undegraded":"2023-11-13T11:01:38.997386+0000","last_fullsized":"2023-11-13T11:01:38.997386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.963540+0000","last_change":"2023-11-13T11:01:37.963540+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.963540+0000","last_undegraded":"2023-11-13T11:01:37.963540+0000","last_fullsized":"2023-11-13T11:01:37.963540+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.966021+0000","last_change":"2023-11-13T11:01:37.978277+0000","last_active":"2023-11-13T11:01:38.966021+0000","last_peered":"2023-11-13T11:01:38.966021+0000","last_clean":"2023-11-13T11:01:38.966021+0000","last_became_active":"2023-11-13T11:01:37.977659+0000","last_became_peered":"2023-11-13T11:01:37.977659+0000","last_unstale":"2023-11-13T11:01:38.966021+0000","last_undegraded":"2023-11-13T11:01:38.966021+0000","last_fullsized":"2023-11-13T11:01:38.966021+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:56.135585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977517+0000","last_change":"2023-11-13T11:01:38.975186+0000","last_active":"2023-11-13T11:01:38.977517+0000","last_peered":"2023-11-13T11:01:38.977517+0000","last_clean":"2023-11-13T11:01:38.977517+0000","last_became_active":"2023-11-13T11:01:38.972436+0000","last_became_peered":"2023-11-13T11:01:38.972436+0000","last_unstale":"2023-11-13T11:01:38.977517+0000","last_undegraded":"2023-11-13T11:01:38.977517+0000","last_fullsized":"2023-11-13T11:01:38.977517+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:45:08.666322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000163+0000","last_change":"2023-11-13T11:01:37.970760+0000","last_active":"2023-11-13T11:01:39.000163+0000","last_peered":"2023-11-13T11:01:39.000163+0000","last_clean":"2023-11-13T11:01:39.000163+0000","last_became_active":"2023-11-13T11:01:37.970465+0000","last_became_peered":"2023-11-13T11:01:37.970465+0000","last_unstale":"2023-11-13T11:01:39.000163+0000","last_undegraded":"2023-11-13T11:01:39.000163+0000","last_fullsized":"2023-11-13T11:01:39.000163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979277+0000","last_change":"2023-11-13T11:01:38.972817+0000","last_active":"2023-11-13T11:01:38.979277+0000","last_peered":"2023-11-13T11:01:38.979277+0000","last_clean":"2023-11-13T11:01:38.979277+0000","last_became_active":"2023-11-13T11:01:38.971691+0000","last_became_peered":"2023-11-13T11:01:38.971691+0000","last_unstale":"2023-11-13T11:01:38.979277+0000","last_undegraded":"2023-11-13T11:01:38.979277+0000","last_fullsized":"2023-11-13T11:01:38.979277+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.978810+0000","last_change":"2023-11-13T11:01:37.982283+0000","last_active":"2023-11-13T11:01:38.978810+0000","last_peered":"2023-11-13T11:01:38.978810+0000","last_clean":"2023-11-13T11:01:38.978810+0000","last_became_active":"2023-11-13T11:01:37.981958+0000","last_became_peered":"2023-11-13T11:01:37.981958+0000","last_unstale":"2023-11-13T11:01:38.978810+0000","last_undegraded":"2023-11-13T11:01:38.978810+0000","last_fullsized":"2023-11-13T11:01:38.978810+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:44:29.604069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991795+0000","last_change":"2023-11-13T11:01:38.982113+0000","last_active":"2023-11-13T11:01:38.991795+0000","last_peered":"2023-11-13T11:01:38.991795+0000","last_clean":"2023-11-13T11:01:38.991795+0000","last_became_active":"2023-11-13T11:01:38.980795+0000","last_became_peered":"2023-11-13T11:01:38.980795+0000","last_unstale":"2023-11-13T11:01:38.991795+0000","last_undegraded":"2023-11-13T11:01:38.991795+0000","last_fullsized":"2023-11-13T11:01:38.991795+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973311+0000","last_change":"2023-11-13T11:01:37.982552+0000","last_active":"2023-11-13T11:01:38.973311+0000","last_peered":"2023-11-13T11:01:38.973311+0000","last_clean":"2023-11-13T11:01:38.973311+0000","last_became_active":"2023-11-13T11:01:37.982269+0000","last_became_peered":"2023-11-13T11:01:37.982269+0000","last_unstale":"2023-11-13T11:01:38.973311+0000","last_undegraded":"2023-11-13T11:01:38.973311+0000","last_fullsized":"2023-11-13T11:01:38.973311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.963480+0000","last_change":"2023-11-13T11:01:37.963480+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.963480+0000","last_undegraded":"2023-11-13T11:01:37.963480+0000","last_fullsized":"2023-11-13T11:01:37.963480+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.975254+0000","last_change":"2023-11-13T11:01:37.971616+0000","last_active":"2023-11-13T11:01:38.975254+0000","last_peered":"2023-11-13T11:01:38.975254+0000","last_clean":"2023-11-13T11:01:38.975254+0000","last_became_active":"2023-11-13T11:01:37.971344+0000","last_became_peered":"2023-11-13T11:01:37.971344+0000","last_unstale":"2023-11-13T11:01:38.975254+0000","last_undegraded":"2023-11-13T11:01:38.975254+0000","last_fullsized":"2023-11-13T11:01:38.975254+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:31.732306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986510+0000","last_change":"2023-11-13T11:01:38.983983+0000","last_active":"2023-11-13T11:01:38.986510+0000","last_peered":"2023-11-13T11:01:38.986510+0000","last_clean":"2023-11-13T11:01:38.986510+0000","last_became_active":"2023-11-13T11:01:38.983698+0000","last_became_peered":"2023-11-13T11:01:38.983698+0000","last_unstale":"2023-11-13T11:01:38.986510+0000","last_undegraded":"2023-11-13T11:01:38.986510+0000","last_fullsized":"2023-11-13T11:01:38.986510+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:46:41.523607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997353+0000","last_change":"2023-11-13T11:01:37.972256+0000","last_active":"2023-11-13T11:01:38.997353+0000","last_peered":"2023-11-13T11:01:38.997353+0000","last_clean":"2023-11-13T11:01:38.997353+0000","last_became_active":"2023-11-13T11:01:37.971890+0000","last_became_peered":"2023-11-13T11:01:37.971890+0000","last_unstale":"2023-11-13T11:01:38.997353+0000","last_undegraded":"2023-11-13T11:01:38.997353+0000","last_fullsized":"2023-11-13T11:01:38.997353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:38.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990549+0000","last_change":"2023-11-13T11:01:38.988092+0000","last_active":"2023-11-13T11:01:38.990549+0000","last_peered":"2023-11-13T11:01:38.990549+0000","last_clean":"2023-11-13T11:01:38.990549+0000","last_became_active":"2023-11-13T11:01:38.987800+0000","last_became_peered":"2023-11-13T11:01:38.987800+0000","last_unstale":"2023-11-13T11:01:38.990549+0000","last_undegraded":"2023-11-13T11:01:38.990549+0000","last_fullsized":"2023-11-13T11:01:38.990549+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:28.542546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.964559+0000","last_change":"2023-11-13T11:01:37.973251+0000","last_active":"2023-11-13T11:01:38.964559+0000","last_peered":"2023-11-13T11:01:38.964559+0000","last_clean":"2023-11-13T11:01:38.964559+0000","last_became_active":"2023-11-13T11:01:37.972947+0000","last_became_peered":"2023-11-13T11:01:37.972947+0000","last_unstale":"2023-11-13T11:01:38.964559+0000","last_undegraded":"2023-11-13T11:01:38.964559+0000","last_fullsized":"2023-11-13T11:01:38.964559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993910+0000","last_change":"2023-11-13T11:01:38.988853+0000","last_active":"2023-11-13T11:01:38.993910+0000","last_peered":"2023-11-13T11:01:38.993910+0000","last_clean":"2023-11-13T11:01:38.993910+0000","last_became_active":"2023-11-13T11:01:38.987558+0000","last_became_peered":"2023-11-13T11:01:38.987558+0000","last_unstale":"2023-11-13T11:01:38.993910+0000","last_undegraded":"2023-11-13T11:01:38.993910+0000","last_fullsized":"2023-11-13T11:01:38.993910+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:38:32.234508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986162+0000","last_change":"2023-11-13T11:01:37.981807+0000","last_active":"2023-11-13T11:01:38.986162+0000","last_peered":"2023-11-13T11:01:38.986162+0000","last_clean":"2023-11-13T11:01:38.986162+0000","last_became_active":"2023-11-13T11:01:37.981480+0000","last_became_peered":"2023-11-13T11:01:37.981480+0000","last_unstale":"2023-11-13T11:01:38.986162+0000","last_undegraded":"2023-11-13T11:01:38.986162+0000","last_fullsized":"2023-11-13T11:01:38.986162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:09:55.684221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.999643+0000","last_change":"2023-11-13T11:01:38.986410+0000","last_active":"2023-11-13T11:01:38.999643+0000","last_peered":"2023-11-13T11:01:38.999643+0000","last_clean":"2023-11-13T11:01:38.999643+0000","last_became_active":"2023-11-13T11:01:38.986131+0000","last_became_peered":"2023-11-13T11:01:38.986131+0000","last_unstale":"2023-11-13T11:01:38.999643+0000","last_undegraded":"2023-11-13T11:01:38.999643+0000","last_fullsized":"2023-11-13T11:01:38.999643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.975456+0000","last_change":"2023-11-13T11:01:37.975627+0000","last_active":"2023-11-13T11:01:38.975456+0000","last_peered":"2023-11-13T11:01:38.975456+0000","last_clean":"2023-11-13T11:01:38.975456+0000","last_became_active":"2023-11-13T11:01:37.974747+0000","last_became_peered":"2023-11-13T11:01:37.974747+0000","last_unstale":"2023-11-13T11:01:38.975456+0000","last_undegraded":"2023-11-13T11:01:38.975456+0000","last_fullsized":"2023-11-13T11:01:38.975456+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990137+0000","last_change":"2023-11-13T11:01:38.989266+0000","last_active":"2023-11-13T11:01:38.990137+0000","last_peered":"2023-11-13T11:01:38.990137+0000","last_clean":"2023-11-13T11:01:38.990137+0000","last_became_active":"2023-11-13T11:01:38.988948+0000","last_became_peered":"2023-11-13T11:01:38.988948+0000","last_unstale":"2023-11-13T11:01:38.990137+0000","last_undegraded":"2023-11-13T11:01:38.990137+0000","last_fullsized":"2023-11-13T11:01:38.990137+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:12.303326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.978974+0000","last_change":"2023-11-13T11:01:37.982077+0000","last_active":"2023-11-13T11:01:38.978974+0000","last_peered":"2023-11-13T11:01:38.978974+0000","last_clean":"2023-11-13T11:01:38.978974+0000","last_became_active":"2023-11-13T11:01:37.981552+0000","last_became_peered":"2023-11-13T11:01:37.981552+0000","last_unstale":"2023-11-13T11:01:38.978974+0000","last_undegraded":"2023-11-13T11:01:38.978974+0000","last_fullsized":"2023-11-13T11:01:38.978974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993927+0000","last_change":"2023-11-13T11:01:38.993394+0000","last_active":"2023-11-13T11:01:38.993927+0000","last_peered":"2023-11-13T11:01:38.993927+0000","last_clean":"2023-11-13T11:01:38.993927+0000","last_became_active":"2023-11-13T11:01:38.992745+0000","last_became_peered":"2023-11-13T11:01:38.992745+0000","last_unstale":"2023-11-13T11:01:38.993927+0000","last_undegraded":"2023-11-13T11:01:38.993927+0000","last_fullsized":"2023-11-13T11:01:38.993927+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.961654+0000","last_change":"2023-11-13T11:01:37.971343+0000","last_active":"2023-11-13T11:01:38.961654+0000","last_peered":"2023-11-13T11:01:38.961654+0000","last_clean":"2023-11-13T11:01:38.961654+0000","last_became_active":"2023-11-13T11:01:37.970761+0000","last_became_peered":"2023-11-13T11:01:37.970761+0000","last_unstale":"2023-11-13T11:01:38.961654+0000","last_undegraded":"2023-11-13T11:01:38.961654+0000","last_fullsized":"2023-11-13T11:01:38.961654+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:47:45.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.964287+0000","last_change":"2023-11-13T11:01:37.964287+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.964287+0000","last_undegraded":"2023-11-13T11:01:37.964287+0000","last_fullsized":"2023-11-13T11:01:37.964287+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.956737+0000","last_change":"2023-11-13T11:01:37.970057+0000","last_active":"2023-11-13T11:01:38.956737+0000","last_peered":"2023-11-13T11:01:38.956737+0000","last_clean":"2023-11-13T11:01:38.956737+0000","last_became_active":"2023-11-13T11:01:37.969819+0000","last_became_peered":"2023-11-13T11:01:37.969819+0000","last_unstale":"2023-11-13T11:01:38.956737+0000","last_undegraded":"2023-11-13T11:01:38.956737+0000","last_fullsized":"2023-11-13T11:01:38.956737+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:45:07.653027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.968087+0000","last_change":"2023-11-13T11:01:38.966353+0000","last_active":"2023-11-13T11:01:38.968087+0000","last_peered":"2023-11-13T11:01:38.968087+0000","last_clean":"2023-11-13T11:01:38.968087+0000","last_became_active":"2023-11-13T11:01:38.966082+0000","last_became_peered":"2023-11-13T11:01:38.966082+0000","last_unstale":"2023-11-13T11:01:38.968087+0000","last_undegraded":"2023-11-13T11:01:38.968087+0000","last_fullsized":"2023-11-13T11:01:38.968087+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977074+0000","last_change":"2023-11-13T11:01:37.970911+0000","last_active":"2023-11-13T11:01:38.977074+0000","last_peered":"2023-11-13T11:01:38.977074+0000","last_clean":"2023-11-13T11:01:38.977074+0000","last_became_active":"2023-11-13T11:01:37.970627+0000","last_became_peered":"2023-11-13T11:01:37.970627+0000","last_unstale":"2023-11-13T11:01:38.977074+0000","last_undegraded":"2023-11-13T11:01:38.977074+0000","last_fullsized":"2023-11-13T11:01:38.977074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.949346+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:01:37.949346+0000","last_peered":"2023-11-13T11:01:37.949346+0000","last_clean":"2023-11-13T11:01:37.949346+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:01:37.949346+0000","last_undegraded":"2023-11-13T11:01:37.949346+0000","last_fullsized":"2023-11-13T11:01:37.949346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0960000000000001,"15min":1.1259999999999999},"min":{"1min":0.84599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.21,"15min":1.1339999999999999},"min":{"1min":0.89800000000000002,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.865,"5min":1.865,"15min":2.2879999999999998},"last":1.353}]},{"osd":1,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.22,"15min":1.175},"min":{"1min":0.91700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8200000000000001,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2969999999999999,"15min":1.2889999999999999},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1070000000000002,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.2789999999999999,"15min":1.2769999999999999},"min":{"1min":0.98099999999999998,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.153,"5min":2.153,"15min":2.1680000000000001},"last":1.244},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3819999999999999,"15min":1.3520000000000001},"min":{"1min":1.1080000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.2400000000000002,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.298}]},{"osd":3,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.407,"15min":1.383},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.63800000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.423,"15min":1.4390000000000001},"min":{"1min":1.167,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.524}]},{"osd":4,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.454,"15min":1.5720000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.577,"5min":1.5129999999999999,"15min":1.5940000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.202}]},{"osd":5,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6000000000000001,"15min":1.6559999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.405},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.72,"15min":1.7589999999999999},"min":{"1min":1.147,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.6339999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.758,"5min":1.659,"15min":1.651},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.754,"5min":1.617,"15min":1.6499999999999999},"min":{"1min":0.96499999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.5940000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1216512,"data_stored":863292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.107,"15min":1.0129999999999999},"min":{"1min":0.82999999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9099999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2070000000000001,"15min":1.2030000000000001},"min":{"1min":0.74299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.639}]},{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.278,"15min":1.2310000000000001},"min":{"1min":0.89800000000000002,"5min":0.71699999999999997,"15min":0.65400000000000003},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.234,"15min":1.238},"min":{"1min":0.878,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8580000000000001,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.2250000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3129999999999999,"15min":1.2509999999999999},"min":{"1min":0.92700000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":1.099},{"interface":"front","average":{"1min":1.375,"5min":1.3120000000000001,"15min":1.2370000000000001},"min":{"1min":0.81599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.5209999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.393,"15min":1.399},"min":{"1min":0.86399999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80700000000000005},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":1.339}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3640000000000001,"15min":1.3340000000000001},"min":{"1min":0.86599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.294,"5min":3.484,"15min":3.484},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.4830000000000001,"15min":1.5349999999999999},"min":{"1min":1.123,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":1.4810000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5049999999999999,"15min":1.6080000000000001},"min":{"1min":1.1539999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.4929999999999999,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.538,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.6120000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6359999999999999,"15min":1.6799999999999999},"min":{"1min":1.167,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.7},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.653,"15min":1.714},"min":{"1min":1.2190000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.4410000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49840,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322000,"statfs":{"total":96636764160,"available":96585728000,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.1599999999999999,"15min":1.0820000000000001},"min":{"1min":0.86399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.103,"15min":1.075},"min":{"1min":0.94399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.218,"15min":1.1579999999999999},"min":{"1min":0.65800000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.7689999999999999,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2689999999999999,"15min":1.2549999999999999},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.821,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.998}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.27,"15min":1.2350000000000001},"min":{"1min":0.98899999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.3200000000000001,"15min":1.3480000000000001},"min":{"1min":0.996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.327}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.46,"15min":1.4139999999999999},"min":{"1min":1.1479999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":2.0049999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.98599999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.742,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.472,"15min":1.502},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0110000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.492},{"interface":"front","average":{"1min":1.504,"5min":1.3480000000000001,"15min":1.401},"min":{"1min":0.98399999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.1910000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.605,"15min":1.629},"min":{"1min":1.0329999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4850000000000001,"15min":1.5389999999999999},"min":{"1min":1.1459999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.6299999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":1.516,"15min":1.591},"min":{"1min":1.139,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.6040000000000001,"15min":1.621},"min":{"1min":1.1020000000000001,"5min":0.63,"15min":0.63},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.571}]}]},{"osd":3,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40124,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331716,"statfs":{"total":96636764160,"available":96595677184,"internally_reserved":0,"allocated":638976,"data_stored":280109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.85299999999999998,"15min":0.86499999999999999},"min":{"1min":0.53000000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.843,"5min":1.867,"15min":1.867},"last":0.68100000000000005},{"interface":"front","average":{"1min":1,"5min":0.98699999999999999,"15min":1.018},"min":{"1min":0.74399999999999999,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4239999999999999,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":0.80300000000000005}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.98599999999999999,"15min":1.04},"min":{"1min":0.65800000000000003,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.603,"5min":1.966,"15min":3.71},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.077,"15min":1.089},"min":{"1min":0.54400000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5660000000000001,"5min":1.923,"15min":2.0190000000000001},"last":0.92300000000000004}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.177,"15min":1.097},"min":{"1min":0.73599999999999999,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":1.5049999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.2909999999999999},"min":{"1min":0.83199999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5389999999999999,"5min":1.952,"15min":3.5710000000000002},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3109999999999999,"15min":1.4179999999999999},"min":{"1min":0.97199999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.5640000000000001,"5min":2.411,"15min":3.5030000000000001},"last":1.143},{"interface":"front","average":{"1min":1.49,"5min":1.345,"15min":1.349},"min":{"1min":1.1319999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.9370000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.615}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.379,"15min":1.476},"min":{"1min":1.073,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.7270000000000001,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.456},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4219999999999999,"15min":1.5149999999999999},"min":{"1min":0.86799999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.72,"5min":2.673,"15min":3.3809999999999998},"last":1.486}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4510000000000001,"15min":1.4870000000000001},"min":{"1min":1.1910000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8260000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.218},{"interface":"front","average":{"1min":1.446,"5min":1.4590000000000001,"15min":1.591},"min":{"1min":0.97099999999999997,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":1.855,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.3109999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.512,"15min":1.6120000000000001},"min":{"1min":1.1819999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.968,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.347},{"interface":"front","average":{"1min":1.647,"5min":1.5680000000000001,"15min":1.5549999999999999},"min":{"1min":1.319,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0059999999999998,"5min":2.75,"15min":2.75},"last":1.5369999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45680,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326160,"statfs":{"total":96636764160,"available":96589987840,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.89500000000000002,"15min":0.92900000000000005},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.9410000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.63},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.95699999999999996,"15min":0.92100000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.7789999999999999,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":0.54400000000000004}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":0.96299999999999997,"15min":0.95199999999999996},"min":{"1min":0.82999999999999996,"5min":0.376,"15min":0.376},"max":{"1min":1.893,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.105,"15min":1.127},"min":{"1min":0.68400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.978,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.133}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.139,"15min":1.2430000000000001},"min":{"1min":0.755,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.796,"5min":2.129,"15min":2.129},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.22,"15min":1.2729999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.867,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":0.98399999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.391,"15min":1.4750000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.9970000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3100000000000001,"15min":1.4039999999999999},"min":{"1min":0.79200000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0449999999999999,"5min":2.3639999999999999,"15min":2.6179999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.3959999999999999,"15min":1.48},"min":{"1min":0.96599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.6600000000000001},"last":1.034},{"interface":"front","average":{"1min":1.633,"5min":1.5409999999999999,"15min":1.5649999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.2450000000000001,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.3240000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.4359999999999999,"15min":1.55},"min":{"1min":0.83099999999999996,"5min":0.78500000000000003,"15min":0.76500000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":4.7569999999999997},"last":1.085},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.538,"15min":1.595},"min":{"1min":1.0229999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.621},"last":1.3620000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.488,"15min":1.6100000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.343,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.573,"15min":1.623},"min":{"1min":1.0740000000000001,"5min":0.85699999999999998,"15min":0.82799999999999996},"max":{"1min":2.2999999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.2909999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509519,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1216512,"data_stored":863292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.84099999999999997,"15min":0.95599999999999996},"min":{"1min":0.59299999999999997,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.47,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.105,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.754}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":0.99099999999999999,"15min":0.98299999999999998},"min":{"1min":0.71599999999999997,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0229999999999999,"15min":0.999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41999999999999998},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.722},"last":1.0029999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7629999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.101,"5min":1,"15min":1.0580000000000001},"min":{"1min":0.57899999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6899999999999999,"5min":1.8149999999999999,"15min":1.9059999999999999},"last":0.91100000000000003}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":0.98399999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.0569999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.363,"15min":1.375},"min":{"1min":1.0860000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.966,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.2290000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.3759999999999999,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83299999999999996},"max":{"1min":2.1629999999999998,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.103},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.4099999999999999},"min":{"1min":0.93999999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.0390000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2909999999999999,"15min":1.407},"min":{"1min":0.91400000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.986,"5min":1.986,"15min":2.3519999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.4339999999999999,"15min":1.4990000000000001},"min":{"1min":0.89800000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.3879999999999999},"last":1.3520000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.4410000000000001,"15min":1.496},"min":{"1min":0.93799999999999994,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.972,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.391,"15min":1.452},"min":{"1min":0.85799999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8660000000000001,"5min":2.202,"15min":2.202},"last":1.3300000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509520,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40108,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331732,"statfs":{"total":96636764160,"available":96595693568,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.90100000000000002},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32100000000000001},"max":{"1min":1.417,"5min":1.74,"15min":1.74},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.022,"15min":1.048},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.7330000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1459999999999999,"15min":1.141},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0089999999999999,"15min":1.018},"min":{"1min":0.54000000000000004,"5min":0.495,"15min":0.371},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.1669999999999998},"last":1.129}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1080000000000001,"15min":1.1020000000000001},"min":{"1min":0.63800000000000001,"5min":0.48999999999999999,"15min":0.45200000000000001},"max":{"1min":1.7450000000000001,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2769999999999999,"15min":1.329},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.827,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.0760000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3300000000000001,"15min":1.3540000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8009999999999999,"5min":2.153,"15min":2.153},"last":4.4279999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.296,"15min":1.4279999999999999},"min":{"1min":0.79400000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.708,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.5029999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.415,"15min":1.5449999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0430000000000001,"5min":2.2719999999999998,"15min":3.7490000000000001},"last":4.0449999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.3839999999999999,"15min":1.484},"min":{"1min":0.89100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.867,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":2.0310000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.48,"15min":1.5229999999999999},"min":{"1min":0.91100000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1389999999999998,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":4.617},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5369999999999999,"15min":1.5740000000000001},"min":{"1min":0.872,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.2810000000000001,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":3.6520000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.5900000000000001,"15min":1.6779999999999999},"min":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.21,"5min":2.742,"15min":3.3540000000000001},"last":1.522},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5800000000000001,"15min":1.6140000000000001},"min":{"1min":0.94999999999999996,"5min":0.88100000000000001,"15min":0.77400000000000002},"max":{"1min":2.2200000000000002,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":3.976}]}]},{"osd":7,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62304,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309536,"statfs":{"total":96636764160,"available":96572964864,"internally_reserved":0,"allocated":610304,"data_stored":266329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0860000000000001,"15min":1.1120000000000001},"min":{"1min":0.77800000000000002,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6970000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.119,"5min":1.087,"15min":1.073},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6080000000000001,"5min":2.266,"15min":2.266},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.147,"15min":1.0589999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6140000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.339,"5min":1.2450000000000001,"15min":1.2310000000000001},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.79800000000000004}]},{"osd":2,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2929999999999999,"15min":1.3029999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8080000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.264,"15min":1.2190000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.86099999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.341,"15min":1.2569999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.829,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5169999999999999,"15min":1.46},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.925,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.96299999999999997}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3680000000000001,"15min":1.3300000000000001},"min":{"1min":0.95199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.907,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":0.629},{"interface":"front","average":{"1min":1.395,"5min":1.4359999999999999,"15min":1.379},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.8420000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":0.70699999999999996}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.5389999999999999,"15min":1.5429999999999999},"min":{"1min":0.92800000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.0640000000000001,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.534,"15min":1.4830000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.9870000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.641,"15min":1.5760000000000001},"min":{"1min":0.84899999999999998,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":2.093,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.5920000000000001,"15min":1.518},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.2349999999999999,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":0.76700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:01:43.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:01:43.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.768+0000 7f69abfff640 1 Processor -- start 2023-11-13T11:01:43.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.768+0000 7f69abfff640 1 -- start start 2023-11-13T11:01:43.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.770+0000 7f69abfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f69ac15cff0 0x7f69ac071560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.770+0000 7f69abfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69ac071aa0 0x7f69ac06a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.770+0000 7f69abfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69ac06a6d0 0x7f69ac0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.770+0000 7f69abfff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ac0d0810 con 0x7f69ac071aa0 2023-11-13T11:01:43.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.770+0000 7f69abfff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ac06ab60 con 0x7f69ac15cff0 2023-11-13T11:01:43.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.770+0000 7f69abfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac0cddb0 con 0x7f69ac06a6d0 2023-11-13T11:01:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69a3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69ac071aa0 0x7f69ac06a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69a3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69ac071aa0 0x7f69ac06a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41902/0 (socket says 172.21.15.16:41902) 2023-11-13T11:01:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69a3fff640 1 -- 172.21.15.16:0/4138515988 learned_addr learned my addr 172.21.15.16:0/4138515988 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:43.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69ab7fe640 1 --2- 172.21.15.16:0/4138515988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69ac06a6d0 0x7f69ac0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69aaffd640 1 --2- 172.21.15.16:0/4138515988 >> v2:172.21.15.184:3300/0 conn(0x7f69ac15cff0 0x7f69ac071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69a3fff640 1 -- 172.21.15.16:0/4138515988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69ac06a6d0 msgr2=0x7f69ac0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:43.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69a3fff640 1 --2- 172.21.15.16:0/4138515988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69ac06a6d0 0x7f69ac0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69a3fff640 1 -- 172.21.15.16:0/4138515988 >> v2:172.21.15.184:3300/0 conn(0x7f69ac15cff0 msgr2=0x7f69ac071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:43.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69a3fff640 1 --2- 172.21.15.16:0/4138515988 >> v2:172.21.15.184:3300/0 conn(0x7f69ac15cff0 0x7f69ac071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.771+0000 7f69a3fff640 1 -- 172.21.15.16:0/4138515988 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac0cfc90 con 0x7f69ac071aa0 2023-11-13T11:01:43.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.772+0000 7f69a3fff640 1 --2- 172.21.15.16:0/4138515988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac071aa0 0x7f69ac06a190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f69ac071f60 tx=0x7f699402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb226a1d5477349 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:43.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.773+0000 7f69aa7fc640 1 -- 172.21.15.16:0/4138515988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994031870 con 0x7f69ac071aa0 2023-11-13T11:01:43.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.773+0000 7f69aa7fc640 1 -- 172.21.15.16:0/4138515988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69940319d0 con 0x7f69ac071aa0 2023-11-13T11:01:43.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.773+0000 7f69aa7fc640 1 -- 172.21.15.16:0/4138515988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994031d60 con 0x7f69ac071aa0 2023-11-13T11:01:43.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.773+0000 7f69abfff640 1 -- 172.21.15.16:0/4138515988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac071aa0 msgr2=0x7f69ac06a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.773+0000 7f69abfff640 1 --2- 172.21.15.16:0/4138515988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac071aa0 0x7f69ac06a190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f69ac071f60 tx=0x7f699402b910 comp rx=0 tx=0).stop 2023-11-13T11:01:43.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.773+0000 7f69abfff640 1 -- 172.21.15.16:0/4138515988 shutdown_connections 2023-11-13T11:01:43.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.773+0000 7f69abfff640 1 --2- 172.21.15.16:0/4138515988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69ac06a6d0 0x7f69ac0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.773+0000 7f69abfff640 1 --2- 172.21.15.16:0/4138515988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac071aa0 0x7f69ac06a190 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.773+0000 7f69abfff640 1 --2- 172.21.15.16:0/4138515988 >> v2:172.21.15.184:3300/0 conn(0x7f69ac15cff0 0x7f69ac071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.774+0000 7f69abfff640 1 -- 172.21.15.16:0/4138515988 >> 172.21.15.16:0/4138515988 conn(0x7f69ac06b2a0 msgr2=0x7f69ac06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:43.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.774+0000 7f69abfff640 1 -- 172.21.15.16:0/4138515988 shutdown_connections 2023-11-13T11:01:43.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.774+0000 7f69abfff640 1 -- 172.21.15.16:0/4138515988 wait complete. 2023-11-13T11:01:43.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.775+0000 7f69abfff640 1 Processor -- start 2023-11-13T11:01:43.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.776+0000 7f69abfff640 1 -- start start 2023-11-13T11:01:43.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.776+0000 7f69abfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac06a6d0 0x7f69ac15b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.776+0000 7f69abfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69ac071aa0 0x7f69ac0c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.776+0000 7f69abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac15cff0 0x7f69ac15fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69abfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac149600 con 0x7f69ac15cff0 2023-11-13T11:01:43.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69abfff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac1489f0 con 0x7f69ac06a6d0 2023-11-13T11:01:43.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69abfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac148cd0 con 0x7f69ac071aa0 2023-11-13T11:01:43.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac15cff0 0x7f69ac15fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac15cff0 0x7f69ac15fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41916/0 (socket says 172.21.15.16:41916) 2023-11-13T11:01:43.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69ab7fe640 1 -- 172.21.15.16:0/4126058053 learned_addr learned my addr 172.21.15.16:0/4126058053 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:43.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69ab7fe640 1 -- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69ac071aa0 msgr2=0x7f69ac0c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69ab7fe640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69ac071aa0 0x7f69ac0c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69ab7fe640 1 -- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac06a6d0 msgr2=0x7f69ac15b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:43.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69aaffd640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac06a6d0 0x7f69ac15b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69ab7fe640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac06a6d0 0x7f69ac15b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.777+0000 7f69ab7fe640 1 -- 172.21.15.16:0/4126058053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69940094d0 con 0x7f69ac15cff0 2023-11-13T11:01:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.778+0000 7f69ab7fe640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac15cff0 0x7f69ac15fb70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007e40 tx=0x7f69a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.778+0000 7f69a37fe640 1 -- 172.21.15.16:0/4126058053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4014620 con 0x7f69ac15cff0 2023-11-13T11:01:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.778+0000 7f69a37fe640 1 -- 172.21.15.16:0/4126058053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a400ee90 con 0x7f69ac15cff0 2023-11-13T11:01:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.778+0000 7f69a37fe640 1 -- 172.21.15.16:0/4126058053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a40149e0 con 0x7f69ac15cff0 2023-11-13T11:01:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.778+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ac148fb0 con 0x7f69ac15cff0 2023-11-13T11:01:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.778+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ac15d460 con 0x7f69ac15cff0 2023-11-13T11:01:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.779+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ac149770 con 0x7f69ac15cff0 2023-11-13T11:01:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.780+0000 7f69a37fe640 1 -- 172.21.15.16:0/4126058053 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69a4014b40 con 0x7f69ac15cff0 2023-11-13T11:01:43.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.781+0000 7f69a37fe640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698c06ae40 0x7f698c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:43.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.781+0000 7f69a37fe640 1 -- 172.21.15.16:0/4126058053 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f69a408a170 con 0x7f69ac15cff0 2023-11-13T11:01:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.782+0000 7f69aaffd640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698c06ae40 0x7f698c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.783+0000 7f69aaffd640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698c06ae40 0x7f698c06d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f699c006fd0 tx=0x7f699c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.786+0000 7f69a37fe640 1 -- 172.21.15.16:0/4126058053 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69a40550e0 con 0x7f69ac15cff0 2023-11-13T11:01:43.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.976+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69ac0d0530 con 0x7f698c06ae40 2023-11-13T11:01:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:43.998+0000 7f69a37fe640 1 -- 172.21.15.16:0/4126058053 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286421 (secure 0 0 0) 0x7f699c00e470 con 0x7f698c06ae40 2023-11-13T11:01:44.008 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:01:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698c06ae40 msgr2=0x7f698c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698c06ae40 0x7f698c06d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f699c006fd0 tx=0x7f699c008040 comp rx=0 tx=0).stop 2023-11-13T11:01:44.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac15cff0 msgr2=0x7f69ac15fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac15cff0 0x7f69ac15fb70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007e40 tx=0x7f69a400e4b0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 shutdown_connections 2023-11-13T11:01:44.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f698c06ae40 0x7f698c06d300 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ac15cff0 0x7f69ac15fb70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69ac071aa0 0x7f69ac0c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 --2- 172.21.15.16:0/4126058053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69ac06a6d0 0x7f69ac15b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 >> 172.21.15.16:0/4126058053 conn(0x7f69ac06b2a0 msgr2=0x7f69ac15c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 shutdown_connections 2023-11-13T11:01:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.005+0000 7f69abfff640 1 -- 172.21.15.16:0/4126058053 wait complete. 2023-11-13T11:01:44.031 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2023-11-13T11:01:42.117958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":184,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":359544,"kb_used_data":6028,"kb_used_omap":12,"kb_used_meta":353395,"kb_avail":754615176,"statfs":{"total":773094113280,"available":772725940224,"internally_reserved":0,"allocated":6172672,"data_stored":3364718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":361877041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.187203"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997544+0000","last_change":"2023-11-13T11:01:38.993640+0000","last_active":"2023-11-13T11:01:38.997544+0000","last_peered":"2023-11-13T11:01:38.997544+0000","last_clean":"2023-11-13T11:01:38.997544+0000","last_became_active":"2023-11-13T11:01:38.991479+0000","last_became_peered":"2023-11-13T11:01:38.991479+0000","last_unstale":"2023-11-13T11:01:38.997544+0000","last_undegraded":"2023-11-13T11:01:38.997544+0000","last_fullsized":"2023-11-13T11:01:38.997544+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977509+0000","last_change":"2023-11-13T11:01:38.975999+0000","last_active":"2023-11-13T11:01:38.977509+0000","last_peered":"2023-11-13T11:01:38.977509+0000","last_clean":"2023-11-13T11:01:38.977509+0000","last_became_active":"2023-11-13T11:01:38.975726+0000","last_became_peered":"2023-11-13T11:01:38.975726+0000","last_unstale":"2023-11-13T11:01:38.977509+0000","last_undegraded":"2023-11-13T11:01:38.977509+0000","last_fullsized":"2023-11-13T11:01:38.977509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:33:09.247389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991945+0000","last_change":"2023-11-13T11:01:38.984201+0000","last_active":"2023-11-13T11:01:38.991945+0000","last_peered":"2023-11-13T11:01:38.991945+0000","last_clean":"2023-11-13T11:01:38.991945+0000","last_became_active":"2023-11-13T11:01:38.983716+0000","last_became_peered":"2023-11-13T11:01:38.983716+0000","last_unstale":"2023-11-13T11:01:38.991945+0000","last_undegraded":"2023-11-13T11:01:38.991945+0000","last_fullsized":"2023-11-13T11:01:38.991945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:37:29.570297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980437+0000","last_change":"2023-11-13T11:01:38.975163+0000","last_active":"2023-11-13T11:01:38.980437+0000","last_peered":"2023-11-13T11:01:38.980437+0000","last_clean":"2023-11-13T11:01:38.980437+0000","last_became_active":"2023-11-13T11:01:38.974878+0000","last_became_peered":"2023-11-13T11:01:38.974878+0000","last_unstale":"2023-11-13T11:01:38.980437+0000","last_undegraded":"2023-11-13T11:01:38.980437+0000","last_fullsized":"2023-11-13T11:01:38.980437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.970996+0000","last_change":"2023-11-13T11:01:38.968974+0000","last_active":"2023-11-13T11:01:38.970996+0000","last_peered":"2023-11-13T11:01:38.970996+0000","last_clean":"2023-11-13T11:01:38.970996+0000","last_became_active":"2023-11-13T11:01:38.968747+0000","last_became_peered":"2023-11-13T11:01:38.968747+0000","last_unstale":"2023-11-13T11:01:38.970996+0000","last_undegraded":"2023-11-13T11:01:38.970996+0000","last_fullsized":"2023-11-13T11:01:38.970996+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:14:08.138073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.961941+0000","last_change":"2023-11-13T11:01:37.961941+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.961941+0000","last_undegraded":"2023-11-13T11:01:37.961941+0000","last_fullsized":"2023-11-13T11:01:37.961941+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.006474+0000","last_change":"2023-11-13T11:01:39.003582+0000","last_active":"2023-11-13T11:01:39.006474+0000","last_peered":"2023-11-13T11:01:39.006474+0000","last_clean":"2023-11-13T11:01:39.006474+0000","last_became_active":"2023-11-13T11:01:39.003324+0000","last_became_peered":"2023-11-13T11:01:39.003324+0000","last_unstale":"2023-11-13T11:01:39.006474+0000","last_undegraded":"2023-11-13T11:01:39.006474+0000","last_fullsized":"2023-11-13T11:01:39.006474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.962816+0000","last_change":"2023-11-13T11:01:37.962816+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.962816+0000","last_undegraded":"2023-11-13T11:01:37.962816+0000","last_fullsized":"2023-11-13T11:01:37.962816+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.003277+0000","last_change":"2023-11-13T11:01:38.997117+0000","last_active":"2023-11-13T11:01:39.003277+0000","last_peered":"2023-11-13T11:01:39.003277+0000","last_clean":"2023-11-13T11:01:39.003277+0000","last_became_active":"2023-11-13T11:01:38.996801+0000","last_became_peered":"2023-11-13T11:01:38.996801+0000","last_unstale":"2023-11-13T11:01:39.003277+0000","last_undegraded":"2023-11-13T11:01:39.003277+0000","last_fullsized":"2023-11-13T11:01:39.003277+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:03:35.448965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990548+0000","last_change":"2023-11-13T11:01:38.987650+0000","last_active":"2023-11-13T11:01:38.990548+0000","last_peered":"2023-11-13T11:01:38.990548+0000","last_clean":"2023-11-13T11:01:38.990548+0000","last_became_active":"2023-11-13T11:01:38.987270+0000","last_became_peered":"2023-11-13T11:01:38.987270+0000","last_unstale":"2023-11-13T11:01:38.990548+0000","last_undegraded":"2023-11-13T11:01:38.990548+0000","last_fullsized":"2023-11-13T11:01:38.990548+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:25.371717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992670+0000","last_change":"2023-11-13T11:01:38.984512+0000","last_active":"2023-11-13T11:01:38.992670+0000","last_peered":"2023-11-13T11:01:38.992670+0000","last_clean":"2023-11-13T11:01:38.992670+0000","last_became_active":"2023-11-13T11:01:38.984243+0000","last_became_peered":"2023-11-13T11:01:38.984243+0000","last_unstale":"2023-11-13T11:01:38.992670+0000","last_undegraded":"2023-11-13T11:01:38.992670+0000","last_fullsized":"2023-11-13T11:01:38.992670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.963379+0000","last_change":"2023-11-13T11:01:37.963379+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.963379+0000","last_undegraded":"2023-11-13T11:01:37.963379+0000","last_fullsized":"2023-11-13T11:01:37.963379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.962841+0000","last_change":"2023-11-13T11:01:37.962841+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.962841+0000","last_undegraded":"2023-11-13T11:01:37.962841+0000","last_fullsized":"2023-11-13T11:01:37.962841+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.994357+0000","last_change":"2023-11-13T11:01:38.992189+0000","last_active":"2023-11-13T11:01:38.994357+0000","last_peered":"2023-11-13T11:01:38.994357+0000","last_clean":"2023-11-13T11:01:38.994357+0000","last_became_active":"2023-11-13T11:01:38.991290+0000","last_became_peered":"2023-11-13T11:01:38.991290+0000","last_unstale":"2023-11-13T11:01:38.994357+0000","last_undegraded":"2023-11-13T11:01:38.994357+0000","last_fullsized":"2023-11-13T11:01:38.994357+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:25.371717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.969075+0000","last_change":"2023-11-13T11:01:38.966119+0000","last_active":"2023-11-13T11:01:38.969075+0000","last_peered":"2023-11-13T11:01:38.969075+0000","last_clean":"2023-11-13T11:01:38.969075+0000","last_became_active":"2023-11-13T11:01:38.965812+0000","last_became_peered":"2023-11-13T11:01:38.965812+0000","last_unstale":"2023-11-13T11:01:38.969075+0000","last_undegraded":"2023-11-13T11:01:38.969075+0000","last_fullsized":"2023-11-13T11:01:38.969075+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.962280+0000","last_change":"2023-11-13T11:01:37.962280+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.962280+0000","last_undegraded":"2023-11-13T11:01:37.962280+0000","last_fullsized":"2023-11-13T11:01:37.962280+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997524+0000","last_change":"2023-11-13T11:01:38.993324+0000","last_active":"2023-11-13T11:01:38.997524+0000","last_peered":"2023-11-13T11:01:38.997524+0000","last_clean":"2023-11-13T11:01:38.997524+0000","last_became_active":"2023-11-13T11:01:38.991072+0000","last_became_peered":"2023-11-13T11:01:38.991072+0000","last_unstale":"2023-11-13T11:01:38.997524+0000","last_undegraded":"2023-11-13T11:01:38.997524+0000","last_fullsized":"2023-11-13T11:01:38.997524+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980403+0000","last_change":"2023-11-13T11:01:38.975150+0000","last_active":"2023-11-13T11:01:38.980403+0000","last_peered":"2023-11-13T11:01:38.980403+0000","last_clean":"2023-11-13T11:01:38.980403+0000","last_became_active":"2023-11-13T11:01:38.974837+0000","last_became_peered":"2023-11-13T11:01:38.974837+0000","last_unstale":"2023-11-13T11:01:38.980403+0000","last_undegraded":"2023-11-13T11:01:38.980403+0000","last_fullsized":"2023-11-13T11:01:38.980403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:19:00.701068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991920+0000","last_change":"2023-11-13T11:01:38.984024+0000","last_active":"2023-11-13T11:01:38.991920+0000","last_peered":"2023-11-13T11:01:38.991920+0000","last_clean":"2023-11-13T11:01:38.991920+0000","last_became_active":"2023-11-13T11:01:38.983338+0000","last_became_peered":"2023-11-13T11:01:38.983338+0000","last_unstale":"2023-11-13T11:01:38.991920+0000","last_undegraded":"2023-11-13T11:01:38.991920+0000","last_fullsized":"2023-11-13T11:01:38.991920+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:06:44.792510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973436+0000","last_change":"2023-11-13T11:01:38.969241+0000","last_active":"2023-11-13T11:01:38.973436+0000","last_peered":"2023-11-13T11:01:38.973436+0000","last_clean":"2023-11-13T11:01:38.973436+0000","last_became_active":"2023-11-13T11:01:38.968942+0000","last_became_peered":"2023-11-13T11:01:38.968942+0000","last_unstale":"2023-11-13T11:01:38.973436+0000","last_undegraded":"2023-11-13T11:01:38.973436+0000","last_fullsized":"2023-11-13T11:01:38.973436+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:19:00.701068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.972503+0000","last_change":"2023-11-13T11:01:38.966987+0000","last_active":"2023-11-13T11:01:38.972503+0000","last_peered":"2023-11-13T11:01:38.972503+0000","last_clean":"2023-11-13T11:01:38.972503+0000","last_became_active":"2023-11-13T11:01:38.966723+0000","last_became_peered":"2023-11-13T11:01:38.966723+0000","last_unstale":"2023-11-13T11:01:38.972503+0000","last_undegraded":"2023-11-13T11:01:38.972503+0000","last_fullsized":"2023-11-13T11:01:38.972503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:33:09.247389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993826+0000","last_change":"2023-11-13T11:01:38.989021+0000","last_active":"2023-11-13T11:01:38.993826+0000","last_peered":"2023-11-13T11:01:38.993826+0000","last_clean":"2023-11-13T11:01:38.993826+0000","last_became_active":"2023-11-13T11:01:38.988715+0000","last_became_peered":"2023-11-13T11:01:38.988715+0000","last_unstale":"2023-11-13T11:01:38.993826+0000","last_undegraded":"2023-11-13T11:01:38.993826+0000","last_fullsized":"2023-11-13T11:01:38.993826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:12:15.067759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993091+0000","last_change":"2023-11-13T11:01:38.986587+0000","last_active":"2023-11-13T11:01:38.993091+0000","last_peered":"2023-11-13T11:01:38.993091+0000","last_clean":"2023-11-13T11:01:38.993091+0000","last_became_active":"2023-11-13T11:01:38.986301+0000","last_became_peered":"2023-11-13T11:01:38.986301+0000","last_unstale":"2023-11-13T11:01:38.993091+0000","last_undegraded":"2023-11-13T11:01:38.993091+0000","last_fullsized":"2023-11-13T11:01:38.993091+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:39.111281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.995951+0000","last_change":"2023-11-13T11:01:38.992108+0000","last_active":"2023-11-13T11:01:38.995951+0000","last_peered":"2023-11-13T11:01:38.995951+0000","last_clean":"2023-11-13T11:01:38.995951+0000","last_became_active":"2023-11-13T11:01:38.988156+0000","last_became_peered":"2023-11-13T11:01:38.988156+0000","last_unstale":"2023-11-13T11:01:38.995951+0000","last_undegraded":"2023-11-13T11:01:38.995951+0000","last_fullsized":"2023-11-13T11:01:38.995951+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:02.526493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000637+0000","last_change":"2023-11-13T11:01:38.996834+0000","last_active":"2023-11-13T11:01:39.000637+0000","last_peered":"2023-11-13T11:01:39.000637+0000","last_clean":"2023-11-13T11:01:39.000637+0000","last_became_active":"2023-11-13T11:01:38.996522+0000","last_became_peered":"2023-11-13T11:01:38.996522+0000","last_unstale":"2023-11-13T11:01:39.000637+0000","last_undegraded":"2023-11-13T11:01:39.000637+0000","last_fullsized":"2023-11-13T11:01:39.000637+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:01.750338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005129+0000","last_change":"2023-11-13T11:01:39.003837+0000","last_active":"2023-11-13T11:01:39.005129+0000","last_peered":"2023-11-13T11:01:39.005129+0000","last_clean":"2023-11-13T11:01:39.005129+0000","last_became_active":"2023-11-13T11:01:39.000372+0000","last_became_peered":"2023-11-13T11:01:39.000372+0000","last_unstale":"2023-11-13T11:01:39.005129+0000","last_undegraded":"2023-11-13T11:01:39.005129+0000","last_fullsized":"2023-11-13T11:01:39.005129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:29:27.396844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.964512+0000","last_change":"2023-11-13T11:01:37.964512+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.964512+0000","last_undegraded":"2023-11-13T11:01:37.964512+0000","last_fullsized":"2023-11-13T11:01:37.964512+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.001088+0000","last_change":"2023-11-13T11:01:39.000662+0000","last_active":"2023-11-13T11:01:39.001088+0000","last_peered":"2023-11-13T11:01:39.001088+0000","last_clean":"2023-11-13T11:01:39.001088+0000","last_became_active":"2023-11-13T11:01:39.000191+0000","last_became_peered":"2023-11-13T11:01:39.000191+0000","last_unstale":"2023-11-13T11:01:39.001088+0000","last_undegraded":"2023-11-13T11:01:39.001088+0000","last_fullsized":"2023-11-13T11:01:39.001088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980942+0000","last_change":"2023-11-13T11:01:38.969203+0000","last_active":"2023-11-13T11:01:38.980942+0000","last_peered":"2023-11-13T11:01:38.980942+0000","last_clean":"2023-11-13T11:01:38.980942+0000","last_became_active":"2023-11-13T11:01:38.968896+0000","last_became_peered":"2023-11-13T11:01:38.968896+0000","last_unstale":"2023-11-13T11:01:38.980942+0000","last_undegraded":"2023-11-13T11:01:38.980942+0000","last_fullsized":"2023-11-13T11:01:38.980942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991585+0000","last_change":"2023-11-13T11:01:38.989974+0000","last_active":"2023-11-13T11:01:38.991585+0000","last_peered":"2023-11-13T11:01:38.991585+0000","last_clean":"2023-11-13T11:01:38.991585+0000","last_became_active":"2023-11-13T11:01:38.989665+0000","last_became_peered":"2023-11-13T11:01:38.989665+0000","last_unstale":"2023-11-13T11:01:38.991585+0000","last_undegraded":"2023-11-13T11:01:38.991585+0000","last_fullsized":"2023-11-13T11:01:38.991585+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:18:26.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000755+0000","last_change":"2023-11-13T11:01:38.995148+0000","last_active":"2023-11-13T11:01:39.000755+0000","last_peered":"2023-11-13T11:01:39.000755+0000","last_clean":"2023-11-13T11:01:39.000755+0000","last_became_active":"2023-11-13T11:01:38.990007+0000","last_became_peered":"2023-11-13T11:01:38.990007+0000","last_unstale":"2023-11-13T11:01:39.000755+0000","last_undegraded":"2023-11-13T11:01:39.000755+0000","last_fullsized":"2023-11-13T11:01:39.000755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:18:26.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.988066+0000","last_change":"2023-11-13T11:01:38.984675+0000","last_active":"2023-11-13T11:01:38.988066+0000","last_peered":"2023-11-13T11:01:38.988066+0000","last_clean":"2023-11-13T11:01:38.988066+0000","last_became_active":"2023-11-13T11:01:38.984073+0000","last_became_peered":"2023-11-13T11:01:38.984073+0000","last_unstale":"2023-11-13T11:01:38.988066+0000","last_undegraded":"2023-11-13T11:01:38.988066+0000","last_fullsized":"2023-11-13T11:01:38.988066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:24:13.781188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979482+0000","last_change":"2023-11-13T11:01:38.973375+0000","last_active":"2023-11-13T11:01:38.979482+0000","last_peered":"2023-11-13T11:01:38.979482+0000","last_clean":"2023-11-13T11:01:38.979482+0000","last_became_active":"2023-11-13T11:01:38.970971+0000","last_became_peered":"2023-11-13T11:01:38.970971+0000","last_unstale":"2023-11-13T11:01:38.979482+0000","last_undegraded":"2023-11-13T11:01:38.979482+0000","last_fullsized":"2023-11-13T11:01:38.979482+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:03:35.448965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.999535+0000","last_change":"2023-11-13T11:01:37.976416+0000","last_active":"2023-11-13T11:01:38.999535+0000","last_peered":"2023-11-13T11:01:38.999535+0000","last_clean":"2023-11-13T11:01:38.999535+0000","last_became_active":"2023-11-13T11:01:37.976080+0000","last_became_peered":"2023-11-13T11:01:37.976080+0000","last_unstale":"2023-11-13T11:01:38.999535+0000","last_undegraded":"2023-11-13T11:01:38.999535+0000","last_fullsized":"2023-11-13T11:01:38.999535+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979796+0000","last_change":"2023-11-13T11:01:38.977746+0000","last_active":"2023-11-13T11:01:38.979796+0000","last_peered":"2023-11-13T11:01:38.979796+0000","last_clean":"2023-11-13T11:01:38.979796+0000","last_became_active":"2023-11-13T11:01:38.977421+0000","last_became_peered":"2023-11-13T11:01:38.977421+0000","last_unstale":"2023-11-13T11:01:38.979796+0000","last_undegraded":"2023-11-13T11:01:38.979796+0000","last_fullsized":"2023-11-13T11:01:38.979796+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:29:27.396844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.985691+0000","last_change":"2023-11-13T11:01:37.984920+0000","last_active":"2023-11-13T11:01:37.985691+0000","last_peered":"2023-11-13T11:01:37.985691+0000","last_clean":"2023-11-13T11:01:37.985691+0000","last_became_active":"2023-11-13T11:01:37.984119+0000","last_became_peered":"2023-11-13T11:01:37.984119+0000","last_unstale":"2023-11-13T11:01:37.985691+0000","last_undegraded":"2023-11-13T11:01:37.985691+0000","last_fullsized":"2023-11-13T11:01:37.985691+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:20:09.176965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977904+0000","last_change":"2023-11-13T11:01:38.971332+0000","last_active":"2023-11-13T11:01:38.977904+0000","last_peered":"2023-11-13T11:01:38.977904+0000","last_clean":"2023-11-13T11:01:38.977904+0000","last_became_active":"2023-11-13T11:01:38.969969+0000","last_became_peered":"2023-11-13T11:01:38.969969+0000","last_unstale":"2023-11-13T11:01:38.977904+0000","last_undegraded":"2023-11-13T11:01:38.977904+0000","last_fullsized":"2023-11-13T11:01:38.977904+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:09:56.697516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000885+0000","last_change":"2023-11-13T11:01:37.970093+0000","last_active":"2023-11-13T11:01:39.000885+0000","last_peered":"2023-11-13T11:01:39.000885+0000","last_clean":"2023-11-13T11:01:39.000885+0000","last_became_active":"2023-11-13T11:01:37.969770+0000","last_became_peered":"2023-11-13T11:01:37.969770+0000","last_unstale":"2023-11-13T11:01:39.000885+0000","last_undegraded":"2023-11-13T11:01:39.000885+0000","last_fullsized":"2023-11-13T11:01:39.000885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:33.254128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990202+0000","last_change":"2023-11-13T11:01:38.980529+0000","last_active":"2023-11-13T11:01:38.990202+0000","last_peered":"2023-11-13T11:01:38.990202+0000","last_clean":"2023-11-13T11:01:38.990202+0000","last_became_active":"2023-11-13T11:01:38.980227+0000","last_became_peered":"2023-11-13T11:01:38.980227+0000","last_unstale":"2023-11-13T11:01:38.990202+0000","last_undegraded":"2023-11-13T11:01:38.990202+0000","last_fullsized":"2023-11-13T11:01:38.990202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:12.303326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.994946+0000","last_change":"2023-11-13T11:01:37.982793+0000","last_active":"2023-11-13T11:01:38.994946+0000","last_peered":"2023-11-13T11:01:38.994946+0000","last_clean":"2023-11-13T11:01:38.994946+0000","last_became_active":"2023-11-13T11:01:37.982111+0000","last_became_peered":"2023-11-13T11:01:37.982111+0000","last_unstale":"2023-11-13T11:01:38.994946+0000","last_undegraded":"2023-11-13T11:01:38.994946+0000","last_fullsized":"2023-11-13T11:01:38.994946+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980711+0000","last_change":"2023-11-13T11:01:38.976171+0000","last_active":"2023-11-13T11:01:38.980711+0000","last_peered":"2023-11-13T11:01:38.980711+0000","last_clean":"2023-11-13T11:01:38.980711+0000","last_became_active":"2023-11-13T11:01:38.975895+0000","last_became_peered":"2023-11-13T11:01:38.975895+0000","last_unstale":"2023-11-13T11:01:38.980711+0000","last_undegraded":"2023-11-13T11:01:38.980711+0000","last_fullsized":"2023-11-13T11:01:38.980711+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.002584+0000","last_change":"2023-11-13T11:01:37.981492+0000","last_active":"2023-11-13T11:01:39.002584+0000","last_peered":"2023-11-13T11:01:39.002584+0000","last_clean":"2023-11-13T11:01:39.002584+0000","last_became_active":"2023-11-13T11:01:37.981149+0000","last_became_peered":"2023-11-13T11:01:37.981149+0000","last_unstale":"2023-11-13T11:01:39.002584+0000","last_undegraded":"2023-11-13T11:01:39.002584+0000","last_fullsized":"2023-11-13T11:01:39.002584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979333+0000","last_change":"2023-11-13T11:01:38.972149+0000","last_active":"2023-11-13T11:01:38.979333+0000","last_peered":"2023-11-13T11:01:38.979333+0000","last_clean":"2023-11-13T11:01:38.979333+0000","last_became_active":"2023-11-13T11:01:38.971860+0000","last_became_peered":"2023-11-13T11:01:38.971860+0000","last_unstale":"2023-11-13T11:01:38.979333+0000","last_undegraded":"2023-11-13T11:01:38.979333+0000","last_fullsized":"2023-11-13T11:01:38.979333+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:44:30.617363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986684+0000","last_change":"2023-11-13T11:01:37.968032+0000","last_active":"2023-11-13T11:01:38.986684+0000","last_peered":"2023-11-13T11:01:38.986684+0000","last_clean":"2023-11-13T11:01:38.986684+0000","last_became_active":"2023-11-13T11:01:37.967709+0000","last_became_peered":"2023-11-13T11:01:37.967709+0000","last_unstale":"2023-11-13T11:01:38.986684+0000","last_undegraded":"2023-11-13T11:01:38.986684+0000","last_fullsized":"2023-11-13T11:01:38.986684+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:44:32.588239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.003718+0000","last_change":"2023-11-13T11:01:39.001367+0000","last_active":"2023-11-13T11:01:39.003718+0000","last_peered":"2023-11-13T11:01:39.003718+0000","last_clean":"2023-11-13T11:01:39.003718+0000","last_became_active":"2023-11-13T11:01:39.001092+0000","last_became_peered":"2023-11-13T11:01:39.001092+0000","last_unstale":"2023-11-13T11:01:39.003718+0000","last_undegraded":"2023-11-13T11:01:39.003718+0000","last_fullsized":"2023-11-13T11:01:39.003718+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.962390+0000","last_change":"2023-11-13T11:01:37.974253+0000","last_active":"2023-11-13T11:01:38.962390+0000","last_peered":"2023-11-13T11:01:38.962390+0000","last_clean":"2023-11-13T11:01:38.962390+0000","last_became_active":"2023-11-13T11:01:37.973388+0000","last_became_peered":"2023-11-13T11:01:37.973388+0000","last_unstale":"2023-11-13T11:01:38.962390+0000","last_undegraded":"2023-11-13T11:01:38.962390+0000","last_fullsized":"2023-11-13T11:01:38.962390+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997619+0000","last_change":"2023-11-13T11:01:38.996067+0000","last_active":"2023-11-13T11:01:38.997619+0000","last_peered":"2023-11-13T11:01:38.997619+0000","last_clean":"2023-11-13T11:01:38.997619+0000","last_became_active":"2023-11-13T11:01:38.995583+0000","last_became_peered":"2023-11-13T11:01:38.995583+0000","last_unstale":"2023-11-13T11:01:38.997619+0000","last_undegraded":"2023-11-13T11:01:38.997619+0000","last_fullsized":"2023-11-13T11:01:38.997619+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973630+0000","last_change":"2023-11-13T11:01:37.980576+0000","last_active":"2023-11-13T11:01:38.973630+0000","last_peered":"2023-11-13T11:01:38.973630+0000","last_clean":"2023-11-13T11:01:38.973630+0000","last_became_active":"2023-11-13T11:01:37.980253+0000","last_became_peered":"2023-11-13T11:01:37.980253+0000","last_unstale":"2023-11-13T11:01:38.973630+0000","last_undegraded":"2023-11-13T11:01:38.973630+0000","last_fullsized":"2023-11-13T11:01:38.973630+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.982849+0000","last_change":"2023-11-13T11:01:38.978906+0000","last_active":"2023-11-13T11:01:38.982849+0000","last_peered":"2023-11-13T11:01:38.982849+0000","last_clean":"2023-11-13T11:01:38.982849+0000","last_became_active":"2023-11-13T11:01:38.978590+0000","last_became_peered":"2023-11-13T11:01:38.978590+0000","last_unstale":"2023-11-13T11:01:38.982849+0000","last_undegraded":"2023-11-13T11:01:38.982849+0000","last_fullsized":"2023-11-13T11:01:38.982849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.957725+0000","last_change":"2023-11-13T11:01:37.967093+0000","last_active":"2023-11-13T11:01:38.957725+0000","last_peered":"2023-11-13T11:01:38.957725+0000","last_clean":"2023-11-13T11:01:38.957725+0000","last_became_active":"2023-11-13T11:01:37.966819+0000","last_became_peered":"2023-11-13T11:01:37.966819+0000","last_unstale":"2023-11-13T11:01:38.957725+0000","last_undegraded":"2023-11-13T11:01:38.957725+0000","last_fullsized":"2023-11-13T11:01:38.957725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980445+0000","last_change":"2023-11-13T11:01:38.978007+0000","last_active":"2023-11-13T11:01:38.980445+0000","last_peered":"2023-11-13T11:01:38.980445+0000","last_clean":"2023-11-13T11:01:38.980445+0000","last_became_active":"2023-11-13T11:01:38.977730+0000","last_became_peered":"2023-11-13T11:01:38.977730+0000","last_unstale":"2023-11-13T11:01:38.980445+0000","last_undegraded":"2023-11-13T11:01:38.980445+0000","last_fullsized":"2023-11-13T11:01:38.980445+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.957148+0000","last_change":"2023-11-13T11:01:37.974872+0000","last_active":"2023-11-13T11:01:38.957148+0000","last_peered":"2023-11-13T11:01:38.957148+0000","last_clean":"2023-11-13T11:01:38.957148+0000","last_became_active":"2023-11-13T11:01:37.973788+0000","last_became_peered":"2023-11-13T11:01:37.973788+0000","last_unstale":"2023-11-13T11:01:38.957148+0000","last_undegraded":"2023-11-13T11:01:38.957148+0000","last_fullsized":"2023-11-13T11:01:38.957148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:11.290032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992368+0000","last_change":"2023-11-13T11:01:38.982930+0000","last_active":"2023-11-13T11:01:38.992368+0000","last_peered":"2023-11-13T11:01:38.992368+0000","last_clean":"2023-11-13T11:01:38.992368+0000","last_became_active":"2023-11-13T11:01:38.982617+0000","last_became_peered":"2023-11-13T11:01:38.982617+0000","last_unstale":"2023-11-13T11:01:38.992368+0000","last_undegraded":"2023-11-13T11:01:38.992368+0000","last_fullsized":"2023-11-13T11:01:38.992368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:35.756266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997924+0000","last_change":"2023-11-13T11:01:37.976846+0000","last_active":"2023-11-13T11:01:38.997924+0000","last_peered":"2023-11-13T11:01:38.997924+0000","last_clean":"2023-11-13T11:01:38.997924+0000","last_became_active":"2023-11-13T11:01:37.976378+0000","last_became_peered":"2023-11-13T11:01:37.976378+0000","last_unstale":"2023-11-13T11:01:38.997924+0000","last_undegraded":"2023-11-13T11:01:38.997924+0000","last_fullsized":"2023-11-13T11:01:38.997924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.985125+0000","last_change":"2023-11-13T11:01:38.983595+0000","last_active":"2023-11-13T11:01:38.985125+0000","last_peered":"2023-11-13T11:01:38.985125+0000","last_clean":"2023-11-13T11:01:38.985125+0000","last_became_active":"2023-11-13T11:01:38.978958+0000","last_became_peered":"2023-11-13T11:01:38.978958+0000","last_unstale":"2023-11-13T11:01:38.985125+0000","last_undegraded":"2023-11-13T11:01:38.985125+0000","last_fullsized":"2023-11-13T11:01:38.985125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:52:25.507954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.008452+0000","last_change":"2023-11-13T11:01:37.983799+0000","last_active":"2023-11-13T11:01:39.008452+0000","last_peered":"2023-11-13T11:01:39.008452+0000","last_clean":"2023-11-13T11:01:39.008452+0000","last_became_active":"2023-11-13T11:01:37.983048+0000","last_became_peered":"2023-11-13T11:01:37.983048+0000","last_unstale":"2023-11-13T11:01:39.008452+0000","last_undegraded":"2023-11-13T11:01:39.008452+0000","last_fullsized":"2023-11-13T11:01:39.008452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:43.793217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979764+0000","last_change":"2023-11-13T11:01:38.972065+0000","last_active":"2023-11-13T11:01:38.979764+0000","last_peered":"2023-11-13T11:01:38.979764+0000","last_clean":"2023-11-13T11:01:38.979764+0000","last_became_active":"2023-11-13T11:01:38.971764+0000","last_became_peered":"2023-11-13T11:01:38.971764+0000","last_unstale":"2023-11-13T11:01:38.979764+0000","last_undegraded":"2023-11-13T11:01:38.979764+0000","last_fullsized":"2023-11-13T11:01:38.979764+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:27:51.217444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.979536+0000","last_change":"2023-11-13T11:01:37.976954+0000","last_active":"2023-11-13T11:01:37.979536+0000","last_peered":"2023-11-13T11:01:37.979536+0000","last_clean":"2023-11-13T11:01:37.979536+0000","last_became_active":"2023-11-13T11:01:37.976659+0000","last_became_peered":"2023-11-13T11:01:37.976659+0000","last_unstale":"2023-11-13T11:01:37.979536+0000","last_undegraded":"2023-11-13T11:01:37.979536+0000","last_fullsized":"2023-11-13T11:01:37.979536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:11:52.113188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000253+0000","last_change":"2023-11-13T11:01:38.995557+0000","last_active":"2023-11-13T11:01:39.000253+0000","last_peered":"2023-11-13T11:01:39.000253+0000","last_clean":"2023-11-13T11:01:39.000253+0000","last_became_active":"2023-11-13T11:01:38.995224+0000","last_became_peered":"2023-11-13T11:01:38.995224+0000","last_unstale":"2023-11-13T11:01:39.000253+0000","last_undegraded":"2023-11-13T11:01:39.000253+0000","last_fullsized":"2023-11-13T11:01:39.000253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.959098+0000","last_change":"2023-11-13T11:01:37.970890+0000","last_active":"2023-11-13T11:01:38.959098+0000","last_peered":"2023-11-13T11:01:38.959098+0000","last_clean":"2023-11-13T11:01:38.959098+0000","last_became_active":"2023-11-13T11:01:37.970585+0000","last_became_peered":"2023-11-13T11:01:37.970585+0000","last_unstale":"2023-11-13T11:01:38.959098+0000","last_undegraded":"2023-11-13T11:01:38.959098+0000","last_fullsized":"2023-11-13T11:01:38.959098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:24.358423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.964731+0000","last_change":"2023-11-13T11:01:37.964731+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.964731+0000","last_undegraded":"2023-11-13T11:01:37.964731+0000","last_fullsized":"2023-11-13T11:01:37.964731+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.961699+0000","last_change":"2023-11-13T11:01:37.966418+0000","last_active":"2023-11-13T11:01:38.961699+0000","last_peered":"2023-11-13T11:01:38.961699+0000","last_clean":"2023-11-13T11:01:38.961699+0000","last_became_active":"2023-11-13T11:01:37.966105+0000","last_became_peered":"2023-11-13T11:01:37.966105+0000","last_unstale":"2023-11-13T11:01:38.961699+0000","last_undegraded":"2023-11-13T11:01:38.961699+0000","last_fullsized":"2023-11-13T11:01:38.961699+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:44:09.646957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991081+0000","last_change":"2023-11-13T11:01:38.985149+0000","last_active":"2023-11-13T11:01:38.991081+0000","last_peered":"2023-11-13T11:01:38.991081+0000","last_clean":"2023-11-13T11:01:38.991081+0000","last_became_active":"2023-11-13T11:01:38.984828+0000","last_became_peered":"2023-11-13T11:01:38.984828+0000","last_unstale":"2023-11-13T11:01:38.991081+0000","last_undegraded":"2023-11-13T11:01:38.991081+0000","last_fullsized":"2023-11-13T11:01:38.991081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:27:51.217444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.985403+0000","last_change":"2023-11-13T11:01:37.983844+0000","last_active":"2023-11-13T11:01:37.985403+0000","last_peered":"2023-11-13T11:01:37.985403+0000","last_clean":"2023-11-13T11:01:37.985403+0000","last_became_active":"2023-11-13T11:01:37.982971+0000","last_became_peered":"2023-11-13T11:01:37.982971+0000","last_unstale":"2023-11-13T11:01:37.985403+0000","last_undegraded":"2023-11-13T11:01:37.985403+0000","last_fullsized":"2023-11-13T11:01:37.985403+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:06:49.184941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990249+0000","last_change":"2023-11-13T11:01:38.984132+0000","last_active":"2023-11-13T11:01:38.990249+0000","last_peered":"2023-11-13T11:01:38.990249+0000","last_clean":"2023-11-13T11:01:38.990249+0000","last_became_active":"2023-11-13T11:01:38.981507+0000","last_became_peered":"2023-11-13T11:01:38.981507+0000","last_unstale":"2023-11-13T11:01:38.990249+0000","last_undegraded":"2023-11-13T11:01:38.990249+0000","last_fullsized":"2023-11-13T11:01:38.990249+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.986923+0000","last_change":"2023-11-13T11:01:37.986267+0000","last_active":"2023-11-13T11:01:37.986923+0000","last_peered":"2023-11-13T11:01:37.986923+0000","last_clean":"2023-11-13T11:01:37.986923+0000","last_became_active":"2023-11-13T11:01:37.985787+0000","last_became_peered":"2023-11-13T11:01:37.985787+0000","last_unstale":"2023-11-13T11:01:37.986923+0000","last_undegraded":"2023-11-13T11:01:37.986923+0000","last_fullsized":"2023-11-13T11:01:37.986923+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:51:14.827858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992067+0000","last_change":"2023-11-13T11:01:38.984798+0000","last_active":"2023-11-13T11:01:38.992067+0000","last_peered":"2023-11-13T11:01:38.992067+0000","last_clean":"2023-11-13T11:01:38.992067+0000","last_became_active":"2023-11-13T11:01:38.984518+0000","last_became_peered":"2023-11-13T11:01:38.984518+0000","last_unstale":"2023-11-13T11:01:38.992067+0000","last_undegraded":"2023-11-13T11:01:38.992067+0000","last_fullsized":"2023-11-13T11:01:38.992067+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.979645+0000","last_change":"2023-11-13T11:01:37.977771+0000","last_active":"2023-11-13T11:01:37.979645+0000","last_peered":"2023-11-13T11:01:37.979645+0000","last_clean":"2023-11-13T11:01:37.979645+0000","last_became_active":"2023-11-13T11:01:37.977506+0000","last_became_peered":"2023-11-13T11:01:37.977506+0000","last_unstale":"2023-11-13T11:01:37.979645+0000","last_undegraded":"2023-11-13T11:01:37.979645+0000","last_fullsized":"2023-11-13T11:01:37.979645+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:48:24.890322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.995489+0000","last_change":"2023-11-13T11:01:38.994873+0000","last_active":"2023-11-13T11:01:38.995489+0000","last_peered":"2023-11-13T11:01:38.995489+0000","last_clean":"2023-11-13T11:01:38.995489+0000","last_became_active":"2023-11-13T11:01:38.994490+0000","last_became_peered":"2023-11-13T11:01:38.994490+0000","last_unstale":"2023-11-13T11:01:38.995489+0000","last_undegraded":"2023-11-13T11:01:38.995489+0000","last_fullsized":"2023-11-13T11:01:38.995489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:46:41.523607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997386+0000","last_change":"2023-11-13T11:01:37.982308+0000","last_active":"2023-11-13T11:01:38.997386+0000","last_peered":"2023-11-13T11:01:38.997386+0000","last_clean":"2023-11-13T11:01:38.997386+0000","last_became_active":"2023-11-13T11:01:37.981963+0000","last_became_peered":"2023-11-13T11:01:37.981963+0000","last_unstale":"2023-11-13T11:01:38.997386+0000","last_undegraded":"2023-11-13T11:01:38.997386+0000","last_fullsized":"2023-11-13T11:01:38.997386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.963540+0000","last_change":"2023-11-13T11:01:37.963540+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.963540+0000","last_undegraded":"2023-11-13T11:01:37.963540+0000","last_fullsized":"2023-11-13T11:01:37.963540+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.966021+0000","last_change":"2023-11-13T11:01:37.978277+0000","last_active":"2023-11-13T11:01:38.966021+0000","last_peered":"2023-11-13T11:01:38.966021+0000","last_clean":"2023-11-13T11:01:38.966021+0000","last_became_active":"2023-11-13T11:01:37.977659+0000","last_became_peered":"2023-11-13T11:01:37.977659+0000","last_unstale":"2023-11-13T11:01:38.966021+0000","last_undegraded":"2023-11-13T11:01:38.966021+0000","last_fullsized":"2023-11-13T11:01:38.966021+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:56.135585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977517+0000","last_change":"2023-11-13T11:01:38.975186+0000","last_active":"2023-11-13T11:01:38.977517+0000","last_peered":"2023-11-13T11:01:38.977517+0000","last_clean":"2023-11-13T11:01:38.977517+0000","last_became_active":"2023-11-13T11:01:38.972436+0000","last_became_peered":"2023-11-13T11:01:38.972436+0000","last_unstale":"2023-11-13T11:01:38.977517+0000","last_undegraded":"2023-11-13T11:01:38.977517+0000","last_fullsized":"2023-11-13T11:01:38.977517+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:45:08.666322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000163+0000","last_change":"2023-11-13T11:01:37.970760+0000","last_active":"2023-11-13T11:01:39.000163+0000","last_peered":"2023-11-13T11:01:39.000163+0000","last_clean":"2023-11-13T11:01:39.000163+0000","last_became_active":"2023-11-13T11:01:37.970465+0000","last_became_peered":"2023-11-13T11:01:37.970465+0000","last_unstale":"2023-11-13T11:01:39.000163+0000","last_undegraded":"2023-11-13T11:01:39.000163+0000","last_fullsized":"2023-11-13T11:01:39.000163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979277+0000","last_change":"2023-11-13T11:01:38.972817+0000","last_active":"2023-11-13T11:01:38.979277+0000","last_peered":"2023-11-13T11:01:38.979277+0000","last_clean":"2023-11-13T11:01:38.979277+0000","last_became_active":"2023-11-13T11:01:38.971691+0000","last_became_peered":"2023-11-13T11:01:38.971691+0000","last_unstale":"2023-11-13T11:01:38.979277+0000","last_undegraded":"2023-11-13T11:01:38.979277+0000","last_fullsized":"2023-11-13T11:01:38.979277+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.978810+0000","last_change":"2023-11-13T11:01:37.982283+0000","last_active":"2023-11-13T11:01:38.978810+0000","last_peered":"2023-11-13T11:01:38.978810+0000","last_clean":"2023-11-13T11:01:38.978810+0000","last_became_active":"2023-11-13T11:01:37.981958+0000","last_became_peered":"2023-11-13T11:01:37.981958+0000","last_unstale":"2023-11-13T11:01:38.978810+0000","last_undegraded":"2023-11-13T11:01:38.978810+0000","last_fullsized":"2023-11-13T11:01:38.978810+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:44:29.604069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991795+0000","last_change":"2023-11-13T11:01:38.982113+0000","last_active":"2023-11-13T11:01:38.991795+0000","last_peered":"2023-11-13T11:01:38.991795+0000","last_clean":"2023-11-13T11:01:38.991795+0000","last_became_active":"2023-11-13T11:01:38.980795+0000","last_became_peered":"2023-11-13T11:01:38.980795+0000","last_unstale":"2023-11-13T11:01:38.991795+0000","last_undegraded":"2023-11-13T11:01:38.991795+0000","last_fullsized":"2023-11-13T11:01:38.991795+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973311+0000","last_change":"2023-11-13T11:01:37.982552+0000","last_active":"2023-11-13T11:01:38.973311+0000","last_peered":"2023-11-13T11:01:38.973311+0000","last_clean":"2023-11-13T11:01:38.973311+0000","last_became_active":"2023-11-13T11:01:37.982269+0000","last_became_peered":"2023-11-13T11:01:37.982269+0000","last_unstale":"2023-11-13T11:01:38.973311+0000","last_undegraded":"2023-11-13T11:01:38.973311+0000","last_fullsized":"2023-11-13T11:01:38.973311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.963480+0000","last_change":"2023-11-13T11:01:37.963480+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.963480+0000","last_undegraded":"2023-11-13T11:01:37.963480+0000","last_fullsized":"2023-11-13T11:01:37.963480+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.975254+0000","last_change":"2023-11-13T11:01:37.971616+0000","last_active":"2023-11-13T11:01:38.975254+0000","last_peered":"2023-11-13T11:01:38.975254+0000","last_clean":"2023-11-13T11:01:38.975254+0000","last_became_active":"2023-11-13T11:01:37.971344+0000","last_became_peered":"2023-11-13T11:01:37.971344+0000","last_unstale":"2023-11-13T11:01:38.975254+0000","last_undegraded":"2023-11-13T11:01:38.975254+0000","last_fullsized":"2023-11-13T11:01:38.975254+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:31.732306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986510+0000","last_change":"2023-11-13T11:01:38.983983+0000","last_active":"2023-11-13T11:01:38.986510+0000","last_peered":"2023-11-13T11:01:38.986510+0000","last_clean":"2023-11-13T11:01:38.986510+0000","last_became_active":"2023-11-13T11:01:38.983698+0000","last_became_peered":"2023-11-13T11:01:38.983698+0000","last_unstale":"2023-11-13T11:01:38.986510+0000","last_undegraded":"2023-11-13T11:01:38.986510+0000","last_fullsized":"2023-11-13T11:01:38.986510+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:46:41.523607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997353+0000","last_change":"2023-11-13T11:01:37.972256+0000","last_active":"2023-11-13T11:01:38.997353+0000","last_peered":"2023-11-13T11:01:38.997353+0000","last_clean":"2023-11-13T11:01:38.997353+0000","last_became_active":"2023-11-13T11:01:37.971890+0000","last_became_peered":"2023-11-13T11:01:37.971890+0000","last_unstale":"2023-11-13T11:01:38.997353+0000","last_undegraded":"2023-11-13T11:01:38.997353+0000","last_fullsized":"2023-11-13T11:01:38.997353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:38.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990549+0000","last_change":"2023-11-13T11:01:38.988092+0000","last_active":"2023-11-13T11:01:38.990549+0000","last_peered":"2023-11-13T11:01:38.990549+0000","last_clean":"2023-11-13T11:01:38.990549+0000","last_became_active":"2023-11-13T11:01:38.987800+0000","last_became_peered":"2023-11-13T11:01:38.987800+0000","last_unstale":"2023-11-13T11:01:38.990549+0000","last_undegraded":"2023-11-13T11:01:38.990549+0000","last_fullsized":"2023-11-13T11:01:38.990549+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:28.542546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.964559+0000","last_change":"2023-11-13T11:01:37.973251+0000","last_active":"2023-11-13T11:01:38.964559+0000","last_peered":"2023-11-13T11:01:38.964559+0000","last_clean":"2023-11-13T11:01:38.964559+0000","last_became_active":"2023-11-13T11:01:37.972947+0000","last_became_peered":"2023-11-13T11:01:37.972947+0000","last_unstale":"2023-11-13T11:01:38.964559+0000","last_undegraded":"2023-11-13T11:01:38.964559+0000","last_fullsized":"2023-11-13T11:01:38.964559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993910+0000","last_change":"2023-11-13T11:01:38.988853+0000","last_active":"2023-11-13T11:01:38.993910+0000","last_peered":"2023-11-13T11:01:38.993910+0000","last_clean":"2023-11-13T11:01:38.993910+0000","last_became_active":"2023-11-13T11:01:38.987558+0000","last_became_peered":"2023-11-13T11:01:38.987558+0000","last_unstale":"2023-11-13T11:01:38.993910+0000","last_undegraded":"2023-11-13T11:01:38.993910+0000","last_fullsized":"2023-11-13T11:01:38.993910+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:38:32.234508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986162+0000","last_change":"2023-11-13T11:01:37.981807+0000","last_active":"2023-11-13T11:01:38.986162+0000","last_peered":"2023-11-13T11:01:38.986162+0000","last_clean":"2023-11-13T11:01:38.986162+0000","last_became_active":"2023-11-13T11:01:37.981480+0000","last_became_peered":"2023-11-13T11:01:37.981480+0000","last_unstale":"2023-11-13T11:01:38.986162+0000","last_undegraded":"2023-11-13T11:01:38.986162+0000","last_fullsized":"2023-11-13T11:01:38.986162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:09:55.684221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.999643+0000","last_change":"2023-11-13T11:01:38.986410+0000","last_active":"2023-11-13T11:01:38.999643+0000","last_peered":"2023-11-13T11:01:38.999643+0000","last_clean":"2023-11-13T11:01:38.999643+0000","last_became_active":"2023-11-13T11:01:38.986131+0000","last_became_peered":"2023-11-13T11:01:38.986131+0000","last_unstale":"2023-11-13T11:01:38.999643+0000","last_undegraded":"2023-11-13T11:01:38.999643+0000","last_fullsized":"2023-11-13T11:01:38.999643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.975456+0000","last_change":"2023-11-13T11:01:37.975627+0000","last_active":"2023-11-13T11:01:38.975456+0000","last_peered":"2023-11-13T11:01:38.975456+0000","last_clean":"2023-11-13T11:01:38.975456+0000","last_became_active":"2023-11-13T11:01:37.974747+0000","last_became_peered":"2023-11-13T11:01:37.974747+0000","last_unstale":"2023-11-13T11:01:38.975456+0000","last_undegraded":"2023-11-13T11:01:38.975456+0000","last_fullsized":"2023-11-13T11:01:38.975456+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990137+0000","last_change":"2023-11-13T11:01:38.989266+0000","last_active":"2023-11-13T11:01:38.990137+0000","last_peered":"2023-11-13T11:01:38.990137+0000","last_clean":"2023-11-13T11:01:38.990137+0000","last_became_active":"2023-11-13T11:01:38.988948+0000","last_became_peered":"2023-11-13T11:01:38.988948+0000","last_unstale":"2023-11-13T11:01:38.990137+0000","last_undegraded":"2023-11-13T11:01:38.990137+0000","last_fullsized":"2023-11-13T11:01:38.990137+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:12.303326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.978974+0000","last_change":"2023-11-13T11:01:37.982077+0000","last_active":"2023-11-13T11:01:38.978974+0000","last_peered":"2023-11-13T11:01:38.978974+0000","last_clean":"2023-11-13T11:01:38.978974+0000","last_became_active":"2023-11-13T11:01:37.981552+0000","last_became_peered":"2023-11-13T11:01:37.981552+0000","last_unstale":"2023-11-13T11:01:38.978974+0000","last_undegraded":"2023-11-13T11:01:38.978974+0000","last_fullsized":"2023-11-13T11:01:38.978974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993927+0000","last_change":"2023-11-13T11:01:38.993394+0000","last_active":"2023-11-13T11:01:38.993927+0000","last_peered":"2023-11-13T11:01:38.993927+0000","last_clean":"2023-11-13T11:01:38.993927+0000","last_became_active":"2023-11-13T11:01:38.992745+0000","last_became_peered":"2023-11-13T11:01:38.992745+0000","last_unstale":"2023-11-13T11:01:38.993927+0000","last_undegraded":"2023-11-13T11:01:38.993927+0000","last_fullsized":"2023-11-13T11:01:38.993927+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.961654+0000","last_change":"2023-11-13T11:01:37.971343+0000","last_active":"2023-11-13T11:01:38.961654+0000","last_peered":"2023-11-13T11:01:38.961654+0000","last_clean":"2023-11-13T11:01:38.961654+0000","last_became_active":"2023-11-13T11:01:37.970761+0000","last_became_peered":"2023-11-13T11:01:37.970761+0000","last_unstale":"2023-11-13T11:01:38.961654+0000","last_undegraded":"2023-11-13T11:01:38.961654+0000","last_fullsized":"2023-11-13T11:01:38.961654+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:47:45.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-13T11:01:37.964287+0000","last_change":"2023-11-13T11:01:37.964287+0000","last_active":"2023-11-13T11:01:37.929718+0000","last_peered":"2023-11-13T11:01:37.929718+0000","last_clean":"2023-11-13T11:01:37.929718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:01:37.964287+0000","last_undegraded":"2023-11-13T11:01:37.964287+0000","last_fullsized":"2023-11-13T11:01:37.964287+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.956737+0000","last_change":"2023-11-13T11:01:37.970057+0000","last_active":"2023-11-13T11:01:38.956737+0000","last_peered":"2023-11-13T11:01:38.956737+0000","last_clean":"2023-11-13T11:01:38.956737+0000","last_became_active":"2023-11-13T11:01:37.969819+0000","last_became_peered":"2023-11-13T11:01:37.969819+0000","last_unstale":"2023-11-13T11:01:38.956737+0000","last_undegraded":"2023-11-13T11:01:38.956737+0000","last_fullsized":"2023-11-13T11:01:38.956737+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:45:07.653027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.968087+0000","last_change":"2023-11-13T11:01:38.966353+0000","last_active":"2023-11-13T11:01:38.968087+0000","last_peered":"2023-11-13T11:01:38.968087+0000","last_clean":"2023-11-13T11:01:38.968087+0000","last_became_active":"2023-11-13T11:01:38.966082+0000","last_became_peered":"2023-11-13T11:01:38.966082+0000","last_unstale":"2023-11-13T11:01:38.968087+0000","last_undegraded":"2023-11-13T11:01:38.968087+0000","last_fullsized":"2023-11-13T11:01:38.968087+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977074+0000","last_change":"2023-11-13T11:01:37.970911+0000","last_active":"2023-11-13T11:01:38.977074+0000","last_peered":"2023-11-13T11:01:38.977074+0000","last_clean":"2023-11-13T11:01:38.977074+0000","last_became_active":"2023-11-13T11:01:37.970627+0000","last_became_peered":"2023-11-13T11:01:37.970627+0000","last_unstale":"2023-11-13T11:01:38.977074+0000","last_undegraded":"2023-11-13T11:01:38.977074+0000","last_fullsized":"2023-11-13T11:01:38.977074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.949346+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:01:37.949346+0000","last_peered":"2023-11-13T11:01:37.949346+0000","last_clean":"2023-11-13T11:01:37.949346+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:01:37.949346+0000","last_undegraded":"2023-11-13T11:01:37.949346+0000","last_fullsized":"2023-11-13T11:01:37.949346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0960000000000001,"15min":1.1259999999999999},"min":{"1min":0.84599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.21,"15min":1.1339999999999999},"min":{"1min":0.89800000000000002,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.865,"5min":1.865,"15min":2.2879999999999998},"last":1.353}]},{"osd":1,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.22,"15min":1.175},"min":{"1min":0.91700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8200000000000001,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2969999999999999,"15min":1.2889999999999999},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1070000000000002,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.2789999999999999,"15min":1.2769999999999999},"min":{"1min":0.98099999999999998,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.153,"5min":2.153,"15min":2.1680000000000001},"last":1.244},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3819999999999999,"15min":1.3520000000000001},"min":{"1min":1.1080000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.2400000000000002,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.298}]},{"osd":3,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.407,"15min":1.383},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.63800000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.423,"15min":1.4390000000000001},"min":{"1min":1.167,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.524}]},{"osd":4,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.454,"15min":1.5720000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.577,"5min":1.5129999999999999,"15min":1.5940000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.202}]},{"osd":5,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6000000000000001,"15min":1.6559999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.405},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.72,"15min":1.7589999999999999},"min":{"1min":1.147,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.6339999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.758,"5min":1.659,"15min":1.651},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.754,"5min":1.617,"15min":1.6499999999999999},"min":{"1min":0.96499999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.5940000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1216512,"data_stored":863292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.107,"15min":1.0129999999999999},"min":{"1min":0.82999999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9099999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2070000000000001,"15min":1.2030000000000001},"min":{"1min":0.74299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.639}]},{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.278,"15min":1.2310000000000001},"min":{"1min":0.89800000000000002,"5min":0.71699999999999997,"15min":0.65400000000000003},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.234,"15min":1.238},"min":{"1min":0.878,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8580000000000001,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.2250000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3129999999999999,"15min":1.2509999999999999},"min":{"1min":0.92700000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":1.099},{"interface":"front","average":{"1min":1.375,"5min":1.3120000000000001,"15min":1.2370000000000001},"min":{"1min":0.81599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.5209999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.393,"15min":1.399},"min":{"1min":0.86399999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80700000000000005},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":1.339}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3640000000000001,"15min":1.3340000000000001},"min":{"1min":0.86599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.294,"5min":3.484,"15min":3.484},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.4830000000000001,"15min":1.5349999999999999},"min":{"1min":1.123,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":1.4810000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5049999999999999,"15min":1.6080000000000001},"min":{"1min":1.1539999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.4929999999999999,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.538,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.6120000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6359999999999999,"15min":1.6799999999999999},"min":{"1min":1.167,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.7},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.653,"15min":1.714},"min":{"1min":1.2190000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.4410000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49840,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322000,"statfs":{"total":96636764160,"available":96585728000,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.1599999999999999,"15min":1.0820000000000001},"min":{"1min":0.86399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.103,"15min":1.075},"min":{"1min":0.94399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.218,"15min":1.1579999999999999},"min":{"1min":0.65800000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.7689999999999999,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2689999999999999,"15min":1.2549999999999999},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.821,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.998}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.27,"15min":1.2350000000000001},"min":{"1min":0.98899999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.3200000000000001,"15min":1.3480000000000001},"min":{"1min":0.996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.327}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.46,"15min":1.4139999999999999},"min":{"1min":1.1479999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":2.0049999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.98599999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.742,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.472,"15min":1.502},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0110000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.492},{"interface":"front","average":{"1min":1.504,"5min":1.3480000000000001,"15min":1.401},"min":{"1min":0.98399999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.1910000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.605,"15min":1.629},"min":{"1min":1.0329999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4850000000000001,"15min":1.5389999999999999},"min":{"1min":1.1459999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.6299999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":1.516,"15min":1.591},"min":{"1min":1.139,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.6040000000000001,"15min":1.621},"min":{"1min":1.1020000000000001,"5min":0.63,"15min":0.63},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.571}]}]},{"osd":3,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40124,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331716,"statfs":{"total":96636764160,"available":96595677184,"internally_reserved":0,"allocated":638976,"data_stored":280109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.85299999999999998,"15min":0.86499999999999999},"min":{"1min":0.53000000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.843,"5min":1.867,"15min":1.867},"last":0.68100000000000005},{"interface":"front","average":{"1min":1,"5min":0.98699999999999999,"15min":1.018},"min":{"1min":0.74399999999999999,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4239999999999999,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":0.80300000000000005}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.98599999999999999,"15min":1.04},"min":{"1min":0.65800000000000003,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.603,"5min":1.966,"15min":3.71},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.077,"15min":1.089},"min":{"1min":0.54400000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5660000000000001,"5min":1.923,"15min":2.0190000000000001},"last":0.92300000000000004}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.177,"15min":1.097},"min":{"1min":0.73599999999999999,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":1.5049999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.2909999999999999},"min":{"1min":0.83199999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5389999999999999,"5min":1.952,"15min":3.5710000000000002},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3109999999999999,"15min":1.4179999999999999},"min":{"1min":0.97199999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.5640000000000001,"5min":2.411,"15min":3.5030000000000001},"last":1.143},{"interface":"front","average":{"1min":1.49,"5min":1.345,"15min":1.349},"min":{"1min":1.1319999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.9370000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.615}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.379,"15min":1.476},"min":{"1min":1.073,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.7270000000000001,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.456},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4219999999999999,"15min":1.5149999999999999},"min":{"1min":0.86799999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.72,"5min":2.673,"15min":3.3809999999999998},"last":1.486}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4510000000000001,"15min":1.4870000000000001},"min":{"1min":1.1910000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8260000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.218},{"interface":"front","average":{"1min":1.446,"5min":1.4590000000000001,"15min":1.591},"min":{"1min":0.97099999999999997,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":1.855,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.3109999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.512,"15min":1.6120000000000001},"min":{"1min":1.1819999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.968,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.347},{"interface":"front","average":{"1min":1.647,"5min":1.5680000000000001,"15min":1.5549999999999999},"min":{"1min":1.319,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0059999999999998,"5min":2.75,"15min":2.75},"last":1.5369999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45680,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326160,"statfs":{"total":96636764160,"available":96589987840,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.89500000000000002,"15min":0.92900000000000005},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.9410000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.63},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.95699999999999996,"15min":0.92100000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.7789999999999999,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":0.54400000000000004}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":0.96299999999999997,"15min":0.95199999999999996},"min":{"1min":0.82999999999999996,"5min":0.376,"15min":0.376},"max":{"1min":1.893,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.105,"15min":1.127},"min":{"1min":0.68400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.978,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.133}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.139,"15min":1.2430000000000001},"min":{"1min":0.755,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.796,"5min":2.129,"15min":2.129},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.22,"15min":1.2729999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.867,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":0.98399999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.391,"15min":1.4750000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.9970000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3100000000000001,"15min":1.4039999999999999},"min":{"1min":0.79200000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0449999999999999,"5min":2.3639999999999999,"15min":2.6179999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.3959999999999999,"15min":1.48},"min":{"1min":0.96599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.6600000000000001},"last":1.034},{"interface":"front","average":{"1min":1.633,"5min":1.5409999999999999,"15min":1.5649999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.2450000000000001,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.3240000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.4359999999999999,"15min":1.55},"min":{"1min":0.83099999999999996,"5min":0.78500000000000003,"15min":0.76500000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":4.7569999999999997},"last":1.085},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.538,"15min":1.595},"min":{"1min":1.0229999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.621},"last":1.3620000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.488,"15min":1.6100000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.343,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.573,"15min":1.623},"min":{"1min":1.0740000000000001,"5min":0.85699999999999998,"15min":0.82799999999999996},"max":{"1min":2.2999999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.2909999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509519,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1216512,"data_stored":863292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.84099999999999997,"15min":0.95599999999999996},"min":{"1min":0.59299999999999997,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.47,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.105,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.754}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":0.99099999999999999,"15min":0.98299999999999998},"min":{"1min":0.71599999999999997,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0229999999999999,"15min":0.999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41999999999999998},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.722},"last":1.0029999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7629999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.101,"5min":1,"15min":1.0580000000000001},"min":{"1min":0.57899999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6899999999999999,"5min":1.8149999999999999,"15min":1.9059999999999999},"last":0.91100000000000003}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":0.98399999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.0569999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.363,"15min":1.375},"min":{"1min":1.0860000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.966,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.2290000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.3759999999999999,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83299999999999996},"max":{"1min":2.1629999999999998,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.103},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.4099999999999999},"min":{"1min":0.93999999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.0390000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2909999999999999,"15min":1.407},"min":{"1min":0.91400000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.986,"5min":1.986,"15min":2.3519999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.4339999999999999,"15min":1.4990000000000001},"min":{"1min":0.89800000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.3879999999999999},"last":1.3520000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.4410000000000001,"15min":1.496},"min":{"1min":0.93799999999999994,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.972,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.391,"15min":1.452},"min":{"1min":0.85799999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8660000000000001,"5min":2.202,"15min":2.202},"last":1.3300000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509520,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40108,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331732,"statfs":{"total":96636764160,"available":96595693568,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.90100000000000002},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32100000000000001},"max":{"1min":1.417,"5min":1.74,"15min":1.74},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.022,"15min":1.048},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.7330000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1459999999999999,"15min":1.141},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0089999999999999,"15min":1.018},"min":{"1min":0.54000000000000004,"5min":0.495,"15min":0.371},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.1669999999999998},"last":1.129}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1080000000000001,"15min":1.1020000000000001},"min":{"1min":0.63800000000000001,"5min":0.48999999999999999,"15min":0.45200000000000001},"max":{"1min":1.7450000000000001,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2769999999999999,"15min":1.329},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.827,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.0760000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3300000000000001,"15min":1.3540000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8009999999999999,"5min":2.153,"15min":2.153},"last":4.4279999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.296,"15min":1.4279999999999999},"min":{"1min":0.79400000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.708,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.5029999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.415,"15min":1.5449999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0430000000000001,"5min":2.2719999999999998,"15min":3.7490000000000001},"last":4.0449999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.3839999999999999,"15min":1.484},"min":{"1min":0.89100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.867,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":2.0310000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.48,"15min":1.5229999999999999},"min":{"1min":0.91100000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1389999999999998,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":4.617},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5369999999999999,"15min":1.5740000000000001},"min":{"1min":0.872,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.2810000000000001,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":3.6520000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.5900000000000001,"15min":1.6779999999999999},"min":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.21,"5min":2.742,"15min":3.3540000000000001},"last":1.522},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5800000000000001,"15min":1.6140000000000001},"min":{"1min":0.94999999999999996,"5min":0.88100000000000001,"15min":0.77400000000000002},"max":{"1min":2.2200000000000002,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":3.976}]}]},{"osd":7,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62304,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309536,"statfs":{"total":96636764160,"available":96572964864,"internally_reserved":0,"allocated":610304,"data_stored":266329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0860000000000001,"15min":1.1120000000000001},"min":{"1min":0.77800000000000002,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6970000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.119,"5min":1.087,"15min":1.073},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6080000000000001,"5min":2.266,"15min":2.266},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.147,"15min":1.0589999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6140000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.339,"5min":1.2450000000000001,"15min":1.2310000000000001},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.79800000000000004}]},{"osd":2,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2929999999999999,"15min":1.3029999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8080000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.264,"15min":1.2190000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.86099999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.341,"15min":1.2569999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.829,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5169999999999999,"15min":1.46},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.925,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.96299999999999997}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3680000000000001,"15min":1.3300000000000001},"min":{"1min":0.95199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.907,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":0.629},{"interface":"front","average":{"1min":1.395,"5min":1.4359999999999999,"15min":1.379},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.8420000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":0.70699999999999996}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.5389999999999999,"15min":1.5429999999999999},"min":{"1min":0.92800000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.0640000000000001,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.534,"15min":1.4830000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.9870000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.641,"15min":1.5760000000000001},"min":{"1min":0.84899999999999998,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":2.093,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.5920000000000001,"15min":1.518},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.2349999999999999,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":0.76700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:01:44.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:01:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.303+0000 7fab84a39640 1 Processor -- start 2023-11-13T11:01:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.304+0000 7fab84a39640 1 -- start start 2023-11-13T11:01:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.305+0000 7fab84a39640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fab8015cfc0 0x7fab800cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.305+0000 7fab84a39640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab800b30e0 0x7fab800d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab84a39640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab800b3cf0 con 0x7fab800d2ae0 2023-11-13T11:01:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab84a39640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab800b3550 con 0x7fab800b30e0 2023-11-13T11:01:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab84a39640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab800d3480 con 0x7fab8015cfc0 2023-11-13T11:01:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab7ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab800b30e0 0x7fab800d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab7ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab800b30e0 0x7fab800d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41922/0 (socket says 172.21.15.16:41922) 2023-11-13T11:01:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab7ffff640 1 -- 172.21.15.16:0/1559034159 learned_addr learned my addr 172.21.15.16:0/1559034159 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab7f7fe640 1 --2- 172.21.15.16:0/1559034159 >> v2:172.21.15.184:3300/0 conn(0x7fab8015cfc0 0x7fab800cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab7ffff640 1 -- 172.21.15.16:0/1559034159 >> v2:172.21.15.184:3300/0 conn(0x7fab8015cfc0 msgr2=0x7fab800cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab7ffff640 1 --2- 172.21.15.16:0/1559034159 >> v2:172.21.15.184:3300/0 conn(0x7fab8015cfc0 0x7fab800cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.306+0000 7fab7ffff640 1 -- 172.21.15.16:0/1559034159 >> v1:172.21.15.184:6789/0 conn(0x7fab800d2ae0 legacy=0x7fab800b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.307+0000 7fab7ffff640 1 -- 172.21.15.16:0/1559034159 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab800b3730 con 0x7fab800b30e0 2023-11-13T11:01:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.308+0000 7fab7ffff640 1 --2- 172.21.15.16:0/1559034159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 0x7fab800d2f40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fab800d0680 tx=0x7fab7802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca93eee3420496e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.308+0000 7fab7effd640 1 -- 172.21.15.16:0/1559034159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab780317e0 con 0x7fab800b30e0 2023-11-13T11:01:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.308+0000 7fab7effd640 1 -- 172.21.15.16:0/1559034159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab78031940 con 0x7fab800b30e0 2023-11-13T11:01:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.308+0000 7fab7effd640 1 -- 172.21.15.16:0/1559034159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab78031cd0 con 0x7fab800b30e0 2023-11-13T11:01:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.309+0000 7fab84a39640 1 -- 172.21.15.16:0/1559034159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 msgr2=0x7fab800d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.309+0000 7fab84a39640 1 --2- 172.21.15.16:0/1559034159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 0x7fab800d2f40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fab800d0680 tx=0x7fab7802b880 comp rx=0 tx=0).stop 2023-11-13T11:01:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.309+0000 7fab84a39640 1 -- 172.21.15.16:0/1559034159 shutdown_connections 2023-11-13T11:01:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.309+0000 7fab84a39640 1 --2- 172.21.15.16:0/1559034159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 0x7fab800d2f40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.309+0000 7fab84a39640 1 --2- 172.21.15.16:0/1559034159 >> v2:172.21.15.184:3300/0 conn(0x7fab8015cfc0 0x7fab800cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.309+0000 7fab84a39640 1 -- 172.21.15.16:0/1559034159 >> 172.21.15.16:0/1559034159 conn(0x7fab8008c570 msgr2=0x7fab8009be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.309+0000 7fab84a39640 1 -- 172.21.15.16:0/1559034159 shutdown_connections 2023-11-13T11:01:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.309+0000 7fab84a39640 1 -- 172.21.15.16:0/1559034159 wait complete. 2023-11-13T11:01:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.311+0000 7fab84a39640 1 Processor -- start 2023-11-13T11:01:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.311+0000 7fab84a39640 1 -- start start 2023-11-13T11:01:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.312+0000 7fab84a39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 0x7fab8015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.312+0000 7fab84a39640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab800d2ae0 0x7fab801e6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.312+0000 7fab84a39640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8015cfc0 0x7fab801e6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.312+0000 7fab84a39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab800b6750 con 0x7fab800b30e0 2023-11-13T11:01:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.312+0000 7fab84a39640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab800b5ad0 con 0x7fab800d2ae0 2023-11-13T11:01:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.312+0000 7fab84a39640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab800b5db0 con 0x7fab8015cfc0 2023-11-13T11:01:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.312+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 0x7fab8015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.312+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 0x7fab8015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41932/0 (socket says 172.21.15.16:41932) 2023-11-13T11:01:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.312+0000 7fab7f7fe640 1 -- 172.21.15.16:0/1005461379 learned_addr learned my addr 172.21.15.16:0/1005461379 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab76ffd640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab800d2ae0 0x7fab801e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab7f7fe640 1 -- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8015cfc0 msgr2=0x7fab801e6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab7f7fe640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8015cfc0 0x7fab801e6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab7f7fe640 1 -- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab800d2ae0 msgr2=0x7fab801e6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab7f7fe640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab800d2ae0 0x7fab801e6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab7f7fe640 1 -- 172.21.15.16:0/1005461379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab780094d0 con 0x7fab800b30e0 2023-11-13T11:01:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab7f7fe640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 0x7fab8015b380 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fab70007e40 tx=0x7fab7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab7cff9640 1 -- 172.21.15.16:0/1005461379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70014620 con 0x7fab800b30e0 2023-11-13T11:01:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab7cff9640 1 -- 172.21.15.16:0/1005461379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7000ee90 con 0x7fab800b30e0 2023-11-13T11:01:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab7cff9640 1 -- 172.21.15.16:0/1005461379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700149c0 con 0x7fab800b30e0 2023-11-13T11:01:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.313+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab800b6090 con 0x7fab800b30e0 2023-11-13T11:01:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.314+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab800b5730 con 0x7fab800b30e0 2023-11-13T11:01:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.315+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab800cf1c0 con 0x7fab800b30e0 2023-11-13T11:01:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.316+0000 7fab7cff9640 1 -- 172.21.15.16:0/1005461379 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab70014b20 con 0x7fab800b30e0 2023-11-13T11:01:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.317+0000 7fab7cff9640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab6006ae40 0x7fab6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.317+0000 7fab7cff9640 1 -- 172.21.15.16:0/1005461379 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fab7008a120 con 0x7fab800b30e0 2023-11-13T11:01:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.318+0000 7fab76ffd640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab6006ae40 0x7fab6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.319+0000 7fab76ffd640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab6006ae40 0x7fab6006d300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fab8006ae40 tx=0x7fab68009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.322+0000 7fab7cff9640 1 -- 172.21.15.16:0/1005461379 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab70054ff0 con 0x7fab800b30e0 2023-11-13T11:01:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.516+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab800b58c0 con 0x7fab6006ae40 2023-11-13T11:01:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.539+0000 7fab7cff9640 1 -- 172.21.15.16:0/1005461379 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287202 (secure 0 0 0) 0x7fab800b58c0 con 0x7fab6006ae40 2023-11-13T11:01:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:44.549 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:01:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab6006ae40 msgr2=0x7fab6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab6006ae40 0x7fab6006d300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fab8006ae40 tx=0x7fab68009040 comp rx=0 tx=0).stop 2023-11-13T11:01:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 msgr2=0x7fab8015b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 0x7fab8015b380 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fab70007e40 tx=0x7fab7000e4b0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 shutdown_connections 2023-11-13T11:01:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab6006ae40 0x7fab6006d300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab8015cfc0 0x7fab801e6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab800d2ae0 0x7fab801e6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 --2- 172.21.15.16:0/1005461379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab800b30e0 0x7fab8015b380 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 >> 172.21.15.16:0/1005461379 conn(0x7fab8008c570 msgr2=0x7fab8006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 shutdown_connections 2023-11-13T11:01:44.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.546+0000 7fab84a39640 1 -- 172.21.15.16:0/1005461379 wait complete. 2023-11-13T11:01:44.573 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2023-11-13T11:01:44.118569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342948,"kb_used_data":6072,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754631772,"statfs":{"total":773094113280,"available":772742934528,"internally_reserved":0,"allocated":6217728,"data_stored":3385084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":344837681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.170123"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997544+0000","last_change":"2023-11-13T11:01:38.993640+0000","last_active":"2023-11-13T11:01:38.997544+0000","last_peered":"2023-11-13T11:01:38.997544+0000","last_clean":"2023-11-13T11:01:38.997544+0000","last_became_active":"2023-11-13T11:01:38.991479+0000","last_became_peered":"2023-11-13T11:01:38.991479+0000","last_unstale":"2023-11-13T11:01:38.997544+0000","last_undegraded":"2023-11-13T11:01:38.997544+0000","last_fullsized":"2023-11-13T11:01:38.997544+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977509+0000","last_change":"2023-11-13T11:01:38.975999+0000","last_active":"2023-11-13T11:01:38.977509+0000","last_peered":"2023-11-13T11:01:38.977509+0000","last_clean":"2023-11-13T11:01:38.977509+0000","last_became_active":"2023-11-13T11:01:38.975726+0000","last_became_peered":"2023-11-13T11:01:38.975726+0000","last_unstale":"2023-11-13T11:01:38.977509+0000","last_undegraded":"2023-11-13T11:01:38.977509+0000","last_fullsized":"2023-11-13T11:01:38.977509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:33:09.247389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991945+0000","last_change":"2023-11-13T11:01:38.984201+0000","last_active":"2023-11-13T11:01:38.991945+0000","last_peered":"2023-11-13T11:01:38.991945+0000","last_clean":"2023-11-13T11:01:38.991945+0000","last_became_active":"2023-11-13T11:01:38.983716+0000","last_became_peered":"2023-11-13T11:01:38.983716+0000","last_unstale":"2023-11-13T11:01:38.991945+0000","last_undegraded":"2023-11-13T11:01:38.991945+0000","last_fullsized":"2023-11-13T11:01:38.991945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:37:29.570297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980437+0000","last_change":"2023-11-13T11:01:38.975163+0000","last_active":"2023-11-13T11:01:38.980437+0000","last_peered":"2023-11-13T11:01:38.980437+0000","last_clean":"2023-11-13T11:01:38.980437+0000","last_became_active":"2023-11-13T11:01:38.974878+0000","last_became_peered":"2023-11-13T11:01:38.974878+0000","last_unstale":"2023-11-13T11:01:38.980437+0000","last_undegraded":"2023-11-13T11:01:38.980437+0000","last_fullsized":"2023-11-13T11:01:38.980437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.970996+0000","last_change":"2023-11-13T11:01:38.968974+0000","last_active":"2023-11-13T11:01:38.970996+0000","last_peered":"2023-11-13T11:01:38.970996+0000","last_clean":"2023-11-13T11:01:38.970996+0000","last_became_active":"2023-11-13T11:01:38.968747+0000","last_became_peered":"2023-11-13T11:01:38.968747+0000","last_unstale":"2023-11-13T11:01:38.970996+0000","last_undegraded":"2023-11-13T11:01:38.970996+0000","last_fullsized":"2023-11-13T11:01:38.970996+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:14:08.138073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011194+0000","last_change":"2023-11-13T11:01:38.993231+0000","last_active":"2023-11-13T11:01:41.011194+0000","last_peered":"2023-11-13T11:01:41.011194+0000","last_clean":"2023-11-13T11:01:41.011194+0000","last_became_active":"2023-11-13T11:01:38.992972+0000","last_became_peered":"2023-11-13T11:01:38.992972+0000","last_unstale":"2023-11-13T11:01:41.011194+0000","last_undegraded":"2023-11-13T11:01:41.011194+0000","last_fullsized":"2023-11-13T11:01:41.011194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:45:42.458182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.006474+0000","last_change":"2023-11-13T11:01:39.003582+0000","last_active":"2023-11-13T11:01:39.006474+0000","last_peered":"2023-11-13T11:01:39.006474+0000","last_clean":"2023-11-13T11:01:39.006474+0000","last_became_active":"2023-11-13T11:01:39.003324+0000","last_became_peered":"2023-11-13T11:01:39.003324+0000","last_unstale":"2023-11-13T11:01:39.006474+0000","last_undegraded":"2023-11-13T11:01:39.006474+0000","last_fullsized":"2023-11-13T11:01:39.006474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.010848+0000","last_change":"2023-11-13T11:01:39.004402+0000","last_active":"2023-11-13T11:01:41.010848+0000","last_peered":"2023-11-13T11:01:41.010848+0000","last_clean":"2023-11-13T11:01:41.010848+0000","last_became_active":"2023-11-13T11:01:39.001982+0000","last_became_peered":"2023-11-13T11:01:39.001982+0000","last_unstale":"2023-11-13T11:01:41.010848+0000","last_undegraded":"2023-11-13T11:01:41.010848+0000","last_fullsized":"2023-11-13T11:01:41.010848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:32:40.380861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.003277+0000","last_change":"2023-11-13T11:01:38.997117+0000","last_active":"2023-11-13T11:01:39.003277+0000","last_peered":"2023-11-13T11:01:39.003277+0000","last_clean":"2023-11-13T11:01:39.003277+0000","last_became_active":"2023-11-13T11:01:38.996801+0000","last_became_peered":"2023-11-13T11:01:38.996801+0000","last_unstale":"2023-11-13T11:01:39.003277+0000","last_undegraded":"2023-11-13T11:01:39.003277+0000","last_fullsized":"2023-11-13T11:01:39.003277+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:03:35.448965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990548+0000","last_change":"2023-11-13T11:01:38.987650+0000","last_active":"2023-11-13T11:01:38.990548+0000","last_peered":"2023-11-13T11:01:38.990548+0000","last_clean":"2023-11-13T11:01:38.990548+0000","last_became_active":"2023-11-13T11:01:38.987270+0000","last_became_peered":"2023-11-13T11:01:38.987270+0000","last_unstale":"2023-11-13T11:01:38.990548+0000","last_undegraded":"2023-11-13T11:01:38.990548+0000","last_fullsized":"2023-11-13T11:01:38.990548+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:25.371717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992670+0000","last_change":"2023-11-13T11:01:38.984512+0000","last_active":"2023-11-13T11:01:38.992670+0000","last_peered":"2023-11-13T11:01:38.992670+0000","last_clean":"2023-11-13T11:01:38.992670+0000","last_became_active":"2023-11-13T11:01:38.984243+0000","last_became_peered":"2023-11-13T11:01:38.984243+0000","last_unstale":"2023-11-13T11:01:38.992670+0000","last_undegraded":"2023-11-13T11:01:38.992670+0000","last_fullsized":"2023-11-13T11:01:38.992670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011670+0000","last_change":"2023-11-13T11:01:38.992022+0000","last_active":"2023-11-13T11:01:41.011670+0000","last_peered":"2023-11-13T11:01:41.011670+0000","last_clean":"2023-11-13T11:01:41.011670+0000","last_became_active":"2023-11-13T11:01:38.991341+0000","last_became_peered":"2023-11-13T11:01:38.991341+0000","last_unstale":"2023-11-13T11:01:41.011670+0000","last_undegraded":"2023-11-13T11:01:41.011670+0000","last_fullsized":"2023-11-13T11:01:41.011670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:53:20.682896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011270+0000","last_change":"2023-11-13T11:01:38.998336+0000","last_active":"2023-11-13T11:01:41.011270+0000","last_peered":"2023-11-13T11:01:41.011270+0000","last_clean":"2023-11-13T11:01:41.011270+0000","last_became_active":"2023-11-13T11:01:38.997894+0000","last_became_peered":"2023-11-13T11:01:38.997894+0000","last_unstale":"2023-11-13T11:01:41.011270+0000","last_undegraded":"2023-11-13T11:01:41.011270+0000","last_fullsized":"2023-11-13T11:01:41.011270+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:42:46.869446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.994357+0000","last_change":"2023-11-13T11:01:38.992189+0000","last_active":"2023-11-13T11:01:38.994357+0000","last_peered":"2023-11-13T11:01:38.994357+0000","last_clean":"2023-11-13T11:01:38.994357+0000","last_became_active":"2023-11-13T11:01:38.991290+0000","last_became_peered":"2023-11-13T11:01:38.991290+0000","last_unstale":"2023-11-13T11:01:38.994357+0000","last_undegraded":"2023-11-13T11:01:38.994357+0000","last_fullsized":"2023-11-13T11:01:38.994357+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:25.371717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.969075+0000","last_change":"2023-11-13T11:01:38.966119+0000","last_active":"2023-11-13T11:01:38.969075+0000","last_peered":"2023-11-13T11:01:38.969075+0000","last_clean":"2023-11-13T11:01:38.969075+0000","last_became_active":"2023-11-13T11:01:38.965812+0000","last_became_peered":"2023-11-13T11:01:38.965812+0000","last_unstale":"2023-11-13T11:01:38.969075+0000","last_undegraded":"2023-11-13T11:01:38.969075+0000","last_fullsized":"2023-11-13T11:01:38.969075+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.010855+0000","last_change":"2023-11-13T11:01:39.002819+0000","last_active":"2023-11-13T11:01:41.010855+0000","last_peered":"2023-11-13T11:01:41.010855+0000","last_clean":"2023-11-13T11:01:41.010855+0000","last_became_active":"2023-11-13T11:01:39.002309+0000","last_became_peered":"2023-11-13T11:01:39.002309+0000","last_unstale":"2023-11-13T11:01:41.010855+0000","last_undegraded":"2023-11-13T11:01:41.010855+0000","last_fullsized":"2023-11-13T11:01:41.010855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:20:55.201885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997524+0000","last_change":"2023-11-13T11:01:38.993324+0000","last_active":"2023-11-13T11:01:38.997524+0000","last_peered":"2023-11-13T11:01:38.997524+0000","last_clean":"2023-11-13T11:01:38.997524+0000","last_became_active":"2023-11-13T11:01:38.991072+0000","last_became_peered":"2023-11-13T11:01:38.991072+0000","last_unstale":"2023-11-13T11:01:38.997524+0000","last_undegraded":"2023-11-13T11:01:38.997524+0000","last_fullsized":"2023-11-13T11:01:38.997524+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980403+0000","last_change":"2023-11-13T11:01:38.975150+0000","last_active":"2023-11-13T11:01:38.980403+0000","last_peered":"2023-11-13T11:01:38.980403+0000","last_clean":"2023-11-13T11:01:38.980403+0000","last_became_active":"2023-11-13T11:01:38.974837+0000","last_became_peered":"2023-11-13T11:01:38.974837+0000","last_unstale":"2023-11-13T11:01:38.980403+0000","last_undegraded":"2023-11-13T11:01:38.980403+0000","last_fullsized":"2023-11-13T11:01:38.980403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:19:00.701068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991920+0000","last_change":"2023-11-13T11:01:38.984024+0000","last_active":"2023-11-13T11:01:38.991920+0000","last_peered":"2023-11-13T11:01:38.991920+0000","last_clean":"2023-11-13T11:01:38.991920+0000","last_became_active":"2023-11-13T11:01:38.983338+0000","last_became_peered":"2023-11-13T11:01:38.983338+0000","last_unstale":"2023-11-13T11:01:38.991920+0000","last_undegraded":"2023-11-13T11:01:38.991920+0000","last_fullsized":"2023-11-13T11:01:38.991920+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:06:44.792510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973436+0000","last_change":"2023-11-13T11:01:38.969241+0000","last_active":"2023-11-13T11:01:38.973436+0000","last_peered":"2023-11-13T11:01:38.973436+0000","last_clean":"2023-11-13T11:01:38.973436+0000","last_became_active":"2023-11-13T11:01:38.968942+0000","last_became_peered":"2023-11-13T11:01:38.968942+0000","last_unstale":"2023-11-13T11:01:38.973436+0000","last_undegraded":"2023-11-13T11:01:38.973436+0000","last_fullsized":"2023-11-13T11:01:38.973436+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:19:00.701068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.972503+0000","last_change":"2023-11-13T11:01:38.966987+0000","last_active":"2023-11-13T11:01:38.972503+0000","last_peered":"2023-11-13T11:01:38.972503+0000","last_clean":"2023-11-13T11:01:38.972503+0000","last_became_active":"2023-11-13T11:01:38.966723+0000","last_became_peered":"2023-11-13T11:01:38.966723+0000","last_unstale":"2023-11-13T11:01:38.972503+0000","last_undegraded":"2023-11-13T11:01:38.972503+0000","last_fullsized":"2023-11-13T11:01:38.972503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:33:09.247389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993826+0000","last_change":"2023-11-13T11:01:38.989021+0000","last_active":"2023-11-13T11:01:38.993826+0000","last_peered":"2023-11-13T11:01:38.993826+0000","last_clean":"2023-11-13T11:01:38.993826+0000","last_became_active":"2023-11-13T11:01:38.988715+0000","last_became_peered":"2023-11-13T11:01:38.988715+0000","last_unstale":"2023-11-13T11:01:38.993826+0000","last_undegraded":"2023-11-13T11:01:38.993826+0000","last_fullsized":"2023-11-13T11:01:38.993826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:12:15.067759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993091+0000","last_change":"2023-11-13T11:01:38.986587+0000","last_active":"2023-11-13T11:01:38.993091+0000","last_peered":"2023-11-13T11:01:38.993091+0000","last_clean":"2023-11-13T11:01:38.993091+0000","last_became_active":"2023-11-13T11:01:38.986301+0000","last_became_peered":"2023-11-13T11:01:38.986301+0000","last_unstale":"2023-11-13T11:01:38.993091+0000","last_undegraded":"2023-11-13T11:01:38.993091+0000","last_fullsized":"2023-11-13T11:01:38.993091+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:39.111281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.995951+0000","last_change":"2023-11-13T11:01:38.992108+0000","last_active":"2023-11-13T11:01:38.995951+0000","last_peered":"2023-11-13T11:01:38.995951+0000","last_clean":"2023-11-13T11:01:38.995951+0000","last_became_active":"2023-11-13T11:01:38.988156+0000","last_became_peered":"2023-11-13T11:01:38.988156+0000","last_unstale":"2023-11-13T11:01:38.995951+0000","last_undegraded":"2023-11-13T11:01:38.995951+0000","last_fullsized":"2023-11-13T11:01:38.995951+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:02.526493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000637+0000","last_change":"2023-11-13T11:01:38.996834+0000","last_active":"2023-11-13T11:01:39.000637+0000","last_peered":"2023-11-13T11:01:39.000637+0000","last_clean":"2023-11-13T11:01:39.000637+0000","last_became_active":"2023-11-13T11:01:38.996522+0000","last_became_peered":"2023-11-13T11:01:38.996522+0000","last_unstale":"2023-11-13T11:01:39.000637+0000","last_undegraded":"2023-11-13T11:01:39.000637+0000","last_fullsized":"2023-11-13T11:01:39.000637+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:01.750338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005129+0000","last_change":"2023-11-13T11:01:39.003837+0000","last_active":"2023-11-13T11:01:39.005129+0000","last_peered":"2023-11-13T11:01:39.005129+0000","last_clean":"2023-11-13T11:01:39.005129+0000","last_became_active":"2023-11-13T11:01:39.000372+0000","last_became_peered":"2023-11-13T11:01:39.000372+0000","last_unstale":"2023-11-13T11:01:39.005129+0000","last_undegraded":"2023-11-13T11:01:39.005129+0000","last_fullsized":"2023-11-13T11:01:39.005129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:29:27.396844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011710+0000","last_change":"2023-11-13T11:01:38.997301+0000","last_active":"2023-11-13T11:01:41.011710+0000","last_peered":"2023-11-13T11:01:41.011710+0000","last_clean":"2023-11-13T11:01:41.011710+0000","last_became_active":"2023-11-13T11:01:38.997039+0000","last_became_peered":"2023-11-13T11:01:38.997039+0000","last_unstale":"2023-11-13T11:01:41.011710+0000","last_undegraded":"2023-11-13T11:01:41.011710+0000","last_fullsized":"2023-11-13T11:01:41.011710+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:07:23.632348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.001088+0000","last_change":"2023-11-13T11:01:39.000662+0000","last_active":"2023-11-13T11:01:39.001088+0000","last_peered":"2023-11-13T11:01:39.001088+0000","last_clean":"2023-11-13T11:01:39.001088+0000","last_became_active":"2023-11-13T11:01:39.000191+0000","last_became_peered":"2023-11-13T11:01:39.000191+0000","last_unstale":"2023-11-13T11:01:39.001088+0000","last_undegraded":"2023-11-13T11:01:39.001088+0000","last_fullsized":"2023-11-13T11:01:39.001088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980942+0000","last_change":"2023-11-13T11:01:38.969203+0000","last_active":"2023-11-13T11:01:38.980942+0000","last_peered":"2023-11-13T11:01:38.980942+0000","last_clean":"2023-11-13T11:01:38.980942+0000","last_became_active":"2023-11-13T11:01:38.968896+0000","last_became_peered":"2023-11-13T11:01:38.968896+0000","last_unstale":"2023-11-13T11:01:38.980942+0000","last_undegraded":"2023-11-13T11:01:38.980942+0000","last_fullsized":"2023-11-13T11:01:38.980942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991585+0000","last_change":"2023-11-13T11:01:38.989974+0000","last_active":"2023-11-13T11:01:38.991585+0000","last_peered":"2023-11-13T11:01:38.991585+0000","last_clean":"2023-11-13T11:01:38.991585+0000","last_became_active":"2023-11-13T11:01:38.989665+0000","last_became_peered":"2023-11-13T11:01:38.989665+0000","last_unstale":"2023-11-13T11:01:38.991585+0000","last_undegraded":"2023-11-13T11:01:38.991585+0000","last_fullsized":"2023-11-13T11:01:38.991585+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:18:26.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000755+0000","last_change":"2023-11-13T11:01:38.995148+0000","last_active":"2023-11-13T11:01:39.000755+0000","last_peered":"2023-11-13T11:01:39.000755+0000","last_clean":"2023-11-13T11:01:39.000755+0000","last_became_active":"2023-11-13T11:01:38.990007+0000","last_became_peered":"2023-11-13T11:01:38.990007+0000","last_unstale":"2023-11-13T11:01:39.000755+0000","last_undegraded":"2023-11-13T11:01:39.000755+0000","last_fullsized":"2023-11-13T11:01:39.000755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:18:26.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.988066+0000","last_change":"2023-11-13T11:01:38.984675+0000","last_active":"2023-11-13T11:01:38.988066+0000","last_peered":"2023-11-13T11:01:38.988066+0000","last_clean":"2023-11-13T11:01:38.988066+0000","last_became_active":"2023-11-13T11:01:38.984073+0000","last_became_peered":"2023-11-13T11:01:38.984073+0000","last_unstale":"2023-11-13T11:01:38.988066+0000","last_undegraded":"2023-11-13T11:01:38.988066+0000","last_fullsized":"2023-11-13T11:01:38.988066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:24:13.781188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979482+0000","last_change":"2023-11-13T11:01:38.973375+0000","last_active":"2023-11-13T11:01:38.979482+0000","last_peered":"2023-11-13T11:01:38.979482+0000","last_clean":"2023-11-13T11:01:38.979482+0000","last_became_active":"2023-11-13T11:01:38.970971+0000","last_became_peered":"2023-11-13T11:01:38.970971+0000","last_unstale":"2023-11-13T11:01:38.979482+0000","last_undegraded":"2023-11-13T11:01:38.979482+0000","last_fullsized":"2023-11-13T11:01:38.979482+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:03:35.448965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.999535+0000","last_change":"2023-11-13T11:01:37.976416+0000","last_active":"2023-11-13T11:01:38.999535+0000","last_peered":"2023-11-13T11:01:38.999535+0000","last_clean":"2023-11-13T11:01:38.999535+0000","last_became_active":"2023-11-13T11:01:37.976080+0000","last_became_peered":"2023-11-13T11:01:37.976080+0000","last_unstale":"2023-11-13T11:01:38.999535+0000","last_undegraded":"2023-11-13T11:01:38.999535+0000","last_fullsized":"2023-11-13T11:01:38.999535+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979796+0000","last_change":"2023-11-13T11:01:38.977746+0000","last_active":"2023-11-13T11:01:38.979796+0000","last_peered":"2023-11-13T11:01:38.979796+0000","last_clean":"2023-11-13T11:01:38.979796+0000","last_became_active":"2023-11-13T11:01:38.977421+0000","last_became_peered":"2023-11-13T11:01:38.977421+0000","last_unstale":"2023-11-13T11:01:38.979796+0000","last_undegraded":"2023-11-13T11:01:38.979796+0000","last_fullsized":"2023-11-13T11:01:38.979796+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:29:27.396844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005578+0000","last_change":"2023-11-13T11:01:37.984920+0000","last_active":"2023-11-13T11:01:39.005578+0000","last_peered":"2023-11-13T11:01:39.005578+0000","last_clean":"2023-11-13T11:01:39.005578+0000","last_became_active":"2023-11-13T11:01:37.984119+0000","last_became_peered":"2023-11-13T11:01:37.984119+0000","last_unstale":"2023-11-13T11:01:39.005578+0000","last_undegraded":"2023-11-13T11:01:39.005578+0000","last_fullsized":"2023-11-13T11:01:39.005578+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:09:29.998731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977904+0000","last_change":"2023-11-13T11:01:38.971332+0000","last_active":"2023-11-13T11:01:38.977904+0000","last_peered":"2023-11-13T11:01:38.977904+0000","last_clean":"2023-11-13T11:01:38.977904+0000","last_became_active":"2023-11-13T11:01:38.969969+0000","last_became_peered":"2023-11-13T11:01:38.969969+0000","last_unstale":"2023-11-13T11:01:38.977904+0000","last_undegraded":"2023-11-13T11:01:38.977904+0000","last_fullsized":"2023-11-13T11:01:38.977904+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:09:56.697516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000885+0000","last_change":"2023-11-13T11:01:37.970093+0000","last_active":"2023-11-13T11:01:39.000885+0000","last_peered":"2023-11-13T11:01:39.000885+0000","last_clean":"2023-11-13T11:01:39.000885+0000","last_became_active":"2023-11-13T11:01:37.969770+0000","last_became_peered":"2023-11-13T11:01:37.969770+0000","last_unstale":"2023-11-13T11:01:39.000885+0000","last_undegraded":"2023-11-13T11:01:39.000885+0000","last_fullsized":"2023-11-13T11:01:39.000885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:33.254128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990202+0000","last_change":"2023-11-13T11:01:38.980529+0000","last_active":"2023-11-13T11:01:38.990202+0000","last_peered":"2023-11-13T11:01:38.990202+0000","last_clean":"2023-11-13T11:01:38.990202+0000","last_became_active":"2023-11-13T11:01:38.980227+0000","last_became_peered":"2023-11-13T11:01:38.980227+0000","last_unstale":"2023-11-13T11:01:38.990202+0000","last_undegraded":"2023-11-13T11:01:38.990202+0000","last_fullsized":"2023-11-13T11:01:38.990202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:12.303326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.994946+0000","last_change":"2023-11-13T11:01:37.982793+0000","last_active":"2023-11-13T11:01:38.994946+0000","last_peered":"2023-11-13T11:01:38.994946+0000","last_clean":"2023-11-13T11:01:38.994946+0000","last_became_active":"2023-11-13T11:01:37.982111+0000","last_became_peered":"2023-11-13T11:01:37.982111+0000","last_unstale":"2023-11-13T11:01:38.994946+0000","last_undegraded":"2023-11-13T11:01:38.994946+0000","last_fullsized":"2023-11-13T11:01:38.994946+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980711+0000","last_change":"2023-11-13T11:01:38.976171+0000","last_active":"2023-11-13T11:01:38.980711+0000","last_peered":"2023-11-13T11:01:38.980711+0000","last_clean":"2023-11-13T11:01:38.980711+0000","last_became_active":"2023-11-13T11:01:38.975895+0000","last_became_peered":"2023-11-13T11:01:38.975895+0000","last_unstale":"2023-11-13T11:01:38.980711+0000","last_undegraded":"2023-11-13T11:01:38.980711+0000","last_fullsized":"2023-11-13T11:01:38.980711+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.002584+0000","last_change":"2023-11-13T11:01:37.981492+0000","last_active":"2023-11-13T11:01:39.002584+0000","last_peered":"2023-11-13T11:01:39.002584+0000","last_clean":"2023-11-13T11:01:39.002584+0000","last_became_active":"2023-11-13T11:01:37.981149+0000","last_became_peered":"2023-11-13T11:01:37.981149+0000","last_unstale":"2023-11-13T11:01:39.002584+0000","last_undegraded":"2023-11-13T11:01:39.002584+0000","last_fullsized":"2023-11-13T11:01:39.002584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979333+0000","last_change":"2023-11-13T11:01:38.972149+0000","last_active":"2023-11-13T11:01:38.979333+0000","last_peered":"2023-11-13T11:01:38.979333+0000","last_clean":"2023-11-13T11:01:38.979333+0000","last_became_active":"2023-11-13T11:01:38.971860+0000","last_became_peered":"2023-11-13T11:01:38.971860+0000","last_unstale":"2023-11-13T11:01:38.979333+0000","last_undegraded":"2023-11-13T11:01:38.979333+0000","last_fullsized":"2023-11-13T11:01:38.979333+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:44:30.617363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986684+0000","last_change":"2023-11-13T11:01:37.968032+0000","last_active":"2023-11-13T11:01:38.986684+0000","last_peered":"2023-11-13T11:01:38.986684+0000","last_clean":"2023-11-13T11:01:38.986684+0000","last_became_active":"2023-11-13T11:01:37.967709+0000","last_became_peered":"2023-11-13T11:01:37.967709+0000","last_unstale":"2023-11-13T11:01:38.986684+0000","last_undegraded":"2023-11-13T11:01:38.986684+0000","last_fullsized":"2023-11-13T11:01:38.986684+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:44:32.588239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.003718+0000","last_change":"2023-11-13T11:01:39.001367+0000","last_active":"2023-11-13T11:01:39.003718+0000","last_peered":"2023-11-13T11:01:39.003718+0000","last_clean":"2023-11-13T11:01:39.003718+0000","last_became_active":"2023-11-13T11:01:39.001092+0000","last_became_peered":"2023-11-13T11:01:39.001092+0000","last_unstale":"2023-11-13T11:01:39.003718+0000","last_undegraded":"2023-11-13T11:01:39.003718+0000","last_fullsized":"2023-11-13T11:01:39.003718+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.962390+0000","last_change":"2023-11-13T11:01:37.974253+0000","last_active":"2023-11-13T11:01:38.962390+0000","last_peered":"2023-11-13T11:01:38.962390+0000","last_clean":"2023-11-13T11:01:38.962390+0000","last_became_active":"2023-11-13T11:01:37.973388+0000","last_became_peered":"2023-11-13T11:01:37.973388+0000","last_unstale":"2023-11-13T11:01:38.962390+0000","last_undegraded":"2023-11-13T11:01:38.962390+0000","last_fullsized":"2023-11-13T11:01:38.962390+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997619+0000","last_change":"2023-11-13T11:01:38.996067+0000","last_active":"2023-11-13T11:01:38.997619+0000","last_peered":"2023-11-13T11:01:38.997619+0000","last_clean":"2023-11-13T11:01:38.997619+0000","last_became_active":"2023-11-13T11:01:38.995583+0000","last_became_peered":"2023-11-13T11:01:38.995583+0000","last_unstale":"2023-11-13T11:01:38.997619+0000","last_undegraded":"2023-11-13T11:01:38.997619+0000","last_fullsized":"2023-11-13T11:01:38.997619+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973630+0000","last_change":"2023-11-13T11:01:37.980576+0000","last_active":"2023-11-13T11:01:38.973630+0000","last_peered":"2023-11-13T11:01:38.973630+0000","last_clean":"2023-11-13T11:01:38.973630+0000","last_became_active":"2023-11-13T11:01:37.980253+0000","last_became_peered":"2023-11-13T11:01:37.980253+0000","last_unstale":"2023-11-13T11:01:38.973630+0000","last_undegraded":"2023-11-13T11:01:38.973630+0000","last_fullsized":"2023-11-13T11:01:38.973630+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.982849+0000","last_change":"2023-11-13T11:01:38.978906+0000","last_active":"2023-11-13T11:01:38.982849+0000","last_peered":"2023-11-13T11:01:38.982849+0000","last_clean":"2023-11-13T11:01:38.982849+0000","last_became_active":"2023-11-13T11:01:38.978590+0000","last_became_peered":"2023-11-13T11:01:38.978590+0000","last_unstale":"2023-11-13T11:01:38.982849+0000","last_undegraded":"2023-11-13T11:01:38.982849+0000","last_fullsized":"2023-11-13T11:01:38.982849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.957725+0000","last_change":"2023-11-13T11:01:37.967093+0000","last_active":"2023-11-13T11:01:38.957725+0000","last_peered":"2023-11-13T11:01:38.957725+0000","last_clean":"2023-11-13T11:01:38.957725+0000","last_became_active":"2023-11-13T11:01:37.966819+0000","last_became_peered":"2023-11-13T11:01:37.966819+0000","last_unstale":"2023-11-13T11:01:38.957725+0000","last_undegraded":"2023-11-13T11:01:38.957725+0000","last_fullsized":"2023-11-13T11:01:38.957725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980445+0000","last_change":"2023-11-13T11:01:38.978007+0000","last_active":"2023-11-13T11:01:38.980445+0000","last_peered":"2023-11-13T11:01:38.980445+0000","last_clean":"2023-11-13T11:01:38.980445+0000","last_became_active":"2023-11-13T11:01:38.977730+0000","last_became_peered":"2023-11-13T11:01:38.977730+0000","last_unstale":"2023-11-13T11:01:38.980445+0000","last_undegraded":"2023-11-13T11:01:38.980445+0000","last_fullsized":"2023-11-13T11:01:38.980445+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.957148+0000","last_change":"2023-11-13T11:01:37.974872+0000","last_active":"2023-11-13T11:01:38.957148+0000","last_peered":"2023-11-13T11:01:38.957148+0000","last_clean":"2023-11-13T11:01:38.957148+0000","last_became_active":"2023-11-13T11:01:37.973788+0000","last_became_peered":"2023-11-13T11:01:37.973788+0000","last_unstale":"2023-11-13T11:01:38.957148+0000","last_undegraded":"2023-11-13T11:01:38.957148+0000","last_fullsized":"2023-11-13T11:01:38.957148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:11.290032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992368+0000","last_change":"2023-11-13T11:01:38.982930+0000","last_active":"2023-11-13T11:01:38.992368+0000","last_peered":"2023-11-13T11:01:38.992368+0000","last_clean":"2023-11-13T11:01:38.992368+0000","last_became_active":"2023-11-13T11:01:38.982617+0000","last_became_peered":"2023-11-13T11:01:38.982617+0000","last_unstale":"2023-11-13T11:01:38.992368+0000","last_undegraded":"2023-11-13T11:01:38.992368+0000","last_fullsized":"2023-11-13T11:01:38.992368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:35.756266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997924+0000","last_change":"2023-11-13T11:01:37.976846+0000","last_active":"2023-11-13T11:01:38.997924+0000","last_peered":"2023-11-13T11:01:38.997924+0000","last_clean":"2023-11-13T11:01:38.997924+0000","last_became_active":"2023-11-13T11:01:37.976378+0000","last_became_peered":"2023-11-13T11:01:37.976378+0000","last_unstale":"2023-11-13T11:01:38.997924+0000","last_undegraded":"2023-11-13T11:01:38.997924+0000","last_fullsized":"2023-11-13T11:01:38.997924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.985125+0000","last_change":"2023-11-13T11:01:38.983595+0000","last_active":"2023-11-13T11:01:38.985125+0000","last_peered":"2023-11-13T11:01:38.985125+0000","last_clean":"2023-11-13T11:01:38.985125+0000","last_became_active":"2023-11-13T11:01:38.978958+0000","last_became_peered":"2023-11-13T11:01:38.978958+0000","last_unstale":"2023-11-13T11:01:38.985125+0000","last_undegraded":"2023-11-13T11:01:38.985125+0000","last_fullsized":"2023-11-13T11:01:38.985125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:52:25.507954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.008452+0000","last_change":"2023-11-13T11:01:37.983799+0000","last_active":"2023-11-13T11:01:39.008452+0000","last_peered":"2023-11-13T11:01:39.008452+0000","last_clean":"2023-11-13T11:01:39.008452+0000","last_became_active":"2023-11-13T11:01:37.983048+0000","last_became_peered":"2023-11-13T11:01:37.983048+0000","last_unstale":"2023-11-13T11:01:39.008452+0000","last_undegraded":"2023-11-13T11:01:39.008452+0000","last_fullsized":"2023-11-13T11:01:39.008452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:43.793217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979764+0000","last_change":"2023-11-13T11:01:38.972065+0000","last_active":"2023-11-13T11:01:38.979764+0000","last_peered":"2023-11-13T11:01:38.979764+0000","last_clean":"2023-11-13T11:01:38.979764+0000","last_became_active":"2023-11-13T11:01:38.971764+0000","last_became_peered":"2023-11-13T11:01:38.971764+0000","last_unstale":"2023-11-13T11:01:38.979764+0000","last_undegraded":"2023-11-13T11:01:38.979764+0000","last_fullsized":"2023-11-13T11:01:38.979764+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:27:51.217444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000285+0000","last_change":"2023-11-13T11:01:37.976954+0000","last_active":"2023-11-13T11:01:39.000285+0000","last_peered":"2023-11-13T11:01:39.000285+0000","last_clean":"2023-11-13T11:01:39.000285+0000","last_became_active":"2023-11-13T11:01:37.976659+0000","last_became_peered":"2023-11-13T11:01:37.976659+0000","last_unstale":"2023-11-13T11:01:39.000285+0000","last_undegraded":"2023-11-13T11:01:39.000285+0000","last_fullsized":"2023-11-13T11:01:39.000285+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:02:54.853648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000253+0000","last_change":"2023-11-13T11:01:38.995557+0000","last_active":"2023-11-13T11:01:39.000253+0000","last_peered":"2023-11-13T11:01:39.000253+0000","last_clean":"2023-11-13T11:01:39.000253+0000","last_became_active":"2023-11-13T11:01:38.995224+0000","last_became_peered":"2023-11-13T11:01:38.995224+0000","last_unstale":"2023-11-13T11:01:39.000253+0000","last_undegraded":"2023-11-13T11:01:39.000253+0000","last_fullsized":"2023-11-13T11:01:39.000253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.959098+0000","last_change":"2023-11-13T11:01:37.970890+0000","last_active":"2023-11-13T11:01:38.959098+0000","last_peered":"2023-11-13T11:01:38.959098+0000","last_clean":"2023-11-13T11:01:38.959098+0000","last_became_active":"2023-11-13T11:01:37.970585+0000","last_became_peered":"2023-11-13T11:01:37.970585+0000","last_unstale":"2023-11-13T11:01:38.959098+0000","last_undegraded":"2023-11-13T11:01:38.959098+0000","last_fullsized":"2023-11-13T11:01:38.959098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:24.358423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011948+0000","last_change":"2023-11-13T11:01:38.994877+0000","last_active":"2023-11-13T11:01:41.011948+0000","last_peered":"2023-11-13T11:01:41.011948+0000","last_clean":"2023-11-13T11:01:41.011948+0000","last_became_active":"2023-11-13T11:01:38.994450+0000","last_became_peered":"2023-11-13T11:01:38.994450+0000","last_unstale":"2023-11-13T11:01:41.011948+0000","last_undegraded":"2023-11-13T11:01:41.011948+0000","last_fullsized":"2023-11-13T11:01:41.011948+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:41:55.799097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.961699+0000","last_change":"2023-11-13T11:01:37.966418+0000","last_active":"2023-11-13T11:01:38.961699+0000","last_peered":"2023-11-13T11:01:38.961699+0000","last_clean":"2023-11-13T11:01:38.961699+0000","last_became_active":"2023-11-13T11:01:37.966105+0000","last_became_peered":"2023-11-13T11:01:37.966105+0000","last_unstale":"2023-11-13T11:01:38.961699+0000","last_undegraded":"2023-11-13T11:01:38.961699+0000","last_fullsized":"2023-11-13T11:01:38.961699+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:44:09.646957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991081+0000","last_change":"2023-11-13T11:01:38.985149+0000","last_active":"2023-11-13T11:01:38.991081+0000","last_peered":"2023-11-13T11:01:38.991081+0000","last_clean":"2023-11-13T11:01:38.991081+0000","last_became_active":"2023-11-13T11:01:38.984828+0000","last_became_peered":"2023-11-13T11:01:38.984828+0000","last_unstale":"2023-11-13T11:01:38.991081+0000","last_undegraded":"2023-11-13T11:01:38.991081+0000","last_fullsized":"2023-11-13T11:01:38.991081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:27:51.217444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.007772+0000","last_change":"2023-11-13T11:01:37.983844+0000","last_active":"2023-11-13T11:01:39.007772+0000","last_peered":"2023-11-13T11:01:39.007772+0000","last_clean":"2023-11-13T11:01:39.007772+0000","last_became_active":"2023-11-13T11:01:37.982971+0000","last_became_peered":"2023-11-13T11:01:37.982971+0000","last_unstale":"2023-11-13T11:01:39.007772+0000","last_undegraded":"2023-11-13T11:01:39.007772+0000","last_fullsized":"2023-11-13T11:01:39.007772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:56:05.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990249+0000","last_change":"2023-11-13T11:01:38.984132+0000","last_active":"2023-11-13T11:01:38.990249+0000","last_peered":"2023-11-13T11:01:38.990249+0000","last_clean":"2023-11-13T11:01:38.990249+0000","last_became_active":"2023-11-13T11:01:38.981507+0000","last_became_peered":"2023-11-13T11:01:38.981507+0000","last_unstale":"2023-11-13T11:01:38.990249+0000","last_undegraded":"2023-11-13T11:01:38.990249+0000","last_fullsized":"2023-11-13T11:01:38.990249+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005279+0000","last_change":"2023-11-13T11:01:37.986267+0000","last_active":"2023-11-13T11:01:39.005279+0000","last_peered":"2023-11-13T11:01:39.005279+0000","last_clean":"2023-11-13T11:01:39.005279+0000","last_became_active":"2023-11-13T11:01:37.985787+0000","last_became_peered":"2023-11-13T11:01:37.985787+0000","last_unstale":"2023-11-13T11:01:39.005279+0000","last_undegraded":"2023-11-13T11:01:39.005279+0000","last_fullsized":"2023-11-13T11:01:39.005279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:10:32.081040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992067+0000","last_change":"2023-11-13T11:01:38.984798+0000","last_active":"2023-11-13T11:01:38.992067+0000","last_peered":"2023-11-13T11:01:38.992067+0000","last_clean":"2023-11-13T11:01:38.992067+0000","last_became_active":"2023-11-13T11:01:38.984518+0000","last_became_peered":"2023-11-13T11:01:38.984518+0000","last_unstale":"2023-11-13T11:01:38.992067+0000","last_undegraded":"2023-11-13T11:01:38.992067+0000","last_fullsized":"2023-11-13T11:01:38.992067+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.004284+0000","last_change":"2023-11-13T11:01:37.977771+0000","last_active":"2023-11-13T11:01:39.004284+0000","last_peered":"2023-11-13T11:01:39.004284+0000","last_clean":"2023-11-13T11:01:39.004284+0000","last_became_active":"2023-11-13T11:01:37.977506+0000","last_became_peered":"2023-11-13T11:01:37.977506+0000","last_unstale":"2023-11-13T11:01:39.004284+0000","last_undegraded":"2023-11-13T11:01:39.004284+0000","last_fullsized":"2023-11-13T11:01:39.004284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:43:08.322589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.995489+0000","last_change":"2023-11-13T11:01:38.994873+0000","last_active":"2023-11-13T11:01:38.995489+0000","last_peered":"2023-11-13T11:01:38.995489+0000","last_clean":"2023-11-13T11:01:38.995489+0000","last_became_active":"2023-11-13T11:01:38.994490+0000","last_became_peered":"2023-11-13T11:01:38.994490+0000","last_unstale":"2023-11-13T11:01:38.995489+0000","last_undegraded":"2023-11-13T11:01:38.995489+0000","last_fullsized":"2023-11-13T11:01:38.995489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:46:41.523607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997386+0000","last_change":"2023-11-13T11:01:37.982308+0000","last_active":"2023-11-13T11:01:38.997386+0000","last_peered":"2023-11-13T11:01:38.997386+0000","last_clean":"2023-11-13T11:01:38.997386+0000","last_became_active":"2023-11-13T11:01:37.981963+0000","last_became_peered":"2023-11-13T11:01:37.981963+0000","last_unstale":"2023-11-13T11:01:38.997386+0000","last_undegraded":"2023-11-13T11:01:38.997386+0000","last_fullsized":"2023-11-13T11:01:38.997386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011071+0000","last_change":"2023-11-13T11:01:38.998159+0000","last_active":"2023-11-13T11:01:41.011071+0000","last_peered":"2023-11-13T11:01:41.011071+0000","last_clean":"2023-11-13T11:01:41.011071+0000","last_became_active":"2023-11-13T11:01:38.997579+0000","last_became_peered":"2023-11-13T11:01:38.997579+0000","last_unstale":"2023-11-13T11:01:41.011071+0000","last_undegraded":"2023-11-13T11:01:41.011071+0000","last_fullsized":"2023-11-13T11:01:41.011071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:52:42.280287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.966021+0000","last_change":"2023-11-13T11:01:37.978277+0000","last_active":"2023-11-13T11:01:38.966021+0000","last_peered":"2023-11-13T11:01:38.966021+0000","last_clean":"2023-11-13T11:01:38.966021+0000","last_became_active":"2023-11-13T11:01:37.977659+0000","last_became_peered":"2023-11-13T11:01:37.977659+0000","last_unstale":"2023-11-13T11:01:38.966021+0000","last_undegraded":"2023-11-13T11:01:38.966021+0000","last_fullsized":"2023-11-13T11:01:38.966021+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:56.135585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977517+0000","last_change":"2023-11-13T11:01:38.975186+0000","last_active":"2023-11-13T11:01:38.977517+0000","last_peered":"2023-11-13T11:01:38.977517+0000","last_clean":"2023-11-13T11:01:38.977517+0000","last_became_active":"2023-11-13T11:01:38.972436+0000","last_became_peered":"2023-11-13T11:01:38.972436+0000","last_unstale":"2023-11-13T11:01:38.977517+0000","last_undegraded":"2023-11-13T11:01:38.977517+0000","last_fullsized":"2023-11-13T11:01:38.977517+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:45:08.666322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000163+0000","last_change":"2023-11-13T11:01:37.970760+0000","last_active":"2023-11-13T11:01:39.000163+0000","last_peered":"2023-11-13T11:01:39.000163+0000","last_clean":"2023-11-13T11:01:39.000163+0000","last_became_active":"2023-11-13T11:01:37.970465+0000","last_became_peered":"2023-11-13T11:01:37.970465+0000","last_unstale":"2023-11-13T11:01:39.000163+0000","last_undegraded":"2023-11-13T11:01:39.000163+0000","last_fullsized":"2023-11-13T11:01:39.000163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979277+0000","last_change":"2023-11-13T11:01:38.972817+0000","last_active":"2023-11-13T11:01:38.979277+0000","last_peered":"2023-11-13T11:01:38.979277+0000","last_clean":"2023-11-13T11:01:38.979277+0000","last_became_active":"2023-11-13T11:01:38.971691+0000","last_became_peered":"2023-11-13T11:01:38.971691+0000","last_unstale":"2023-11-13T11:01:38.979277+0000","last_undegraded":"2023-11-13T11:01:38.979277+0000","last_fullsized":"2023-11-13T11:01:38.979277+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.978810+0000","last_change":"2023-11-13T11:01:37.982283+0000","last_active":"2023-11-13T11:01:38.978810+0000","last_peered":"2023-11-13T11:01:38.978810+0000","last_clean":"2023-11-13T11:01:38.978810+0000","last_became_active":"2023-11-13T11:01:37.981958+0000","last_became_peered":"2023-11-13T11:01:37.981958+0000","last_unstale":"2023-11-13T11:01:38.978810+0000","last_undegraded":"2023-11-13T11:01:38.978810+0000","last_fullsized":"2023-11-13T11:01:38.978810+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:44:29.604069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991795+0000","last_change":"2023-11-13T11:01:38.982113+0000","last_active":"2023-11-13T11:01:38.991795+0000","last_peered":"2023-11-13T11:01:38.991795+0000","last_clean":"2023-11-13T11:01:38.991795+0000","last_became_active":"2023-11-13T11:01:38.980795+0000","last_became_peered":"2023-11-13T11:01:38.980795+0000","last_unstale":"2023-11-13T11:01:38.991795+0000","last_undegraded":"2023-11-13T11:01:38.991795+0000","last_fullsized":"2023-11-13T11:01:38.991795+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973311+0000","last_change":"2023-11-13T11:01:37.982552+0000","last_active":"2023-11-13T11:01:38.973311+0000","last_peered":"2023-11-13T11:01:38.973311+0000","last_clean":"2023-11-13T11:01:38.973311+0000","last_became_active":"2023-11-13T11:01:37.982269+0000","last_became_peered":"2023-11-13T11:01:37.982269+0000","last_unstale":"2023-11-13T11:01:38.973311+0000","last_undegraded":"2023-11-13T11:01:38.973311+0000","last_fullsized":"2023-11-13T11:01:38.973311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.012056+0000","last_change":"2023-11-13T11:01:38.995167+0000","last_active":"2023-11-13T11:01:41.012056+0000","last_peered":"2023-11-13T11:01:41.012056+0000","last_clean":"2023-11-13T11:01:41.012056+0000","last_became_active":"2023-11-13T11:01:38.994892+0000","last_became_peered":"2023-11-13T11:01:38.994892+0000","last_unstale":"2023-11-13T11:01:41.012056+0000","last_undegraded":"2023-11-13T11:01:41.012056+0000","last_fullsized":"2023-11-13T11:01:41.012056+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:28:05.690927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.975254+0000","last_change":"2023-11-13T11:01:37.971616+0000","last_active":"2023-11-13T11:01:38.975254+0000","last_peered":"2023-11-13T11:01:38.975254+0000","last_clean":"2023-11-13T11:01:38.975254+0000","last_became_active":"2023-11-13T11:01:37.971344+0000","last_became_peered":"2023-11-13T11:01:37.971344+0000","last_unstale":"2023-11-13T11:01:38.975254+0000","last_undegraded":"2023-11-13T11:01:38.975254+0000","last_fullsized":"2023-11-13T11:01:38.975254+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:31.732306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986510+0000","last_change":"2023-11-13T11:01:38.983983+0000","last_active":"2023-11-13T11:01:38.986510+0000","last_peered":"2023-11-13T11:01:38.986510+0000","last_clean":"2023-11-13T11:01:38.986510+0000","last_became_active":"2023-11-13T11:01:38.983698+0000","last_became_peered":"2023-11-13T11:01:38.983698+0000","last_unstale":"2023-11-13T11:01:38.986510+0000","last_undegraded":"2023-11-13T11:01:38.986510+0000","last_fullsized":"2023-11-13T11:01:38.986510+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:46:41.523607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997353+0000","last_change":"2023-11-13T11:01:37.972256+0000","last_active":"2023-11-13T11:01:38.997353+0000","last_peered":"2023-11-13T11:01:38.997353+0000","last_clean":"2023-11-13T11:01:38.997353+0000","last_became_active":"2023-11-13T11:01:37.971890+0000","last_became_peered":"2023-11-13T11:01:37.971890+0000","last_unstale":"2023-11-13T11:01:38.997353+0000","last_undegraded":"2023-11-13T11:01:38.997353+0000","last_fullsized":"2023-11-13T11:01:38.997353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:38.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990549+0000","last_change":"2023-11-13T11:01:38.988092+0000","last_active":"2023-11-13T11:01:38.990549+0000","last_peered":"2023-11-13T11:01:38.990549+0000","last_clean":"2023-11-13T11:01:38.990549+0000","last_became_active":"2023-11-13T11:01:38.987800+0000","last_became_peered":"2023-11-13T11:01:38.987800+0000","last_unstale":"2023-11-13T11:01:38.990549+0000","last_undegraded":"2023-11-13T11:01:38.990549+0000","last_fullsized":"2023-11-13T11:01:38.990549+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:28.542546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.964559+0000","last_change":"2023-11-13T11:01:37.973251+0000","last_active":"2023-11-13T11:01:38.964559+0000","last_peered":"2023-11-13T11:01:38.964559+0000","last_clean":"2023-11-13T11:01:38.964559+0000","last_became_active":"2023-11-13T11:01:37.972947+0000","last_became_peered":"2023-11-13T11:01:37.972947+0000","last_unstale":"2023-11-13T11:01:38.964559+0000","last_undegraded":"2023-11-13T11:01:38.964559+0000","last_fullsized":"2023-11-13T11:01:38.964559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993910+0000","last_change":"2023-11-13T11:01:38.988853+0000","last_active":"2023-11-13T11:01:38.993910+0000","last_peered":"2023-11-13T11:01:38.993910+0000","last_clean":"2023-11-13T11:01:38.993910+0000","last_became_active":"2023-11-13T11:01:38.987558+0000","last_became_peered":"2023-11-13T11:01:38.987558+0000","last_unstale":"2023-11-13T11:01:38.993910+0000","last_undegraded":"2023-11-13T11:01:38.993910+0000","last_fullsized":"2023-11-13T11:01:38.993910+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:38:32.234508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986162+0000","last_change":"2023-11-13T11:01:37.981807+0000","last_active":"2023-11-13T11:01:38.986162+0000","last_peered":"2023-11-13T11:01:38.986162+0000","last_clean":"2023-11-13T11:01:38.986162+0000","last_became_active":"2023-11-13T11:01:37.981480+0000","last_became_peered":"2023-11-13T11:01:37.981480+0000","last_unstale":"2023-11-13T11:01:38.986162+0000","last_undegraded":"2023-11-13T11:01:38.986162+0000","last_fullsized":"2023-11-13T11:01:38.986162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:09:55.684221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.999643+0000","last_change":"2023-11-13T11:01:38.986410+0000","last_active":"2023-11-13T11:01:38.999643+0000","last_peered":"2023-11-13T11:01:38.999643+0000","last_clean":"2023-11-13T11:01:38.999643+0000","last_became_active":"2023-11-13T11:01:38.986131+0000","last_became_peered":"2023-11-13T11:01:38.986131+0000","last_unstale":"2023-11-13T11:01:38.999643+0000","last_undegraded":"2023-11-13T11:01:38.999643+0000","last_fullsized":"2023-11-13T11:01:38.999643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.975456+0000","last_change":"2023-11-13T11:01:37.975627+0000","last_active":"2023-11-13T11:01:38.975456+0000","last_peered":"2023-11-13T11:01:38.975456+0000","last_clean":"2023-11-13T11:01:38.975456+0000","last_became_active":"2023-11-13T11:01:37.974747+0000","last_became_peered":"2023-11-13T11:01:37.974747+0000","last_unstale":"2023-11-13T11:01:38.975456+0000","last_undegraded":"2023-11-13T11:01:38.975456+0000","last_fullsized":"2023-11-13T11:01:38.975456+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990137+0000","last_change":"2023-11-13T11:01:38.989266+0000","last_active":"2023-11-13T11:01:38.990137+0000","last_peered":"2023-11-13T11:01:38.990137+0000","last_clean":"2023-11-13T11:01:38.990137+0000","last_became_active":"2023-11-13T11:01:38.988948+0000","last_became_peered":"2023-11-13T11:01:38.988948+0000","last_unstale":"2023-11-13T11:01:38.990137+0000","last_undegraded":"2023-11-13T11:01:38.990137+0000","last_fullsized":"2023-11-13T11:01:38.990137+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:12.303326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.978974+0000","last_change":"2023-11-13T11:01:37.982077+0000","last_active":"2023-11-13T11:01:38.978974+0000","last_peered":"2023-11-13T11:01:38.978974+0000","last_clean":"2023-11-13T11:01:38.978974+0000","last_became_active":"2023-11-13T11:01:37.981552+0000","last_became_peered":"2023-11-13T11:01:37.981552+0000","last_unstale":"2023-11-13T11:01:38.978974+0000","last_undegraded":"2023-11-13T11:01:38.978974+0000","last_fullsized":"2023-11-13T11:01:38.978974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993927+0000","last_change":"2023-11-13T11:01:38.993394+0000","last_active":"2023-11-13T11:01:38.993927+0000","last_peered":"2023-11-13T11:01:38.993927+0000","last_clean":"2023-11-13T11:01:38.993927+0000","last_became_active":"2023-11-13T11:01:38.992745+0000","last_became_peered":"2023-11-13T11:01:38.992745+0000","last_unstale":"2023-11-13T11:01:38.993927+0000","last_undegraded":"2023-11-13T11:01:38.993927+0000","last_fullsized":"2023-11-13T11:01:38.993927+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.961654+0000","last_change":"2023-11-13T11:01:37.971343+0000","last_active":"2023-11-13T11:01:38.961654+0000","last_peered":"2023-11-13T11:01:38.961654+0000","last_clean":"2023-11-13T11:01:38.961654+0000","last_became_active":"2023-11-13T11:01:37.970761+0000","last_became_peered":"2023-11-13T11:01:37.970761+0000","last_unstale":"2023-11-13T11:01:38.961654+0000","last_undegraded":"2023-11-13T11:01:38.961654+0000","last_fullsized":"2023-11-13T11:01:38.961654+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:47:45.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.012076+0000","last_change":"2023-11-13T11:01:39.001466+0000","last_active":"2023-11-13T11:01:41.012076+0000","last_peered":"2023-11-13T11:01:41.012076+0000","last_clean":"2023-11-13T11:01:41.012076+0000","last_became_active":"2023-11-13T11:01:39.001200+0000","last_became_peered":"2023-11-13T11:01:39.001200+0000","last_unstale":"2023-11-13T11:01:41.012076+0000","last_undegraded":"2023-11-13T11:01:41.012076+0000","last_fullsized":"2023-11-13T11:01:41.012076+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:28:45.331614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.956737+0000","last_change":"2023-11-13T11:01:37.970057+0000","last_active":"2023-11-13T11:01:38.956737+0000","last_peered":"2023-11-13T11:01:38.956737+0000","last_clean":"2023-11-13T11:01:38.956737+0000","last_became_active":"2023-11-13T11:01:37.969819+0000","last_became_peered":"2023-11-13T11:01:37.969819+0000","last_unstale":"2023-11-13T11:01:38.956737+0000","last_undegraded":"2023-11-13T11:01:38.956737+0000","last_fullsized":"2023-11-13T11:01:38.956737+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:45:07.653027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.968087+0000","last_change":"2023-11-13T11:01:38.966353+0000","last_active":"2023-11-13T11:01:38.968087+0000","last_peered":"2023-11-13T11:01:38.968087+0000","last_clean":"2023-11-13T11:01:38.968087+0000","last_became_active":"2023-11-13T11:01:38.966082+0000","last_became_peered":"2023-11-13T11:01:38.966082+0000","last_unstale":"2023-11-13T11:01:38.968087+0000","last_undegraded":"2023-11-13T11:01:38.968087+0000","last_fullsized":"2023-11-13T11:01:38.968087+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977074+0000","last_change":"2023-11-13T11:01:37.970911+0000","last_active":"2023-11-13T11:01:38.977074+0000","last_peered":"2023-11-13T11:01:38.977074+0000","last_clean":"2023-11-13T11:01:38.977074+0000","last_became_active":"2023-11-13T11:01:37.970627+0000","last_became_peered":"2023-11-13T11:01:37.970627+0000","last_unstale":"2023-11-13T11:01:38.977074+0000","last_undegraded":"2023-11-13T11:01:38.977074+0000","last_fullsized":"2023-11-13T11:01:38.977074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.949346+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:01:37.949346+0000","last_peered":"2023-11-13T11:01:37.949346+0000","last_clean":"2023-11-13T11:01:37.949346+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:01:37.949346+0000","last_undegraded":"2023-11-13T11:01:37.949346+0000","last_fullsized":"2023-11-13T11:01:37.949346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0960000000000001,"15min":1.1259999999999999},"min":{"1min":0.84599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.21,"15min":1.1339999999999999},"min":{"1min":0.89800000000000002,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.865,"5min":1.865,"15min":2.2879999999999998},"last":1.353}]},{"osd":1,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.22,"15min":1.175},"min":{"1min":0.91700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8200000000000001,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2969999999999999,"15min":1.2889999999999999},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1070000000000002,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.2789999999999999,"15min":1.2769999999999999},"min":{"1min":0.98099999999999998,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.153,"5min":2.153,"15min":2.1680000000000001},"last":1.244},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3819999999999999,"15min":1.3520000000000001},"min":{"1min":1.1080000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.2400000000000002,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.298}]},{"osd":3,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.407,"15min":1.383},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.63800000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.423,"15min":1.4390000000000001},"min":{"1min":1.167,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.524}]},{"osd":4,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.454,"15min":1.5720000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.577,"5min":1.5129999999999999,"15min":1.5940000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.202}]},{"osd":5,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6000000000000001,"15min":1.6559999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.405},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.72,"15min":1.7589999999999999},"min":{"1min":1.147,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.6339999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.758,"5min":1.659,"15min":1.651},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.754,"5min":1.617,"15min":1.6499999999999999},"min":{"1min":0.96499999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.5940000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1216512,"data_stored":863292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.107,"15min":1.0129999999999999},"min":{"1min":0.82999999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9099999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2070000000000001,"15min":1.2030000000000001},"min":{"1min":0.74299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.639}]},{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.278,"15min":1.2310000000000001},"min":{"1min":0.89800000000000002,"5min":0.71699999999999997,"15min":0.65400000000000003},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.234,"15min":1.238},"min":{"1min":0.878,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8580000000000001,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.2250000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3129999999999999,"15min":1.2509999999999999},"min":{"1min":0.92700000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":1.099},{"interface":"front","average":{"1min":1.375,"5min":1.3120000000000001,"15min":1.2370000000000001},"min":{"1min":0.81599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.5209999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.393,"15min":1.399},"min":{"1min":0.86399999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80700000000000005},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":1.339}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3640000000000001,"15min":1.3340000000000001},"min":{"1min":0.86599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.294,"5min":3.484,"15min":3.484},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.4830000000000001,"15min":1.5349999999999999},"min":{"1min":1.123,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":1.4810000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5049999999999999,"15min":1.6080000000000001},"min":{"1min":1.1539999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.4929999999999999,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.538,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.6120000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6359999999999999,"15min":1.6799999999999999},"min":{"1min":1.167,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.7},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.653,"15min":1.714},"min":{"1min":1.2190000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.4410000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49840,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322000,"statfs":{"total":96636764160,"available":96585728000,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.1599999999999999,"15min":1.0820000000000001},"min":{"1min":0.86399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.103,"15min":1.075},"min":{"1min":0.94399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.218,"15min":1.1579999999999999},"min":{"1min":0.65800000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.7689999999999999,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2689999999999999,"15min":1.2549999999999999},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.821,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.998}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.27,"15min":1.2350000000000001},"min":{"1min":0.98899999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.3200000000000001,"15min":1.3480000000000001},"min":{"1min":0.996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.327}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.46,"15min":1.4139999999999999},"min":{"1min":1.1479999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":2.0049999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.98599999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.742,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.472,"15min":1.502},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0110000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.492},{"interface":"front","average":{"1min":1.504,"5min":1.3480000000000001,"15min":1.401},"min":{"1min":0.98399999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.1910000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.605,"15min":1.629},"min":{"1min":1.0329999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4850000000000001,"15min":1.5389999999999999},"min":{"1min":1.1459999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.6299999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":1.516,"15min":1.591},"min":{"1min":1.139,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.6040000000000001,"15min":1.621},"min":{"1min":1.1020000000000001,"5min":0.63,"15min":0.63},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.571}]}]},{"osd":3,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40124,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331716,"statfs":{"total":96636764160,"available":96595677184,"internally_reserved":0,"allocated":638976,"data_stored":280109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.85299999999999998,"15min":0.86499999999999999},"min":{"1min":0.53000000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.843,"5min":1.867,"15min":1.867},"last":0.68100000000000005},{"interface":"front","average":{"1min":1,"5min":0.98699999999999999,"15min":1.018},"min":{"1min":0.74399999999999999,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4239999999999999,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":0.80300000000000005}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.98599999999999999,"15min":1.04},"min":{"1min":0.65800000000000003,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.603,"5min":1.966,"15min":3.71},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.077,"15min":1.089},"min":{"1min":0.54400000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5660000000000001,"5min":1.923,"15min":2.0190000000000001},"last":0.92300000000000004}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.177,"15min":1.097},"min":{"1min":0.73599999999999999,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":1.5049999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.2909999999999999},"min":{"1min":0.83199999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5389999999999999,"5min":1.952,"15min":3.5710000000000002},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3109999999999999,"15min":1.4179999999999999},"min":{"1min":0.97199999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.5640000000000001,"5min":2.411,"15min":3.5030000000000001},"last":1.143},{"interface":"front","average":{"1min":1.49,"5min":1.345,"15min":1.349},"min":{"1min":1.1319999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.9370000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.615}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.379,"15min":1.476},"min":{"1min":1.073,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.7270000000000001,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.456},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4219999999999999,"15min":1.5149999999999999},"min":{"1min":0.86799999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.72,"5min":2.673,"15min":3.3809999999999998},"last":1.486}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4510000000000001,"15min":1.4870000000000001},"min":{"1min":1.1910000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8260000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.218},{"interface":"front","average":{"1min":1.446,"5min":1.4590000000000001,"15min":1.591},"min":{"1min":0.97099999999999997,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":1.855,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.3109999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.512,"15min":1.6120000000000001},"min":{"1min":1.1819999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.968,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.347},{"interface":"front","average":{"1min":1.647,"5min":1.5680000000000001,"15min":1.5549999999999999},"min":{"1min":1.319,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0059999999999998,"5min":2.75,"15min":2.75},"last":1.5369999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45680,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326160,"statfs":{"total":96636764160,"available":96589987840,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.89500000000000002,"15min":0.92900000000000005},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.9410000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.63},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.95699999999999996,"15min":0.92100000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.7789999999999999,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":0.54400000000000004}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":0.96299999999999997,"15min":0.95199999999999996},"min":{"1min":0.82999999999999996,"5min":0.376,"15min":0.376},"max":{"1min":1.893,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.105,"15min":1.127},"min":{"1min":0.68400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.978,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.133}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.139,"15min":1.2430000000000001},"min":{"1min":0.755,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.796,"5min":2.129,"15min":2.129},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.22,"15min":1.2729999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.867,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":0.98399999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.391,"15min":1.4750000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.9970000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3100000000000001,"15min":1.4039999999999999},"min":{"1min":0.79200000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0449999999999999,"5min":2.3639999999999999,"15min":2.6179999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.3959999999999999,"15min":1.48},"min":{"1min":0.96599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.6600000000000001},"last":1.034},{"interface":"front","average":{"1min":1.633,"5min":1.5409999999999999,"15min":1.5649999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.2450000000000001,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.3240000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.4359999999999999,"15min":1.55},"min":{"1min":0.83099999999999996,"5min":0.78500000000000003,"15min":0.76500000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":4.7569999999999997},"last":1.085},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.538,"15min":1.595},"min":{"1min":1.0229999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.621},"last":1.3620000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.488,"15min":1.6100000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.343,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.573,"15min":1.623},"min":{"1min":1.0740000000000001,"5min":0.85699999999999998,"15min":0.82799999999999996},"max":{"1min":2.2999999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.2909999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509519,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1216512,"data_stored":863292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.84099999999999997,"15min":0.95599999999999996},"min":{"1min":0.59299999999999997,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.47,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.105,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.754}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":0.99099999999999999,"15min":0.98299999999999998},"min":{"1min":0.71599999999999997,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0229999999999999,"15min":0.999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41999999999999998},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.722},"last":1.0029999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7629999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.101,"5min":1,"15min":1.0580000000000001},"min":{"1min":0.57899999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6899999999999999,"5min":1.8149999999999999,"15min":1.9059999999999999},"last":0.91100000000000003}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":0.98399999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.0569999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.363,"15min":1.375},"min":{"1min":1.0860000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.966,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.2290000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.3759999999999999,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83299999999999996},"max":{"1min":2.1629999999999998,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.103},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.4099999999999999},"min":{"1min":0.93999999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.0390000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2909999999999999,"15min":1.407},"min":{"1min":0.91400000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.986,"5min":1.986,"15min":2.3519999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.4339999999999999,"15min":1.4990000000000001},"min":{"1min":0.89800000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.3879999999999999},"last":1.3520000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.4410000000000001,"15min":1.496},"min":{"1min":0.93799999999999994,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.972,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.391,"15min":1.452},"min":{"1min":0.85799999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8660000000000001,"5min":2.202,"15min":2.202},"last":1.3300000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509520,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40108,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331732,"statfs":{"total":96636764160,"available":96595693568,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.90100000000000002},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32100000000000001},"max":{"1min":1.417,"5min":1.74,"15min":1.74},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.022,"15min":1.048},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.7330000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1459999999999999,"15min":1.141},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0089999999999999,"15min":1.018},"min":{"1min":0.54000000000000004,"5min":0.495,"15min":0.371},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.1669999999999998},"last":1.129}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1080000000000001,"15min":1.1020000000000001},"min":{"1min":0.63800000000000001,"5min":0.48999999999999999,"15min":0.45200000000000001},"max":{"1min":1.7450000000000001,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2769999999999999,"15min":1.329},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.827,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.0760000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3300000000000001,"15min":1.3540000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8009999999999999,"5min":2.153,"15min":2.153},"last":4.4279999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.296,"15min":1.4279999999999999},"min":{"1min":0.79400000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.708,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.5029999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.415,"15min":1.5449999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0430000000000001,"5min":2.2719999999999998,"15min":3.7490000000000001},"last":4.0449999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.3839999999999999,"15min":1.484},"min":{"1min":0.89100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.867,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":2.0310000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.48,"15min":1.5229999999999999},"min":{"1min":0.91100000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1389999999999998,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":4.617},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5369999999999999,"15min":1.5740000000000001},"min":{"1min":0.872,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.2810000000000001,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":3.6520000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.5900000000000001,"15min":1.6779999999999999},"min":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.21,"5min":2.742,"15min":3.3540000000000001},"last":1.522},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5800000000000001,"15min":1.6140000000000001},"min":{"1min":0.94999999999999996,"5min":0.88100000000000001,"15min":0.77400000000000002},"max":{"1min":2.2200000000000002,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":3.976}]}]},{"osd":7,"up_from":14,"seq":60129542225,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45708,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326132,"statfs":{"total":96636764160,"available":96589959168,"internally_reserved":0,"allocated":655360,"data_stored":286695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0860000000000001,"15min":1.1120000000000001},"min":{"1min":0.77800000000000002,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6970000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.087,"15min":1.073},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6080000000000001,"5min":2.266,"15min":2.266},"last":1.0620000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.147,"15min":1.0589999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6140000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.294},{"interface":"front","average":{"1min":1.339,"5min":1.2450000000000001,"15min":1.2310000000000001},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3939999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2929999999999999,"15min":1.3029999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8080000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.264,"15min":1.2190000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.4390000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.341,"15min":1.2569999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.829,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.534},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5169999999999999,"15min":1.46},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.925,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.5089999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3680000000000001,"15min":1.3300000000000001},"min":{"1min":0.95199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.907,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":1.337},{"interface":"front","average":{"1min":1.395,"5min":1.4359999999999999,"15min":1.379},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.8420000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":1.1579999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.5389999999999999,"15min":1.5429999999999999},"min":{"1min":0.92800000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.0640000000000001,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.534,"15min":1.4830000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.9870000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.2010000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.641,"15min":1.5760000000000001},"min":{"1min":0.84899999999999998,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":2.093,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.569},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.5920000000000001,"15min":1.518},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.2349999999999999,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":1.4790000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:01:44.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-13T11:01:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.841+0000 7fdd4b4b4640 1 Processor -- start 2023-11-13T11:01:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.842+0000 7fdd4b4b4640 1 -- start start 2023-11-13T11:01:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.843+0000 7fdd4b4b4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdd44146320 0x7fdd440b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.843+0000 7fdd4b4b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd440ce4a0 0x7fdd440ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.843+0000 7fdd4b4b4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4406c940 con 0x7fdd4406bbd0 2023-11-13T11:01:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.843+0000 7fdd4b4b4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd440cee60 con 0x7fdd440ce4a0 2023-11-13T11:01:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.843+0000 7fdd4b4b4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4406c720 con 0x7fdd44146320 2023-11-13T11:01:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.844+0000 7fdd49cb1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd440ce4a0 0x7fdd440ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.844+0000 7fdd4acb3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdd4406bbd0 0x7fdd4406c010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59942/0 (socket says 172.21.15.16:59942) 2023-11-13T11:01:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.844+0000 7fdd4acb3640 1 -- 172.21.15.16:0/2690146550 learned_addr learned my addr 172.21.15.16:0/2690146550 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.844+0000 7fdd4a4b2640 1 --2- 172.21.15.16:0/2690146550 >> v2:172.21.15.184:3300/0 conn(0x7fdd44146320 0x7fdd440b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.844+0000 7fdd49cb1640 1 -- 172.21.15.16:0/2690146550 >> v2:172.21.15.184:3300/0 conn(0x7fdd44146320 msgr2=0x7fdd440b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.844+0000 7fdd49cb1640 1 --2- 172.21.15.16:0/2690146550 >> v2:172.21.15.184:3300/0 conn(0x7fdd44146320 0x7fdd440b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.844+0000 7fdd49cb1640 1 -- 172.21.15.16:0/2690146550 >> v1:172.21.15.16:6789/0 conn(0x7fdd4406bbd0 legacy=0x7fdd4406c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.844+0000 7fdd49cb1640 1 -- 172.21.15.16:0/2690146550 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44070480 con 0x7fdd440ce4a0 2023-11-13T11:01:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.844+0000 7fdd494b0640 1 -- 172.21.15.16:0/2690146550 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058935207 0 0) 0x7fdd4406c940 con 0x7fdd4406bbd0 2023-11-13T11:01:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.845+0000 7fdd49cb1640 1 --2- 172.21.15.16:0/2690146550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd440ce4a0 0x7fdd440ce920 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b1aa0 tx=0x7fdd3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ba617eb3c7a5cb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:44.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.846+0000 7fdd494b0640 1 -- 172.21.15.16:0/2690146550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34031830 con 0x7fdd440ce4a0 2023-11-13T11:01:44.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.846+0000 7fdd494b0640 1 -- 172.21.15.16:0/2690146550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd34031990 con 0x7fdd440ce4a0 2023-11-13T11:01:44.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.846+0000 7fdd494b0640 1 -- 172.21.15.16:0/2690146550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34031d20 con 0x7fdd440ce4a0 2023-11-13T11:01:44.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.846+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/2690146550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd440ce4a0 msgr2=0x7fdd440ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.846+0000 7fdd4b4b4640 1 --2- 172.21.15.16:0/2690146550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd440ce4a0 0x7fdd440ce920 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b1aa0 tx=0x7fdd3402b8d0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.846+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/2690146550 shutdown_connections 2023-11-13T11:01:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.846+0000 7fdd4b4b4640 1 --2- 172.21.15.16:0/2690146550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd440ce4a0 0x7fdd440ce920 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.846+0000 7fdd4b4b4640 1 --2- 172.21.15.16:0/2690146550 >> v2:172.21.15.184:3300/0 conn(0x7fdd44146320 0x7fdd440b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.846+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/2690146550 >> 172.21.15.16:0/2690146550 conn(0x7fdd4406a190 msgr2=0x7fdd4406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.847+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/2690146550 shutdown_connections 2023-11-13T11:01:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.847+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/2690146550 wait complete. 2023-11-13T11:01:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.848+0000 7fdd4b4b4640 1 Processor -- start 2023-11-13T11:01:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.849+0000 7fdd4b4b4640 1 -- start start 2023-11-13T11:01:44.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4b4b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4406bbd0 0x7fdd441590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4b4b4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd440ce4a0 0x7fdd440b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4b4b4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd44146320 0x7fdd440b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4b4b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd441f3740 con 0x7fdd4406bbd0 2023-11-13T11:01:44.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4b4b4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44159b80 con 0x7fdd440ce4a0 2023-11-13T11:01:44.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4b4b4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44159e60 con 0x7fdd44146320 2023-11-13T11:01:44.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4a4b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4406bbd0 0x7fdd441590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4a4b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4406bbd0 0x7fdd441590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41964/0 (socket says 172.21.15.16:41964) 2023-11-13T11:01:44.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4a4b2640 1 -- 172.21.15.16:0/709801828 learned_addr learned my addr 172.21.15.16:0/709801828 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:44.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd49cb1640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd440ce4a0 0x7fdd440b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4acb3640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd44146320 0x7fdd440b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4a4b2640 1 -- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd44146320 msgr2=0x7fdd440b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4a4b2640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd44146320 0x7fdd440b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4a4b2640 1 -- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd440ce4a0 msgr2=0x7fdd440b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:44.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4a4b2640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd440ce4a0 0x7fdd440b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:44.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4a4b2640 1 -- 172.21.15.16:0/709801828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd34009400 con 0x7fdd4406bbd0 2023-11-13T11:01:44.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd49cb1640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd440ce4a0 0x7fdd440b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:44.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.850+0000 7fdd4acb3640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd44146320 0x7fdd440b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:44.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.851+0000 7fdd4a4b2640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4406bbd0 0x7fdd441590e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000d5f0 tx=0x7fdd4000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:44.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.851+0000 7fdd3affd640 1 -- 172.21.15.16:0/709801828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd400125f0 con 0x7fdd4406bbd0 2023-11-13T11:01:44.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.851+0000 7fdd3affd640 1 -- 172.21.15.16:0/709801828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd40012750 con 0x7fdd4406bbd0 2023-11-13T11:01:44.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.851+0000 7fdd3affd640 1 -- 172.21.15.16:0/709801828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40012b00 con 0x7fdd4406bbd0 2023-11-13T11:01:44.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.851+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4415a140 con 0x7fdd4406bbd0 2023-11-13T11:01:44.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.851+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd44158bc0 con 0x7fdd4406bbd0 2023-11-13T11:01:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.852+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4415a2d0 con 0x7fdd4406bbd0 2023-11-13T11:01:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.853+0000 7fdd3affd640 1 -- 172.21.15.16:0/709801828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd40012c60 con 0x7fdd4406bbd0 2023-11-13T11:01:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.854+0000 7fdd3affd640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd2806ae40 0x7fdd2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.854+0000 7fdd3affd640 1 -- 172.21.15.16:0/709801828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdd40089c10 con 0x7fdd4406bbd0 2023-11-13T11:01:44.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.855+0000 7fdd49cb1640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd2806ae40 0x7fdd2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:44.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.856+0000 7fdd49cb1640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd2806ae40 0x7fdd2806d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd34036000 tx=0x7fdd34008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:44.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:44.858+0000 7fdd3affd640 1 -- 172.21.15.16:0/709801828 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd40054c10 con 0x7fdd4406bbd0 2023-11-13T11:01:44.885 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-11-13T11:01:44.880+0000 7f0f7526b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-13T11:01:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.122+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdd44160590 con 0x7fdd4406bbd0 2023-11-13T11:01:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.123+0000 7fdd3affd640 1 -- 172.21.15.16:0/709801828 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1012 (secure 0 0 0) 0x7fdd40058230 con 0x7fdd4406bbd0 2023-11-13T11:01:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:45.129 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":404,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1699872913,"num_in_osds":8,"osd_in_since":1699872904,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":351178752,"bytes_avail":772742934528,"bytes_total":773094113280,"write_bytes_sec":2323,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":28,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5272}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-11-13T10:58:24.041972+0000","services":{}},"progress_events":{}} 2023-11-13T11:01:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd2806ae40 msgr2=0x7fdd2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd2806ae40 0x7fdd2806d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd34036000 tx=0x7fdd34008030 comp rx=0 tx=0).stop 2023-11-13T11:01:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4406bbd0 msgr2=0x7fdd441590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4406bbd0 0x7fdd441590e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000d5f0 tx=0x7fdd4000dab0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 shutdown_connections 2023-11-13T11:01:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdd2806ae40 0x7fdd2806d300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd44146320 0x7fdd440b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd440ce4a0 0x7fdd440b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 --2- 172.21.15.16:0/709801828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4406bbd0 0x7fdd441590e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 >> 172.21.15.16:0/709801828 conn(0x7fdd4406a190 msgr2=0x7fdd4406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 shutdown_connections 2023-11-13T11:01:45.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.126+0000 7fdd4b4b4640 1 -- 172.21.15.16:0/709801828 wait complete. 2023-11-13T11:01:45.150 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-13T11:01:45.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:01:45.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.374+0000 7f4f08a14640 1 Processor -- start 2023-11-13T11:01:45.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.374+0000 7f4f08a14640 1 -- start start 2023-11-13T11:01:45.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.375+0000 7f4f08a14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 0x7f4f0406a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:45.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.375+0000 7f4f08a14640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f0406a6d0 0x7f4f04070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:45.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.375+0000 7f4f08a14640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0406d680 con 0x7f4f040708a0 2023-11-13T11:01:45.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.375+0000 7f4f08a14640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f0406ab50 con 0x7f4f0406a6d0 2023-11-13T11:01:45.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.375+0000 7f4f08a14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f04070d30 con 0x7f4f0415aa90 2023-11-13T11:01:45.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.376+0000 7f4efbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f0406a6d0 0x7f4f04070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.376+0000 7f4efbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f0406a6d0 0x7f4f04070360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41972/0 (socket says 172.21.15.16:41972) 2023-11-13T11:01:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.376+0000 7f4efbfff640 1 -- 172.21.15.16:0/1928237246 learned_addr learned my addr 172.21.15.16:0/1928237246 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.376+0000 7f4f02d9d640 1 --2- 172.21.15.16:0/1928237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 0x7f4f0406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.376+0000 7f4efbfff640 1 -- 172.21.15.16:0/1928237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 msgr2=0x7f4f0406a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.376+0000 7f4efbfff640 1 --2- 172.21.15.16:0/1928237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 0x7f4f0406a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.376+0000 7f4efbfff640 1 -- 172.21.15.16:0/1928237246 >> v1:172.21.15.184:6789/0 conn(0x7f4f040708a0 legacy=0x7f4f040b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.376+0000 7f4efbfff640 1 -- 172.21.15.16:0/1928237246 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0415d630 con 0x7f4f0406a6d0 2023-11-13T11:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.376+0000 7f4f02d9d640 1 --2- 172.21.15.16:0/1928237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 0x7f4f0406a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.377+0000 7f4efbfff640 1 --2- 172.21.15.16:0/1928237246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f0406a6d0 0x7f4f04070360 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009870 tx=0x7f4eec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fee1d4a93d23b16 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f0259c640 1 -- 172.21.15.16:0/1928237246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0317c0 con 0x7f4f0406a6d0 2023-11-13T11:01:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f0259c640 1 -- 172.21.15.16:0/1928237246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eec031920 con 0x7f4f0406a6d0 2023-11-13T11:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f0259c640 1 -- 172.21.15.16:0/1928237246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec031cb0 con 0x7f4f0406a6d0 2023-11-13T11:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f08a14640 1 -- 172.21.15.16:0/1928237246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f0406a6d0 msgr2=0x7f4f04070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f08a14640 1 --2- 172.21.15.16:0/1928237246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f0406a6d0 0x7f4f04070360 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009870 tx=0x7f4eec02b860 comp rx=0 tx=0).stop 2023-11-13T11:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f08a14640 1 -- 172.21.15.16:0/1928237246 shutdown_connections 2023-11-13T11:01:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f08a14640 1 --2- 172.21.15.16:0/1928237246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f0406a6d0 0x7f4f04070360 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f08a14640 1 --2- 172.21.15.16:0/1928237246 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 0x7f4f0406a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f08a14640 1 -- 172.21.15.16:0/1928237246 >> 172.21.15.16:0/1928237246 conn(0x7f4f040b04a0 msgr2=0x7f4f040b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f08a14640 1 -- 172.21.15.16:0/1928237246 shutdown_connections 2023-11-13T11:01:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.378+0000 7f4f08a14640 1 -- 172.21.15.16:0/1928237246 wait complete. 2023-11-13T11:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.380+0000 7f4f08a14640 1 Processor -- start 2023-11-13T11:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.380+0000 7f4f08a14640 1 -- start start 2023-11-13T11:01:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.381+0000 7f4f08a14640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f0406a6d0 0x7f4f041488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.381+0000 7f4f08a14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f040708a0 0x7f4f040cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.381+0000 7f4f08a14640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 0x7f4f040cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.381+0000 7f4f08a14640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040ce070 con 0x7f4f040708a0 2023-11-13T11:01:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.381+0000 7f4f08a14640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040cdbe0 con 0x7f4f0406a6d0 2023-11-13T11:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.381+0000 7f4f08a14640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f04158e70 con 0x7f4f0415aa90 2023-11-13T11:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4efbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f040708a0 0x7f4f040cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4efbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f040708a0 0x7f4f040cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41980/0 (socket says 172.21.15.16:41980) 2023-11-13T11:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4efbfff640 1 -- 172.21.15.16:0/3511481058 learned_addr learned my addr 172.21.15.16:0/3511481058 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4efbfff640 1 -- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 msgr2=0x7f4f040cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4efbfff640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 0x7f4f040cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4efbfff640 1 -- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f0406a6d0 msgr2=0x7f4f041488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4f02d9d640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f0406a6d0 0x7f4f041488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4efbfff640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f0406a6d0 0x7f4f041488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4efbfff640 1 -- 172.21.15.16:0/3511481058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec0094d0 con 0x7f4f040708a0 2023-11-13T11:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.382+0000 7f4efbfff640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f040708a0 0x7f4f040cc460 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4eec000c00 tx=0x7f4eec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.383+0000 7f4efb7fe640 1 -- 172.21.15.16:0/3511481058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec032860 con 0x7f4f040708a0 2023-11-13T11:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.383+0000 7f4efb7fe640 1 -- 172.21.15.16:0/3511481058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eec0329e0 con 0x7f4f040708a0 2023-11-13T11:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.383+0000 7f4efb7fe640 1 -- 172.21.15.16:0/3511481058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec032d90 con 0x7f4f040708a0 2023-11-13T11:01:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.383+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f040ccfd0 con 0x7f4f040708a0 2023-11-13T11:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.383+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f040cd590 con 0x7f4f040708a0 2023-11-13T11:01:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.384+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0406dca0 con 0x7f4f040708a0 2023-11-13T11:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.385+0000 7f4efb7fe640 1 -- 172.21.15.16:0/3511481058 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4eec04d3f0 con 0x7f4f040708a0 2023-11-13T11:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.386+0000 7f4efb7fe640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ee406ae40 0x7f4ee406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.386+0000 7f4efb7fe640 1 -- 172.21.15.16:0/3511481058 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4eec0ab740 con 0x7f4f040708a0 2023-11-13T11:01:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.387+0000 7f4f02d9d640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ee406ae40 0x7f4ee406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.388+0000 7f4f02d9d640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ee406ae40 0x7f4ee406d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4008850 tx=0x7f4ef4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.391+0000 7f4efb7fe640 1 -- 172.21.15.16:0/3511481058 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4eec076610 con 0x7f4f040708a0 2023-11-13T11:01:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.588+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f0415ded0 con 0x7f4ee406ae40 2023-11-13T11:01:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.611+0000 7f4efb7fe640 1 -- 172.21.15.16:0/3511481058 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287202 (secure 0 0 0) 0x7f4ef400c040 con 0x7f4ee406ae40 2023-11-13T11:01:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:01:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ee406ae40 msgr2=0x7f4ee406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ee406ae40 0x7f4ee406d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4008850 tx=0x7f4ef4008040 comp rx=0 tx=0).stop 2023-11-13T11:01:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f040708a0 msgr2=0x7f4f040cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f040708a0 0x7f4f040cc460 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4eec000c00 tx=0x7f4eec0046c0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 shutdown_connections 2023-11-13T11:01:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4ee406ae40 0x7f4ee406d300 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f0415aa90 0x7f4f040cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f040708a0 0x7f4f040cc460 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 --2- 172.21.15.16:0/3511481058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f0406a6d0 0x7f4f041488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 >> 172.21.15.16:0/3511481058 conn(0x7f4f040b04a0 msgr2=0x7f4f040b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 shutdown_connections 2023-11-13T11:01:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:45.618+0000 7f4f08a14640 1 -- 172.21.15.16:0/3511481058 wait complete. 2023-11-13T11:01:45.645 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2023-11-13T11:01:44.118569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342948,"kb_used_data":6072,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754631772,"statfs":{"total":773094113280,"available":772742934528,"internally_reserved":0,"allocated":6217728,"data_stored":3385084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":344837681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.170123"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997544+0000","last_change":"2023-11-13T11:01:38.993640+0000","last_active":"2023-11-13T11:01:38.997544+0000","last_peered":"2023-11-13T11:01:38.997544+0000","last_clean":"2023-11-13T11:01:38.997544+0000","last_became_active":"2023-11-13T11:01:38.991479+0000","last_became_peered":"2023-11-13T11:01:38.991479+0000","last_unstale":"2023-11-13T11:01:38.997544+0000","last_undegraded":"2023-11-13T11:01:38.997544+0000","last_fullsized":"2023-11-13T11:01:38.997544+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977509+0000","last_change":"2023-11-13T11:01:38.975999+0000","last_active":"2023-11-13T11:01:38.977509+0000","last_peered":"2023-11-13T11:01:38.977509+0000","last_clean":"2023-11-13T11:01:38.977509+0000","last_became_active":"2023-11-13T11:01:38.975726+0000","last_became_peered":"2023-11-13T11:01:38.975726+0000","last_unstale":"2023-11-13T11:01:38.977509+0000","last_undegraded":"2023-11-13T11:01:38.977509+0000","last_fullsized":"2023-11-13T11:01:38.977509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:33:09.247389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991945+0000","last_change":"2023-11-13T11:01:38.984201+0000","last_active":"2023-11-13T11:01:38.991945+0000","last_peered":"2023-11-13T11:01:38.991945+0000","last_clean":"2023-11-13T11:01:38.991945+0000","last_became_active":"2023-11-13T11:01:38.983716+0000","last_became_peered":"2023-11-13T11:01:38.983716+0000","last_unstale":"2023-11-13T11:01:38.991945+0000","last_undegraded":"2023-11-13T11:01:38.991945+0000","last_fullsized":"2023-11-13T11:01:38.991945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:37:29.570297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980437+0000","last_change":"2023-11-13T11:01:38.975163+0000","last_active":"2023-11-13T11:01:38.980437+0000","last_peered":"2023-11-13T11:01:38.980437+0000","last_clean":"2023-11-13T11:01:38.980437+0000","last_became_active":"2023-11-13T11:01:38.974878+0000","last_became_peered":"2023-11-13T11:01:38.974878+0000","last_unstale":"2023-11-13T11:01:38.980437+0000","last_undegraded":"2023-11-13T11:01:38.980437+0000","last_fullsized":"2023-11-13T11:01:38.980437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.970996+0000","last_change":"2023-11-13T11:01:38.968974+0000","last_active":"2023-11-13T11:01:38.970996+0000","last_peered":"2023-11-13T11:01:38.970996+0000","last_clean":"2023-11-13T11:01:38.970996+0000","last_became_active":"2023-11-13T11:01:38.968747+0000","last_became_peered":"2023-11-13T11:01:38.968747+0000","last_unstale":"2023-11-13T11:01:38.970996+0000","last_undegraded":"2023-11-13T11:01:38.970996+0000","last_fullsized":"2023-11-13T11:01:38.970996+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:14:08.138073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011194+0000","last_change":"2023-11-13T11:01:38.993231+0000","last_active":"2023-11-13T11:01:41.011194+0000","last_peered":"2023-11-13T11:01:41.011194+0000","last_clean":"2023-11-13T11:01:41.011194+0000","last_became_active":"2023-11-13T11:01:38.992972+0000","last_became_peered":"2023-11-13T11:01:38.992972+0000","last_unstale":"2023-11-13T11:01:41.011194+0000","last_undegraded":"2023-11-13T11:01:41.011194+0000","last_fullsized":"2023-11-13T11:01:41.011194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:45:42.458182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.006474+0000","last_change":"2023-11-13T11:01:39.003582+0000","last_active":"2023-11-13T11:01:39.006474+0000","last_peered":"2023-11-13T11:01:39.006474+0000","last_clean":"2023-11-13T11:01:39.006474+0000","last_became_active":"2023-11-13T11:01:39.003324+0000","last_became_peered":"2023-11-13T11:01:39.003324+0000","last_unstale":"2023-11-13T11:01:39.006474+0000","last_undegraded":"2023-11-13T11:01:39.006474+0000","last_fullsized":"2023-11-13T11:01:39.006474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.010848+0000","last_change":"2023-11-13T11:01:39.004402+0000","last_active":"2023-11-13T11:01:41.010848+0000","last_peered":"2023-11-13T11:01:41.010848+0000","last_clean":"2023-11-13T11:01:41.010848+0000","last_became_active":"2023-11-13T11:01:39.001982+0000","last_became_peered":"2023-11-13T11:01:39.001982+0000","last_unstale":"2023-11-13T11:01:41.010848+0000","last_undegraded":"2023-11-13T11:01:41.010848+0000","last_fullsized":"2023-11-13T11:01:41.010848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:32:40.380861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.003277+0000","last_change":"2023-11-13T11:01:38.997117+0000","last_active":"2023-11-13T11:01:39.003277+0000","last_peered":"2023-11-13T11:01:39.003277+0000","last_clean":"2023-11-13T11:01:39.003277+0000","last_became_active":"2023-11-13T11:01:38.996801+0000","last_became_peered":"2023-11-13T11:01:38.996801+0000","last_unstale":"2023-11-13T11:01:39.003277+0000","last_undegraded":"2023-11-13T11:01:39.003277+0000","last_fullsized":"2023-11-13T11:01:39.003277+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:03:35.448965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990548+0000","last_change":"2023-11-13T11:01:38.987650+0000","last_active":"2023-11-13T11:01:38.990548+0000","last_peered":"2023-11-13T11:01:38.990548+0000","last_clean":"2023-11-13T11:01:38.990548+0000","last_became_active":"2023-11-13T11:01:38.987270+0000","last_became_peered":"2023-11-13T11:01:38.987270+0000","last_unstale":"2023-11-13T11:01:38.990548+0000","last_undegraded":"2023-11-13T11:01:38.990548+0000","last_fullsized":"2023-11-13T11:01:38.990548+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:25.371717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992670+0000","last_change":"2023-11-13T11:01:38.984512+0000","last_active":"2023-11-13T11:01:38.992670+0000","last_peered":"2023-11-13T11:01:38.992670+0000","last_clean":"2023-11-13T11:01:38.992670+0000","last_became_active":"2023-11-13T11:01:38.984243+0000","last_became_peered":"2023-11-13T11:01:38.984243+0000","last_unstale":"2023-11-13T11:01:38.992670+0000","last_undegraded":"2023-11-13T11:01:38.992670+0000","last_fullsized":"2023-11-13T11:01:38.992670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011670+0000","last_change":"2023-11-13T11:01:38.992022+0000","last_active":"2023-11-13T11:01:41.011670+0000","last_peered":"2023-11-13T11:01:41.011670+0000","last_clean":"2023-11-13T11:01:41.011670+0000","last_became_active":"2023-11-13T11:01:38.991341+0000","last_became_peered":"2023-11-13T11:01:38.991341+0000","last_unstale":"2023-11-13T11:01:41.011670+0000","last_undegraded":"2023-11-13T11:01:41.011670+0000","last_fullsized":"2023-11-13T11:01:41.011670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:53:20.682896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011270+0000","last_change":"2023-11-13T11:01:38.998336+0000","last_active":"2023-11-13T11:01:41.011270+0000","last_peered":"2023-11-13T11:01:41.011270+0000","last_clean":"2023-11-13T11:01:41.011270+0000","last_became_active":"2023-11-13T11:01:38.997894+0000","last_became_peered":"2023-11-13T11:01:38.997894+0000","last_unstale":"2023-11-13T11:01:41.011270+0000","last_undegraded":"2023-11-13T11:01:41.011270+0000","last_fullsized":"2023-11-13T11:01:41.011270+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:42:46.869446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.994357+0000","last_change":"2023-11-13T11:01:38.992189+0000","last_active":"2023-11-13T11:01:38.994357+0000","last_peered":"2023-11-13T11:01:38.994357+0000","last_clean":"2023-11-13T11:01:38.994357+0000","last_became_active":"2023-11-13T11:01:38.991290+0000","last_became_peered":"2023-11-13T11:01:38.991290+0000","last_unstale":"2023-11-13T11:01:38.994357+0000","last_undegraded":"2023-11-13T11:01:38.994357+0000","last_fullsized":"2023-11-13T11:01:38.994357+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:25.371717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.969075+0000","last_change":"2023-11-13T11:01:38.966119+0000","last_active":"2023-11-13T11:01:38.969075+0000","last_peered":"2023-11-13T11:01:38.969075+0000","last_clean":"2023-11-13T11:01:38.969075+0000","last_became_active":"2023-11-13T11:01:38.965812+0000","last_became_peered":"2023-11-13T11:01:38.965812+0000","last_unstale":"2023-11-13T11:01:38.969075+0000","last_undegraded":"2023-11-13T11:01:38.969075+0000","last_fullsized":"2023-11-13T11:01:38.969075+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.010855+0000","last_change":"2023-11-13T11:01:39.002819+0000","last_active":"2023-11-13T11:01:41.010855+0000","last_peered":"2023-11-13T11:01:41.010855+0000","last_clean":"2023-11-13T11:01:41.010855+0000","last_became_active":"2023-11-13T11:01:39.002309+0000","last_became_peered":"2023-11-13T11:01:39.002309+0000","last_unstale":"2023-11-13T11:01:41.010855+0000","last_undegraded":"2023-11-13T11:01:41.010855+0000","last_fullsized":"2023-11-13T11:01:41.010855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:20:55.201885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997524+0000","last_change":"2023-11-13T11:01:38.993324+0000","last_active":"2023-11-13T11:01:38.997524+0000","last_peered":"2023-11-13T11:01:38.997524+0000","last_clean":"2023-11-13T11:01:38.997524+0000","last_became_active":"2023-11-13T11:01:38.991072+0000","last_became_peered":"2023-11-13T11:01:38.991072+0000","last_unstale":"2023-11-13T11:01:38.997524+0000","last_undegraded":"2023-11-13T11:01:38.997524+0000","last_fullsized":"2023-11-13T11:01:38.997524+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980403+0000","last_change":"2023-11-13T11:01:38.975150+0000","last_active":"2023-11-13T11:01:38.980403+0000","last_peered":"2023-11-13T11:01:38.980403+0000","last_clean":"2023-11-13T11:01:38.980403+0000","last_became_active":"2023-11-13T11:01:38.974837+0000","last_became_peered":"2023-11-13T11:01:38.974837+0000","last_unstale":"2023-11-13T11:01:38.980403+0000","last_undegraded":"2023-11-13T11:01:38.980403+0000","last_fullsized":"2023-11-13T11:01:38.980403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:19:00.701068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991920+0000","last_change":"2023-11-13T11:01:38.984024+0000","last_active":"2023-11-13T11:01:38.991920+0000","last_peered":"2023-11-13T11:01:38.991920+0000","last_clean":"2023-11-13T11:01:38.991920+0000","last_became_active":"2023-11-13T11:01:38.983338+0000","last_became_peered":"2023-11-13T11:01:38.983338+0000","last_unstale":"2023-11-13T11:01:38.991920+0000","last_undegraded":"2023-11-13T11:01:38.991920+0000","last_fullsized":"2023-11-13T11:01:38.991920+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:06:44.792510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973436+0000","last_change":"2023-11-13T11:01:38.969241+0000","last_active":"2023-11-13T11:01:38.973436+0000","last_peered":"2023-11-13T11:01:38.973436+0000","last_clean":"2023-11-13T11:01:38.973436+0000","last_became_active":"2023-11-13T11:01:38.968942+0000","last_became_peered":"2023-11-13T11:01:38.968942+0000","last_unstale":"2023-11-13T11:01:38.973436+0000","last_undegraded":"2023-11-13T11:01:38.973436+0000","last_fullsized":"2023-11-13T11:01:38.973436+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:19:00.701068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.972503+0000","last_change":"2023-11-13T11:01:38.966987+0000","last_active":"2023-11-13T11:01:38.972503+0000","last_peered":"2023-11-13T11:01:38.972503+0000","last_clean":"2023-11-13T11:01:38.972503+0000","last_became_active":"2023-11-13T11:01:38.966723+0000","last_became_peered":"2023-11-13T11:01:38.966723+0000","last_unstale":"2023-11-13T11:01:38.972503+0000","last_undegraded":"2023-11-13T11:01:38.972503+0000","last_fullsized":"2023-11-13T11:01:38.972503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:33:09.247389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993826+0000","last_change":"2023-11-13T11:01:38.989021+0000","last_active":"2023-11-13T11:01:38.993826+0000","last_peered":"2023-11-13T11:01:38.993826+0000","last_clean":"2023-11-13T11:01:38.993826+0000","last_became_active":"2023-11-13T11:01:38.988715+0000","last_became_peered":"2023-11-13T11:01:38.988715+0000","last_unstale":"2023-11-13T11:01:38.993826+0000","last_undegraded":"2023-11-13T11:01:38.993826+0000","last_fullsized":"2023-11-13T11:01:38.993826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:12:15.067759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993091+0000","last_change":"2023-11-13T11:01:38.986587+0000","last_active":"2023-11-13T11:01:38.993091+0000","last_peered":"2023-11-13T11:01:38.993091+0000","last_clean":"2023-11-13T11:01:38.993091+0000","last_became_active":"2023-11-13T11:01:38.986301+0000","last_became_peered":"2023-11-13T11:01:38.986301+0000","last_unstale":"2023-11-13T11:01:38.993091+0000","last_undegraded":"2023-11-13T11:01:38.993091+0000","last_fullsized":"2023-11-13T11:01:38.993091+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:39.111281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.995951+0000","last_change":"2023-11-13T11:01:38.992108+0000","last_active":"2023-11-13T11:01:38.995951+0000","last_peered":"2023-11-13T11:01:38.995951+0000","last_clean":"2023-11-13T11:01:38.995951+0000","last_became_active":"2023-11-13T11:01:38.988156+0000","last_became_peered":"2023-11-13T11:01:38.988156+0000","last_unstale":"2023-11-13T11:01:38.995951+0000","last_undegraded":"2023-11-13T11:01:38.995951+0000","last_fullsized":"2023-11-13T11:01:38.995951+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:02.526493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000637+0000","last_change":"2023-11-13T11:01:38.996834+0000","last_active":"2023-11-13T11:01:39.000637+0000","last_peered":"2023-11-13T11:01:39.000637+0000","last_clean":"2023-11-13T11:01:39.000637+0000","last_became_active":"2023-11-13T11:01:38.996522+0000","last_became_peered":"2023-11-13T11:01:38.996522+0000","last_unstale":"2023-11-13T11:01:39.000637+0000","last_undegraded":"2023-11-13T11:01:39.000637+0000","last_fullsized":"2023-11-13T11:01:39.000637+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:01.750338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005129+0000","last_change":"2023-11-13T11:01:39.003837+0000","last_active":"2023-11-13T11:01:39.005129+0000","last_peered":"2023-11-13T11:01:39.005129+0000","last_clean":"2023-11-13T11:01:39.005129+0000","last_became_active":"2023-11-13T11:01:39.000372+0000","last_became_peered":"2023-11-13T11:01:39.000372+0000","last_unstale":"2023-11-13T11:01:39.005129+0000","last_undegraded":"2023-11-13T11:01:39.005129+0000","last_fullsized":"2023-11-13T11:01:39.005129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:29:27.396844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011710+0000","last_change":"2023-11-13T11:01:38.997301+0000","last_active":"2023-11-13T11:01:41.011710+0000","last_peered":"2023-11-13T11:01:41.011710+0000","last_clean":"2023-11-13T11:01:41.011710+0000","last_became_active":"2023-11-13T11:01:38.997039+0000","last_became_peered":"2023-11-13T11:01:38.997039+0000","last_unstale":"2023-11-13T11:01:41.011710+0000","last_undegraded":"2023-11-13T11:01:41.011710+0000","last_fullsized":"2023-11-13T11:01:41.011710+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:07:23.632348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.001088+0000","last_change":"2023-11-13T11:01:39.000662+0000","last_active":"2023-11-13T11:01:39.001088+0000","last_peered":"2023-11-13T11:01:39.001088+0000","last_clean":"2023-11-13T11:01:39.001088+0000","last_became_active":"2023-11-13T11:01:39.000191+0000","last_became_peered":"2023-11-13T11:01:39.000191+0000","last_unstale":"2023-11-13T11:01:39.001088+0000","last_undegraded":"2023-11-13T11:01:39.001088+0000","last_fullsized":"2023-11-13T11:01:39.001088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980942+0000","last_change":"2023-11-13T11:01:38.969203+0000","last_active":"2023-11-13T11:01:38.980942+0000","last_peered":"2023-11-13T11:01:38.980942+0000","last_clean":"2023-11-13T11:01:38.980942+0000","last_became_active":"2023-11-13T11:01:38.968896+0000","last_became_peered":"2023-11-13T11:01:38.968896+0000","last_unstale":"2023-11-13T11:01:38.980942+0000","last_undegraded":"2023-11-13T11:01:38.980942+0000","last_fullsized":"2023-11-13T11:01:38.980942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991585+0000","last_change":"2023-11-13T11:01:38.989974+0000","last_active":"2023-11-13T11:01:38.991585+0000","last_peered":"2023-11-13T11:01:38.991585+0000","last_clean":"2023-11-13T11:01:38.991585+0000","last_became_active":"2023-11-13T11:01:38.989665+0000","last_became_peered":"2023-11-13T11:01:38.989665+0000","last_unstale":"2023-11-13T11:01:38.991585+0000","last_undegraded":"2023-11-13T11:01:38.991585+0000","last_fullsized":"2023-11-13T11:01:38.991585+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:18:26.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000755+0000","last_change":"2023-11-13T11:01:38.995148+0000","last_active":"2023-11-13T11:01:39.000755+0000","last_peered":"2023-11-13T11:01:39.000755+0000","last_clean":"2023-11-13T11:01:39.000755+0000","last_became_active":"2023-11-13T11:01:38.990007+0000","last_became_peered":"2023-11-13T11:01:38.990007+0000","last_unstale":"2023-11-13T11:01:39.000755+0000","last_undegraded":"2023-11-13T11:01:39.000755+0000","last_fullsized":"2023-11-13T11:01:39.000755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:18:26.064560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.988066+0000","last_change":"2023-11-13T11:01:38.984675+0000","last_active":"2023-11-13T11:01:38.988066+0000","last_peered":"2023-11-13T11:01:38.988066+0000","last_clean":"2023-11-13T11:01:38.988066+0000","last_became_active":"2023-11-13T11:01:38.984073+0000","last_became_peered":"2023-11-13T11:01:38.984073+0000","last_unstale":"2023-11-13T11:01:38.988066+0000","last_undegraded":"2023-11-13T11:01:38.988066+0000","last_fullsized":"2023-11-13T11:01:38.988066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:24:13.781188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979482+0000","last_change":"2023-11-13T11:01:38.973375+0000","last_active":"2023-11-13T11:01:38.979482+0000","last_peered":"2023-11-13T11:01:38.979482+0000","last_clean":"2023-11-13T11:01:38.979482+0000","last_became_active":"2023-11-13T11:01:38.970971+0000","last_became_peered":"2023-11-13T11:01:38.970971+0000","last_unstale":"2023-11-13T11:01:38.979482+0000","last_undegraded":"2023-11-13T11:01:38.979482+0000","last_fullsized":"2023-11-13T11:01:38.979482+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:03:35.448965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.999535+0000","last_change":"2023-11-13T11:01:37.976416+0000","last_active":"2023-11-13T11:01:38.999535+0000","last_peered":"2023-11-13T11:01:38.999535+0000","last_clean":"2023-11-13T11:01:38.999535+0000","last_became_active":"2023-11-13T11:01:37.976080+0000","last_became_peered":"2023-11-13T11:01:37.976080+0000","last_unstale":"2023-11-13T11:01:38.999535+0000","last_undegraded":"2023-11-13T11:01:38.999535+0000","last_fullsized":"2023-11-13T11:01:38.999535+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979796+0000","last_change":"2023-11-13T11:01:38.977746+0000","last_active":"2023-11-13T11:01:38.979796+0000","last_peered":"2023-11-13T11:01:38.979796+0000","last_clean":"2023-11-13T11:01:38.979796+0000","last_became_active":"2023-11-13T11:01:38.977421+0000","last_became_peered":"2023-11-13T11:01:38.977421+0000","last_unstale":"2023-11-13T11:01:38.979796+0000","last_undegraded":"2023-11-13T11:01:38.979796+0000","last_fullsized":"2023-11-13T11:01:38.979796+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:29:27.396844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005578+0000","last_change":"2023-11-13T11:01:37.984920+0000","last_active":"2023-11-13T11:01:39.005578+0000","last_peered":"2023-11-13T11:01:39.005578+0000","last_clean":"2023-11-13T11:01:39.005578+0000","last_became_active":"2023-11-13T11:01:37.984119+0000","last_became_peered":"2023-11-13T11:01:37.984119+0000","last_unstale":"2023-11-13T11:01:39.005578+0000","last_undegraded":"2023-11-13T11:01:39.005578+0000","last_fullsized":"2023-11-13T11:01:39.005578+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:09:29.998731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977904+0000","last_change":"2023-11-13T11:01:38.971332+0000","last_active":"2023-11-13T11:01:38.977904+0000","last_peered":"2023-11-13T11:01:38.977904+0000","last_clean":"2023-11-13T11:01:38.977904+0000","last_became_active":"2023-11-13T11:01:38.969969+0000","last_became_peered":"2023-11-13T11:01:38.969969+0000","last_unstale":"2023-11-13T11:01:38.977904+0000","last_undegraded":"2023-11-13T11:01:38.977904+0000","last_fullsized":"2023-11-13T11:01:38.977904+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:09:56.697516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000885+0000","last_change":"2023-11-13T11:01:37.970093+0000","last_active":"2023-11-13T11:01:39.000885+0000","last_peered":"2023-11-13T11:01:39.000885+0000","last_clean":"2023-11-13T11:01:39.000885+0000","last_became_active":"2023-11-13T11:01:37.969770+0000","last_became_peered":"2023-11-13T11:01:37.969770+0000","last_unstale":"2023-11-13T11:01:39.000885+0000","last_undegraded":"2023-11-13T11:01:39.000885+0000","last_fullsized":"2023-11-13T11:01:39.000885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:33.254128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990202+0000","last_change":"2023-11-13T11:01:38.980529+0000","last_active":"2023-11-13T11:01:38.990202+0000","last_peered":"2023-11-13T11:01:38.990202+0000","last_clean":"2023-11-13T11:01:38.990202+0000","last_became_active":"2023-11-13T11:01:38.980227+0000","last_became_peered":"2023-11-13T11:01:38.980227+0000","last_unstale":"2023-11-13T11:01:38.990202+0000","last_undegraded":"2023-11-13T11:01:38.990202+0000","last_fullsized":"2023-11-13T11:01:38.990202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:12.303326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.994946+0000","last_change":"2023-11-13T11:01:37.982793+0000","last_active":"2023-11-13T11:01:38.994946+0000","last_peered":"2023-11-13T11:01:38.994946+0000","last_clean":"2023-11-13T11:01:38.994946+0000","last_became_active":"2023-11-13T11:01:37.982111+0000","last_became_peered":"2023-11-13T11:01:37.982111+0000","last_unstale":"2023-11-13T11:01:38.994946+0000","last_undegraded":"2023-11-13T11:01:38.994946+0000","last_fullsized":"2023-11-13T11:01:38.994946+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980711+0000","last_change":"2023-11-13T11:01:38.976171+0000","last_active":"2023-11-13T11:01:38.980711+0000","last_peered":"2023-11-13T11:01:38.980711+0000","last_clean":"2023-11-13T11:01:38.980711+0000","last_became_active":"2023-11-13T11:01:38.975895+0000","last_became_peered":"2023-11-13T11:01:38.975895+0000","last_unstale":"2023-11-13T11:01:38.980711+0000","last_undegraded":"2023-11-13T11:01:38.980711+0000","last_fullsized":"2023-11-13T11:01:38.980711+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.002584+0000","last_change":"2023-11-13T11:01:37.981492+0000","last_active":"2023-11-13T11:01:39.002584+0000","last_peered":"2023-11-13T11:01:39.002584+0000","last_clean":"2023-11-13T11:01:39.002584+0000","last_became_active":"2023-11-13T11:01:37.981149+0000","last_became_peered":"2023-11-13T11:01:37.981149+0000","last_unstale":"2023-11-13T11:01:39.002584+0000","last_undegraded":"2023-11-13T11:01:39.002584+0000","last_fullsized":"2023-11-13T11:01:39.002584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979333+0000","last_change":"2023-11-13T11:01:38.972149+0000","last_active":"2023-11-13T11:01:38.979333+0000","last_peered":"2023-11-13T11:01:38.979333+0000","last_clean":"2023-11-13T11:01:38.979333+0000","last_became_active":"2023-11-13T11:01:38.971860+0000","last_became_peered":"2023-11-13T11:01:38.971860+0000","last_unstale":"2023-11-13T11:01:38.979333+0000","last_undegraded":"2023-11-13T11:01:38.979333+0000","last_fullsized":"2023-11-13T11:01:38.979333+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:44:30.617363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986684+0000","last_change":"2023-11-13T11:01:37.968032+0000","last_active":"2023-11-13T11:01:38.986684+0000","last_peered":"2023-11-13T11:01:38.986684+0000","last_clean":"2023-11-13T11:01:38.986684+0000","last_became_active":"2023-11-13T11:01:37.967709+0000","last_became_peered":"2023-11-13T11:01:37.967709+0000","last_unstale":"2023-11-13T11:01:38.986684+0000","last_undegraded":"2023-11-13T11:01:38.986684+0000","last_fullsized":"2023-11-13T11:01:38.986684+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:44:32.588239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.003718+0000","last_change":"2023-11-13T11:01:39.001367+0000","last_active":"2023-11-13T11:01:39.003718+0000","last_peered":"2023-11-13T11:01:39.003718+0000","last_clean":"2023-11-13T11:01:39.003718+0000","last_became_active":"2023-11-13T11:01:39.001092+0000","last_became_peered":"2023-11-13T11:01:39.001092+0000","last_unstale":"2023-11-13T11:01:39.003718+0000","last_undegraded":"2023-11-13T11:01:39.003718+0000","last_fullsized":"2023-11-13T11:01:39.003718+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.962390+0000","last_change":"2023-11-13T11:01:37.974253+0000","last_active":"2023-11-13T11:01:38.962390+0000","last_peered":"2023-11-13T11:01:38.962390+0000","last_clean":"2023-11-13T11:01:38.962390+0000","last_became_active":"2023-11-13T11:01:37.973388+0000","last_became_peered":"2023-11-13T11:01:37.973388+0000","last_unstale":"2023-11-13T11:01:38.962390+0000","last_undegraded":"2023-11-13T11:01:38.962390+0000","last_fullsized":"2023-11-13T11:01:38.962390+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997619+0000","last_change":"2023-11-13T11:01:38.996067+0000","last_active":"2023-11-13T11:01:38.997619+0000","last_peered":"2023-11-13T11:01:38.997619+0000","last_clean":"2023-11-13T11:01:38.997619+0000","last_became_active":"2023-11-13T11:01:38.995583+0000","last_became_peered":"2023-11-13T11:01:38.995583+0000","last_unstale":"2023-11-13T11:01:38.997619+0000","last_undegraded":"2023-11-13T11:01:38.997619+0000","last_fullsized":"2023-11-13T11:01:38.997619+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973630+0000","last_change":"2023-11-13T11:01:37.980576+0000","last_active":"2023-11-13T11:01:38.973630+0000","last_peered":"2023-11-13T11:01:38.973630+0000","last_clean":"2023-11-13T11:01:38.973630+0000","last_became_active":"2023-11-13T11:01:37.980253+0000","last_became_peered":"2023-11-13T11:01:37.980253+0000","last_unstale":"2023-11-13T11:01:38.973630+0000","last_undegraded":"2023-11-13T11:01:38.973630+0000","last_fullsized":"2023-11-13T11:01:38.973630+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.982849+0000","last_change":"2023-11-13T11:01:38.978906+0000","last_active":"2023-11-13T11:01:38.982849+0000","last_peered":"2023-11-13T11:01:38.982849+0000","last_clean":"2023-11-13T11:01:38.982849+0000","last_became_active":"2023-11-13T11:01:38.978590+0000","last_became_peered":"2023-11-13T11:01:38.978590+0000","last_unstale":"2023-11-13T11:01:38.982849+0000","last_undegraded":"2023-11-13T11:01:38.982849+0000","last_fullsized":"2023-11-13T11:01:38.982849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.957725+0000","last_change":"2023-11-13T11:01:37.967093+0000","last_active":"2023-11-13T11:01:38.957725+0000","last_peered":"2023-11-13T11:01:38.957725+0000","last_clean":"2023-11-13T11:01:38.957725+0000","last_became_active":"2023-11-13T11:01:37.966819+0000","last_became_peered":"2023-11-13T11:01:37.966819+0000","last_unstale":"2023-11-13T11:01:38.957725+0000","last_undegraded":"2023-11-13T11:01:38.957725+0000","last_fullsized":"2023-11-13T11:01:38.957725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.980445+0000","last_change":"2023-11-13T11:01:38.978007+0000","last_active":"2023-11-13T11:01:38.980445+0000","last_peered":"2023-11-13T11:01:38.980445+0000","last_clean":"2023-11-13T11:01:38.980445+0000","last_became_active":"2023-11-13T11:01:38.977730+0000","last_became_peered":"2023-11-13T11:01:38.977730+0000","last_unstale":"2023-11-13T11:01:38.980445+0000","last_undegraded":"2023-11-13T11:01:38.980445+0000","last_fullsized":"2023-11-13T11:01:38.980445+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.957148+0000","last_change":"2023-11-13T11:01:37.974872+0000","last_active":"2023-11-13T11:01:38.957148+0000","last_peered":"2023-11-13T11:01:38.957148+0000","last_clean":"2023-11-13T11:01:38.957148+0000","last_became_active":"2023-11-13T11:01:37.973788+0000","last_became_peered":"2023-11-13T11:01:37.973788+0000","last_unstale":"2023-11-13T11:01:38.957148+0000","last_undegraded":"2023-11-13T11:01:38.957148+0000","last_fullsized":"2023-11-13T11:01:38.957148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:11.290032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992368+0000","last_change":"2023-11-13T11:01:38.982930+0000","last_active":"2023-11-13T11:01:38.992368+0000","last_peered":"2023-11-13T11:01:38.992368+0000","last_clean":"2023-11-13T11:01:38.992368+0000","last_became_active":"2023-11-13T11:01:38.982617+0000","last_became_peered":"2023-11-13T11:01:38.982617+0000","last_unstale":"2023-11-13T11:01:38.992368+0000","last_undegraded":"2023-11-13T11:01:38.992368+0000","last_fullsized":"2023-11-13T11:01:38.992368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:35.756266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997924+0000","last_change":"2023-11-13T11:01:37.976846+0000","last_active":"2023-11-13T11:01:38.997924+0000","last_peered":"2023-11-13T11:01:38.997924+0000","last_clean":"2023-11-13T11:01:38.997924+0000","last_became_active":"2023-11-13T11:01:37.976378+0000","last_became_peered":"2023-11-13T11:01:37.976378+0000","last_unstale":"2023-11-13T11:01:38.997924+0000","last_undegraded":"2023-11-13T11:01:38.997924+0000","last_fullsized":"2023-11-13T11:01:38.997924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.985125+0000","last_change":"2023-11-13T11:01:38.983595+0000","last_active":"2023-11-13T11:01:38.985125+0000","last_peered":"2023-11-13T11:01:38.985125+0000","last_clean":"2023-11-13T11:01:38.985125+0000","last_became_active":"2023-11-13T11:01:38.978958+0000","last_became_peered":"2023-11-13T11:01:38.978958+0000","last_unstale":"2023-11-13T11:01:38.985125+0000","last_undegraded":"2023-11-13T11:01:38.985125+0000","last_fullsized":"2023-11-13T11:01:38.985125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:52:25.507954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.008452+0000","last_change":"2023-11-13T11:01:37.983799+0000","last_active":"2023-11-13T11:01:39.008452+0000","last_peered":"2023-11-13T11:01:39.008452+0000","last_clean":"2023-11-13T11:01:39.008452+0000","last_became_active":"2023-11-13T11:01:37.983048+0000","last_became_peered":"2023-11-13T11:01:37.983048+0000","last_unstale":"2023-11-13T11:01:39.008452+0000","last_undegraded":"2023-11-13T11:01:39.008452+0000","last_fullsized":"2023-11-13T11:01:39.008452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:43.793217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979764+0000","last_change":"2023-11-13T11:01:38.972065+0000","last_active":"2023-11-13T11:01:38.979764+0000","last_peered":"2023-11-13T11:01:38.979764+0000","last_clean":"2023-11-13T11:01:38.979764+0000","last_became_active":"2023-11-13T11:01:38.971764+0000","last_became_peered":"2023-11-13T11:01:38.971764+0000","last_unstale":"2023-11-13T11:01:38.979764+0000","last_undegraded":"2023-11-13T11:01:38.979764+0000","last_fullsized":"2023-11-13T11:01:38.979764+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:27:51.217444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000285+0000","last_change":"2023-11-13T11:01:37.976954+0000","last_active":"2023-11-13T11:01:39.000285+0000","last_peered":"2023-11-13T11:01:39.000285+0000","last_clean":"2023-11-13T11:01:39.000285+0000","last_became_active":"2023-11-13T11:01:37.976659+0000","last_became_peered":"2023-11-13T11:01:37.976659+0000","last_unstale":"2023-11-13T11:01:39.000285+0000","last_undegraded":"2023-11-13T11:01:39.000285+0000","last_fullsized":"2023-11-13T11:01:39.000285+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:02:54.853648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000253+0000","last_change":"2023-11-13T11:01:38.995557+0000","last_active":"2023-11-13T11:01:39.000253+0000","last_peered":"2023-11-13T11:01:39.000253+0000","last_clean":"2023-11-13T11:01:39.000253+0000","last_became_active":"2023-11-13T11:01:38.995224+0000","last_became_peered":"2023-11-13T11:01:38.995224+0000","last_unstale":"2023-11-13T11:01:39.000253+0000","last_undegraded":"2023-11-13T11:01:39.000253+0000","last_fullsized":"2023-11-13T11:01:39.000253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.959098+0000","last_change":"2023-11-13T11:01:37.970890+0000","last_active":"2023-11-13T11:01:38.959098+0000","last_peered":"2023-11-13T11:01:38.959098+0000","last_clean":"2023-11-13T11:01:38.959098+0000","last_became_active":"2023-11-13T11:01:37.970585+0000","last_became_peered":"2023-11-13T11:01:37.970585+0000","last_unstale":"2023-11-13T11:01:38.959098+0000","last_undegraded":"2023-11-13T11:01:38.959098+0000","last_fullsized":"2023-11-13T11:01:38.959098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:24.358423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011948+0000","last_change":"2023-11-13T11:01:38.994877+0000","last_active":"2023-11-13T11:01:41.011948+0000","last_peered":"2023-11-13T11:01:41.011948+0000","last_clean":"2023-11-13T11:01:41.011948+0000","last_became_active":"2023-11-13T11:01:38.994450+0000","last_became_peered":"2023-11-13T11:01:38.994450+0000","last_unstale":"2023-11-13T11:01:41.011948+0000","last_undegraded":"2023-11-13T11:01:41.011948+0000","last_fullsized":"2023-11-13T11:01:41.011948+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:41:55.799097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.961699+0000","last_change":"2023-11-13T11:01:37.966418+0000","last_active":"2023-11-13T11:01:38.961699+0000","last_peered":"2023-11-13T11:01:38.961699+0000","last_clean":"2023-11-13T11:01:38.961699+0000","last_became_active":"2023-11-13T11:01:37.966105+0000","last_became_peered":"2023-11-13T11:01:37.966105+0000","last_unstale":"2023-11-13T11:01:38.961699+0000","last_undegraded":"2023-11-13T11:01:38.961699+0000","last_fullsized":"2023-11-13T11:01:38.961699+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:44:09.646957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991081+0000","last_change":"2023-11-13T11:01:38.985149+0000","last_active":"2023-11-13T11:01:38.991081+0000","last_peered":"2023-11-13T11:01:38.991081+0000","last_clean":"2023-11-13T11:01:38.991081+0000","last_became_active":"2023-11-13T11:01:38.984828+0000","last_became_peered":"2023-11-13T11:01:38.984828+0000","last_unstale":"2023-11-13T11:01:38.991081+0000","last_undegraded":"2023-11-13T11:01:38.991081+0000","last_fullsized":"2023-11-13T11:01:38.991081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:27:51.217444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.007772+0000","last_change":"2023-11-13T11:01:37.983844+0000","last_active":"2023-11-13T11:01:39.007772+0000","last_peered":"2023-11-13T11:01:39.007772+0000","last_clean":"2023-11-13T11:01:39.007772+0000","last_became_active":"2023-11-13T11:01:37.982971+0000","last_became_peered":"2023-11-13T11:01:37.982971+0000","last_unstale":"2023-11-13T11:01:39.007772+0000","last_undegraded":"2023-11-13T11:01:39.007772+0000","last_fullsized":"2023-11-13T11:01:39.007772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:56:05.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990249+0000","last_change":"2023-11-13T11:01:38.984132+0000","last_active":"2023-11-13T11:01:38.990249+0000","last_peered":"2023-11-13T11:01:38.990249+0000","last_clean":"2023-11-13T11:01:38.990249+0000","last_became_active":"2023-11-13T11:01:38.981507+0000","last_became_peered":"2023-11-13T11:01:38.981507+0000","last_unstale":"2023-11-13T11:01:38.990249+0000","last_undegraded":"2023-11-13T11:01:38.990249+0000","last_fullsized":"2023-11-13T11:01:38.990249+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:19:31.415898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005279+0000","last_change":"2023-11-13T11:01:37.986267+0000","last_active":"2023-11-13T11:01:39.005279+0000","last_peered":"2023-11-13T11:01:39.005279+0000","last_clean":"2023-11-13T11:01:39.005279+0000","last_became_active":"2023-11-13T11:01:37.985787+0000","last_became_peered":"2023-11-13T11:01:37.985787+0000","last_unstale":"2023-11-13T11:01:39.005279+0000","last_undegraded":"2023-11-13T11:01:39.005279+0000","last_fullsized":"2023-11-13T11:01:39.005279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:10:32.081040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.992067+0000","last_change":"2023-11-13T11:01:38.984798+0000","last_active":"2023-11-13T11:01:38.992067+0000","last_peered":"2023-11-13T11:01:38.992067+0000","last_clean":"2023-11-13T11:01:38.992067+0000","last_became_active":"2023-11-13T11:01:38.984518+0000","last_became_peered":"2023-11-13T11:01:38.984518+0000","last_unstale":"2023-11-13T11:01:38.992067+0000","last_undegraded":"2023-11-13T11:01:38.992067+0000","last_fullsized":"2023-11-13T11:01:38.992067+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.004284+0000","last_change":"2023-11-13T11:01:37.977771+0000","last_active":"2023-11-13T11:01:39.004284+0000","last_peered":"2023-11-13T11:01:39.004284+0000","last_clean":"2023-11-13T11:01:39.004284+0000","last_became_active":"2023-11-13T11:01:37.977506+0000","last_became_peered":"2023-11-13T11:01:37.977506+0000","last_unstale":"2023-11-13T11:01:39.004284+0000","last_undegraded":"2023-11-13T11:01:39.004284+0000","last_fullsized":"2023-11-13T11:01:39.004284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:43:08.322589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.995489+0000","last_change":"2023-11-13T11:01:38.994873+0000","last_active":"2023-11-13T11:01:38.995489+0000","last_peered":"2023-11-13T11:01:38.995489+0000","last_clean":"2023-11-13T11:01:38.995489+0000","last_became_active":"2023-11-13T11:01:38.994490+0000","last_became_peered":"2023-11-13T11:01:38.994490+0000","last_unstale":"2023-11-13T11:01:38.995489+0000","last_undegraded":"2023-11-13T11:01:38.995489+0000","last_fullsized":"2023-11-13T11:01:38.995489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:46:41.523607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997386+0000","last_change":"2023-11-13T11:01:37.982308+0000","last_active":"2023-11-13T11:01:38.997386+0000","last_peered":"2023-11-13T11:01:38.997386+0000","last_clean":"2023-11-13T11:01:38.997386+0000","last_became_active":"2023-11-13T11:01:37.981963+0000","last_became_peered":"2023-11-13T11:01:37.981963+0000","last_unstale":"2023-11-13T11:01:38.997386+0000","last_undegraded":"2023-11-13T11:01:38.997386+0000","last_fullsized":"2023-11-13T11:01:38.997386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011071+0000","last_change":"2023-11-13T11:01:38.998159+0000","last_active":"2023-11-13T11:01:41.011071+0000","last_peered":"2023-11-13T11:01:41.011071+0000","last_clean":"2023-11-13T11:01:41.011071+0000","last_became_active":"2023-11-13T11:01:38.997579+0000","last_became_peered":"2023-11-13T11:01:38.997579+0000","last_unstale":"2023-11-13T11:01:41.011071+0000","last_undegraded":"2023-11-13T11:01:41.011071+0000","last_fullsized":"2023-11-13T11:01:41.011071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:52:42.280287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.966021+0000","last_change":"2023-11-13T11:01:37.978277+0000","last_active":"2023-11-13T11:01:38.966021+0000","last_peered":"2023-11-13T11:01:38.966021+0000","last_clean":"2023-11-13T11:01:38.966021+0000","last_became_active":"2023-11-13T11:01:37.977659+0000","last_became_peered":"2023-11-13T11:01:37.977659+0000","last_unstale":"2023-11-13T11:01:38.966021+0000","last_undegraded":"2023-11-13T11:01:38.966021+0000","last_fullsized":"2023-11-13T11:01:38.966021+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:56.135585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977517+0000","last_change":"2023-11-13T11:01:38.975186+0000","last_active":"2023-11-13T11:01:38.977517+0000","last_peered":"2023-11-13T11:01:38.977517+0000","last_clean":"2023-11-13T11:01:38.977517+0000","last_became_active":"2023-11-13T11:01:38.972436+0000","last_became_peered":"2023-11-13T11:01:38.972436+0000","last_unstale":"2023-11-13T11:01:38.977517+0000","last_undegraded":"2023-11-13T11:01:38.977517+0000","last_fullsized":"2023-11-13T11:01:38.977517+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:45:08.666322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000163+0000","last_change":"2023-11-13T11:01:37.970760+0000","last_active":"2023-11-13T11:01:39.000163+0000","last_peered":"2023-11-13T11:01:39.000163+0000","last_clean":"2023-11-13T11:01:39.000163+0000","last_became_active":"2023-11-13T11:01:37.970465+0000","last_became_peered":"2023-11-13T11:01:37.970465+0000","last_unstale":"2023-11-13T11:01:39.000163+0000","last_undegraded":"2023-11-13T11:01:39.000163+0000","last_fullsized":"2023-11-13T11:01:39.000163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.979277+0000","last_change":"2023-11-13T11:01:38.972817+0000","last_active":"2023-11-13T11:01:38.979277+0000","last_peered":"2023-11-13T11:01:38.979277+0000","last_clean":"2023-11-13T11:01:38.979277+0000","last_became_active":"2023-11-13T11:01:38.971691+0000","last_became_peered":"2023-11-13T11:01:38.971691+0000","last_unstale":"2023-11-13T11:01:38.979277+0000","last_undegraded":"2023-11-13T11:01:38.979277+0000","last_fullsized":"2023-11-13T11:01:38.979277+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.978810+0000","last_change":"2023-11-13T11:01:37.982283+0000","last_active":"2023-11-13T11:01:38.978810+0000","last_peered":"2023-11-13T11:01:38.978810+0000","last_clean":"2023-11-13T11:01:38.978810+0000","last_became_active":"2023-11-13T11:01:37.981958+0000","last_became_peered":"2023-11-13T11:01:37.981958+0000","last_unstale":"2023-11-13T11:01:38.978810+0000","last_undegraded":"2023-11-13T11:01:38.978810+0000","last_fullsized":"2023-11-13T11:01:38.978810+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:44:29.604069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.991795+0000","last_change":"2023-11-13T11:01:38.982113+0000","last_active":"2023-11-13T11:01:38.991795+0000","last_peered":"2023-11-13T11:01:38.991795+0000","last_clean":"2023-11-13T11:01:38.991795+0000","last_became_active":"2023-11-13T11:01:38.980795+0000","last_became_peered":"2023-11-13T11:01:38.980795+0000","last_unstale":"2023-11-13T11:01:38.991795+0000","last_undegraded":"2023-11-13T11:01:38.991795+0000","last_fullsized":"2023-11-13T11:01:38.991795+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.973311+0000","last_change":"2023-11-13T11:01:37.982552+0000","last_active":"2023-11-13T11:01:38.973311+0000","last_peered":"2023-11-13T11:01:38.973311+0000","last_clean":"2023-11-13T11:01:38.973311+0000","last_became_active":"2023-11-13T11:01:37.982269+0000","last_became_peered":"2023-11-13T11:01:37.982269+0000","last_unstale":"2023-11-13T11:01:38.973311+0000","last_undegraded":"2023-11-13T11:01:38.973311+0000","last_fullsized":"2023-11-13T11:01:38.973311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.012056+0000","last_change":"2023-11-13T11:01:38.995167+0000","last_active":"2023-11-13T11:01:41.012056+0000","last_peered":"2023-11-13T11:01:41.012056+0000","last_clean":"2023-11-13T11:01:41.012056+0000","last_became_active":"2023-11-13T11:01:38.994892+0000","last_became_peered":"2023-11-13T11:01:38.994892+0000","last_unstale":"2023-11-13T11:01:41.012056+0000","last_undegraded":"2023-11-13T11:01:41.012056+0000","last_fullsized":"2023-11-13T11:01:41.012056+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:28:05.690927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.975254+0000","last_change":"2023-11-13T11:01:37.971616+0000","last_active":"2023-11-13T11:01:38.975254+0000","last_peered":"2023-11-13T11:01:38.975254+0000","last_clean":"2023-11-13T11:01:38.975254+0000","last_became_active":"2023-11-13T11:01:37.971344+0000","last_became_peered":"2023-11-13T11:01:37.971344+0000","last_unstale":"2023-11-13T11:01:38.975254+0000","last_undegraded":"2023-11-13T11:01:38.975254+0000","last_fullsized":"2023-11-13T11:01:38.975254+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:31.732306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986510+0000","last_change":"2023-11-13T11:01:38.983983+0000","last_active":"2023-11-13T11:01:38.986510+0000","last_peered":"2023-11-13T11:01:38.986510+0000","last_clean":"2023-11-13T11:01:38.986510+0000","last_became_active":"2023-11-13T11:01:38.983698+0000","last_became_peered":"2023-11-13T11:01:38.983698+0000","last_unstale":"2023-11-13T11:01:38.986510+0000","last_undegraded":"2023-11-13T11:01:38.986510+0000","last_fullsized":"2023-11-13T11:01:38.986510+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:46:41.523607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.997353+0000","last_change":"2023-11-13T11:01:37.972256+0000","last_active":"2023-11-13T11:01:38.997353+0000","last_peered":"2023-11-13T11:01:38.997353+0000","last_clean":"2023-11-13T11:01:38.997353+0000","last_became_active":"2023-11-13T11:01:37.971890+0000","last_became_peered":"2023-11-13T11:01:37.971890+0000","last_unstale":"2023-11-13T11:01:38.997353+0000","last_undegraded":"2023-11-13T11:01:38.997353+0000","last_fullsized":"2023-11-13T11:01:38.997353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:38.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990549+0000","last_change":"2023-11-13T11:01:38.988092+0000","last_active":"2023-11-13T11:01:38.990549+0000","last_peered":"2023-11-13T11:01:38.990549+0000","last_clean":"2023-11-13T11:01:38.990549+0000","last_became_active":"2023-11-13T11:01:38.987800+0000","last_became_peered":"2023-11-13T11:01:38.987800+0000","last_unstale":"2023-11-13T11:01:38.990549+0000","last_undegraded":"2023-11-13T11:01:38.990549+0000","last_fullsized":"2023-11-13T11:01:38.990549+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:28.542546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.964559+0000","last_change":"2023-11-13T11:01:37.973251+0000","last_active":"2023-11-13T11:01:38.964559+0000","last_peered":"2023-11-13T11:01:38.964559+0000","last_clean":"2023-11-13T11:01:38.964559+0000","last_became_active":"2023-11-13T11:01:37.972947+0000","last_became_peered":"2023-11-13T11:01:37.972947+0000","last_unstale":"2023-11-13T11:01:38.964559+0000","last_undegraded":"2023-11-13T11:01:38.964559+0000","last_fullsized":"2023-11-13T11:01:38.964559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993910+0000","last_change":"2023-11-13T11:01:38.988853+0000","last_active":"2023-11-13T11:01:38.993910+0000","last_peered":"2023-11-13T11:01:38.993910+0000","last_clean":"2023-11-13T11:01:38.993910+0000","last_became_active":"2023-11-13T11:01:38.987558+0000","last_became_peered":"2023-11-13T11:01:38.987558+0000","last_unstale":"2023-11-13T11:01:38.993910+0000","last_undegraded":"2023-11-13T11:01:38.993910+0000","last_fullsized":"2023-11-13T11:01:38.993910+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:38:32.234508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.986162+0000","last_change":"2023-11-13T11:01:37.981807+0000","last_active":"2023-11-13T11:01:38.986162+0000","last_peered":"2023-11-13T11:01:38.986162+0000","last_clean":"2023-11-13T11:01:38.986162+0000","last_became_active":"2023-11-13T11:01:37.981480+0000","last_became_peered":"2023-11-13T11:01:37.981480+0000","last_unstale":"2023-11-13T11:01:38.986162+0000","last_undegraded":"2023-11-13T11:01:38.986162+0000","last_fullsized":"2023-11-13T11:01:38.986162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:09:55.684221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.999643+0000","last_change":"2023-11-13T11:01:38.986410+0000","last_active":"2023-11-13T11:01:38.999643+0000","last_peered":"2023-11-13T11:01:38.999643+0000","last_clean":"2023-11-13T11:01:38.999643+0000","last_became_active":"2023-11-13T11:01:38.986131+0000","last_became_peered":"2023-11-13T11:01:38.986131+0000","last_unstale":"2023-11-13T11:01:38.999643+0000","last_undegraded":"2023-11-13T11:01:38.999643+0000","last_fullsized":"2023-11-13T11:01:38.999643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:59:36.431142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.975456+0000","last_change":"2023-11-13T11:01:37.975627+0000","last_active":"2023-11-13T11:01:38.975456+0000","last_peered":"2023-11-13T11:01:38.975456+0000","last_clean":"2023-11-13T11:01:38.975456+0000","last_became_active":"2023-11-13T11:01:37.974747+0000","last_became_peered":"2023-11-13T11:01:37.974747+0000","last_unstale":"2023-11-13T11:01:38.975456+0000","last_undegraded":"2023-11-13T11:01:38.975456+0000","last_fullsized":"2023-11-13T11:01:38.975456+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.990137+0000","last_change":"2023-11-13T11:01:38.989266+0000","last_active":"2023-11-13T11:01:38.990137+0000","last_peered":"2023-11-13T11:01:38.990137+0000","last_clean":"2023-11-13T11:01:38.990137+0000","last_became_active":"2023-11-13T11:01:38.988948+0000","last_became_peered":"2023-11-13T11:01:38.988948+0000","last_unstale":"2023-11-13T11:01:38.990137+0000","last_undegraded":"2023-11-13T11:01:38.990137+0000","last_fullsized":"2023-11-13T11:01:38.990137+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:12.303326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.978974+0000","last_change":"2023-11-13T11:01:37.982077+0000","last_active":"2023-11-13T11:01:38.978974+0000","last_peered":"2023-11-13T11:01:38.978974+0000","last_clean":"2023-11-13T11:01:38.978974+0000","last_became_active":"2023-11-13T11:01:37.981552+0000","last_became_peered":"2023-11-13T11:01:37.981552+0000","last_unstale":"2023-11-13T11:01:38.978974+0000","last_undegraded":"2023-11-13T11:01:38.978974+0000","last_fullsized":"2023-11-13T11:01:38.978974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.993927+0000","last_change":"2023-11-13T11:01:38.993394+0000","last_active":"2023-11-13T11:01:38.993927+0000","last_peered":"2023-11-13T11:01:38.993927+0000","last_clean":"2023-11-13T11:01:38.993927+0000","last_became_active":"2023-11-13T11:01:38.992745+0000","last_became_peered":"2023-11-13T11:01:38.992745+0000","last_unstale":"2023-11-13T11:01:38.993927+0000","last_undegraded":"2023-11-13T11:01:38.993927+0000","last_fullsized":"2023-11-13T11:01:38.993927+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:57.148879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.961654+0000","last_change":"2023-11-13T11:01:37.971343+0000","last_active":"2023-11-13T11:01:38.961654+0000","last_peered":"2023-11-13T11:01:38.961654+0000","last_clean":"2023-11-13T11:01:38.961654+0000","last_became_active":"2023-11-13T11:01:37.970761+0000","last_became_peered":"2023-11-13T11:01:37.970761+0000","last_unstale":"2023-11-13T11:01:38.961654+0000","last_undegraded":"2023-11-13T11:01:38.961654+0000","last_fullsized":"2023-11-13T11:01:38.961654+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:47:45.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.012076+0000","last_change":"2023-11-13T11:01:39.001466+0000","last_active":"2023-11-13T11:01:41.012076+0000","last_peered":"2023-11-13T11:01:41.012076+0000","last_clean":"2023-11-13T11:01:41.012076+0000","last_became_active":"2023-11-13T11:01:39.001200+0000","last_became_peered":"2023-11-13T11:01:39.001200+0000","last_unstale":"2023-11-13T11:01:41.012076+0000","last_undegraded":"2023-11-13T11:01:41.012076+0000","last_fullsized":"2023-11-13T11:01:41.012076+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:28:45.331614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.956737+0000","last_change":"2023-11-13T11:01:37.970057+0000","last_active":"2023-11-13T11:01:38.956737+0000","last_peered":"2023-11-13T11:01:38.956737+0000","last_clean":"2023-11-13T11:01:38.956737+0000","last_became_active":"2023-11-13T11:01:37.969819+0000","last_became_peered":"2023-11-13T11:01:37.969819+0000","last_unstale":"2023-11-13T11:01:38.956737+0000","last_undegraded":"2023-11-13T11:01:38.956737+0000","last_fullsized":"2023-11-13T11:01:38.956737+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:45:07.653027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.968087+0000","last_change":"2023-11-13T11:01:38.966353+0000","last_active":"2023-11-13T11:01:38.968087+0000","last_peered":"2023-11-13T11:01:38.968087+0000","last_clean":"2023-11-13T11:01:38.968087+0000","last_became_active":"2023-11-13T11:01:38.966082+0000","last_became_peered":"2023-11-13T11:01:38.966082+0000","last_unstale":"2023-11-13T11:01:38.968087+0000","last_undegraded":"2023-11-13T11:01:38.968087+0000","last_fullsized":"2023-11-13T11:01:38.968087+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.977074+0000","last_change":"2023-11-13T11:01:37.970911+0000","last_active":"2023-11-13T11:01:38.977074+0000","last_peered":"2023-11-13T11:01:38.977074+0000","last_clean":"2023-11-13T11:01:38.977074+0000","last_became_active":"2023-11-13T11:01:37.970627+0000","last_became_peered":"2023-11-13T11:01:37.970627+0000","last_unstale":"2023-11-13T11:01:38.977074+0000","last_undegraded":"2023-11-13T11:01:38.977074+0000","last_fullsized":"2023-11-13T11:01:38.977074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-13T11:01:37.949346+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:01:37.949346+0000","last_peered":"2023-11-13T11:01:37.949346+0000","last_clean":"2023-11-13T11:01:37.949346+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:01:37.949346+0000","last_undegraded":"2023-11-13T11:01:37.949346+0000","last_fullsized":"2023-11-13T11:01:37.949346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0960000000000001,"15min":1.1259999999999999},"min":{"1min":0.84599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.21,"15min":1.1339999999999999},"min":{"1min":0.89800000000000002,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.865,"5min":1.865,"15min":2.2879999999999998},"last":1.353}]},{"osd":1,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.22,"15min":1.175},"min":{"1min":0.91700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8200000000000001,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2969999999999999,"15min":1.2889999999999999},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1070000000000002,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.2789999999999999,"15min":1.2769999999999999},"min":{"1min":0.98099999999999998,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.153,"5min":2.153,"15min":2.1680000000000001},"last":1.244},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3819999999999999,"15min":1.3520000000000001},"min":{"1min":1.1080000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.2400000000000002,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.298}]},{"osd":3,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.407,"15min":1.383},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.63800000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.423,"15min":1.4390000000000001},"min":{"1min":1.167,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.524}]},{"osd":4,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.454,"15min":1.5720000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.577,"5min":1.5129999999999999,"15min":1.5940000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.202}]},{"osd":5,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6000000000000001,"15min":1.6559999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.405},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.72,"15min":1.7589999999999999},"min":{"1min":1.147,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.6339999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.758,"5min":1.659,"15min":1.651},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.754,"5min":1.617,"15min":1.6499999999999999},"min":{"1min":0.96499999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.5940000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1216512,"data_stored":863292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.107,"15min":1.0129999999999999},"min":{"1min":0.82999999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9099999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2070000000000001,"15min":1.2030000000000001},"min":{"1min":0.74299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.639}]},{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.278,"15min":1.2310000000000001},"min":{"1min":0.89800000000000002,"5min":0.71699999999999997,"15min":0.65400000000000003},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.234,"15min":1.238},"min":{"1min":0.878,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8580000000000001,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.2250000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3129999999999999,"15min":1.2509999999999999},"min":{"1min":0.92700000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":1.099},{"interface":"front","average":{"1min":1.375,"5min":1.3120000000000001,"15min":1.2370000000000001},"min":{"1min":0.81599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.5209999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.393,"15min":1.399},"min":{"1min":0.86399999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80700000000000005},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":1.339}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3640000000000001,"15min":1.3340000000000001},"min":{"1min":0.86599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.294,"5min":3.484,"15min":3.484},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.4830000000000001,"15min":1.5349999999999999},"min":{"1min":1.123,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":1.4810000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5049999999999999,"15min":1.6080000000000001},"min":{"1min":1.1539999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.4929999999999999,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.538,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.6120000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6359999999999999,"15min":1.6799999999999999},"min":{"1min":1.167,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.7},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.653,"15min":1.714},"min":{"1min":1.2190000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.4410000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49840,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322000,"statfs":{"total":96636764160,"available":96585728000,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.1599999999999999,"15min":1.0820000000000001},"min":{"1min":0.86399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.103,"15min":1.075},"min":{"1min":0.94399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.218,"15min":1.1579999999999999},"min":{"1min":0.65800000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.7689999999999999,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2689999999999999,"15min":1.2549999999999999},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.821,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.998}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.27,"15min":1.2350000000000001},"min":{"1min":0.98899999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.3200000000000001,"15min":1.3480000000000001},"min":{"1min":0.996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.327}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.46,"15min":1.4139999999999999},"min":{"1min":1.1479999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":2.0049999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.98599999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.742,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.472,"15min":1.502},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0110000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.492},{"interface":"front","average":{"1min":1.504,"5min":1.3480000000000001,"15min":1.401},"min":{"1min":0.98399999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.1910000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.605,"15min":1.629},"min":{"1min":1.0329999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4850000000000001,"15min":1.5389999999999999},"min":{"1min":1.1459999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.6299999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":1.516,"15min":1.591},"min":{"1min":1.139,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.6040000000000001,"15min":1.621},"min":{"1min":1.1020000000000001,"5min":0.63,"15min":0.63},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.571}]}]},{"osd":3,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40124,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331716,"statfs":{"total":96636764160,"available":96595677184,"internally_reserved":0,"allocated":638976,"data_stored":280109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.85299999999999998,"15min":0.86499999999999999},"min":{"1min":0.53000000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.843,"5min":1.867,"15min":1.867},"last":0.68100000000000005},{"interface":"front","average":{"1min":1,"5min":0.98699999999999999,"15min":1.018},"min":{"1min":0.74399999999999999,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4239999999999999,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":0.80300000000000005}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.98599999999999999,"15min":1.04},"min":{"1min":0.65800000000000003,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.603,"5min":1.966,"15min":3.71},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.077,"15min":1.089},"min":{"1min":0.54400000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5660000000000001,"5min":1.923,"15min":2.0190000000000001},"last":0.92300000000000004}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.177,"15min":1.097},"min":{"1min":0.73599999999999999,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":1.5049999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.2909999999999999},"min":{"1min":0.83199999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5389999999999999,"5min":1.952,"15min":3.5710000000000002},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3109999999999999,"15min":1.4179999999999999},"min":{"1min":0.97199999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.5640000000000001,"5min":2.411,"15min":3.5030000000000001},"last":1.143},{"interface":"front","average":{"1min":1.49,"5min":1.345,"15min":1.349},"min":{"1min":1.1319999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.9370000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.615}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.379,"15min":1.476},"min":{"1min":1.073,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.7270000000000001,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.456},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4219999999999999,"15min":1.5149999999999999},"min":{"1min":0.86799999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.72,"5min":2.673,"15min":3.3809999999999998},"last":1.486}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4510000000000001,"15min":1.4870000000000001},"min":{"1min":1.1910000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8260000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.218},{"interface":"front","average":{"1min":1.446,"5min":1.4590000000000001,"15min":1.591},"min":{"1min":0.97099999999999997,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":1.855,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.3109999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.512,"15min":1.6120000000000001},"min":{"1min":1.1819999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.968,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.347},{"interface":"front","average":{"1min":1.647,"5min":1.5680000000000001,"15min":1.5549999999999999},"min":{"1min":1.319,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0059999999999998,"5min":2.75,"15min":2.75},"last":1.5369999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45680,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326160,"statfs":{"total":96636764160,"available":96589987840,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.89500000000000002,"15min":0.92900000000000005},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.9410000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.63},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.95699999999999996,"15min":0.92100000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.7789999999999999,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":0.54400000000000004}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":0.96299999999999997,"15min":0.95199999999999996},"min":{"1min":0.82999999999999996,"5min":0.376,"15min":0.376},"max":{"1min":1.893,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.105,"15min":1.127},"min":{"1min":0.68400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.978,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.133}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.139,"15min":1.2430000000000001},"min":{"1min":0.755,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.796,"5min":2.129,"15min":2.129},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.22,"15min":1.2729999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.867,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":0.98399999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.391,"15min":1.4750000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.9970000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3100000000000001,"15min":1.4039999999999999},"min":{"1min":0.79200000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0449999999999999,"5min":2.3639999999999999,"15min":2.6179999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.3959999999999999,"15min":1.48},"min":{"1min":0.96599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.6600000000000001},"last":1.034},{"interface":"front","average":{"1min":1.633,"5min":1.5409999999999999,"15min":1.5649999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.2450000000000001,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.3240000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.4359999999999999,"15min":1.55},"min":{"1min":0.83099999999999996,"5min":0.78500000000000003,"15min":0.76500000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":4.7569999999999997},"last":1.085},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.538,"15min":1.595},"min":{"1min":1.0229999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.621},"last":1.3620000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.488,"15min":1.6100000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.343,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.573,"15min":1.623},"min":{"1min":1.0740000000000001,"5min":0.85699999999999998,"15min":0.82799999999999996},"max":{"1min":2.2999999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.2909999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509519,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1216512,"data_stored":863292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.84099999999999997,"15min":0.95599999999999996},"min":{"1min":0.59299999999999997,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.47,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.105,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.754}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":0.99099999999999999,"15min":0.98299999999999998},"min":{"1min":0.71599999999999997,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0229999999999999,"15min":0.999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41999999999999998},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.722},"last":1.0029999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7629999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.101,"5min":1,"15min":1.0580000000000001},"min":{"1min":0.57899999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6899999999999999,"5min":1.8149999999999999,"15min":1.9059999999999999},"last":0.91100000000000003}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":0.98399999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.0569999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.363,"15min":1.375},"min":{"1min":1.0860000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.966,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.2290000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.3759999999999999,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83299999999999996},"max":{"1min":2.1629999999999998,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.103},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.4099999999999999},"min":{"1min":0.93999999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.0390000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2909999999999999,"15min":1.407},"min":{"1min":0.91400000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.986,"5min":1.986,"15min":2.3519999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.4339999999999999,"15min":1.4990000000000001},"min":{"1min":0.89800000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.3879999999999999},"last":1.3520000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.4410000000000001,"15min":1.496},"min":{"1min":0.93799999999999994,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.972,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.391,"15min":1.452},"min":{"1min":0.85799999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8660000000000001,"5min":2.202,"15min":2.202},"last":1.3300000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509520,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40108,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331732,"statfs":{"total":96636764160,"available":96595693568,"internally_reserved":0,"allocated":622592,"data_stored":272924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.90100000000000002},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32100000000000001},"max":{"1min":1.417,"5min":1.74,"15min":1.74},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.022,"15min":1.048},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.7330000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1459999999999999,"15min":1.141},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0089999999999999,"15min":1.018},"min":{"1min":0.54000000000000004,"5min":0.495,"15min":0.371},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.1669999999999998},"last":1.129}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1080000000000001,"15min":1.1020000000000001},"min":{"1min":0.63800000000000001,"5min":0.48999999999999999,"15min":0.45200000000000001},"max":{"1min":1.7450000000000001,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2769999999999999,"15min":1.329},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.827,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.0760000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3300000000000001,"15min":1.3540000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8009999999999999,"5min":2.153,"15min":2.153},"last":4.4279999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.296,"15min":1.4279999999999999},"min":{"1min":0.79400000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.708,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.5029999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.415,"15min":1.5449999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0430000000000001,"5min":2.2719999999999998,"15min":3.7490000000000001},"last":4.0449999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.3839999999999999,"15min":1.484},"min":{"1min":0.89100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.867,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":2.0310000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.48,"15min":1.5229999999999999},"min":{"1min":0.91100000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1389999999999998,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":4.617},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5369999999999999,"15min":1.5740000000000001},"min":{"1min":0.872,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.2810000000000001,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":3.6520000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.5900000000000001,"15min":1.6779999999999999},"min":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.21,"5min":2.742,"15min":3.3540000000000001},"last":1.522},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5800000000000001,"15min":1.6140000000000001},"min":{"1min":0.94999999999999996,"5min":0.88100000000000001,"15min":0.77400000000000002},"max":{"1min":2.2200000000000002,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":3.976}]}]},{"osd":7,"up_from":14,"seq":60129542225,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45708,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326132,"statfs":{"total":96636764160,"available":96589959168,"internally_reserved":0,"allocated":655360,"data_stored":286695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0860000000000001,"15min":1.1120000000000001},"min":{"1min":0.77800000000000002,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6970000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.087,"15min":1.073},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6080000000000001,"5min":2.266,"15min":2.266},"last":1.0620000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.147,"15min":1.0589999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6140000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.294},{"interface":"front","average":{"1min":1.339,"5min":1.2450000000000001,"15min":1.2310000000000001},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3939999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2929999999999999,"15min":1.3029999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8080000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.264,"15min":1.2190000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.4390000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.341,"15min":1.2569999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.829,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.534},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5169999999999999,"15min":1.46},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.925,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.5089999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3680000000000001,"15min":1.3300000000000001},"min":{"1min":0.95199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.907,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":1.337},{"interface":"front","average":{"1min":1.395,"5min":1.4359999999999999,"15min":1.379},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.8420000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":1.1579999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.5389999999999999,"15min":1.5429999999999999},"min":{"1min":0.92800000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.0640000000000001,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.534,"15min":1.4830000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.9870000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.2010000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.641,"15min":1.5760000000000001},"min":{"1min":0.84899999999999998,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":2.093,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.569},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.5920000000000001,"15min":1.518},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.2349999999999999,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":1.4790000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:01:48.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:01:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.883+0000 7fc80bfff640 1 Processor -- start 2023-11-13T11:01:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.883+0000 7fc80bfff640 1 -- start start 2023-11-13T11:01:48.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.885+0000 7fc80bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc80c0cb190 0x7fc80c15b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:48.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.885+0000 7fc80bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc80c0716b0 0x7fc80c06f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:48.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.885+0000 7fc80bfff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80c158c20 con 0x7fc80c06a190 2023-11-13T11:01:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.885+0000 7fc80bfff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80c0b1830 con 0x7fc80c0cb190 2023-11-13T11:01:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.885+0000 7fc80bfff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80c15b780 con 0x7fc80c0716b0 2023-11-13T11:01:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.885+0000 7fc80affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc80c0cb190 0x7fc80c15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.885+0000 7fc80affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc80c0cb190 0x7fc80c15b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41992/0 (socket says 172.21.15.16:41992) 2023-11-13T11:01:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.885+0000 7fc80affd640 1 -- 172.21.15.16:0/3741588301 learned_addr learned my addr 172.21.15.16:0/3741588301 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.886+0000 7fc80a7fc640 1 --2- 172.21.15.16:0/3741588301 >> v2:172.21.15.184:3300/0 conn(0x7fc80c0716b0 0x7fc80c06f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.886+0000 7fc80affd640 1 -- 172.21.15.16:0/3741588301 >> v2:172.21.15.184:3300/0 conn(0x7fc80c0716b0 msgr2=0x7fc80c06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.886+0000 7fc80affd640 1 --2- 172.21.15.16:0/3741588301 >> v2:172.21.15.184:3300/0 conn(0x7fc80c0716b0 0x7fc80c06f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:48.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.886+0000 7fc80affd640 1 -- 172.21.15.16:0/3741588301 >> v1:172.21.15.184:6789/0 conn(0x7fc80c06a190 legacy=0x7fc80c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:48.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.886+0000 7fc80affd640 1 -- 172.21.15.16:0/3741588301 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c15ba60 con 0x7fc80c0cb190 2023-11-13T11:01:48.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.887+0000 7fc80affd640 1 --2- 172.21.15.16:0/3741588301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 0x7fc80c15b190 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0ae3e0 tx=0x7fc7fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf30ca047ba97ed server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:48.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.887+0000 7fc809ffb640 1 -- 172.21.15.16:0/3741588301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc031870 con 0x7fc80c0cb190 2023-11-13T11:01:48.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.887+0000 7fc809ffb640 1 -- 172.21.15.16:0/3741588301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7fc0319d0 con 0x7fc80c0cb190 2023-11-13T11:01:48.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.887+0000 7fc809ffb640 1 -- 172.21.15.16:0/3741588301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc031d60 con 0x7fc80c0cb190 2023-11-13T11:01:48.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.888+0000 7fc80bfff640 1 -- 172.21.15.16:0/3741588301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 msgr2=0x7fc80c15b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:48.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.888+0000 7fc80bfff640 1 --2- 172.21.15.16:0/3741588301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 0x7fc80c15b190 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0ae3e0 tx=0x7fc7fc02b910 comp rx=0 tx=0).stop 2023-11-13T11:01:48.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.888+0000 7fc80bfff640 1 -- 172.21.15.16:0/3741588301 shutdown_connections 2023-11-13T11:01:48.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.888+0000 7fc80bfff640 1 --2- 172.21.15.16:0/3741588301 >> v2:172.21.15.184:3300/0 conn(0x7fc80c0716b0 0x7fc80c06f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.888+0000 7fc80bfff640 1 --2- 172.21.15.16:0/3741588301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 0x7fc80c15b190 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.888+0000 7fc80bfff640 1 -- 172.21.15.16:0/3741588301 >> 172.21.15.16:0/3741588301 conn(0x7fc80c0b04a0 msgr2=0x7fc80c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.888+0000 7fc80bfff640 1 -- 172.21.15.16:0/3741588301 shutdown_connections 2023-11-13T11:01:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.888+0000 7fc80bfff640 1 -- 172.21.15.16:0/3741588301 wait complete. 2023-11-13T11:01:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.890+0000 7fc80bfff640 1 Processor -- start 2023-11-13T11:01:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.890+0000 7fc80bfff640 1 -- start start 2023-11-13T11:01:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.891+0000 7fc80bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c06a190 0x7fc80c0aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.891+0000 7fc80bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0716b0 0x7fc80c0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.891+0000 7fc80bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 0x7fc80c1f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.891+0000 7fc80bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0a9650 con 0x7fc80c0cb190 2023-11-13T11:01:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.891+0000 7fc80bfff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c1f43c0 con 0x7fc80c06a190 2023-11-13T11:01:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.891+0000 7fc80bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c1f46a0 con 0x7fc80c0716b0 2023-11-13T11:01:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 0x7fc80c1f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 0x7fc80c1f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42006/0 (socket says 172.21.15.16:42006) 2023-11-13T11:01:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80b7fe640 1 -- 172.21.15.16:0/1403228930 learned_addr learned my addr 172.21.15.16:0/1403228930 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80affd640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c06a190 0x7fc80c0aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80b7fe640 1 -- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0716b0 msgr2=0x7fc80c0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80b7fe640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0716b0 0x7fc80c0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80b7fe640 1 -- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c06a190 msgr2=0x7fc80c0aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80b7fe640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c06a190 0x7fc80c0aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80b7fe640 1 -- 172.21.15.16:0/1403228930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc0094d0 con 0x7fc80c0cb190 2023-11-13T11:01:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80affd640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c06a190 0x7fc80c0aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.892+0000 7fc80b7fe640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 0x7fc80c1f3900 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc80400b970 tx=0x7fc80400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.893+0000 7fc7f37fe640 1 -- 172.21.15.16:0/1403228930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80400cb40 con 0x7fc80c0cb190 2023-11-13T11:01:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.893+0000 7fc7f37fe640 1 -- 172.21.15.16:0/1403228930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80400cca0 con 0x7fc80c0cb190 2023-11-13T11:01:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.893+0000 7fc7f37fe640 1 -- 172.21.15.16:0/1403228930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804014680 con 0x7fc80c0cb190 2023-11-13T11:01:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.893+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c0a9860 con 0x7fc80c0cb190 2023-11-13T11:01:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.893+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c0aa5a0 con 0x7fc80c0cb190 2023-11-13T11:01:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.894+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80c0a9d80 con 0x7fc80c0cb190 2023-11-13T11:01:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.895+0000 7fc7f37fe640 1 -- 172.21.15.16:0/1403228930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc80401d3b0 con 0x7fc80c0cb190 2023-11-13T11:01:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.896+0000 7fc7f37fe640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc7f806ae40 0x7fc7f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.896+0000 7fc7f37fe640 1 -- 172.21.15.16:0/1403228930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc8040895e0 con 0x7fc80c0cb190 2023-11-13T11:01:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.897+0000 7fc80affd640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc7f806ae40 0x7fc7f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.898+0000 7fc80affd640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc7f806ae40 0x7fc7f806d300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc036000 tx=0x7fc7fc03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:48.901+0000 7fc7f37fe640 1 -- 172.21.15.16:0/1403228930 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc80c0a9d80 con 0x7fc80c0cb190 2023-11-13T11:01:49.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.093+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc80c0cb570 con 0x7fc7f806ae40 2023-11-13T11:01:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.116+0000 7fc7f37fe640 1 -- 172.21.15.16:0/1403228930 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287301 (secure 0 0 0) 0x7fc7fc031c40 con 0x7fc7f806ae40 2023-11-13T11:01:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:49.126 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:01:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.121+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc7f806ae40 msgr2=0x7fc7f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.121+0000 7fc80bfff640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc7f806ae40 0x7fc7f806d300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc036000 tx=0x7fc7fc03c040 comp rx=0 tx=0).stop 2023-11-13T11:01:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 msgr2=0x7fc80c1f3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 0x7fc80c1f3900 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc80400b970 tx=0x7fc80400be30 comp rx=0 tx=0).stop 2023-11-13T11:01:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 shutdown_connections 2023-11-13T11:01:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc7f806ae40 0x7fc7f806d300 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c0cb190 0x7fc80c1f3900 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0716b0 0x7fc80c0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 --2- 172.21.15.16:0/1403228930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c06a190 0x7fc80c0aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 >> 172.21.15.16:0/1403228930 conn(0x7fc80c0b04a0 msgr2=0x7fc80c15a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 shutdown_connections 2023-11-13T11:01:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.122+0000 7fc80bfff640 1 -- 172.21.15.16:0/1403228930 wait complete. 2023-11-13T11:01:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2023-11-13T11:01:48.120253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":343172,"kb_used_data":6280,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754631548,"statfs":{"total":773094113280,"available":772742705152,"internally_reserved":0,"allocated":6430720,"data_stored":3478668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":344837681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.166576"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.987704+0000","last_change":"2023-11-13T11:01:38.993640+0000","last_active":"2023-11-13T11:01:39.987704+0000","last_peered":"2023-11-13T11:01:39.987704+0000","last_clean":"2023-11-13T11:01:39.987704+0000","last_became_active":"2023-11-13T11:01:38.991479+0000","last_became_peered":"2023-11-13T11:01:38.991479+0000","last_unstale":"2023-11-13T11:01:39.987704+0000","last_undegraded":"2023-11-13T11:01:39.987704+0000","last_fullsized":"2023-11-13T11:01:39.987704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:55:39.320111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.510700+0000","last_change":"2023-11-13T11:01:38.975999+0000","last_active":"2023-11-13T11:01:40.510700+0000","last_peered":"2023-11-13T11:01:40.510700+0000","last_clean":"2023-11-13T11:01:40.510700+0000","last_became_active":"2023-11-13T11:01:38.975726+0000","last_became_peered":"2023-11-13T11:01:38.975726+0000","last_unstale":"2023-11-13T11:01:40.510700+0000","last_undegraded":"2023-11-13T11:01:40.510700+0000","last_fullsized":"2023-11-13T11:01:40.510700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.886558+0000","last_change":"2023-11-13T11:01:38.984201+0000","last_active":"2023-11-13T11:01:40.886558+0000","last_peered":"2023-11-13T11:01:40.886558+0000","last_clean":"2023-11-13T11:01:40.886558+0000","last_became_active":"2023-11-13T11:01:38.983716+0000","last_became_peered":"2023-11-13T11:01:38.983716+0000","last_unstale":"2023-11-13T11:01:40.886558+0000","last_undegraded":"2023-11-13T11:01:40.886558+0000","last_fullsized":"2023-11-13T11:01:40.886558+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:35:23.626812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.982331+0000","last_change":"2023-11-13T11:01:38.975163+0000","last_active":"2023-11-13T11:01:39.982331+0000","last_peered":"2023-11-13T11:01:39.982331+0000","last_clean":"2023-11-13T11:01:39.982331+0000","last_became_active":"2023-11-13T11:01:38.974878+0000","last_became_peered":"2023-11-13T11:01:38.974878+0000","last_unstale":"2023-11-13T11:01:39.982331+0000","last_undegraded":"2023-11-13T11:01:39.982331+0000","last_fullsized":"2023-11-13T11:01:39.982331+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:01.750338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.510414+0000","last_change":"2023-11-13T11:01:38.968974+0000","last_active":"2023-11-13T11:01:40.510414+0000","last_peered":"2023-11-13T11:01:40.510414+0000","last_clean":"2023-11-13T11:01:40.510414+0000","last_became_active":"2023-11-13T11:01:38.968747+0000","last_became_peered":"2023-11-13T11:01:38.968747+0000","last_unstale":"2023-11-13T11:01:40.510414+0000","last_undegraded":"2023-11-13T11:01:40.510414+0000","last_fullsized":"2023-11-13T11:01:40.510414+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:07:48.926499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011194+0000","last_change":"2023-11-13T11:01:38.993231+0000","last_active":"2023-11-13T11:01:41.011194+0000","last_peered":"2023-11-13T11:01:41.011194+0000","last_clean":"2023-11-13T11:01:41.011194+0000","last_became_active":"2023-11-13T11:01:38.992972+0000","last_became_peered":"2023-11-13T11:01:38.992972+0000","last_unstale":"2023-11-13T11:01:41.011194+0000","last_undegraded":"2023-11-13T11:01:41.011194+0000","last_fullsized":"2023-11-13T11:01:41.011194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:45:42.458182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.988259+0000","last_change":"2023-11-13T11:01:39.003582+0000","last_active":"2023-11-13T11:01:39.988259+0000","last_peered":"2023-11-13T11:01:39.988259+0000","last_clean":"2023-11-13T11:01:39.988259+0000","last_became_active":"2023-11-13T11:01:39.003324+0000","last_became_peered":"2023-11-13T11:01:39.003324+0000","last_unstale":"2023-11-13T11:01:39.988259+0000","last_undegraded":"2023-11-13T11:01:39.988259+0000","last_fullsized":"2023-11-13T11:01:39.988259+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:51:36.644389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.010848+0000","last_change":"2023-11-13T11:01:39.004402+0000","last_active":"2023-11-13T11:01:41.010848+0000","last_peered":"2023-11-13T11:01:41.010848+0000","last_clean":"2023-11-13T11:01:41.010848+0000","last_became_active":"2023-11-13T11:01:39.001982+0000","last_became_peered":"2023-11-13T11:01:39.001982+0000","last_unstale":"2023-11-13T11:01:41.010848+0000","last_undegraded":"2023-11-13T11:01:41.010848+0000","last_fullsized":"2023-11-13T11:01:41.010848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:32:40.380861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.887035+0000","last_change":"2023-11-13T11:01:38.997117+0000","last_active":"2023-11-13T11:01:40.887035+0000","last_peered":"2023-11-13T11:01:40.887035+0000","last_clean":"2023-11-13T11:01:40.887035+0000","last_became_active":"2023-11-13T11:01:38.996801+0000","last_became_peered":"2023-11-13T11:01:38.996801+0000","last_unstale":"2023-11-13T11:01:40.887035+0000","last_undegraded":"2023-11-13T11:01:40.887035+0000","last_fullsized":"2023-11-13T11:01:40.887035+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:15:49.912064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.989664+0000","last_change":"2023-11-13T11:01:38.987650+0000","last_active":"2023-11-13T11:01:39.989664+0000","last_peered":"2023-11-13T11:01:39.989664+0000","last_clean":"2023-11-13T11:01:39.989664+0000","last_became_active":"2023-11-13T11:01:38.987270+0000","last_became_peered":"2023-11-13T11:01:38.987270+0000","last_unstale":"2023-11-13T11:01:39.989664+0000","last_undegraded":"2023-11-13T11:01:39.989664+0000","last_fullsized":"2023-11-13T11:01:39.989664+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:33:03.468642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.987266+0000","last_change":"2023-11-13T11:01:38.984512+0000","last_active":"2023-11-13T11:01:39.987266+0000","last_peered":"2023-11-13T11:01:39.987266+0000","last_clean":"2023-11-13T11:01:39.987266+0000","last_became_active":"2023-11-13T11:01:38.984243+0000","last_became_peered":"2023-11-13T11:01:38.984243+0000","last_unstale":"2023-11-13T11:01:39.987266+0000","last_undegraded":"2023-11-13T11:01:39.987266+0000","last_fullsized":"2023-11-13T11:01:39.987266+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:33:03.468642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011670+0000","last_change":"2023-11-13T11:01:38.992022+0000","last_active":"2023-11-13T11:01:41.011670+0000","last_peered":"2023-11-13T11:01:41.011670+0000","last_clean":"2023-11-13T11:01:41.011670+0000","last_became_active":"2023-11-13T11:01:38.991341+0000","last_became_peered":"2023-11-13T11:01:38.991341+0000","last_unstale":"2023-11-13T11:01:41.011670+0000","last_undegraded":"2023-11-13T11:01:41.011670+0000","last_fullsized":"2023-11-13T11:01:41.011670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:53:20.682896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011270+0000","last_change":"2023-11-13T11:01:38.998336+0000","last_active":"2023-11-13T11:01:41.011270+0000","last_peered":"2023-11-13T11:01:41.011270+0000","last_clean":"2023-11-13T11:01:41.011270+0000","last_became_active":"2023-11-13T11:01:38.997894+0000","last_became_peered":"2023-11-13T11:01:38.997894+0000","last_unstale":"2023-11-13T11:01:41.011270+0000","last_undegraded":"2023-11-13T11:01:41.011270+0000","last_fullsized":"2023-11-13T11:01:41.011270+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:42:46.869446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.986826+0000","last_change":"2023-11-13T11:01:38.992189+0000","last_active":"2023-11-13T11:01:39.986826+0000","last_peered":"2023-11-13T11:01:39.986826+0000","last_clean":"2023-11-13T11:01:39.986826+0000","last_became_active":"2023-11-13T11:01:38.991290+0000","last_became_peered":"2023-11-13T11:01:38.991290+0000","last_unstale":"2023-11-13T11:01:39.986826+0000","last_undegraded":"2023-11-13T11:01:39.986826+0000","last_fullsized":"2023-11-13T11:01:39.986826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:18.523871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.982094+0000","last_change":"2023-11-13T11:01:38.966119+0000","last_active":"2023-11-13T11:01:39.982094+0000","last_peered":"2023-11-13T11:01:39.982094+0000","last_clean":"2023-11-13T11:01:39.982094+0000","last_became_active":"2023-11-13T11:01:38.965812+0000","last_became_peered":"2023-11-13T11:01:38.965812+0000","last_unstale":"2023-11-13T11:01:39.982094+0000","last_undegraded":"2023-11-13T11:01:39.982094+0000","last_fullsized":"2023-11-13T11:01:39.982094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.010855+0000","last_change":"2023-11-13T11:01:39.002819+0000","last_active":"2023-11-13T11:01:41.010855+0000","last_peered":"2023-11-13T11:01:41.010855+0000","last_clean":"2023-11-13T11:01:41.010855+0000","last_became_active":"2023-11-13T11:01:39.002309+0000","last_became_peered":"2023-11-13T11:01:39.002309+0000","last_unstale":"2023-11-13T11:01:41.010855+0000","last_undegraded":"2023-11-13T11:01:41.010855+0000","last_fullsized":"2023-11-13T11:01:41.010855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:20:55.201885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.988313+0000","last_change":"2023-11-13T11:01:38.993324+0000","last_active":"2023-11-13T11:01:39.988313+0000","last_peered":"2023-11-13T11:01:39.988313+0000","last_clean":"2023-11-13T11:01:39.988313+0000","last_became_active":"2023-11-13T11:01:38.991072+0000","last_became_peered":"2023-11-13T11:01:38.991072+0000","last_unstale":"2023-11-13T11:01:39.988313+0000","last_undegraded":"2023-11-13T11:01:39.988313+0000","last_fullsized":"2023-11-13T11:01:39.988313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:30:19.613020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.982490+0000","last_change":"2023-11-13T11:01:38.975150+0000","last_active":"2023-11-13T11:01:39.982490+0000","last_peered":"2023-11-13T11:01:39.982490+0000","last_clean":"2023-11-13T11:01:39.982490+0000","last_became_active":"2023-11-13T11:01:38.974837+0000","last_became_peered":"2023-11-13T11:01:38.974837+0000","last_unstale":"2023-11-13T11:01:39.982490+0000","last_undegraded":"2023-11-13T11:01:39.982490+0000","last_fullsized":"2023-11-13T11:01:39.982490+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:47.666680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.887048+0000","last_change":"2023-11-13T11:01:38.984024+0000","last_active":"2023-11-13T11:01:40.887048+0000","last_peered":"2023-11-13T11:01:40.887048+0000","last_clean":"2023-11-13T11:01:40.887048+0000","last_became_active":"2023-11-13T11:01:38.983338+0000","last_became_peered":"2023-11-13T11:01:38.983338+0000","last_unstale":"2023-11-13T11:01:40.887048+0000","last_undegraded":"2023-11-13T11:01:40.887048+0000","last_fullsized":"2023-11-13T11:01:40.887048+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:15:33.302947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.510527+0000","last_change":"2023-11-13T11:01:38.969241+0000","last_active":"2023-11-13T11:01:40.510527+0000","last_peered":"2023-11-13T11:01:40.510527+0000","last_clean":"2023-11-13T11:01:40.510527+0000","last_became_active":"2023-11-13T11:01:38.968942+0000","last_became_peered":"2023-11-13T11:01:38.968942+0000","last_unstale":"2023-11-13T11:01:40.510527+0000","last_undegraded":"2023-11-13T11:01:40.510527+0000","last_fullsized":"2023-11-13T11:01:40.510527+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:35.756266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.982219+0000","last_change":"2023-11-13T11:01:38.966987+0000","last_active":"2023-11-13T11:01:39.982219+0000","last_peered":"2023-11-13T11:01:39.982219+0000","last_clean":"2023-11-13T11:01:39.982219+0000","last_became_active":"2023-11-13T11:01:38.966723+0000","last_became_peered":"2023-11-13T11:01:38.966723+0000","last_unstale":"2023-11-13T11:01:39.982219+0000","last_undegraded":"2023-11-13T11:01:39.982219+0000","last_fullsized":"2023-11-13T11:01:39.982219+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:32.745601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.886216+0000","last_change":"2023-11-13T11:01:38.989021+0000","last_active":"2023-11-13T11:01:40.886216+0000","last_peered":"2023-11-13T11:01:40.886216+0000","last_clean":"2023-11-13T11:01:40.886216+0000","last_became_active":"2023-11-13T11:01:38.988715+0000","last_became_peered":"2023-11-13T11:01:38.988715+0000","last_unstale":"2023-11-13T11:01:40.886216+0000","last_undegraded":"2023-11-13T11:01:40.886216+0000","last_fullsized":"2023-11-13T11:01:40.886216+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:22:27.706599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.885837+0000","last_change":"2023-11-13T11:01:38.986587+0000","last_active":"2023-11-13T11:01:40.885837+0000","last_peered":"2023-11-13T11:01:40.885837+0000","last_clean":"2023-11-13T11:01:40.885837+0000","last_became_active":"2023-11-13T11:01:38.986301+0000","last_became_peered":"2023-11-13T11:01:38.986301+0000","last_unstale":"2023-11-13T11:01:40.885837+0000","last_undegraded":"2023-11-13T11:01:40.885837+0000","last_fullsized":"2023-11-13T11:01:40.885837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:05:00.330485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.886274+0000","last_change":"2023-11-13T11:01:38.992108+0000","last_active":"2023-11-13T11:01:40.886274+0000","last_peered":"2023-11-13T11:01:40.886274+0000","last_clean":"2023-11-13T11:01:40.886274+0000","last_became_active":"2023-11-13T11:01:38.988156+0000","last_became_peered":"2023-11-13T11:01:38.988156+0000","last_unstale":"2023-11-13T11:01:40.886274+0000","last_undegraded":"2023-11-13T11:01:40.886274+0000","last_fullsized":"2023-11-13T11:01:40.886274+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:04:10.017117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.987707+0000","last_change":"2023-11-13T11:01:38.996834+0000","last_active":"2023-11-13T11:01:39.987707+0000","last_peered":"2023-11-13T11:01:39.987707+0000","last_clean":"2023-11-13T11:01:39.987707+0000","last_became_active":"2023-11-13T11:01:38.996522+0000","last_became_peered":"2023-11-13T11:01:38.996522+0000","last_unstale":"2023-11-13T11:01:39.987707+0000","last_undegraded":"2023-11-13T11:01:39.987707+0000","last_fullsized":"2023-11-13T11:01:39.987707+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:00:47.616351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.887177+0000","last_change":"2023-11-13T11:01:39.003837+0000","last_active":"2023-11-13T11:01:40.887177+0000","last_peered":"2023-11-13T11:01:40.887177+0000","last_clean":"2023-11-13T11:01:40.887177+0000","last_became_active":"2023-11-13T11:01:39.000372+0000","last_became_peered":"2023-11-13T11:01:39.000372+0000","last_unstale":"2023-11-13T11:01:40.887177+0000","last_undegraded":"2023-11-13T11:01:40.887177+0000","last_fullsized":"2023-11-13T11:01:40.887177+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:59:01.124681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011710+0000","last_change":"2023-11-13T11:01:38.997301+0000","last_active":"2023-11-13T11:01:41.011710+0000","last_peered":"2023-11-13T11:01:41.011710+0000","last_clean":"2023-11-13T11:01:41.011710+0000","last_became_active":"2023-11-13T11:01:38.997039+0000","last_became_peered":"2023-11-13T11:01:38.997039+0000","last_unstale":"2023-11-13T11:01:41.011710+0000","last_undegraded":"2023-11-13T11:01:41.011710+0000","last_fullsized":"2023-11-13T11:01:41.011710+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:07:23.632348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.510720+0000","last_change":"2023-11-13T11:01:39.000662+0000","last_active":"2023-11-13T11:01:40.510720+0000","last_peered":"2023-11-13T11:01:40.510720+0000","last_clean":"2023-11-13T11:01:40.510720+0000","last_became_active":"2023-11-13T11:01:39.000191+0000","last_became_peered":"2023-11-13T11:01:39.000191+0000","last_unstale":"2023-11-13T11:01:40.510720+0000","last_undegraded":"2023-11-13T11:01:40.510720+0000","last_fullsized":"2023-11-13T11:01:40.510720+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.986690+0000","last_change":"2023-11-13T11:01:38.969203+0000","last_active":"2023-11-13T11:01:39.986690+0000","last_peered":"2023-11-13T11:01:39.986690+0000","last_clean":"2023-11-13T11:01:39.986690+0000","last_became_active":"2023-11-13T11:01:38.968896+0000","last_became_peered":"2023-11-13T11:01:38.968896+0000","last_unstale":"2023-11-13T11:01:39.986690+0000","last_undegraded":"2023-11-13T11:01:39.986690+0000","last_fullsized":"2023-11-13T11:01:39.986690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:27:04.009871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.173173+0000","last_change":"2023-11-13T11:01:38.989974+0000","last_active":"2023-11-13T11:01:40.173173+0000","last_peered":"2023-11-13T11:01:40.173173+0000","last_clean":"2023-11-13T11:01:40.173173+0000","last_became_active":"2023-11-13T11:01:38.989665+0000","last_became_peered":"2023-11-13T11:01:38.989665+0000","last_unstale":"2023-11-13T11:01:40.173173+0000","last_undegraded":"2023-11-13T11:01:40.173173+0000","last_fullsized":"2023-11-13T11:01:40.173173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:41:22.236439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.886087+0000","last_change":"2023-11-13T11:01:38.995148+0000","last_active":"2023-11-13T11:01:40.886087+0000","last_peered":"2023-11-13T11:01:40.886087+0000","last_clean":"2023-11-13T11:01:40.886087+0000","last_became_active":"2023-11-13T11:01:38.990007+0000","last_became_peered":"2023-11-13T11:01:38.990007+0000","last_unstale":"2023-11-13T11:01:40.886087+0000","last_undegraded":"2023-11-13T11:01:40.886087+0000","last_fullsized":"2023-11-13T11:01:40.886087+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:41:22.236439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.509705+0000","last_change":"2023-11-13T11:01:38.984675+0000","last_active":"2023-11-13T11:01:40.509705+0000","last_peered":"2023-11-13T11:01:40.509705+0000","last_clean":"2023-11-13T11:01:40.509705+0000","last_became_active":"2023-11-13T11:01:38.984073+0000","last_became_peered":"2023-11-13T11:01:38.984073+0000","last_unstale":"2023-11-13T11:01:40.509705+0000","last_undegraded":"2023-11-13T11:01:40.509705+0000","last_fullsized":"2023-11-13T11:01:40.509705+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:06:04.973523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.173813+0000","last_change":"2023-11-13T11:01:38.973375+0000","last_active":"2023-11-13T11:01:40.173813+0000","last_peered":"2023-11-13T11:01:40.173813+0000","last_clean":"2023-11-13T11:01:40.173813+0000","last_became_active":"2023-11-13T11:01:38.970971+0000","last_became_peered":"2023-11-13T11:01:38.970971+0000","last_unstale":"2023-11-13T11:01:40.173813+0000","last_undegraded":"2023-11-13T11:01:40.173813+0000","last_fullsized":"2023-11-13T11:01:40.173813+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:35:23.626812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.150046+0000","last_change":"2023-11-13T11:01:37.976416+0000","last_active":"2023-11-13T11:01:39.150046+0000","last_peered":"2023-11-13T11:01:39.150046+0000","last_clean":"2023-11-13T11:01:39.150046+0000","last_became_active":"2023-11-13T11:01:37.976080+0000","last_became_peered":"2023-11-13T11:01:37.976080+0000","last_unstale":"2023-11-13T11:01:39.150046+0000","last_undegraded":"2023-11-13T11:01:39.150046+0000","last_fullsized":"2023-11-13T11:01:39.150046+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.173388+0000","last_change":"2023-11-13T11:01:38.977746+0000","last_active":"2023-11-13T11:01:40.173388+0000","last_peered":"2023-11-13T11:01:40.173388+0000","last_clean":"2023-11-13T11:01:40.173388+0000","last_became_active":"2023-11-13T11:01:38.977421+0000","last_became_peered":"2023-11-13T11:01:38.977421+0000","last_unstale":"2023-11-13T11:01:40.173388+0000","last_undegraded":"2023-11-13T11:01:40.173388+0000","last_fullsized":"2023-11-13T11:01:40.173388+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:04:10.017117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005578+0000","last_change":"2023-11-13T11:01:37.984920+0000","last_active":"2023-11-13T11:01:39.005578+0000","last_peered":"2023-11-13T11:01:39.005578+0000","last_clean":"2023-11-13T11:01:39.005578+0000","last_became_active":"2023-11-13T11:01:37.984119+0000","last_became_peered":"2023-11-13T11:01:37.984119+0000","last_unstale":"2023-11-13T11:01:39.005578+0000","last_undegraded":"2023-11-13T11:01:39.005578+0000","last_fullsized":"2023-11-13T11:01:39.005578+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:09:29.998731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.173192+0000","last_change":"2023-11-13T11:01:38.971332+0000","last_active":"2023-11-13T11:01:40.173192+0000","last_peered":"2023-11-13T11:01:40.173192+0000","last_clean":"2023-11-13T11:01:40.173192+0000","last_became_active":"2023-11-13T11:01:38.969969+0000","last_became_peered":"2023-11-13T11:01:38.969969+0000","last_unstale":"2023-11-13T11:01:40.173192+0000","last_undegraded":"2023-11-13T11:01:40.173192+0000","last_fullsized":"2023-11-13T11:01:40.173192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:22:27.706599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.142342+0000","last_change":"2023-11-13T11:01:37.970093+0000","last_active":"2023-11-13T11:01:39.142342+0000","last_peered":"2023-11-13T11:01:39.142342+0000","last_clean":"2023-11-13T11:01:39.142342+0000","last_became_active":"2023-11-13T11:01:37.969770+0000","last_became_peered":"2023-11-13T11:01:37.969770+0000","last_unstale":"2023-11-13T11:01:39.142342+0000","last_undegraded":"2023-11-13T11:01:39.142342+0000","last_fullsized":"2023-11-13T11:01:39.142342+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:33.254128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.989227+0000","last_change":"2023-11-13T11:01:38.980529+0000","last_active":"2023-11-13T11:01:39.989227+0000","last_peered":"2023-11-13T11:01:39.989227+0000","last_clean":"2023-11-13T11:01:39.989227+0000","last_became_active":"2023-11-13T11:01:38.980227+0000","last_became_peered":"2023-11-13T11:01:38.980227+0000","last_unstale":"2023-11-13T11:01:39.989227+0000","last_undegraded":"2023-11-13T11:01:39.989227+0000","last_fullsized":"2023-11-13T11:01:39.989227+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:18.523871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.149752+0000","last_change":"2023-11-13T11:01:37.982793+0000","last_active":"2023-11-13T11:01:39.149752+0000","last_peered":"2023-11-13T11:01:39.149752+0000","last_clean":"2023-11-13T11:01:39.149752+0000","last_became_active":"2023-11-13T11:01:37.982111+0000","last_became_peered":"2023-11-13T11:01:37.982111+0000","last_unstale":"2023-11-13T11:01:39.149752+0000","last_undegraded":"2023-11-13T11:01:39.149752+0000","last_fullsized":"2023-11-13T11:01:39.149752+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.010727+0000","last_change":"2023-11-13T11:01:38.976171+0000","last_active":"2023-11-13T11:01:41.010727+0000","last_peered":"2023-11-13T11:01:41.010727+0000","last_clean":"2023-11-13T11:01:41.010727+0000","last_became_active":"2023-11-13T11:01:38.975895+0000","last_became_peered":"2023-11-13T11:01:38.975895+0000","last_unstale":"2023-11-13T11:01:41.010727+0000","last_undegraded":"2023-11-13T11:01:41.010727+0000","last_fullsized":"2023-11-13T11:01:41.010727+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:27:04.009871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.169713+0000","last_change":"2023-11-13T11:01:37.981492+0000","last_active":"2023-11-13T11:01:39.169713+0000","last_peered":"2023-11-13T11:01:39.169713+0000","last_clean":"2023-11-13T11:01:39.169713+0000","last_became_active":"2023-11-13T11:01:37.981149+0000","last_became_peered":"2023-11-13T11:01:37.981149+0000","last_unstale":"2023-11-13T11:01:39.169713+0000","last_undegraded":"2023-11-13T11:01:39.169713+0000","last_fullsized":"2023-11-13T11:01:39.169713+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.173942+0000","last_change":"2023-11-13T11:01:38.972149+0000","last_active":"2023-11-13T11:01:40.173942+0000","last_peered":"2023-11-13T11:01:40.173942+0000","last_clean":"2023-11-13T11:01:40.173942+0000","last_became_active":"2023-11-13T11:01:38.971860+0000","last_became_peered":"2023-11-13T11:01:38.971860+0000","last_unstale":"2023-11-13T11:01:40.173942+0000","last_undegraded":"2023-11-13T11:01:40.173942+0000","last_fullsized":"2023-11-13T11:01:40.173942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:15:49.912064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.156538+0000","last_change":"2023-11-13T11:01:37.968032+0000","last_active":"2023-11-13T11:01:39.156538+0000","last_peered":"2023-11-13T11:01:39.156538+0000","last_clean":"2023-11-13T11:01:39.156538+0000","last_became_active":"2023-11-13T11:01:37.967709+0000","last_became_peered":"2023-11-13T11:01:37.967709+0000","last_unstale":"2023-11-13T11:01:39.156538+0000","last_undegraded":"2023-11-13T11:01:39.156538+0000","last_fullsized":"2023-11-13T11:01:39.156538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:44:32.588239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.990010+0000","last_change":"2023-11-13T11:01:39.001367+0000","last_active":"2023-11-13T11:01:39.990010+0000","last_peered":"2023-11-13T11:01:39.990010+0000","last_clean":"2023-11-13T11:01:39.990010+0000","last_became_active":"2023-11-13T11:01:39.001092+0000","last_became_peered":"2023-11-13T11:01:39.001092+0000","last_unstale":"2023-11-13T11:01:39.990010+0000","last_undegraded":"2023-11-13T11:01:39.990010+0000","last_fullsized":"2023-11-13T11:01:39.990010+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:39:32.034804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.156365+0000","last_change":"2023-11-13T11:01:37.974253+0000","last_active":"2023-11-13T11:01:39.156365+0000","last_peered":"2023-11-13T11:01:39.156365+0000","last_clean":"2023-11-13T11:01:39.156365+0000","last_became_active":"2023-11-13T11:01:37.973388+0000","last_became_peered":"2023-11-13T11:01:37.973388+0000","last_unstale":"2023-11-13T11:01:39.156365+0000","last_undegraded":"2023-11-13T11:01:39.156365+0000","last_fullsized":"2023-11-13T11:01:39.156365+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.982489+0000","last_change":"2023-11-13T11:01:38.996067+0000","last_active":"2023-11-13T11:01:39.982489+0000","last_peered":"2023-11-13T11:01:39.982489+0000","last_clean":"2023-11-13T11:01:39.982489+0000","last_became_active":"2023-11-13T11:01:38.995583+0000","last_became_peered":"2023-11-13T11:01:38.995583+0000","last_unstale":"2023-11-13T11:01:39.982489+0000","last_undegraded":"2023-11-13T11:01:39.982489+0000","last_fullsized":"2023-11-13T11:01:39.982489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:32:34.267423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.169415+0000","last_change":"2023-11-13T11:01:37.980576+0000","last_active":"2023-11-13T11:01:39.169415+0000","last_peered":"2023-11-13T11:01:39.169415+0000","last_clean":"2023-11-13T11:01:39.169415+0000","last_became_active":"2023-11-13T11:01:37.980253+0000","last_became_peered":"2023-11-13T11:01:37.980253+0000","last_unstale":"2023-11-13T11:01:39.169415+0000","last_undegraded":"2023-11-13T11:01:39.169415+0000","last_fullsized":"2023-11-13T11:01:39.169415+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:50:19.867127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.010538+0000","last_change":"2023-11-13T11:01:38.978906+0000","last_active":"2023-11-13T11:01:41.010538+0000","last_peered":"2023-11-13T11:01:41.010538+0000","last_clean":"2023-11-13T11:01:41.010538+0000","last_became_active":"2023-11-13T11:01:38.978590+0000","last_became_peered":"2023-11-13T11:01:38.978590+0000","last_unstale":"2023-11-13T11:01:41.010538+0000","last_undegraded":"2023-11-13T11:01:41.010538+0000","last_fullsized":"2023-11-13T11:01:41.010538+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:51:36.644389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.149994+0000","last_change":"2023-11-13T11:01:37.967093+0000","last_active":"2023-11-13T11:01:39.149994+0000","last_peered":"2023-11-13T11:01:39.149994+0000","last_clean":"2023-11-13T11:01:39.149994+0000","last_became_active":"2023-11-13T11:01:37.966819+0000","last_became_peered":"2023-11-13T11:01:37.966819+0000","last_unstale":"2023-11-13T11:01:39.149994+0000","last_undegraded":"2023-11-13T11:01:39.149994+0000","last_fullsized":"2023-11-13T11:01:39.149994+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:32:27.529252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.982625+0000","last_change":"2023-11-13T11:01:38.978007+0000","last_active":"2023-11-13T11:01:39.982625+0000","last_peered":"2023-11-13T11:01:39.982625+0000","last_clean":"2023-11-13T11:01:39.982625+0000","last_became_active":"2023-11-13T11:01:38.977730+0000","last_became_peered":"2023-11-13T11:01:38.977730+0000","last_unstale":"2023-11-13T11:01:39.982625+0000","last_undegraded":"2023-11-13T11:01:39.982625+0000","last_fullsized":"2023-11-13T11:01:39.982625+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:51:36.644389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.149905+0000","last_change":"2023-11-13T11:01:37.974872+0000","last_active":"2023-11-13T11:01:39.149905+0000","last_peered":"2023-11-13T11:01:39.149905+0000","last_clean":"2023-11-13T11:01:39.149905+0000","last_became_active":"2023-11-13T11:01:37.973788+0000","last_became_peered":"2023-11-13T11:01:37.973788+0000","last_unstale":"2023-11-13T11:01:39.149905+0000","last_undegraded":"2023-11-13T11:01:39.149905+0000","last_fullsized":"2023-11-13T11:01:39.149905+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:22:11.290032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.990043+0000","last_change":"2023-11-13T11:01:38.982930+0000","last_active":"2023-11-13T11:01:39.990043+0000","last_peered":"2023-11-13T11:01:39.990043+0000","last_clean":"2023-11-13T11:01:39.990043+0000","last_became_active":"2023-11-13T11:01:38.982617+0000","last_became_peered":"2023-11-13T11:01:38.982617+0000","last_unstale":"2023-11-13T11:01:39.990043+0000","last_undegraded":"2023-11-13T11:01:39.990043+0000","last_fullsized":"2023-11-13T11:01:39.990043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:13:17.824685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.156673+0000","last_change":"2023-11-13T11:01:37.976846+0000","last_active":"2023-11-13T11:01:39.156673+0000","last_peered":"2023-11-13T11:01:39.156673+0000","last_clean":"2023-11-13T11:01:39.156673+0000","last_became_active":"2023-11-13T11:01:37.976378+0000","last_became_peered":"2023-11-13T11:01:37.976378+0000","last_unstale":"2023-11-13T11:01:39.156673+0000","last_undegraded":"2023-11-13T11:01:39.156673+0000","last_fullsized":"2023-11-13T11:01:39.156673+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.510499+0000","last_change":"2023-11-13T11:01:38.983595+0000","last_active":"2023-11-13T11:01:40.510499+0000","last_peered":"2023-11-13T11:01:40.510499+0000","last_clean":"2023-11-13T11:01:40.510499+0000","last_became_active":"2023-11-13T11:01:38.978958+0000","last_became_peered":"2023-11-13T11:01:38.978958+0000","last_unstale":"2023-11-13T11:01:40.510499+0000","last_undegraded":"2023-11-13T11:01:40.510499+0000","last_fullsized":"2023-11-13T11:01:40.510499+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:44.806511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.142316+0000","last_change":"2023-11-13T11:01:37.983799+0000","last_active":"2023-11-13T11:01:39.142316+0000","last_peered":"2023-11-13T11:01:39.142316+0000","last_clean":"2023-11-13T11:01:39.142316+0000","last_became_active":"2023-11-13T11:01:37.983048+0000","last_became_peered":"2023-11-13T11:01:37.983048+0000","last_unstale":"2023-11-13T11:01:39.142316+0000","last_undegraded":"2023-11-13T11:01:39.142316+0000","last_fullsized":"2023-11-13T11:01:39.142316+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:55:43.793217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.173830+0000","last_change":"2023-11-13T11:01:38.972065+0000","last_active":"2023-11-13T11:01:40.173830+0000","last_peered":"2023-11-13T11:01:40.173830+0000","last_clean":"2023-11-13T11:01:40.173830+0000","last_became_active":"2023-11-13T11:01:38.971764+0000","last_became_peered":"2023-11-13T11:01:38.971764+0000","last_unstale":"2023-11-13T11:01:40.173830+0000","last_undegraded":"2023-11-13T11:01:40.173830+0000","last_fullsized":"2023-11-13T11:01:40.173830+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:51:09.389900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.000285+0000","last_change":"2023-11-13T11:01:37.976954+0000","last_active":"2023-11-13T11:01:39.000285+0000","last_peered":"2023-11-13T11:01:39.000285+0000","last_clean":"2023-11-13T11:01:39.000285+0000","last_became_active":"2023-11-13T11:01:37.976659+0000","last_became_peered":"2023-11-13T11:01:37.976659+0000","last_unstale":"2023-11-13T11:01:39.000285+0000","last_undegraded":"2023-11-13T11:01:39.000285+0000","last_fullsized":"2023-11-13T11:01:39.000285+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:02:54.853648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.987425+0000","last_change":"2023-11-13T11:01:38.995557+0000","last_active":"2023-11-13T11:01:39.987425+0000","last_peered":"2023-11-13T11:01:39.987425+0000","last_clean":"2023-11-13T11:01:39.987425+0000","last_became_active":"2023-11-13T11:01:38.995224+0000","last_became_peered":"2023-11-13T11:01:38.995224+0000","last_unstale":"2023-11-13T11:01:39.987425+0000","last_undegraded":"2023-11-13T11:01:39.987425+0000","last_fullsized":"2023-11-13T11:01:39.987425+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:39:32.034804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.149611+0000","last_change":"2023-11-13T11:01:37.970890+0000","last_active":"2023-11-13T11:01:39.149611+0000","last_peered":"2023-11-13T11:01:39.149611+0000","last_clean":"2023-11-13T11:01:39.149611+0000","last_became_active":"2023-11-13T11:01:37.970585+0000","last_became_peered":"2023-11-13T11:01:37.970585+0000","last_unstale":"2023-11-13T11:01:39.149611+0000","last_undegraded":"2023-11-13T11:01:39.149611+0000","last_fullsized":"2023-11-13T11:01:39.149611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:24.358423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011948+0000","last_change":"2023-11-13T11:01:38.994877+0000","last_active":"2023-11-13T11:01:41.011948+0000","last_peered":"2023-11-13T11:01:41.011948+0000","last_clean":"2023-11-13T11:01:41.011948+0000","last_became_active":"2023-11-13T11:01:38.994450+0000","last_became_peered":"2023-11-13T11:01:38.994450+0000","last_unstale":"2023-11-13T11:01:41.011948+0000","last_undegraded":"2023-11-13T11:01:41.011948+0000","last_fullsized":"2023-11-13T11:01:41.011948+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:41:55.799097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.149337+0000","last_change":"2023-11-13T11:01:37.966418+0000","last_active":"2023-11-13T11:01:39.149337+0000","last_peered":"2023-11-13T11:01:39.149337+0000","last_clean":"2023-11-13T11:01:39.149337+0000","last_became_active":"2023-11-13T11:01:37.966105+0000","last_became_peered":"2023-11-13T11:01:37.966105+0000","last_unstale":"2023-11-13T11:01:39.149337+0000","last_undegraded":"2023-11-13T11:01:39.149337+0000","last_fullsized":"2023-11-13T11:01:39.149337+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:44:09.646957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.887096+0000","last_change":"2023-11-13T11:01:38.985149+0000","last_active":"2023-11-13T11:01:40.887096+0000","last_peered":"2023-11-13T11:01:40.887096+0000","last_clean":"2023-11-13T11:01:40.887096+0000","last_became_active":"2023-11-13T11:01:38.984828+0000","last_became_peered":"2023-11-13T11:01:38.984828+0000","last_unstale":"2023-11-13T11:01:40.887096+0000","last_undegraded":"2023-11-13T11:01:40.887096+0000","last_fullsized":"2023-11-13T11:01:40.887096+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:02:23.803734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.007772+0000","last_change":"2023-11-13T11:01:37.983844+0000","last_active":"2023-11-13T11:01:39.007772+0000","last_peered":"2023-11-13T11:01:39.007772+0000","last_clean":"2023-11-13T11:01:39.007772+0000","last_became_active":"2023-11-13T11:01:37.982971+0000","last_became_peered":"2023-11-13T11:01:37.982971+0000","last_unstale":"2023-11-13T11:01:39.007772+0000","last_undegraded":"2023-11-13T11:01:39.007772+0000","last_fullsized":"2023-11-13T11:01:39.007772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:56:05.286095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.988260+0000","last_change":"2023-11-13T11:01:38.984132+0000","last_active":"2023-11-13T11:01:39.988260+0000","last_peered":"2023-11-13T11:01:39.988260+0000","last_clean":"2023-11-13T11:01:39.988260+0000","last_became_active":"2023-11-13T11:01:38.981507+0000","last_became_peered":"2023-11-13T11:01:38.981507+0000","last_unstale":"2023-11-13T11:01:39.988260+0000","last_undegraded":"2023-11-13T11:01:39.988260+0000","last_fullsized":"2023-11-13T11:01:39.988260+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:26:39.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.005279+0000","last_change":"2023-11-13T11:01:37.986267+0000","last_active":"2023-11-13T11:01:39.005279+0000","last_peered":"2023-11-13T11:01:39.005279+0000","last_clean":"2023-11-13T11:01:39.005279+0000","last_became_active":"2023-11-13T11:01:37.985787+0000","last_became_peered":"2023-11-13T11:01:37.985787+0000","last_unstale":"2023-11-13T11:01:39.005279+0000","last_undegraded":"2023-11-13T11:01:39.005279+0000","last_fullsized":"2023-11-13T11:01:39.005279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:10:32.081040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.986209+0000","last_change":"2023-11-13T11:01:38.984798+0000","last_active":"2023-11-13T11:01:39.986209+0000","last_peered":"2023-11-13T11:01:39.986209+0000","last_clean":"2023-11-13T11:01:39.986209+0000","last_became_active":"2023-11-13T11:01:38.984518+0000","last_became_peered":"2023-11-13T11:01:38.984518+0000","last_unstale":"2023-11-13T11:01:39.986209+0000","last_undegraded":"2023-11-13T11:01:39.986209+0000","last_fullsized":"2023-11-13T11:01:39.986209+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:51:36.644389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.004284+0000","last_change":"2023-11-13T11:01:37.977771+0000","last_active":"2023-11-13T11:01:39.004284+0000","last_peered":"2023-11-13T11:01:39.004284+0000","last_clean":"2023-11-13T11:01:39.004284+0000","last_became_active":"2023-11-13T11:01:37.977506+0000","last_became_peered":"2023-11-13T11:01:37.977506+0000","last_unstale":"2023-11-13T11:01:39.004284+0000","last_undegraded":"2023-11-13T11:01:39.004284+0000","last_fullsized":"2023-11-13T11:01:39.004284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:43:08.322589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.510015+0000","last_change":"2023-11-13T11:01:38.994873+0000","last_active":"2023-11-13T11:01:40.510015+0000","last_peered":"2023-11-13T11:01:40.510015+0000","last_clean":"2023-11-13T11:01:40.510015+0000","last_became_active":"2023-11-13T11:01:38.994490+0000","last_became_peered":"2023-11-13T11:01:38.994490+0000","last_unstale":"2023-11-13T11:01:40.510015+0000","last_undegraded":"2023-11-13T11:01:40.510015+0000","last_fullsized":"2023-11-13T11:01:40.510015+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:33:25.371717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.149811+0000","last_change":"2023-11-13T11:01:37.982308+0000","last_active":"2023-11-13T11:01:39.149811+0000","last_peered":"2023-11-13T11:01:39.149811+0000","last_clean":"2023-11-13T11:01:39.149811+0000","last_became_active":"2023-11-13T11:01:37.981963+0000","last_became_peered":"2023-11-13T11:01:37.981963+0000","last_unstale":"2023-11-13T11:01:39.149811+0000","last_undegraded":"2023-11-13T11:01:39.149811+0000","last_fullsized":"2023-11-13T11:01:39.149811+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.011071+0000","last_change":"2023-11-13T11:01:38.998159+0000","last_active":"2023-11-13T11:01:41.011071+0000","last_peered":"2023-11-13T11:01:41.011071+0000","last_clean":"2023-11-13T11:01:41.011071+0000","last_became_active":"2023-11-13T11:01:38.997579+0000","last_became_peered":"2023-11-13T11:01:38.997579+0000","last_unstale":"2023-11-13T11:01:41.011071+0000","last_undegraded":"2023-11-13T11:01:41.011071+0000","last_fullsized":"2023-11-13T11:01:41.011071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:52:42.280287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.149800+0000","last_change":"2023-11-13T11:01:37.978277+0000","last_active":"2023-11-13T11:01:39.149800+0000","last_peered":"2023-11-13T11:01:39.149800+0000","last_clean":"2023-11-13T11:01:39.149800+0000","last_became_active":"2023-11-13T11:01:37.977659+0000","last_became_peered":"2023-11-13T11:01:37.977659+0000","last_unstale":"2023-11-13T11:01:39.149800+0000","last_undegraded":"2023-11-13T11:01:39.149800+0000","last_fullsized":"2023-11-13T11:01:39.149800+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:20:56.135585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.510739+0000","last_change":"2023-11-13T11:01:38.975186+0000","last_active":"2023-11-13T11:01:40.510739+0000","last_peered":"2023-11-13T11:01:40.510739+0000","last_clean":"2023-11-13T11:01:40.510739+0000","last_became_active":"2023-11-13T11:01:38.972436+0000","last_became_peered":"2023-11-13T11:01:38.972436+0000","last_unstale":"2023-11-13T11:01:40.510739+0000","last_undegraded":"2023-11-13T11:01:40.510739+0000","last_fullsized":"2023-11-13T11:01:40.510739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:01.750338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.151834+0000","last_change":"2023-11-13T11:01:37.970760+0000","last_active":"2023-11-13T11:01:39.151834+0000","last_peered":"2023-11-13T11:01:39.151834+0000","last_clean":"2023-11-13T11:01:39.151834+0000","last_became_active":"2023-11-13T11:01:37.970465+0000","last_became_peered":"2023-11-13T11:01:37.970465+0000","last_unstale":"2023-11-13T11:01:39.151834+0000","last_undegraded":"2023-11-13T11:01:39.151834+0000","last_fullsized":"2023-11-13T11:01:39.151834+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.982716+0000","last_change":"2023-11-13T11:01:38.972817+0000","last_active":"2023-11-13T11:01:39.982716+0000","last_peered":"2023-11-13T11:01:39.982716+0000","last_clean":"2023-11-13T11:01:39.982716+0000","last_became_active":"2023-11-13T11:01:38.971691+0000","last_became_peered":"2023-11-13T11:01:38.971691+0000","last_unstale":"2023-11-13T11:01:39.982716+0000","last_undegraded":"2023-11-13T11:01:39.982716+0000","last_fullsized":"2023-11-13T11:01:39.982716+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:27:04.009871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.169584+0000","last_change":"2023-11-13T11:01:37.982283+0000","last_active":"2023-11-13T11:01:39.169584+0000","last_peered":"2023-11-13T11:01:39.169584+0000","last_clean":"2023-11-13T11:01:39.169584+0000","last_became_active":"2023-11-13T11:01:37.981958+0000","last_became_peered":"2023-11-13T11:01:37.981958+0000","last_unstale":"2023-11-13T11:01:39.169584+0000","last_undegraded":"2023-11-13T11:01:39.169584+0000","last_fullsized":"2023-11-13T11:01:39.169584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:44:29.604069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.986852+0000","last_change":"2023-11-13T11:01:38.982113+0000","last_active":"2023-11-13T11:01:39.986852+0000","last_peered":"2023-11-13T11:01:39.986852+0000","last_clean":"2023-11-13T11:01:39.986852+0000","last_became_active":"2023-11-13T11:01:38.980795+0000","last_became_peered":"2023-11-13T11:01:38.980795+0000","last_unstale":"2023-11-13T11:01:39.986852+0000","last_undegraded":"2023-11-13T11:01:39.986852+0000","last_fullsized":"2023-11-13T11:01:39.986852+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:13:04.096588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.142196+0000","last_change":"2023-11-13T11:01:37.982552+0000","last_active":"2023-11-13T11:01:39.142196+0000","last_peered":"2023-11-13T11:01:39.142196+0000","last_clean":"2023-11-13T11:01:39.142196+0000","last_became_active":"2023-11-13T11:01:37.982269+0000","last_became_peered":"2023-11-13T11:01:37.982269+0000","last_unstale":"2023-11-13T11:01:39.142196+0000","last_undegraded":"2023-11-13T11:01:39.142196+0000","last_fullsized":"2023-11-13T11:01:39.142196+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:53:34.742971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.012056+0000","last_change":"2023-11-13T11:01:38.995167+0000","last_active":"2023-11-13T11:01:41.012056+0000","last_peered":"2023-11-13T11:01:41.012056+0000","last_clean":"2023-11-13T11:01:41.012056+0000","last_became_active":"2023-11-13T11:01:38.994892+0000","last_became_peered":"2023-11-13T11:01:38.994892+0000","last_unstale":"2023-11-13T11:01:41.012056+0000","last_undegraded":"2023-11-13T11:01:41.012056+0000","last_fullsized":"2023-11-13T11:01:41.012056+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:28:05.690927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.142148+0000","last_change":"2023-11-13T11:01:37.971616+0000","last_active":"2023-11-13T11:01:39.142148+0000","last_peered":"2023-11-13T11:01:39.142148+0000","last_clean":"2023-11-13T11:01:39.142148+0000","last_became_active":"2023-11-13T11:01:37.971344+0000","last_became_peered":"2023-11-13T11:01:37.971344+0000","last_unstale":"2023-11-13T11:01:39.142148+0000","last_undegraded":"2023-11-13T11:01:39.142148+0000","last_fullsized":"2023-11-13T11:01:39.142148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:33:31.732306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.982516+0000","last_change":"2023-11-13T11:01:38.983983+0000","last_active":"2023-11-13T11:01:39.982516+0000","last_peered":"2023-11-13T11:01:39.982516+0000","last_clean":"2023-11-13T11:01:39.982516+0000","last_became_active":"2023-11-13T11:01:38.983698+0000","last_became_peered":"2023-11-13T11:01:38.983698+0000","last_unstale":"2023-11-13T11:01:39.982516+0000","last_undegraded":"2023-11-13T11:01:39.982516+0000","last_fullsized":"2023-11-13T11:01:39.982516+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:26:39.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.156774+0000","last_change":"2023-11-13T11:01:37.972256+0000","last_active":"2023-11-13T11:01:39.156774+0000","last_peered":"2023-11-13T11:01:39.156774+0000","last_clean":"2023-11-13T11:01:39.156774+0000","last_became_active":"2023-11-13T11:01:37.971890+0000","last_became_peered":"2023-11-13T11:01:37.971890+0000","last_unstale":"2023-11-13T11:01:39.156774+0000","last_undegraded":"2023-11-13T11:01:39.156774+0000","last_fullsized":"2023-11-13T11:01:39.156774+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:38.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.988909+0000","last_change":"2023-11-13T11:01:38.988092+0000","last_active":"2023-11-13T11:01:39.988909+0000","last_peered":"2023-11-13T11:01:39.988909+0000","last_clean":"2023-11-13T11:01:39.988909+0000","last_became_active":"2023-11-13T11:01:38.987800+0000","last_became_peered":"2023-11-13T11:01:38.987800+0000","last_unstale":"2023-11-13T11:01:39.988909+0000","last_undegraded":"2023-11-13T11:01:39.988909+0000","last_fullsized":"2023-11-13T11:01:39.988909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:27:04.009871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.156729+0000","last_change":"2023-11-13T11:01:37.973251+0000","last_active":"2023-11-13T11:01:39.156729+0000","last_peered":"2023-11-13T11:01:39.156729+0000","last_clean":"2023-11-13T11:01:39.156729+0000","last_became_active":"2023-11-13T11:01:37.972947+0000","last_became_peered":"2023-11-13T11:01:37.972947+0000","last_unstale":"2023-11-13T11:01:39.156729+0000","last_undegraded":"2023-11-13T11:01:39.156729+0000","last_fullsized":"2023-11-13T11:01:39.156729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:53:33.150341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.886851+0000","last_change":"2023-11-13T11:01:38.988853+0000","last_active":"2023-11-13T11:01:40.886851+0000","last_peered":"2023-11-13T11:01:40.886851+0000","last_clean":"2023-11-13T11:01:40.886851+0000","last_became_active":"2023-11-13T11:01:38.987558+0000","last_became_peered":"2023-11-13T11:01:38.987558+0000","last_unstale":"2023-11-13T11:01:40.886851+0000","last_undegraded":"2023-11-13T11:01:40.886851+0000","last_fullsized":"2023-11-13T11:01:40.886851+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:51:09.389900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.169759+0000","last_change":"2023-11-13T11:01:37.981807+0000","last_active":"2023-11-13T11:01:39.169759+0000","last_peered":"2023-11-13T11:01:39.169759+0000","last_clean":"2023-11-13T11:01:39.169759+0000","last_became_active":"2023-11-13T11:01:37.981480+0000","last_became_peered":"2023-11-13T11:01:37.981480+0000","last_unstale":"2023-11-13T11:01:39.169759+0000","last_undegraded":"2023-11-13T11:01:39.169759+0000","last_fullsized":"2023-11-13T11:01:39.169759+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:09:55.684221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.989349+0000","last_change":"2023-11-13T11:01:38.986410+0000","last_active":"2023-11-13T11:01:39.989349+0000","last_peered":"2023-11-13T11:01:39.989349+0000","last_clean":"2023-11-13T11:01:39.989349+0000","last_became_active":"2023-11-13T11:01:38.986131+0000","last_became_peered":"2023-11-13T11:01:38.986131+0000","last_unstale":"2023-11-13T11:01:39.989349+0000","last_undegraded":"2023-11-13T11:01:39.989349+0000","last_fullsized":"2023-11-13T11:01:39.989349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:13:04.096588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.142263+0000","last_change":"2023-11-13T11:01:37.975627+0000","last_active":"2023-11-13T11:01:39.142263+0000","last_peered":"2023-11-13T11:01:39.142263+0000","last_clean":"2023-11-13T11:01:39.142263+0000","last_became_active":"2023-11-13T11:01:37.974747+0000","last_became_peered":"2023-11-13T11:01:37.974747+0000","last_unstale":"2023-11-13T11:01:39.142263+0000","last_undegraded":"2023-11-13T11:01:39.142263+0000","last_fullsized":"2023-11-13T11:01:39.142263+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:00:00.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:40.511105+0000","last_change":"2023-11-13T11:01:38.989266+0000","last_active":"2023-11-13T11:01:40.511105+0000","last_peered":"2023-11-13T11:01:40.511105+0000","last_clean":"2023-11-13T11:01:40.511105+0000","last_became_active":"2023-11-13T11:01:38.988948+0000","last_became_peered":"2023-11-13T11:01:38.988948+0000","last_unstale":"2023-11-13T11:01:40.511105+0000","last_undegraded":"2023-11-13T11:01:40.511105+0000","last_fullsized":"2023-11-13T11:01:40.511105+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:47.666680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.169626+0000","last_change":"2023-11-13T11:01:37.982077+0000","last_active":"2023-11-13T11:01:39.169626+0000","last_peered":"2023-11-13T11:01:39.169626+0000","last_clean":"2023-11-13T11:01:39.169626+0000","last_became_active":"2023-11-13T11:01:37.981552+0000","last_became_peered":"2023-11-13T11:01:37.981552+0000","last_unstale":"2023-11-13T11:01:39.169626+0000","last_undegraded":"2023-11-13T11:01:39.169626+0000","last_fullsized":"2023-11-13T11:01:39.169626+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:52:51.994067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.990293+0000","last_change":"2023-11-13T11:01:38.993394+0000","last_active":"2023-11-13T11:01:39.990293+0000","last_peered":"2023-11-13T11:01:39.990293+0000","last_clean":"2023-11-13T11:01:39.990293+0000","last_became_active":"2023-11-13T11:01:38.992745+0000","last_became_peered":"2023-11-13T11:01:38.992745+0000","last_unstale":"2023-11-13T11:01:39.990293+0000","last_undegraded":"2023-11-13T11:01:39.990293+0000","last_fullsized":"2023-11-13T11:01:39.990293+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:00:47.616351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.149383+0000","last_change":"2023-11-13T11:01:37.971343+0000","last_active":"2023-11-13T11:01:39.149383+0000","last_peered":"2023-11-13T11:01:39.149383+0000","last_clean":"2023-11-13T11:01:39.149383+0000","last_became_active":"2023-11-13T11:01:37.970761+0000","last_became_peered":"2023-11-13T11:01:37.970761+0000","last_unstale":"2023-11-13T11:01:39.149383+0000","last_undegraded":"2023-11-13T11:01:39.149383+0000","last_fullsized":"2023-11-13T11:01:39.149383+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:47:45.202186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.012076+0000","last_change":"2023-11-13T11:01:39.001466+0000","last_active":"2023-11-13T11:01:41.012076+0000","last_peered":"2023-11-13T11:01:41.012076+0000","last_clean":"2023-11-13T11:01:41.012076+0000","last_became_active":"2023-11-13T11:01:39.001200+0000","last_became_peered":"2023-11-13T11:01:39.001200+0000","last_unstale":"2023-11-13T11:01:41.012076+0000","last_undegraded":"2023-11-13T11:01:41.012076+0000","last_fullsized":"2023-11-13T11:01:41.012076+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:28:45.331614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.149387+0000","last_change":"2023-11-13T11:01:37.970057+0000","last_active":"2023-11-13T11:01:39.149387+0000","last_peered":"2023-11-13T11:01:39.149387+0000","last_clean":"2023-11-13T11:01:39.149387+0000","last_became_active":"2023-11-13T11:01:37.969819+0000","last_became_peered":"2023-11-13T11:01:37.969819+0000","last_unstale":"2023-11-13T11:01:39.149387+0000","last_undegraded":"2023-11-13T11:01:39.149387+0000","last_fullsized":"2023-11-13T11:01:39.149387+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:45:07.653027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:41.010515+0000","last_change":"2023-11-13T11:01:38.966353+0000","last_active":"2023-11-13T11:01:41.010515+0000","last_peered":"2023-11-13T11:01:41.010515+0000","last_clean":"2023-11-13T11:01:41.010515+0000","last_became_active":"2023-11-13T11:01:38.966082+0000","last_became_peered":"2023-11-13T11:01:38.966082+0000","last_unstale":"2023-11-13T11:01:41.010515+0000","last_undegraded":"2023-11-13T11:01:41.010515+0000","last_fullsized":"2023-11-13T11:01:41.010515+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:37.929718+0000","last_clean_scrub_stamp":"2023-11-13T11:01:37.929718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:26:39.719636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"55'1","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-13T11:01:39.990434+0000","last_change":"2023-11-13T11:01:37.970911+0000","last_active":"2023-11-13T11:01:39.990434+0000","last_peered":"2023-11-13T11:01:39.990434+0000","last_clean":"2023-11-13T11:01:39.990434+0000","last_became_active":"2023-11-13T11:01:37.970627+0000","last_became_peered":"2023-11-13T11:01:37.970627+0000","last_unstale":"2023-11-13T11:01:39.990434+0000","last_undegraded":"2023-11-13T11:01:39.990434+0000","last_fullsized":"2023-11-13T11:01:39.990434+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:01:36.916424+0000","last_clean_scrub_stamp":"2023-11-13T11:01:36.916424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:55:46.653386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":195,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-13T11:01:38.956826+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:01:38.956826+0000","last_peered":"2023-11-13T11:01:38.956826+0000","last_clean":"2023-11-13T11:01:38.956826+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:01:38.956826+0000","last_undegraded":"2023-11-13T11:01:38.956826+0000","last_fullsized":"2023-11-13T11:01:38.956826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509521,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":651264,"data_stored":287221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0960000000000001,"15min":1.1259999999999999},"min":{"1min":0.84599999999999997,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.21,"15min":1.1339999999999999},"min":{"1min":0.89800000000000002,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.865,"5min":1.865,"15min":2.2879999999999998},"last":0.86499999999999999}]},{"osd":1,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.22,"15min":1.175},"min":{"1min":0.91700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8200000000000001,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2969999999999999,"15min":1.2889999999999999},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1070000000000002,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.67700000000000005}]},{"osd":2,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.2789999999999999,"15min":1.2769999999999999},"min":{"1min":0.98099999999999998,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.153,"5min":2.153,"15min":2.1680000000000001},"last":0.623},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.3819999999999999,"15min":1.3520000000000001},"min":{"1min":1.1080000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.2400000000000002,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.526}]},{"osd":3,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.407,"15min":1.383},"min":{"1min":0.76300000000000001,"5min":0.751,"15min":0.63800000000000001},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.528,"5min":1.423,"15min":1.4390000000000001},"min":{"1min":1.167,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.99299999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.454,"15min":1.5720000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.597},{"interface":"front","average":{"1min":1.577,"5min":1.5129999999999999,"15min":1.5940000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.1599999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6000000000000001,"15min":1.6559999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.72,"15min":1.7589999999999999},"min":{"1min":1.147,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.7070000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:29 2023","interfaces":[{"interface":"back","average":{"1min":1.758,"5min":1.659,"15min":1.651},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.754,"5min":1.617,"15min":1.6499999999999999},"min":{"1min":0.96499999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.2749999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509522,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40736,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331104,"statfs":{"total":96636764160,"available":96595050496,"internally_reserved":0,"allocated":1257472,"data_stored":878709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.107,"15min":1.0129999999999999},"min":{"1min":0.82999999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.9099999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2070000000000001,"15min":1.2030000000000001},"min":{"1min":0.74299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.639}]},{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.278,"15min":1.2310000000000001},"min":{"1min":0.89800000000000002,"5min":0.71699999999999997,"15min":0.65400000000000003},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.234,"15min":1.238},"min":{"1min":0.878,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8580000000000001,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.2250000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3129999999999999,"15min":1.2509999999999999},"min":{"1min":0.92700000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":1.099},{"interface":"front","average":{"1min":1.375,"5min":1.3120000000000001,"15min":1.2370000000000001},"min":{"1min":0.81599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.5209999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.393,"15min":1.399},"min":{"1min":0.86399999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.80700000000000005},"max":{"1min":3.7890000000000001,"5min":3.7890000000000001,"15min":3.7890000000000001},"last":1.339}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3640000000000001,"15min":1.3340000000000001},"min":{"1min":0.86599999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.294,"5min":3.484,"15min":3.484},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.4830000000000001,"15min":1.5349999999999999},"min":{"1min":1.123,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":1.4810000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5049999999999999,"15min":1.6080000000000001},"min":{"1min":1.1539999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.4929999999999999,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.538,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.6120000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.6359999999999999,"15min":1.6799999999999999},"min":{"1min":1.167,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.7},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.653,"15min":1.714},"min":{"1min":1.2190000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.4410000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509522,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":667648,"data_stored":288343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.1599999999999999,"15min":1.0820000000000001},"min":{"1min":0.86399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.103,"15min":1.075},"min":{"1min":0.94399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":2.2400000000000002}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.218,"15min":1.1579999999999999},"min":{"1min":0.65800000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.7689999999999999,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.131},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2689999999999999,"15min":1.2549999999999999},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.821,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.4750000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.27,"15min":1.2350000000000001},"min":{"1min":0.98899999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.3200000000000001,"15min":1.3480000000000001},"min":{"1min":0.996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.3839999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.46,"15min":1.4139999999999999},"min":{"1min":1.1479999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":2.0049999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.98599999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.742,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.841}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.472,"15min":1.502},"min":{"1min":0.92500000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0110000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.228},{"interface":"front","average":{"1min":1.504,"5min":1.3480000000000001,"15min":1.401},"min":{"1min":0.98399999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":2.0529999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.605,"15min":1.629},"min":{"1min":1.0329999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4850000000000001,"15min":1.5389999999999999},"min":{"1min":1.1459999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.286}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":1.516,"15min":1.591},"min":{"1min":1.139,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":2.1480000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.6040000000000001,"15min":1.621},"min":{"1min":1.1020000000000001,"5min":0.63,"15min":0.63},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.528}]}]},{"osd":3,"up_from":15,"seq":64424509521,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":651264,"data_stored":286729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.85299999999999998,"15min":0.86499999999999999},"min":{"1min":0.53000000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.843,"5min":1.867,"15min":1.867},"last":1.1890000000000001},{"interface":"front","average":{"1min":1,"5min":0.98699999999999999,"15min":1.018},"min":{"1min":0.74399999999999999,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4239999999999999,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":1.254}]},{"osd":1,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.98599999999999999,"15min":1.04},"min":{"1min":0.65800000000000003,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.603,"5min":1.966,"15min":3.71},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.149,"5min":1.077,"15min":1.089},"min":{"1min":0.54400000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5660000000000001,"5min":1.923,"15min":2.0190000000000001},"last":1.4019999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.177,"15min":1.097},"min":{"1min":0.73599999999999999,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":1.5049999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.137},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.2909999999999999},"min":{"1min":0.83199999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5389999999999999,"5min":1.952,"15min":3.5710000000000002},"last":1.1040000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3109999999999999,"15min":1.4179999999999999},"min":{"1min":0.97199999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.5640000000000001,"5min":2.411,"15min":3.5030000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.345,"15min":1.349},"min":{"1min":1.1319999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.9370000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.3220000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.379,"15min":1.476},"min":{"1min":1.073,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.7270000000000001,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.72},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4219999999999999,"15min":1.5149999999999999},"min":{"1min":0.86799999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.72,"5min":2.673,"15min":3.3809999999999998},"last":2.0649999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.4510000000000001,"15min":1.4870000000000001},"min":{"1min":1.1910000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.8260000000000001,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.4590000000000001,"15min":1.591},"min":{"1min":0.97099999999999997,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":1.855,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.54}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.512,"15min":1.6120000000000001},"min":{"1min":1.1819999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.968,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":2.1339999999999999},{"interface":"front","average":{"1min":1.647,"5min":1.5680000000000001,"15min":1.5549999999999999},"min":{"1min":1.319,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.0059999999999998,"5min":2.75,"15min":2.75},"last":2.1040000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509521,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45704,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326136,"statfs":{"total":96636764160,"available":96589963264,"internally_reserved":0,"allocated":651264,"data_stored":286673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.89500000000000002,"15min":0.92900000000000005},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.9410000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.95699999999999996,"15min":0.92100000000000004},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.7789999999999999,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":1.071}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":0.96299999999999997,"15min":0.95199999999999996},"min":{"1min":0.82999999999999996,"5min":0.376,"15min":0.376},"max":{"1min":1.893,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.105,"15min":1.127},"min":{"1min":0.68400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.978,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.2629999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.139,"15min":1.2430000000000001},"min":{"1min":0.755,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.796,"5min":2.129,"15min":2.129},"last":1.506},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.22,"15min":1.2729999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.867,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.3360000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.391,"15min":1.4750000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.9970000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.907},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3100000000000001,"15min":1.4039999999999999},"min":{"1min":0.79200000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0449999999999999,"5min":2.3639999999999999,"15min":2.6179999999999999},"last":1.964}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.3959999999999999,"15min":1.48},"min":{"1min":0.96599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.6600000000000001},"last":1.627},{"interface":"front","average":{"1min":1.633,"5min":1.5409999999999999,"15min":1.5649999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.2450000000000001,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.7549999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.4359999999999999,"15min":1.55},"min":{"1min":0.83099999999999996,"5min":0.78500000000000003,"15min":0.76500000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":4.7569999999999997},"last":2.0489999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.538,"15min":1.595},"min":{"1min":1.0229999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.621},"last":2.1190000000000002}]},{"osd":7,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.488,"15min":1.6100000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.343,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.573,"15min":1.623},"min":{"1min":1.0740000000000001,"5min":0.85699999999999998,"15min":0.82799999999999996},"max":{"1min":2.2999999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.9950000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509520,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40724,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331116,"statfs":{"total":96636764160,"available":96595062784,"internally_reserved":0,"allocated":1245184,"data_stored":877077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.84099999999999997,"15min":0.95599999999999996},"min":{"1min":0.59299999999999997,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.47,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.105,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.0489999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":0.99099999999999999,"15min":0.98299999999999998},"min":{"1min":0.71599999999999997,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0229999999999999,"15min":0.999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41999999999999998},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.722},"last":0.85699999999999998}]},{"osd":3,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7629999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.101,"5min":1,"15min":1.0580000000000001},"min":{"1min":0.57899999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6899999999999999,"5min":1.8149999999999999,"15min":1.9059999999999999},"last":0.97299999999999998}]},{"osd":4,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":0.98399999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.0569999999999999},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.363,"15min":1.375},"min":{"1min":1.0860000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.966,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.5109999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.3759999999999999,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83299999999999996},"max":{"1min":2.1629999999999998,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.4099999999999999},"min":{"1min":0.93999999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.0390000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.9410000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2909999999999999,"15min":1.407},"min":{"1min":0.91400000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.986,"5min":1.986,"15min":2.3519999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.4339999999999999,"15min":1.4990000000000001},"min":{"1min":0.89800000000000002,"5min":0.752,"15min":0.752},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.3879999999999999},"last":1.367}]},{"osd":7,"last update":"Mon Nov 13 11:01:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.4410000000000001,"15min":1.496},"min":{"1min":0.93799999999999994,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.972,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":2.0609999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.391,"15min":1.452},"min":{"1min":0.85799999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8660000000000001,"5min":2.202,"15min":2.202},"last":1.4510000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509521,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":651264,"data_stored":287221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.90100000000000002},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32100000000000001},"max":{"1min":1.417,"5min":1.74,"15min":1.74},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.022,"15min":1.048},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.7330000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.89700000000000002}]},{"osd":2,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1459999999999999,"15min":1.141},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":1.391},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0089999999999999,"15min":1.018},"min":{"1min":0.54000000000000004,"5min":0.495,"15min":0.371},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.1669999999999998},"last":0.84399999999999997}]},{"osd":3,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1080000000000001,"15min":1.1020000000000001},"min":{"1min":0.63800000000000001,"5min":0.48999999999999999,"15min":0.45200000000000001},"max":{"1min":1.7450000000000001,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2769999999999999,"15min":1.329},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.827,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.2050000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3300000000000001,"15min":1.3540000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8009999999999999,"5min":2.153,"15min":2.153},"last":1.044},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.296,"15min":1.4279999999999999},"min":{"1min":0.79400000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.708,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.96799999999999997}]},{"osd":5,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.415,"15min":1.5449999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.0430000000000001,"5min":2.2719999999999998,"15min":3.7490000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.3839999999999999,"15min":1.484},"min":{"1min":0.89100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.867,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.4330000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.48,"15min":1.5229999999999999},"min":{"1min":0.91100000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1389999999999998,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5369999999999999,"15min":1.5740000000000001},"min":{"1min":0.872,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.2810000000000001,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.131}]},{"osd":7,"last update":"Mon Nov 13 11:01:25 2023","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.5900000000000001,"15min":1.6779999999999999},"min":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.21,"5min":2.742,"15min":3.3540000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5800000000000001,"15min":1.6140000000000001},"min":{"1min":0.94999999999999996,"5min":0.88100000000000001,"15min":0.77400000000000002},"max":{"1min":2.2200000000000002,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.468}]}]},{"osd":7,"up_from":14,"seq":60129542225,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45708,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326132,"statfs":{"total":96636764160,"available":96589959168,"internally_reserved":0,"allocated":655360,"data_stored":286695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0860000000000001,"15min":1.1120000000000001},"min":{"1min":0.77800000000000002,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6970000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.087,"15min":1.073},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6080000000000001,"5min":2.266,"15min":2.266},"last":1.0620000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.147,"15min":1.0589999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6140000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.294},{"interface":"front","average":{"1min":1.339,"5min":1.2450000000000001,"15min":1.2310000000000001},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3939999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2929999999999999,"15min":1.3029999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8080000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.264,"15min":1.2190000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.4390000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.341,"15min":1.2569999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.829,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.534},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5169999999999999,"15min":1.46},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.925,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":1.5089999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3680000000000001,"15min":1.3300000000000001},"min":{"1min":0.95199999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.907,"5min":3.0190000000000001,"15min":3.0190000000000001},"last":1.337},{"interface":"front","average":{"1min":1.395,"5min":1.4359999999999999,"15min":1.379},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.8420000000000001,"5min":3.2530000000000001,"15min":3.2530000000000001},"last":1.1579999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.5389999999999999,"15min":1.5429999999999999},"min":{"1min":0.92800000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.0640000000000001,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.534,"15min":1.4830000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.9870000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.2010000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:01:28 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.641,"15min":1.5760000000000001},"min":{"1min":0.84899999999999998,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":2.093,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.569},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.5920000000000001,"15min":1.518},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.2349999999999999,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":1.4790000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:01:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-13T11:01:49.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-13T11:01:49.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:01:49.266 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:01:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.411+0000 7f3c67213640 1 Processor -- start 2023-11-13T11:01:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.412+0000 7f3c67213640 1 -- start start 2023-11-13T11:01:49.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.413+0000 7f3c67213640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c6015cf90 0x7f3c60160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.413+0000 7f3c67213640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c60160fa0 0x7f3c601613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.413+0000 7f3c67213640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6006ea90 con 0x7f3c600cf2b0 2023-11-13T11:01:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.413+0000 7f3c67213640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c6015d370 con 0x7f3c60160fa0 2023-11-13T11:01:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.413+0000 7f3c67213640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6006de40 con 0x7f3c6015cf90 2023-11-13T11:01:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.414+0000 7f3c65a10640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c60160fa0 0x7f3c601613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.414+0000 7f3c65a10640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c60160fa0 0x7f3c601613e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42018/0 (socket says 172.21.15.16:42018) 2023-11-13T11:01:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.414+0000 7f3c65a10640 1 -- 172.21.15.16:0/4156122212 learned_addr learned my addr 172.21.15.16:0/4156122212 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.414+0000 7f3c65a10640 1 -- 172.21.15.16:0/4156122212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c6015cf90 msgr2=0x7f3c60160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.414+0000 7f3c65a10640 1 --2- 172.21.15.16:0/4156122212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c6015cf90 0x7f3c60160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.414+0000 7f3c65a10640 1 -- 172.21.15.16:0/4156122212 >> v1:172.21.15.184:6789/0 conn(0x7f3c600cf2b0 legacy=0x7f3c600cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.414+0000 7f3c65a10640 1 -- 172.21.15.16:0/4156122212 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6006e120 con 0x7f3c60160fa0 2023-11-13T11:01:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.415+0000 7f3c65a10640 1 --2- 172.21.15.16:0/4156122212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c60160fa0 0x7f3c601613e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009870 tx=0x7f3c5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321c2283e1cc3cdb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.416+0000 7f3c6520f640 1 -- 172.21.15.16:0/4156122212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50031640 con 0x7f3c60160fa0 2023-11-13T11:01:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.416+0000 7f3c6520f640 1 -- 172.21.15.16:0/4156122212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c500317a0 con 0x7f3c60160fa0 2023-11-13T11:01:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.416+0000 7f3c6520f640 1 -- 172.21.15.16:0/4156122212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50031b30 con 0x7f3c60160fa0 2023-11-13T11:01:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.416+0000 7f3c67213640 1 -- 172.21.15.16:0/4156122212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c60160fa0 msgr2=0x7f3c601613e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.416+0000 7f3c67213640 1 --2- 172.21.15.16:0/4156122212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c60160fa0 0x7f3c601613e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009870 tx=0x7f3c5002b8f0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.417+0000 7f3c67213640 1 -- 172.21.15.16:0/4156122212 shutdown_connections 2023-11-13T11:01:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.417+0000 7f3c67213640 1 --2- 172.21.15.16:0/4156122212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c60160fa0 0x7f3c601613e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.417+0000 7f3c67213640 1 --2- 172.21.15.16:0/4156122212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c6015cf90 0x7f3c60160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.417+0000 7f3c67213640 1 -- 172.21.15.16:0/4156122212 >> 172.21.15.16:0/4156122212 conn(0x7f3c600b24b0 msgr2=0x7f3c600b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:49.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.417+0000 7f3c67213640 1 -- 172.21.15.16:0/4156122212 shutdown_connections 2023-11-13T11:01:49.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.417+0000 7f3c67213640 1 -- 172.21.15.16:0/4156122212 wait complete. 2023-11-13T11:01:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.419+0000 7f3c67213640 1 Processor -- start 2023-11-13T11:01:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.419+0000 7f3c67213640 1 -- start start 2023-11-13T11:01:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c67213640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c600cf2b0 0x7f3c6015b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c67213640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c6015cf90 0x7f3c6015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c67213640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c60160fa0 0x7f3c600c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c67213640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6015b820 con 0x7f3c600cf2b0 2023-11-13T11:01:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c67213640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600c7360 con 0x7f3c60160fa0 2023-11-13T11:01:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c67213640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600c6de0 con 0x7f3c6015cf90 2023-11-13T11:01:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c66211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c600cf2b0 0x7f3c6015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c66211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c600cf2b0 0x7f3c6015b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42026/0 (socket says 172.21.15.16:42026) 2023-11-13T11:01:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c66211640 1 -- 172.21.15.16:0/2127101972 learned_addr learned my addr 172.21.15.16:0/2127101972 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c66211640 1 -- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c6015cf90 msgr2=0x7f3c6015b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.420+0000 7f3c66211640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c6015cf90 0x7f3c6015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c66211640 1 -- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c60160fa0 msgr2=0x7f3c600c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c66a12640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c60160fa0 0x7f3c600c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c66211640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c60160fa0 0x7f3c600c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c66211640 1 -- 172.21.15.16:0/2127101972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c500094d0 con 0x7f3c600cf2b0 2023-11-13T11:01:49.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c66a12640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c60160fa0 0x7f3c600c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:49.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c66211640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c600cf2b0 0x7f3c6015b990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00b4a0 tx=0x7f3c5c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:49.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c56ffd640 1 -- 172.21.15.16:0/2127101972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c010610 con 0x7f3c600cf2b0 2023-11-13T11:01:49.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c56ffd640 1 -- 172.21.15.16:0/2127101972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c010770 con 0x7f3c600cf2b0 2023-11-13T11:01:49.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c56ffd640 1 -- 172.21.15.16:0/2127101972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c010b20 con 0x7f3c600cf2b0 2023-11-13T11:01:49.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c600c7080 con 0x7f3c600cf2b0 2023-11-13T11:01:49.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.421+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c600c5360 con 0x7f3c600cf2b0 2023-11-13T11:01:49.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.423+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c600c4ee0 con 0x7f3c600cf2b0 2023-11-13T11:01:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.424+0000 7f3c56ffd640 1 -- 172.21.15.16:0/2127101972 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c5c010c80 con 0x7f3c600cf2b0 2023-11-13T11:01:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.425+0000 7f3c56ffd640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.425+0000 7f3c56ffd640 1 -- 172.21.15.16:0/2127101972 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3c5c0893c0 con 0x7f3c600cf2b0 2023-11-13T11:01:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.425+0000 7f3c65a10640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.427+0000 7f3c65a10640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c4406ae40 0x7f3c4406d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3c50036000 tx=0x7f3c50030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:49.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.429+0000 7f3c56ffd640 1 -- 172.21.15.16:0/2127101972 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c600c4ee0 con 0x7f3c600cf2b0 2023-11-13T11:01:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.697+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3c600c4ee0 con 0x7f3c600cf2b0 2023-11-13T11:01:49.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.702+0000 7f3c56ffd640 1 -- 172.21.15.16:0/2127101972 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3c600c4ee0 con 0x7f3c600cf2b0 2023-11-13T11:01:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-13T11:01:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBdAFJl9p66LRAAYRThccLplmNpsr+W647dOA== 2023-11-13T11:01:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2023-11-13T11:01:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2023-11-13T11:01:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-13T11:01:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-11-13T11:01:49.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.705+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c4406ae40 msgr2=0x7f3c4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.705+0000 7f3c67213640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c4406ae40 0x7f3c4406d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3c50036000 tx=0x7f3c50030000 comp rx=0 tx=0).stop 2023-11-13T11:01:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.705+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c600cf2b0 msgr2=0x7f3c6015b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.705+0000 7f3c67213640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c600cf2b0 0x7f3c6015b990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00b4a0 tx=0x7f3c5c00b960 comp rx=0 tx=0).stop 2023-11-13T11:01:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.706+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 shutdown_connections 2023-11-13T11:01:49.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.706+0000 7f3c67213640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.706+0000 7f3c67213640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c60160fa0 0x7f3c600c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.706+0000 7f3c67213640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c6015cf90 0x7f3c6015b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.706+0000 7f3c67213640 1 --2- 172.21.15.16:0/2127101972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c600cf2b0 0x7f3c6015b990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.706+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 >> 172.21.15.16:0/2127101972 conn(0x7f3c600b24b0 msgr2=0x7f3c6006bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:49.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.706+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 shutdown_connections 2023-11-13T11:01:49.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.706+0000 7f3c67213640 1 -- 172.21.15.16:0/2127101972 wait complete. 2023-11-13T11:01:49.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:01:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.935+0000 7f546451d640 1 Processor -- start 2023-11-13T11:01:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.936+0000 7f546451d640 1 -- start start 2023-11-13T11:01:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.937+0000 7f546451d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f545c06c3c0 0x7f545c15c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.937+0000 7f546451d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0b6890 con 0x7f545c06b4b0 2023-11-13T11:01:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.937+0000 7f546451d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0b42d0 con 0x7f545c0b64b0 2023-11-13T11:01:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.937+0000 7f546451d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c0d0180 con 0x7f545c06c3c0 2023-11-13T11:01:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.938+0000 7f5461ab8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f545c06b4b0 0x7f545c06bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59954/0 (socket says 172.21.15.16:59954) 2023-11-13T11:01:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.938+0000 7f5461ab8640 1 -- 172.21.15.16:0/1229813734 learned_addr learned my addr 172.21.15.16:0/1229813734 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.938+0000 7f5462aba640 1 --2- 172.21.15.16:0/1229813734 >> v2:172.21.15.184:3300/0 conn(0x7f545c06c3c0 0x7f545c15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.938+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065037151 0 0) 0x7f544c001110 con 0x7f545c06b4b0 2023-11-13T11:01:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.938+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5448003520 con 0x7f545c06b4b0 2023-11-13T11:01:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3377118225 0 0) 0x7f5448003520 con 0x7f545c06b4b0 2023-11-13T11:01:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 >> v2:172.21.15.184:3300/0 conn(0x7f545c06c3c0 msgr2=0x7f545c15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 --2- 172.21.15.16:0/1229813734 >> v2:172.21.15.184:3300/0 conn(0x7f545c06c3c0 0x7f545c15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 >> v1:172.21.15.184:6789/0 conn(0x7f545c0b64b0 legacy=0x7f545c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0b39a0 con 0x7f545c06b4b0 2023-11-13T11:01:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f544c001800 con 0x7f545c06b4b0 2023-11-13T11:01:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 >> v1:172.21.15.16:6789/0 conn(0x7f545c06b4b0 legacy=0x7f545c06bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480039c0 0x7f5448003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480047c0 0x7f5448004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5448005180 0x7f544800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0d0180 con 0x7f54480047c0 2023-11-13T11:01:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544800c080 con 0x7f54480039c0 2023-11-13T11:01:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544800c240 con 0x7f5448005180 2023-11-13T11:01:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.939+0000 7f5461ab8640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480047c0 0x7f5448004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.940+0000 7f54622b9640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480039c0 0x7f5448003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.940+0000 7f5461ab8640 1 -- 172.21.15.16:0/1229813734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5448005180 msgr2=0x7f544800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.940+0000 7f5461ab8640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5448005180 0x7f544800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.940+0000 7f5461ab8640 1 -- 172.21.15.16:0/1229813734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480039c0 msgr2=0x7f5448003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.940+0000 7f5461ab8640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480039c0 0x7f5448003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.940+0000 7f5461ab8640 1 -- 172.21.15.16:0/1229813734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5448003520 con 0x7f54480047c0 2023-11-13T11:01:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.940+0000 7f54622b9640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480039c0 0x7f5448003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.940+0000 7f5462aba640 1 -- 172.21.15.16:0/1229813734 reap_dead start 2023-11-13T11:01:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.941+0000 7f5461ab8640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480047c0 0x7f5448004c40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f544c007fc0 tx=0x7f54480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fcf77d83c519f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.941+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c005ad0 con 0x7f54480047c0 2023-11-13T11:01:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.941+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c005c50 con 0x7f54480047c0 2023-11-13T11:01:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.941+0000 7f54612b7640 1 -- 172.21.15.16:0/1229813734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c02c7c0 con 0x7f54480047c0 2023-11-13T11:01:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.941+0000 7f546451d640 1 -- 172.21.15.16:0/1229813734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480047c0 msgr2=0x7f5448004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.942+0000 7f546451d640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480047c0 0x7f5448004c40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f544c007fc0 tx=0x7f54480057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.942+0000 7f546451d640 1 -- 172.21.15.16:0/1229813734 shutdown_connections 2023-11-13T11:01:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.942+0000 7f546451d640 1 --2- 172.21.15.16:0/1229813734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54480047c0 0x7f5448004c40 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.942+0000 7f546451d640 1 -- 172.21.15.16:0/1229813734 >> 172.21.15.16:0/1229813734 conn(0x7f545c0b2600 msgr2=0x7f545c0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.942+0000 7f546451d640 1 -- 172.21.15.16:0/1229813734 shutdown_connections 2023-11-13T11:01:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.942+0000 7f546451d640 1 -- 172.21.15.16:0/1229813734 wait complete. 2023-11-13T11:01:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.944+0000 7f546451d640 1 Processor -- start 2023-11-13T11:01:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.944+0000 7f546451d640 1 -- start start 2023-11-13T11:01:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f546451d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480047c0 0x7f545c14bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f546451d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f545c158540 0x7f545c09d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f546451d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545c09d810 0x7f545c15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f546451d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c09e8a0 con 0x7f545c09d810 2023-11-13T11:01:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f546451d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c09dd20 con 0x7f54480047c0 2023-11-13T11:01:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f546451d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c09e000 con 0x7f545c158540 2023-11-13T11:01:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f5462aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545c09d810 0x7f545c15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f5462aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545c09d810 0x7f545c15ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42050/0 (socket says 172.21.15.16:42050) 2023-11-13T11:01:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f5462aba640 1 -- 172.21.15.16:0/349811501 learned_addr learned my addr 172.21.15.16:0/349811501 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f5462aba640 1 -- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f545c158540 msgr2=0x7f545c09d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f5462aba640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f545c158540 0x7f545c09d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f54622b9640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480047c0 0x7f545c14bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.945+0000 7f5462aba640 1 -- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480047c0 msgr2=0x7f545c14bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.946+0000 7f5462aba640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480047c0 0x7f545c14bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.946+0000 7f5462aba640 1 -- 172.21.15.16:0/349811501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c09e2e0 con 0x7f545c09d810 2023-11-13T11:01:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.946+0000 7f54622b9640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480047c0 0x7f545c14bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.946+0000 7f5462aba640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545c09d810 0x7f545c15ec40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f545c0ce560 tx=0x7f545400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.946+0000 7f5452ffd640 1 -- 172.21.15.16:0/349811501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454010040 con 0x7f545c09d810 2023-11-13T11:01:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.946+0000 7f5452ffd640 1 -- 172.21.15.16:0/349811501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545400bc80 con 0x7f545c09d810 2023-11-13T11:01:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.946+0000 7f5452ffd640 1 -- 172.21.15.16:0/349811501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454014400 con 0x7f545c09d810 2023-11-13T11:01:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.946+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c09e5c0 con 0x7f545c09d810 2023-11-13T11:01:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.946+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c15f360 con 0x7f545c09d810 2023-11-13T11:01:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.948+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c067090 con 0x7f545c09d810 2023-11-13T11:01:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.949+0000 7f5452ffd640 1 -- 172.21.15.16:0/349811501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5454014560 con 0x7f545c09d810 2023-11-13T11:01:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.950+0000 7f5452ffd640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f544006ae40 0x7f544006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.950+0000 7f5452ffd640 1 -- 172.21.15.16:0/349811501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5454089fb0 con 0x7f545c09d810 2023-11-13T11:01:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.950+0000 7f54622b9640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f544006ae40 0x7f544006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.952+0000 7f54622b9640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f544006ae40 0x7f544006d300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f545c063e10 tx=0x7f5458009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:49.954+0000 7f5452ffd640 1 -- 172.21.15.16:0/349811501 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5454054f10 con 0x7f545c09d810 2023-11-13T11:01:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.186+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f545c06fb90 con 0x7f545c09d810 2023-11-13T11:01:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.187+0000 7f5452ffd640 1 -- 172.21.15.16:0/349811501 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f5454018090 con 0x7f545c09d810 2023-11-13T11:01:50.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:01:50.193 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:01:41.484670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:01:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2023-11-13T11:01:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f544006ae40 msgr2=0x7f544006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f544006ae40 0x7f544006d300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f545c063e10 tx=0x7f5458009040 comp rx=0 tx=0).stop 2023-11-13T11:01:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545c09d810 msgr2=0x7f545c15ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545c09d810 0x7f545c15ec40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f545c0ce560 tx=0x7f545400b480 comp rx=0 tx=0).stop 2023-11-13T11:01:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 shutdown_connections 2023-11-13T11:01:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f544006ae40 0x7f544006d300 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545c09d810 0x7f545c15ec40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f545c158540 0x7f545c09d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 --2- 172.21.15.16:0/349811501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54480047c0 0x7f545c14bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 >> 172.21.15.16:0/349811501 conn(0x7f545c0b2600 msgr2=0x7f545c0c48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 shutdown_connections 2023-11-13T11:01:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.190+0000 7f546451d640 1 -- 172.21.15.16:0/349811501 wait complete. 2023-11-13T11:01:50.618 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-13T11:01:38.428966+0000', 'modified': '2023-11-13T11:01:41.484670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5272': {'gid': 5272, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/3328237273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 3328237273}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 3328237273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-13T11:01:50.618 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-13T11:01:50.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-13T11:01:50.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.855+0000 7f34fffe2640 1 Processor -- start 2023-11-13T11:01:50.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.855+0000 7f34fffe2640 1 -- start start 2023-11-13T11:01:50.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.856+0000 7f34fffe2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f34f80ccc60 0x7f34f8071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.856+0000 7f34fffe2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34f8071980 0x7f34f806fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.856+0000 7f34fffe2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f80cd110 con 0x7f34f8155410 2023-11-13T11:01:50.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.856+0000 7f34fffe2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f8071e10 con 0x7f34f80ccc60 2023-11-13T11:01:50.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.856+0000 7f34fffe2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f815b500 con 0x7f34f8071980 2023-11-13T11:01:50.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.857+0000 7f34fdd7e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f34f8155410 0x7f34f80ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59960/0 (socket says 172.21.15.16:59960) 2023-11-13T11:01:50.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.857+0000 7f34fdd7e640 1 -- 172.21.15.16:0/1157390377 learned_addr learned my addr 172.21.15.16:0/1157390377 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:50.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.857+0000 7f34fd57d640 1 --2- 172.21.15.16:0/1157390377 >> v2:172.21.15.184:3300/0 conn(0x7f34f80ccc60 0x7f34f8071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.857+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242711834 0 0) 0x7f34f80cd110 con 0x7f34f8155410 2023-11-13T11:01:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.857+0000 7f34fe57f640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34f8071980 0x7f34f806fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.857+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e4003520 con 0x7f34f8155410 2023-11-13T11:01:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652738798 0 0) 0x7f34e4003520 con 0x7f34f8155410 2023-11-13T11:01:50.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34f8071980 msgr2=0x7f34f806fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34f8071980 0x7f34f806fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 >> v2:172.21.15.184:3300/0 conn(0x7f34f80ccc60 msgr2=0x7f34f8071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 --2- 172.21.15.16:0/1157390377 >> v2:172.21.15.184:3300/0 conn(0x7f34f80ccc60 0x7f34f8071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f815b7e0 con 0x7f34f8155410 2023-11-13T11:01:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f34f40019f0 con 0x7f34f8155410 2023-11-13T11:01:50.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 >> v1:172.21.15.16:6789/0 conn(0x7f34f8155410 legacy=0x7f34f80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e4003970 0x7f34e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34e40047c0 0x7f34e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34f8071980 0x7f34e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8071e10 con 0x7f34e4003970 2023-11-13T11:01:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f815b500 con 0x7f34e40047c0 2023-11-13T11:01:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.858+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e400ba00 con 0x7f34f8071980 2023-11-13T11:01:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.859+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e4003970 0x7f34e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.859+0000 7f34fe57f640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34f8071980 0x7f34e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.859+0000 7f34fd57d640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34e40047c0 0x7f34e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.859+0000 7f34fdd7e640 1 -- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34f8071980 msgr2=0x7f34e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.859+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34f8071980 0x7f34e4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.859+0000 7f34fdd7e640 1 -- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34e40047c0 msgr2=0x7f34e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.859+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34e40047c0 0x7f34e4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.859+0000 7f34fdd7e640 1 -- 172.21.15.16:0/1157390377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4003520 con 0x7f34e4003970 2023-11-13T11:01:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.860+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e4003970 0x7f34e4003d90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f34f400a780 tx=0x7f34e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8877411178c71b15 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.860+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4034630 con 0x7f34e4003970 2023-11-13T11:01:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.860+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f402f070 con 0x7f34e4003970 2023-11-13T11:01:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.860+0000 7f34fcd7c640 1 -- 172.21.15.16:0/1157390377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f40349d0 con 0x7f34e4003970 2023-11-13T11:01:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.861+0000 7f34fffe2640 1 -- 172.21.15.16:0/1157390377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e4003970 msgr2=0x7f34e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.861+0000 7f34fffe2640 1 --2- 172.21.15.16:0/1157390377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e4003970 0x7f34e4003d90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f34f400a780 tx=0x7f34e40057f0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.861+0000 7f34fe57f640 1 -- 172.21.15.16:0/1157390377 reap_dead start 2023-11-13T11:01:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.861+0000 7f34fffe2640 1 -- 172.21.15.16:0/1157390377 shutdown_connections 2023-11-13T11:01:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.861+0000 7f34fffe2640 1 -- 172.21.15.16:0/1157390377 >> 172.21.15.16:0/1157390377 conn(0x7f34f806a190 msgr2=0x7f34f806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.861+0000 7f34fffe2640 1 -- 172.21.15.16:0/1157390377 shutdown_connections 2023-11-13T11:01:50.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.861+0000 7f34fffe2640 1 -- 172.21.15.16:0/1157390377 wait complete. 2023-11-13T11:01:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.863+0000 7f34fffe2640 1 Processor -- start 2023-11-13T11:01:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.863+0000 7f34fffe2640 1 -- start start 2023-11-13T11:01:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.864+0000 7f34fffe2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34e4003970 0x7f34f815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.864+0000 7f34fffe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8150ff0 0x7f34f8151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.864+0000 7f34fffe2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f8148d90 0x7f34f8149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.864+0000 7f34fffe2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f806c1c0 con 0x7f34f8150ff0 2023-11-13T11:01:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.864+0000 7f34fffe2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f81559f0 con 0x7f34f8148d90 2023-11-13T11:01:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.864+0000 7f34fffe2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f806bb90 con 0x7f34e4003970 2023-11-13T11:01:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.864+0000 7f34fd57d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8150ff0 0x7f34f8151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.864+0000 7f34fd57d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8150ff0 0x7f34f8151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42068/0 (socket says 172.21.15.16:42068) 2023-11-13T11:01:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.864+0000 7f34fd57d640 1 -- 172.21.15.16:0/2496369753 learned_addr learned my addr 172.21.15.16:0/2496369753 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fe57f640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f8148d90 0x7f34f8149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34e4003970 0x7f34f815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fd57d640 1 -- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34e4003970 msgr2=0x7f34f815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fd57d640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34e4003970 0x7f34f815d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fd57d640 1 -- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f8148d90 msgr2=0x7f34f8149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fd57d640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f8148d90 0x7f34f8149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fd57d640 1 -- 172.21.15.16:0/2496369753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8149750 con 0x7f34f8150ff0 2023-11-13T11:01:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fd57d640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8150ff0 0x7f34f8151470 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f34e80108e0 tx=0x7f34e8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:50.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34ee7fc640 1 -- 172.21.15.16:0/2496369753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800eb60 con 0x7f34f8150ff0 2023-11-13T11:01:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34ee7fc640 1 -- 172.21.15.16:0/2496369753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e800ecc0 con 0x7f34f8150ff0 2023-11-13T11:01:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34ee7fc640 1 -- 172.21.15.16:0/2496369753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e801c400 con 0x7f34f8150ff0 2023-11-13T11:01:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fffe2640 1 -- 172.21.15.16:0/2496369753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f8151a70 con 0x7f34f8150ff0 2023-11-13T11:01:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.865+0000 7f34fffe2640 1 -- 172.21.15.16:0/2496369753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f806bde0 con 0x7f34f8150ff0 2023-11-13T11:01:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.868+0000 7f34ee7fc640 1 -- 172.21.15.16:0/2496369753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f34e801c560 con 0x7f34f8150ff0 2023-11-13T11:01:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.869+0000 7f34ee7fc640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f34dc06ae40 0x7f34dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.869+0000 7f34ee7fc640 1 -- 172.21.15.16:0/2496369753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f34e8089dd0 con 0x7f34f8150ff0 2023-11-13T11:01:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.870+0000 7f3502b79740 1 Processor -- start 2023-11-13T11:01:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.870+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f34dc06ae40 0x7f34dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.870+0000 7f3502b79740 1 -- start start 2023-11-13T11:01:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f3502b79740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6ede0 0x55d8cab6f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f3502b79740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab69ae0 0x55d8cab6dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f3502b79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6e520 0x55d8cabb24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f3502b79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8caacd880 con 0x55d8cab6e520 2023-11-13T11:01:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f3502b79740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8caadcae0 con 0x55d8cab69ae0 2023-11-13T11:01:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f3502b79740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8caada1d0 con 0x55d8cab6ede0 2023-11-13T11:01:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f34dc06ae40 0x7f34dc06d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f34f40013d0 tx=0x7f34f4009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f34fe57f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6e520 0x55d8cabb24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f34fe57f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6e520 0x55d8cabb24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42076/0 (socket says 172.21.15.16:42076) 2023-11-13T11:01:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.871+0000 7f34fe57f640 1 -- 172.21.15.16:0/3826793445 learned_addr learned my addr 172.21.15.16:0/3826793445 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.872+0000 7f34fd57d640 1 --2- 172.21.15.16:0/3826793445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab69ae0 0x55d8cab6dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.872+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/3826793445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6ede0 0x55d8cab6f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.872+0000 7f34fe57f640 1 -- 172.21.15.16:0/3826793445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6ede0 msgr2=0x55d8cab6f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.872+0000 7f34fe57f640 1 --2- 172.21.15.16:0/3826793445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6ede0 0x55d8cab6f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.872+0000 7f34fe57f640 1 -- 172.21.15.16:0/3826793445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab69ae0 msgr2=0x55d8cab6dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.872+0000 7f34fe57f640 1 --2- 172.21.15.16:0/3826793445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab69ae0 0x55d8cab6dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.872+0000 7f34fe57f640 1 -- 172.21.15.16:0/3826793445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8caa9d860 con 0x55d8cab6e520 2023-11-13T11:01:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.872+0000 7f34fe57f640 1 --2- 172.21.15.16:0/3826793445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6e520 0x55d8cabb24a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f34f000f440 tx=0x7f34f000f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3d3978aa1f5acc server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f34db7fe640 1 -- 172.21.15.16:0/3826793445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f00134c0 con 0x55d8cab6e520 2023-11-13T11:01:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f34db7fe640 1 -- 172.21.15.16:0/3826793445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f000c030 con 0x55d8cab6e520 2023-11-13T11:01:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f34db7fe640 1 -- 172.21.15.16:0/3826793445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0013880 con 0x55d8cab6e520 2023-11-13T11:01:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f3502b79740 1 -- 172.21.15.16:0/3826793445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6e520 msgr2=0x55d8cabb24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f3502b79740 1 --2- 172.21.15.16:0/3826793445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6e520 0x55d8cabb24a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f34f000f440 tx=0x7f34f000f900 comp rx=0 tx=0).stop 2023-11-13T11:01:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f3502b79740 1 -- 172.21.15.16:0/3826793445 shutdown_connections 2023-11-13T11:01:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f3502b79740 1 --2- 172.21.15.16:0/3826793445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6e520 0x55d8cabb24a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f3502b79740 1 --2- 172.21.15.16:0/3826793445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab69ae0 0x55d8cab6dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f3502b79740 1 --2- 172.21.15.16:0/3826793445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6ede0 0x55d8cab6f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f3502b79740 1 -- 172.21.15.16:0/3826793445 >> 172.21.15.16:0/3826793445 conn(0x55d8cab30bb0 msgr2=0x55d8cabb48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:50.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f3502b79740 1 -- 172.21.15.16:0/3826793445 shutdown_connections 2023-11-13T11:01:50.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.873+0000 7f3502b79740 1 -- 172.21.15.16:0/3826793445 wait complete. 2023-11-13T11:01:50.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.874+0000 7f3502b79740 1 -- start start 2023-11-13T11:01:50.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.874+0000 7f3502b79740 1 Processor -- start 2023-11-13T11:01:50.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.875+0000 7f3502b79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab85f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.875+0000 7f3502b79740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 0x55d8cab971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.875+0000 7f3502b79740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6ede0 0x55d8cabb7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.875+0000 7f3502b79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8caad5ac0 con 0x55d8cab69ae0 2023-11-13T11:01:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.875+0000 7f3502b79740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8caa722f0 con 0x55d8cab6e520 2023-11-13T11:01:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.875+0000 7f3502b79740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8caa8fc70 con 0x55d8cab6ede0 2023-11-13T11:01:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.875+0000 7f3502b79740 10 client.0 _ll_register_callbacks cb 0x55d8caa943b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:01:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.876+0000 7f34fdd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab85f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.876+0000 7f34fdd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab85f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42092/0 (socket says 172.21.15.16:42092) 2023-11-13T11:01:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.876+0000 7f34fdd7e640 1 -- 172.21.15.16:0/2102833868 learned_addr learned my addr 172.21.15.16:0/2102833868 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.876+0000 7f34fdd7e640 1 -- 172.21.15.16:0/2102833868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6ede0 msgr2=0x55d8cabb7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.876+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6ede0 0x55d8cabb7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.876+0000 7f34fdd7e640 1 -- 172.21.15.16:0/2102833868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 msgr2=0x55d8cab971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.876+0000 7f34fd57d640 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 0x55d8cab971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.876+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 0x55d8cab971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.876+0000 7f34fdd7e640 1 -- 172.21.15.16:0/2102833868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f000e040 con 0x55d8cab69ae0 2023-11-13T11:01:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab85f50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f34f402cba0 tx=0x7f34f4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af61f773d41a98a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f34d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:01:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2102833868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4034b00 con 0x55d8cab69ae0 2023-11-13T11:01:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2102833868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f34f4045040 con 0x55d8cab69ae0 2023-11-13T11:01:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f3502b79740 1 -- 172.21.15.16:0/2102833868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8caaceaa0 con 0x55d8cab69ae0 2023-11-13T11:01:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2102833868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34f4034c60 con 0x55d8cab69ae0 2023-11-13T11:01:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f3502b79740 10 client.6159 fetch_fsmap learned FSMap version 28 2023-11-13T11:01:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f3502b79740 1 -- 172.21.15.16:0/2102833868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55d8caad4140 con 0x55d8cab69ae0 2023-11-13T11:01:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2102833868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8caaceaa0 con 0x55d8cab69ae0 2023-11-13T11:01:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.877+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2102833868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f34f404a430 con 0x55d8cab69ae0 2023-11-13T11:01:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.878+0000 7f3502b79740 10 client.6159 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:01:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.878+0000 7f3502b79740 10 client.6159 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T11:01:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.878+0000 7f3502b79740 20 client.6159 populate_metadata read hostname 'smithi016' 2023-11-13T11:01:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.878+0000 7f3502b79740 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d8cabb7f50 0x55d8cabb9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.878+0000 7f3502b79740 4 client.6159 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:01:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.878+0000 7f3502b79740 1 -- 172.21.15.16:0/2102833868 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8caace7c0 con 0x55d8cabb7f50 2023-11-13T11:01:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.878+0000 7f34fd57d640 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d8cabb7f50 0x55d8cabb9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.878+0000 7f34fd57d640 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d8cabb7f50 0x55d8cabb9ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64bfd18ac7ae211f server_cookie=772c763bb35ca3d5 in_seq=0 out_seq=0 2023-11-13T11:01:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.878+0000 7f34d8ff9640 10 client.6159 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:01:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2102833868 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8caace7c0 con 0x55d8cabb7f50 2023-11-13T11:01:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f34d8ff9640 10 client.6159 handle_command_reply: tid=0 2023-11-13T11:01:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f3502b79740 1 client.6159 shutdown 2023-11-13T11:01:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f3502b79740 1 -- 172.21.15.16:0/2102833868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 msgr2=0x55d8cab85f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f3502b79740 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab85f50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f34f402cba0 tx=0x7f34f4001cb0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f3502b79740 1 -- 172.21.15.16:0/2102833868 shutdown_connections 2023-11-13T11:01:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f3502b79740 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6ede0 0x55d8cabb7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f3502b79740 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 0x55d8cab971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f3502b79740 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d8cabb7f50 0x55d8cabb9ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f3502b79740 1 --2- 172.21.15.16:0/2102833868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab85f50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.879+0000 7f3502b79740 1 -- 172.21.15.16:0/2102833868 >> 172.21.15.16:0/2102833868 conn(0x55d8cab30bb0 msgr2=0x55d8cabb4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.880+0000 7f34d8ff9640 0 client.6159 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:01:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.880+0000 7f3502b79740 1 -- 172.21.15.16:0/2102833868 shutdown_connections 2023-11-13T11:01:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.880+0000 7f3502b79740 1 -- 172.21.15.16:0/2102833868 wait complete. 2023-11-13T11:01:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.880+0000 7f3502b79740 20 client.6159 trim_cache size 0 max 16384 2023-11-13T11:01:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.897+0000 7f3502b79740 1 Processor -- start 2023-11-13T11:01:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.897+0000 7f3502b79740 1 -- start start 2023-11-13T11:01:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.898+0000 7f3502b79740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab69ae0 0x55d8cab6a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.898+0000 7f3502b79740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 0x55d8cab68ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.898+0000 7f3502b79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6ede0 0x55d8cab69000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.898+0000 7f3502b79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ca838f20 con 0x55d8cab6ede0 2023-11-13T11:01:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.898+0000 7f3502b79740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cab6b890 con 0x55d8cab6e520 2023-11-13T11:01:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.898+0000 7f3502b79740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8caade250 con 0x55d8cab69ae0 2023-11-13T11:01:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.898+0000 7f34fe57f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6ede0 0x55d8cab69000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.898+0000 7f34fe57f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6ede0 0x55d8cab69000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42104/0 (socket says 172.21.15.16:42104) 2023-11-13T11:01:50.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.898+0000 7f34fe57f640 1 -- 172.21.15.16:0/184580534 learned_addr learned my addr 172.21.15.16:0/184580534 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:50.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34fd57d640 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 0x55d8cab68ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab69ae0 0x55d8cab6a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34fe57f640 1 -- 172.21.15.16:0/184580534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab69ae0 msgr2=0x55d8cab6a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34fe57f640 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab69ae0 0x55d8cab6a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34fe57f640 1 -- 172.21.15.16:0/184580534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 msgr2=0x55d8cab68ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34fe57f640 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 0x55d8cab68ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34fe57f640 1 -- 172.21.15.16:0/184580534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8caa71b80 con 0x55d8cab6ede0 2023-11-13T11:01:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab69ae0 0x55d8cab6a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:50.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34fe57f640 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6ede0 0x55d8cab69000 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f34f0017040 tx=0x7f34f0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841d179f9381c5aa server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.899+0000 7f34dbfff640 1 -- 172.21.15.16:0/184580534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0013500 con 0x55d8cab6ede0 2023-11-13T11:01:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f34dbfff640 1 -- 172.21.15.16:0/184580534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f0013680 con 0x55d8cab6ede0 2023-11-13T11:01:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f34dbfff640 1 -- 172.21.15.16:0/184580534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0013a30 con 0x55d8cab6ede0 2023-11-13T11:01:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f3502b79740 1 -- 172.21.15.16:0/184580534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6ede0 msgr2=0x55d8cab69000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f3502b79740 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6ede0 0x55d8cab69000 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f34f0017040 tx=0x7f34f0012000 comp rx=0 tx=0).stop 2023-11-13T11:01:50.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f3502b79740 1 -- 172.21.15.16:0/184580534 shutdown_connections 2023-11-13T11:01:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f3502b79740 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab6ede0 0x55d8cab69000 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f3502b79740 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6e520 0x55d8cab68ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f3502b79740 1 --2- 172.21.15.16:0/184580534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab69ae0 0x55d8cab6a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f3502b79740 1 -- 172.21.15.16:0/184580534 >> 172.21.15.16:0/184580534 conn(0x55d8cab30bb0 msgr2=0x55d8cab6fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f3502b79740 1 -- 172.21.15.16:0/184580534 shutdown_connections 2023-11-13T11:01:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.900+0000 7f3502b79740 1 -- 172.21.15.16:0/184580534 wait complete. 2023-11-13T11:01:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.901+0000 7f3502b79740 1 -- start start 2023-11-13T11:01:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.901+0000 7f3502b79740 1 Processor -- start 2023-11-13T11:01:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f3502b79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab68ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f3502b79740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6e520 0x55d8cac217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f3502b79740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6ede0 0x55d8cac21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f3502b79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cac1e4b0 con 0x55d8cab69ae0 2023-11-13T11:01:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f3502b79740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cabbb300 con 0x55d8cab6ede0 2023-11-13T11:01:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f3502b79740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8cabbb550 con 0x55d8cab6e520 2023-11-13T11:01:50.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f3502b79740 10 client.0 _ll_register_callbacks cb 0x7f34e40036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:01:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f34fdd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab68ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f34fdd7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab68ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42120/0 (socket says 172.21.15.16:42120) 2023-11-13T11:01:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.902+0000 7f34fdd7e640 1 -- 172.21.15.16:0/2283963510 learned_addr learned my addr 172.21.15.16:0/2283963510 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34fd57d640 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6e520 0x55d8cac217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34fe57f640 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6ede0 0x55d8cac21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34fdd7e640 1 -- 172.21.15.16:0/2283963510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6e520 msgr2=0x55d8cac217e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6e520 0x55d8cac217e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34fdd7e640 1 -- 172.21.15.16:0/2283963510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6ede0 msgr2=0x55d8cac21d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:50.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6ede0 0x55d8cac21d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34fdd7e640 1 -- 172.21.15.16:0/2283963510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8caa9d860 con 0x55d8cab69ae0 2023-11-13T11:01:50.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34fe57f640 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6ede0 0x55d8cac21d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:50.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34fdd7e640 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab68ec0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f34f4002230 tx=0x7f34f4002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88349e084c5d27 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:50.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.903+0000 7f34d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:01:50.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2283963510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f404ab40 con 0x55d8cab69ae0 2023-11-13T11:01:50.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2283963510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f34f405f070 con 0x55d8cab69ae0 2023-11-13T11:01:50.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f3502b79740 1 -- 172.21.15.16:0/2283963510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34f000c030 con 0x55d8cab69ae0 2023-11-13T11:01:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2283963510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34f4034e90 con 0x55d8cab69ae0 2023-11-13T11:01:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f3502b79740 10 client.6165 fetch_fsmap learned FSMap version 28 2023-11-13T11:01:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f3502b79740 1 -- 172.21.15.16:0/2283963510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55d8caa97170 con 0x55d8cab69ae0 2023-11-13T11:01:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2283963510 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f000c030 con 0x55d8cab69ae0 2023-11-13T11:01:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2283963510 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f34f404aca0 con 0x55d8cab69ae0 2023-11-13T11:01:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f3502b79740 10 client.6165 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:01:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f3502b79740 10 client.6165 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T11:01:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f3502b79740 20 client.6165 populate_metadata read hostname 'smithi016' 2023-11-13T11:01:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.904+0000 7f3502b79740 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d8cac1d1f0 0x55d8cac1d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.905+0000 7f3502b79740 4 client.6165 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-13T11:01:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.905+0000 7f3502b79740 1 -- 172.21.15.16:0/2283963510 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d8ca7d66c0 con 0x55d8cac1d1f0 2023-11-13T11:01:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.905+0000 7f34fd57d640 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d8cac1d1f0 0x55d8cac1d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.905+0000 7f34fd57d640 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d8cac1d1f0 0x55d8cac1d5d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4ecbf59f27a1f1 server_cookie=74dc142565eec374 in_seq=0 out_seq=0 2023-11-13T11:01:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.905+0000 7f34d8ff9640 10 client.6165 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:01:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.905+0000 7f34d8ff9640 1 -- 172.21.15.16:0/2283963510 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d8ca7d66c0 con 0x55d8cac1d1f0 2023-11-13T11:01:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f34d8ff9640 10 client.6165 handle_command_reply: tid=0 2023-11-13T11:01:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 client.6165 shutdown 2023-11-13T11:01:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 -- 172.21.15.16:0/2283963510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 msgr2=0x55d8cab68ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab68ec0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f34f4002230 tx=0x7f34f4002180 comp rx=0 tx=0).stop 2023-11-13T11:01:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 -- 172.21.15.16:0/2283963510 shutdown_connections 2023-11-13T11:01:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d8cab6ede0 0x55d8cac21d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d8cab6e520 0x55d8cac217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d8cac1d1f0 0x55d8cac1d5d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 --2- 172.21.15.16:0/2283963510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8cab69ae0 0x55d8cab68ec0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f34d8ff9640 0 client.6165 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:01:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 -- 172.21.15.16:0/2283963510 >> 172.21.15.16:0/2283963510 conn(0x55d8cab30bb0 msgr2=0x55d8cab6a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 -- 172.21.15.16:0/2283963510 shutdown_connections 2023-11-13T11:01:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 1 -- 172.21.15.16:0/2283963510 wait complete. 2023-11-13T11:01:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.906+0000 7f3502b79740 20 client.6165 trim_cache size 0 max 16384 2023-11-13T11:01:50.928 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:01:50.928 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9", 2023-11-13T11:01:50.929 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-13T11:01:50.929 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5272, 2023-11-13T11:01:50.929 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-13T11:01:50.929 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-13T11:01:50.929 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-13T11:01:50.930 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 11.95436402, 2023-11-13T11:01:50.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2023-11-13T11:01:50.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 55, 2023-11-13T11:01:50.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 55, 2023-11-13T11:01:50.930 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 17.779772822999998 2023-11-13T11:01:50.931 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:01:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 -- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f34dc06ae40 msgr2=0x7f34dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f34dc06ae40 0x7f34dc06d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f34f40013d0 tx=0x7f34f4009030 comp rx=0 tx=0).stop 2023-11-13T11:01:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 -- 172.21.15.16:0/2496369753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8150ff0 msgr2=0x7f34f8151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8150ff0 0x7f34f8151470 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f34e80108e0 tx=0x7f34e8010da0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 -- 172.21.15.16:0/2496369753 shutdown_connections 2023-11-13T11:01:50.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f34dc06ae40 0x7f34dc06d300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34f8148d90 0x7f34f8149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8150ff0 0x7f34f8151470 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 --2- 172.21.15.16:0/2496369753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f34e4003970 0x7f34f815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:50.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 -- 172.21.15.16:0/2496369753 >> 172.21.15.16:0/2496369753 conn(0x7f34f806a190 msgr2=0x7f34f806fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:50.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 -- 172.21.15.16:0/2496369753 shutdown_connections 2023-11-13T11:01:50.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:50.907+0000 7f34fffe2640 1 -- 172.21.15.16:0/2496369753 wait complete. 2023-11-13T11:01:50.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-13T11:01:51.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.158+0000 7f013359e640 1 Processor -- start 2023-11-13T11:01:51.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.159+0000 7f013359e640 1 -- start start 2023-11-13T11:01:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.160+0000 7f013359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0134159230 0x7f013406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.160+0000 7f013359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 0x7f01340700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.160+0000 7f013359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01340b18d0 con 0x7f013406bfb0 2023-11-13T11:01:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.160+0000 7f013359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01340caf10 con 0x7f0134159230 2023-11-13T11:01:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.160+0000 7f013359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013406cea0 con 0x7f01340adc80 2023-11-13T11:01:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.160+0000 7f013259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0134159230 0x7f013406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.160+0000 7f0132d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f013406bfb0 0x7f013415cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59974/0 (socket says 172.21.15.16:59974) 2023-11-13T11:01:51.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.160+0000 7f0132d9d640 1 -- 172.21.15.16:0/1307475053 learned_addr learned my addr 172.21.15.16:0/1307475053 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:01:51.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.161+0000 7f012bfff640 1 --2- 172.21.15.16:0/1307475053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 0x7f01340700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.161+0000 7f013259c640 1 -- 172.21.15.16:0/1307475053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 msgr2=0x7f01340700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:01:51.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.161+0000 7f013259c640 1 --2- 172.21.15.16:0/1307475053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 0x7f01340700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.161+0000 7f013259c640 1 -- 172.21.15.16:0/1307475053 >> v1:172.21.15.16:6789/0 conn(0x7f013406bfb0 legacy=0x7f013415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:51.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.161+0000 7f013259c640 1 -- 172.21.15.16:0/1307475053 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013406d9c0 con 0x7f0134159230 2023-11-13T11:01:51.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.161+0000 7f012bfff640 1 --2- 172.21.15.16:0/1307475053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 0x7f01340700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:01:51.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.161+0000 7f0131d9b640 1 -- 172.21.15.16:0/1307475053 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869113011 0 0) 0x7f01340b18d0 con 0x7f013406bfb0 2023-11-13T11:01:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.162+0000 7f013259c640 1 --2- 172.21.15.16:0/1307475053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 0x7f013406fb80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f012400bfa0 tx=0x7f012402e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ccb68e6c980bc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.162+0000 7f0131d9b640 1 -- 172.21.15.16:0/1307475053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012402ee50 con 0x7f0134159230 2023-11-13T11:01:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f0131d9b640 1 -- 172.21.15.16:0/1307475053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124032070 con 0x7f0134159230 2023-11-13T11:01:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f0131d9b640 1 -- 172.21.15.16:0/1307475053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124038620 con 0x7f0134159230 2023-11-13T11:01:51.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f013359e640 1 -- 172.21.15.16:0/1307475053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 msgr2=0x7f013406fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:51.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f013359e640 1 --2- 172.21.15.16:0/1307475053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 0x7f013406fb80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f012400bfa0 tx=0x7f012402e0a0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f013359e640 1 -- 172.21.15.16:0/1307475053 shutdown_connections 2023-11-13T11:01:51.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f013359e640 1 --2- 172.21.15.16:0/1307475053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 0x7f01340700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f013359e640 1 --2- 172.21.15.16:0/1307475053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 0x7f013406fb80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f013359e640 1 -- 172.21.15.16:0/1307475053 >> 172.21.15.16:0/1307475053 conn(0x7f01340b04a0 msgr2=0x7f01340b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:51.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f013359e640 1 -- 172.21.15.16:0/1307475053 shutdown_connections 2023-11-13T11:01:51.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.163+0000 7f013359e640 1 -- 172.21.15.16:0/1307475053 wait complete. 2023-11-13T11:01:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.166+0000 7f013359e640 1 Processor -- start 2023-11-13T11:01:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.166+0000 7f013359e640 1 -- start start 2023-11-13T11:01:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.167+0000 7f013359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f013406bfb0 0x7f0134156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.167+0000 7f013359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 0x7f0134157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f013359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 0x7f0134157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f013359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134157250 con 0x7f0134159230 2023-11-13T11:01:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f013359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01340cc010 con 0x7f013406bfb0 2023-11-13T11:01:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f013359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01340cc2f0 con 0x7f01340adc80 2023-11-13T11:01:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f0132d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 0x7f0134157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f0132d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 0x7f0134157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42142/0 (socket says 172.21.15.16:42142) 2023-11-13T11:01:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f0132d9d640 1 -- 172.21.15.16:0/1305570735 learned_addr learned my addr 172.21.15.16:0/1305570735 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:01:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f012bfff640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 0x7f0134157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f013259c640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f013406bfb0 0x7f0134156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f0132d9d640 1 -- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 msgr2=0x7f0134157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.168+0000 7f0132d9d640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 0x7f0134157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.169+0000 7f0132d9d640 1 -- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f013406bfb0 msgr2=0x7f0134156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.169+0000 7f0132d9d640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f013406bfb0 0x7f0134156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.169+0000 7f0132d9d640 1 -- 172.21.15.16:0/1305570735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012400bc80 con 0x7f0134159230 2023-11-13T11:01:51.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.169+0000 7f0132d9d640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 0x7f0134157aa0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c480 tx=0x7f012c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:51.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.169+0000 7f012affd640 1 -- 172.21.15.16:0/1305570735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0074e0 con 0x7f0134159230 2023-11-13T11:01:51.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.169+0000 7f012affd640 1 -- 172.21.15.16:0/1305570735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c007640 con 0x7f0134159230 2023-11-13T11:01:51.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.169+0000 7f012affd640 1 -- 172.21.15.16:0/1305570735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0049e0 con 0x7f0134159230 2023-11-13T11:01:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.169+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01340cc5d0 con 0x7f0134159230 2023-11-13T11:01:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.169+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01340ccb90 con 0x7f0134159230 2023-11-13T11:01:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.171+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01340ccdb0 con 0x7f0134159230 2023-11-13T11:01:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.172+0000 7f012affd640 1 -- 172.21.15.16:0/1305570735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f012c004b40 con 0x7f0134159230 2023-11-13T11:01:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.173+0000 7f012affd640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f011406ae40 0x7f011406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:01:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.173+0000 7f012affd640 1 -- 172.21.15.16:0/1305570735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f012c0889f0 con 0x7f0134159230 2023-11-13T11:01:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.174+0000 7f013259c640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f011406ae40 0x7f011406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:01:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.175+0000 7f013259c640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f011406ae40 0x7f011406d300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0124039000 tx=0x7f0124034000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:01:51.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.178+0000 7f012affd640 1 -- 172.21.15.16:0/1305570735 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f012c0573e0 con 0x7f0134159230 2023-11-13T11:01:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.416+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f013404e4d0 con 0x7f0134159230 2023-11-13T11:01:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.417+0000 7f012affd640 1 -- 172.21.15.16:0/1305570735 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+73 (secure 0 0 0) 0x7f012c018070 con 0x7f0134159230 2023-11-13T11:01:51.422 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-13T11:01:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f011406ae40 msgr2=0x7f011406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f011406ae40 0x7f011406d300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0124039000 tx=0x7f0124034000 comp rx=0 tx=0).stop 2023-11-13T11:01:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 msgr2=0x7f0134157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:01:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 0x7f0134157aa0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c480 tx=0x7f012c00c940 comp rx=0 tx=0).stop 2023-11-13T11:01:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 shutdown_connections 2023-11-13T11:01:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f011406ae40 0x7f011406d300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0134159230 0x7f0134157aa0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01340adc80 0x7f0134157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 --2- 172.21.15.16:0/1305570735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f013406bfb0 0x7f0134156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:01:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 >> 172.21.15.16:0/1305570735 conn(0x7f01340b04a0 msgr2=0x7f013415b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:01:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 shutdown_connections 2023-11-13T11:01:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:51.420+0000 7f013359e640 1 -- 172.21.15.16:0/1305570735 wait complete. 2023-11-13T11:01:51.446 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-13T11:01:51.446 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-13T11:01:51.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-13T11:01:51.447 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-13T11:01:51.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-13T11:01:51.448 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-13T11:01:51.448 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-13T11:01:51.448 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-13T11:01:51.449 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-13T11:01:51.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:51.449 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-13T11:01:51.505 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T11:01:51.505 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T11:01:51.506 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T11:01:51.506 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:01:51.506 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-13T11:01:51.506 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:01:51.506 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T11:01:51.507 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:01:51.507 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-13T11:01:51.507 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T11:01:51.507 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:01:51.507 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-13T11:01:51.508 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-13T11:01:51.508 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:01:51.508 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-13T11:01:51.508 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-13T11:01:51.508 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41931sec preferred_lft 41931sec 2023-11-13T11:01:51.509 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-13T11:01:51.509 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:01:51.509 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T11:01:51.509 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:01:51.509 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-13T11:01:51.510 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-13T11:01:51.510 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1a:84:8e:80:25:76 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:01:51.510 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-13T11:01:51.510 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:01:51.510 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5426:dff:feb9:4760/64 scope link 2023-11-13T11:01:51.511 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:01:51.511 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-13T11:01:51.511 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1a:84:8e:80:25:76 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:01:51.511 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::1884:8eff:fe80:2576/64 scope link 2023-11-13T11:01:51.511 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:01:51.512 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-13T11:01:51.512 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether ae:42:49:3f:88:fc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:01:51.512 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ac42:49ff:fe3f:88fc/64 scope link 2023-11-13T11:01:51.512 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:01:51.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:51.514 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-13T11:01:51.612 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-13T11:01:51.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:51.617 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-13T11:01:51.676 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-13T11:01:51.676 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-13T11:01:51.677 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-13T11:01:51.677 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-13T11:01:51.677 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-13T11:01:51.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:51.678 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T11:01:51.678 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T11:01:51.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-13T11:01:51.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-13T11:01:51.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-13T11:01:51.680 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T11:01:51.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:51.883 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-13T11:01:51.939 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-13T11:01:51.940 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-13T11:01:51.941 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-13T11:01:51.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:51.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-13T11:01:52.041 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-13T11:01:52.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T11:01:52.042 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-13T11:01:52.098 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-13T11:01:52.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T11:01:52.099 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-13T11:01:52.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T11:01:52.257 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T11:01:52.318 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T11:01:52.319 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T11:01:52.319 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T11:01:52.319 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T11:01:52.319 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T11:01:52.320 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T11:01:52.320 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T11:01:52.320 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T11:01:52.320 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T11:01:52.320 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T11:01:52.321 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T11:01:52.321 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T11:01:52.321 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T11:01:52.321 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T11:01:52.322 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T11:01:52.322 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T11:01:52.322 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T11:01:52.322 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T11:01:52.323 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T11:01:52.323 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T11:01:52.323 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T11:01:52.323 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T11:01:52.323 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T11:01:52.324 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T11:01:52.324 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T11:01:52.324 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T11:01:52.324 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T11:01:52.324 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T11:01:52.325 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T11:01:52.325 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:01:52.325 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:01:52.325 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:01:52.325 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:01:52.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:52.327 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T11:01:52.385 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-13T11:01:52.386 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-13T11:01:52.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T11:01:52.475 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T11:01:52.519 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T11:01:52.519 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T11:01:52.519 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T11:01:52.520 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T11:01:52.520 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T11:01:52.520 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T11:01:52.520 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T11:01:52.521 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T11:01:52.521 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T11:01:52.521 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T11:01:52.521 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T11:01:52.521 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T11:01:52.522 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T11:01:52.522 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T11:01:52.522 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T11:01:52.522 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T11:01:52.522 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T11:01:52.523 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T11:01:52.523 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T11:01:52.523 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T11:01:52.523 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T11:01:52.523 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T11:01:52.524 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T11:01:52.524 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T11:01:52.524 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T11:01:52.524 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T11:01:52.524 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T11:01:52.525 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T11:01:52.525 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T11:01:52.525 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:01:52.525 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:01:52.525 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:01:52.526 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:01:52.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:52.527 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T11:01:52.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-13T11:01:52.587+0000 7fc8ed5db480 -1 init, newargv = 0x55753aa80190 newargc=15 2023-11-13T11:01:52.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[111941]: starting ceph client 2023-11-13T11:01:52.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[111941]: starting fuse 2023-11-13T11:01:52.616 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-13T11:01:52.616 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-13T11:01:52.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-13T11:01:52.617 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-13T11:01:52.618 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-13T11:01:52.618 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T11:01:52.619 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-13T11:01:52.619 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:01:52.619 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-13T11:01:52.620 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-13T11:01:52.620 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-13T11:01:52.621 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-13T11:01:52.621 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:01:52.621 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-13T11:01:52.622 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-13T11:01:52.622 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-13T11:01:52.622 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-13T11:01:52.623 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:01:52.623 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-13T11:01:52.624 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-13T11:01:52.624 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-13T11:01:52.624 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-13T11:01:52.625 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-13T11:01:52.625 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-13T11:01:52.625 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-13T11:01:52.626 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-13T11:01:52.626 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:01:52.626 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-13T11:01:52.627 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T11:01:52.830 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.111941.asok 2023-11-13T11:01:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T11:01:52.848 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.111941.asok 2023-11-13T11:01:52.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:52.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.111941.asok status 2023-11-13T11:01:53.101 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:01:53.101 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-13T11:01:53.102 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "35f0debb3cc28955b7132ef4c509cd138a09d56b", 2023-11-13T11:01:53.102 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7288-g35f0debb (35f0debb3cc28955b7132ef4c509cd138a09d56b) reef (dev)", 2023-11-13T11:01:53.102 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-13T11:01:53.102 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-13T11:01:53.103 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-13T11:01:53.103 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "111941", 2023-11-13T11:01:53.103 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-13T11:01:53.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:53.103 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-13T11:01:53.104 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-13T11:01:53.104 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6183, 2023-11-13T11:01:53.104 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-13T11:01:53.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-13T11:01:53.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-13T11:01:53.105 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 6183 2023-11-13T11:01:53.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:53.105 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T11:01:53.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T11:01:53.105 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T11:01:53.106 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2043171909 2023-11-13T11:01:53.106 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:01:53.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:53.106 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T11:01:53.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T11:01:53.107 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T11:01:53.107 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2043171909 2023-11-13T11:01:53.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:01:53.107 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.6183 192.168.0.1:0/2043171909", 2023-11-13T11:01:53.107 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2043171909", 2023-11-13T11:01:53.108 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-13T11:01:53.108 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 28, 2023-11-13T11:01:53.108 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 55, 2023-11-13T11:01:53.108 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-13T11:01:53.108 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-13T11:01:53.109 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-13T11:01:53.109 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:01:53.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:53.118 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-13T11:01:53.118 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-13T11:01:53.178 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-13T11:01:53.178 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-13T11:01:53.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:53.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-13T11:01:53.273 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2023-11-13T11:01:53.273 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2023-11-13T11:01:53.274 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2023-11-13T11:01:53.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:53.274 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2023-11-13T11:01:53.341 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2023-11-13T11:01:53.341 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T11:01:53.342 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2023-11-13T11:01:53.342 DEBUG:teuthology.orchestra.run.smithi016:> import json 2023-11-13T11:01:53.343 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2023-11-13T11:01:53.343 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:01:53.343 DEBUG:teuthology.orchestra.run.smithi016:> try: 2023-11-13T11:01:53.344 DEBUG:teuthology.orchestra.run.smithi016:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2023-11-13T11:01:53.344 DEBUG:teuthology.orchestra.run.smithi016:> except OSError as e: 2023-11-13T11:01:53.345 DEBUG:teuthology.orchestra.run.smithi016:> sys.exit(e.errno) 2023-11-13T11:01:53.345 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:01:53.345 DEBUG:teuthology.orchestra.run.smithi016:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-13T11:01:53.346 DEBUG:teuthology.orchestra.run.smithi016:> print(json.dumps( 2023-11-13T11:01:53.346 DEBUG:teuthology.orchestra.run.smithi016:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-13T11:01:53.347 DEBUG:teuthology.orchestra.run.smithi016:> indent=2)) 2023-11-13T11:01:53.347 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T11:01:53.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:01:53 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2023-11-13T11:01:53.516 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-13T11:01:53.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:01:53.517 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2023-11-13T11:01:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2023-11-13T11:01:54.576 INFO:teuthology.orchestra.run.smithi016.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2023-11-13T11:02:19.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:02:19.456 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:02:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:Updating files: 5% (75/1398) Updating files: 6% (84/1398) Updating files: 7% (98/1398) Updating files: 8% (112/1398) Updating files: 9% (126/1398) Updating files: 10% (140/1398) Updating files: 10% (143/1398) Updating files: 11% (154/1398) Updating files: 12% (168/1398) Updating files: 13% (182/1398) Updating files: 14% (196/1398) Updating files: 14% (203/1398) Updating files: 15% (210/1398) Updating files: 16% (224/1398) Updating files: 17% (238/1398) Updating files: 18% (252/1398) Updating files: 18% (259/1398) Updating files: 19% (266/1398) Updating files: 20% (280/1398) Updating files: 21% (294/1398) Updating files: 22% (308/1398) Updating files: 22% (315/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 26% (372/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 33% (462/1398) Updating files: 33% (467/1398) Updating files: 34% (476/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 42% (599/1398) Updating files: 43% (602/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 46% (644/1398) Updating files: 46% (656/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 50% (702/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 56% (788/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 60% (849/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 65% (912/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 68% (962/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 72% (1018/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 77% (1088/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 81% (1134/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 83% (1169/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 86% (1208/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 89% (1250/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 93% (1310/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2023-11-13T11:02:29.901 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2023-11-13T11:02:29.902 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2023-11-13T11:02:29.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:30.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.120+0000 7f405ce59640 1 Processor -- start 2023-11-13T11:02:30.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.120+0000 7f405ce59640 1 -- start start 2023-11-13T11:02:30.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.122+0000 7f405ce59640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f405806a8a0 0x7f40580b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.122+0000 7f405ce59640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4058070cf0 0x7f40580b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.122+0000 7f405ce59640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405806fc50 con 0x7f405815cc90 2023-11-13T11:02:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.122+0000 7f405ce59640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405806ff30 con 0x7f405806a8a0 2023-11-13T11:02:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.122+0000 7f405ce59640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4058070210 con 0x7f4058070cf0 2023-11-13T11:02:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.123+0000 7f404ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f405806a8a0 0x7f40580b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.123+0000 7f40577fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f405815cc90 0x7f405806a190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54560/0 (socket says 172.21.15.16:54560) 2023-11-13T11:02:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.123+0000 7f40577fe640 1 -- 172.21.15.16:0/2258056974 learned_addr learned my addr 172.21.15.16:0/2258056974 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.123+0000 7f4057fff640 1 --2- 172.21.15.16:0/2258056974 >> v2:172.21.15.184:3300/0 conn(0x7f4058070cf0 0x7f40580b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.123+0000 7f404ffff640 1 -- 172.21.15.16:0/2258056974 >> v2:172.21.15.184:3300/0 conn(0x7f4058070cf0 msgr2=0x7f40580b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.123+0000 7f404ffff640 1 --2- 172.21.15.16:0/2258056974 >> v2:172.21.15.184:3300/0 conn(0x7f4058070cf0 0x7f40580b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.123+0000 7f404ffff640 1 -- 172.21.15.16:0/2258056974 >> v1:172.21.15.16:6789/0 conn(0x7f405815cc90 legacy=0x7f405806a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.123+0000 7f404ffff640 1 -- 172.21.15.16:0/2258056974 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40580704f0 con 0x7f405806a8a0 2023-11-13T11:02:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.123+0000 7f4057fff640 1 --2- 172.21.15.16:0/2258056974 >> v2:172.21.15.184:3300/0 conn(0x7f4058070cf0 0x7f40580b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.124+0000 7f404ffff640 1 --2- 172.21.15.16:0/2258056974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 0x7f40580b2110 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f40580ce730 tx=0x7f404002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76aa9dfa1c2a5a3b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.125+0000 7f4056ffd640 1 -- 172.21.15.16:0/2258056974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040031830 con 0x7f405806a8a0 2023-11-13T11:02:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.125+0000 7f4056ffd640 1 -- 172.21.15.16:0/2258056974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040031990 con 0x7f405806a8a0 2023-11-13T11:02:30.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.125+0000 7f4056ffd640 1 -- 172.21.15.16:0/2258056974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040031d20 con 0x7f405806a8a0 2023-11-13T11:02:30.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.126+0000 7f405ce59640 1 -- 172.21.15.16:0/2258056974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 msgr2=0x7f40580b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.126+0000 7f405ce59640 1 --2- 172.21.15.16:0/2258056974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 0x7f40580b2110 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f40580ce730 tx=0x7f404002b8d0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.126+0000 7f405ce59640 1 -- 172.21.15.16:0/2258056974 shutdown_connections 2023-11-13T11:02:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.126+0000 7f405ce59640 1 --2- 172.21.15.16:0/2258056974 >> v2:172.21.15.184:3300/0 conn(0x7f4058070cf0 0x7f40580b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.126+0000 7f405ce59640 1 --2- 172.21.15.16:0/2258056974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 0x7f40580b2110 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.126+0000 7f405ce59640 1 -- 172.21.15.16:0/2258056974 >> 172.21.15.16:0/2258056974 conn(0x7f40580b04a0 msgr2=0x7f40580b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.126+0000 7f405ce59640 1 -- 172.21.15.16:0/2258056974 shutdown_connections 2023-11-13T11:02:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.126+0000 7f405ce59640 1 -- 172.21.15.16:0/2258056974 wait complete. 2023-11-13T11:02:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.128+0000 7f405ce59640 1 Processor -- start 2023-11-13T11:02:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.128+0000 7f405ce59640 1 -- start start 2023-11-13T11:02:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f405ce59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 0x7f4058145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f405ce59640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4058070cf0 0x7f4058154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f405ce59640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f405815cc90 0x7f40581547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f405ce59640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40581460d0 con 0x7f405806a8a0 2023-11-13T11:02:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f405ce59640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40580cac90 con 0x7f4058070cf0 2023-11-13T11:02:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f405ce59640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4058147ad0 con 0x7f405815cc90 2023-11-13T11:02:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f40577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 0x7f4058145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f40577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 0x7f4058145b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59796/0 (socket says 172.21.15.16:59796) 2023-11-13T11:02:30.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f40577fe640 1 -- 172.21.15.16:0/1804701964 learned_addr learned my addr 172.21.15.16:0/1804701964 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:30.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.129+0000 7f404ffff640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4058070cf0 0x7f4058154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f4057fff640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f405815cc90 0x7f40581547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f40577fe640 1 -- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f405815cc90 msgr2=0x7f40581547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f40577fe640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f405815cc90 0x7f40581547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f40577fe640 1 -- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4058070cf0 msgr2=0x7f4058154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f40577fe640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4058070cf0 0x7f4058154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f40577fe640 1 -- 172.21.15.16:0/1804701964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4040009400 con 0x7f405806a8a0 2023-11-13T11:02:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f4057fff640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f405815cc90 0x7f40581547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f40577fe640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 0x7f4058145b90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f404800c480 tx=0x7f404800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f4054ff9640 1 -- 172.21.15.16:0/1804701964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048005ce0 con 0x7f405806a8a0 2023-11-13T11:02:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f4054ff9640 1 -- 172.21.15.16:0/1804701964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4048005e40 con 0x7f405806a8a0 2023-11-13T11:02:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f4054ff9640 1 -- 172.21.15.16:0/1804701964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480072f0 con 0x7f405806a8a0 2023-11-13T11:02:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4058147db0 con 0x7f405806a8a0 2023-11-13T11:02:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.130+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40580caf30 con 0x7f405806a8a0 2023-11-13T11:02:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.132+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405806d320 con 0x7f405806a8a0 2023-11-13T11:02:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.133+0000 7f4054ff9640 1 -- 172.21.15.16:0/1804701964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4048007450 con 0x7f405806a8a0 2023-11-13T11:02:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.134+0000 7f4054ff9640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f403806ae70 0x7f403806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.134+0000 7f4054ff9640 1 -- 172.21.15.16:0/1804701964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4048088b90 con 0x7f405806a8a0 2023-11-13T11:02:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.134+0000 7f404ffff640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f403806ae70 0x7f403806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.136+0000 7f404ffff640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f403806ae70 0x7f403806d330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4040036000 tx=0x7f404003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.140+0000 7f4054ff9640 1 -- 172.21.15.16:0/1804701964 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4048053c00 con 0x7f405806a8a0 2023-11-13T11:02:30.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.375+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40580cb120 con 0x7f405806a8a0 2023-11-13T11:02:30.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.376+0000 7f4054ff9640 1 -- 172.21.15.16:0/1804701964 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f4048017020 con 0x7f405806a8a0 2023-11-13T11:02:30.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:30.381 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:01:41.484670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:30.381 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2023-11-13T11:02:30.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.379+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f403806ae70 msgr2=0x7f403806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.379+0000 7f405ce59640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f403806ae70 0x7f403806d330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4040036000 tx=0x7f404003c040 comp rx=0 tx=0).stop 2023-11-13T11:02:30.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.379+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 msgr2=0x7f4058145b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.379+0000 7f405ce59640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 0x7f4058145b90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f404800c480 tx=0x7f404800c940 comp rx=0 tx=0).stop 2023-11-13T11:02:30.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.380+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 shutdown_connections 2023-11-13T11:02:30.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.380+0000 7f405ce59640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f403806ae70 0x7f403806d330 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.380+0000 7f405ce59640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f405815cc90 0x7f40581547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.380+0000 7f405ce59640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4058070cf0 0x7f4058154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.380+0000 7f405ce59640 1 --2- 172.21.15.16:0/1804701964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f405806a8a0 0x7f4058145b90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.380+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 >> 172.21.15.16:0/1804701964 conn(0x7f40580b04a0 msgr2=0x7f405815b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.380+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 shutdown_connections 2023-11-13T11:02:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.380+0000 7f405ce59640 1 -- 172.21.15.16:0/1804701964 wait complete. 2023-11-13T11:02:30.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2023-11-13T11:02:30.619 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:30.620 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": -2 2023-11-13T11:02:30.620 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:30.635 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2023-11-13T11:02:30.635 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2023-11-13T11:02:30.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2023-11-13T11:02:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.872+0000 7effdc1d5640 1 Processor -- start 2023-11-13T11:02:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.872+0000 7effdc1d5640 1 -- start start 2023-11-13T11:02:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.873+0000 7effdc1d5640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7effd4156e30 0x7effd4159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.873+0000 7effdc1d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effd415a4d0 0x7effd40703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.873+0000 7effdc1d5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd40b2d20 con 0x7effd40ce210 2023-11-13T11:02:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.873+0000 7effdc1d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd40ce780 con 0x7effd415a4d0 2023-11-13T11:02:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.873+0000 7effdc1d5640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd40b2480 con 0x7effd4156e30 2023-11-13T11:02:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effda772640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effd415a4d0 0x7effd40703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effda772640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effd415a4d0 0x7effd40703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59798/0 (socket says 172.21.15.16:59798) 2023-11-13T11:02:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effda772640 1 -- 172.21.15.16:0/2841464527 learned_addr learned my addr 172.21.15.16:0/2841464527 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effd9f71640 1 --2- 172.21.15.16:0/2841464527 >> v2:172.21.15.184:3300/0 conn(0x7effd4156e30 0x7effd4159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effda772640 1 -- 172.21.15.16:0/2841464527 >> v2:172.21.15.184:3300/0 conn(0x7effd4156e30 msgr2=0x7effd4159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effda772640 1 --2- 172.21.15.16:0/2841464527 >> v2:172.21.15.184:3300/0 conn(0x7effd4156e30 0x7effd4159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effda772640 1 -- 172.21.15.16:0/2841464527 >> v1:172.21.15.16:6789/0 conn(0x7effd40ce210 legacy=0x7effd4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effda772640 1 -- 172.21.15.16:0/2841464527 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40b2760 con 0x7effd415a4d0 2023-11-13T11:02:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effd8f6f640 1 -- 172.21.15.16:0/2841464527 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000697328 0 0) 0x7effd40b2d20 con 0x7effd40ce210 2023-11-13T11:02:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.874+0000 7effd9f71640 1 --2- 172.21.15.16:0/2841464527 >> v2:172.21.15.184:3300/0 conn(0x7effd4156e30 0x7effd4159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.875+0000 7effda772640 1 --2- 172.21.15.16:0/2841464527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 0x7effd40703e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7effd40b40c0 tx=0x7effcc02e210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ac2cd868b9820e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effd8f6f640 1 -- 172.21.15.16:0/2841464527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc038400 con 0x7effd415a4d0 2023-11-13T11:02:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effd8f6f640 1 -- 172.21.15.16:0/2841464527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc032070 con 0x7effd415a4d0 2023-11-13T11:02:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effd8f6f640 1 -- 172.21.15.16:0/2841464527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc038780 con 0x7effd415a4d0 2023-11-13T11:02:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effdc1d5640 1 -- 172.21.15.16:0/2841464527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 msgr2=0x7effd40703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effdc1d5640 1 --2- 172.21.15.16:0/2841464527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 0x7effd40703e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7effd40b40c0 tx=0x7effcc02e210 comp rx=0 tx=0).stop 2023-11-13T11:02:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effdc1d5640 1 -- 172.21.15.16:0/2841464527 shutdown_connections 2023-11-13T11:02:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effdc1d5640 1 --2- 172.21.15.16:0/2841464527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 0x7effd40703e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effdc1d5640 1 --2- 172.21.15.16:0/2841464527 >> v2:172.21.15.184:3300/0 conn(0x7effd4156e30 0x7effd4159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effdc1d5640 1 -- 172.21.15.16:0/2841464527 >> 172.21.15.16:0/2841464527 conn(0x7effd406a190 msgr2=0x7effd406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effdc1d5640 1 -- 172.21.15.16:0/2841464527 shutdown_connections 2023-11-13T11:02:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.876+0000 7effdc1d5640 1 -- 172.21.15.16:0/2841464527 wait complete. 2023-11-13T11:02:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.878+0000 7effdc1d5640 1 Processor -- start 2023-11-13T11:02:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.878+0000 7effdc1d5640 1 -- start start 2023-11-13T11:02:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.879+0000 7effdc1d5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effd40ce210 0x7effd4156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.879+0000 7effdc1d5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4156e30 0x7effd41f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.879+0000 7effdc1d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 0x7effd40cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.879+0000 7effdc1d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd40cc270 con 0x7effd415a4d0 2023-11-13T11:02:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.879+0000 7effdc1d5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd40cba10 con 0x7effd4156e30 2023-11-13T11:02:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.879+0000 7effdc1d5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd41f30c0 con 0x7effd40ce210 2023-11-13T11:02:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.879+0000 7effda772640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 0x7effd40cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.880+0000 7effda772640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 0x7effd40cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59800/0 (socket says 172.21.15.16:59800) 2023-11-13T11:02:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.880+0000 7effda772640 1 -- 172.21.15.16:0/4071544738 learned_addr learned my addr 172.21.15.16:0/4071544738 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.880+0000 7effd9770640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4156e30 0x7effd41f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.880+0000 7effda772640 1 -- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effd40ce210 msgr2=0x7effd4156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.880+0000 7effda772640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effd40ce210 0x7effd4156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.880+0000 7effda772640 1 -- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4156e30 msgr2=0x7effd41f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.880+0000 7effda772640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4156e30 0x7effd41f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.880+0000 7effda772640 1 -- 172.21.15.16:0/4071544738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc00bd40 con 0x7effd415a4d0 2023-11-13T11:02:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.880+0000 7effda772640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 0x7effd40cb4d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7effcc00d240 tx=0x7effcc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.881+0000 7effca7fc640 1 -- 172.21.15.16:0/4071544738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc008d40 con 0x7effd415a4d0 2023-11-13T11:02:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.881+0000 7effca7fc640 1 -- 172.21.15.16:0/4071544738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc042400 con 0x7effd415a4d0 2023-11-13T11:02:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.881+0000 7effdc1d5640 1 -- 172.21.15.16:0/4071544738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd40cbcb0 con 0x7effd415a4d0 2023-11-13T11:02:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.881+0000 7effca7fc640 1 -- 172.21.15.16:0/4071544738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc0427b0 con 0x7effd415a4d0 2023-11-13T11:02:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.881+0000 7effdc1d5640 1 -- 172.21.15.16:0/4071544738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd41f3380 con 0x7effd415a4d0 2023-11-13T11:02:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.883+0000 7effca7fc640 1 -- 172.21.15.16:0/4071544738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effcc042910 con 0x7effd415a4d0 2023-11-13T11:02:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.884+0000 7effca7fc640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effb806ae40 0x7effb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.884+0000 7effca7fc640 1 -- 172.21.15.16:0/4071544738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7effcc0ac4c0 con 0x7effd415a4d0 2023-11-13T11:02:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.885+0000 7effded68740 1 Processor -- start 2023-11-13T11:02:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.885+0000 7effd9f71640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effb806ae40 0x7effb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.885+0000 7effded68740 1 -- start start 2023-11-13T11:02:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effded68740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579a1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effded68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effded68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579a1b60 0x5578579d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effded68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557857962530 con 0x5578579cea60 2023-11-13T11:02:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effded68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557857913ef0 con 0x5578579a1b60 2023-11-13T11:02:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effded68740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578579115e0 con 0x5578579ce680 2023-11-13T11:02:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effd9770640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effd9770640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59806/0 (socket says 172.21.15.16:59806) 2023-11-13T11:02:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effd9770640 1 -- 172.21.15.16:0/140315265 learned_addr learned my addr 172.21.15.16:0/140315265 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effd9f71640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effb806ae40 0x7effb806d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7effd0006fd0 tx=0x7effd0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effda772640 1 --2- 172.21.15.16:0/140315265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579a1b60 0x5578579d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effd9770640 1 -- 172.21.15.16:0/140315265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 msgr2=0x5578579a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.886+0000 7effd9770640 1 --2- 172.21.15.16:0/140315265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.887+0000 7effd9770640 1 -- 172.21.15.16:0/140315265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579a1b60 msgr2=0x5578579d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.887+0000 7effd9770640 1 --2- 172.21.15.16:0/140315265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579a1b60 0x5578579d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.887+0000 7effd9770640 1 -- 172.21.15.16:0/140315265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578578d2d40 con 0x5578579cea60 2023-11-13T11:02:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.887+0000 7effd9770640 1 --2- 172.21.15.16:0/140315265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579d1410 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7effc4001750 tx=0x7effc4001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e8d65dd1013db server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.887+0000 7effb77fe640 1 -- 172.21.15.16:0/140315265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400b070 con 0x5578579cea60 2023-11-13T11:02:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.887+0000 7effb77fe640 1 -- 172.21.15.16:0/140315265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc400d730 con 0x5578579cea60 2023-11-13T11:02:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.887+0000 7effb77fe640 1 -- 172.21.15.16:0/140315265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400daa0 con 0x5578579cea60 2023-11-13T11:02:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.888+0000 7effded68740 1 -- 172.21.15.16:0/140315265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 msgr2=0x5578579d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.888+0000 7effded68740 1 --2- 172.21.15.16:0/140315265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579d1410 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7effc4001750 tx=0x7effc4001c10 comp rx=0 tx=0).stop 2023-11-13T11:02:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.888+0000 7effded68740 1 -- 172.21.15.16:0/140315265 shutdown_connections 2023-11-13T11:02:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.888+0000 7effded68740 1 --2- 172.21.15.16:0/140315265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579a1b60 0x5578579d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.888+0000 7effded68740 1 --2- 172.21.15.16:0/140315265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579d1410 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.888+0000 7effded68740 1 --2- 172.21.15.16:0/140315265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.888+0000 7effded68740 1 -- 172.21.15.16:0/140315265 >> 172.21.15.16:0/140315265 conn(0x5578579a3890 msgr2=0x5578579ec410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.888+0000 7effded68740 1 -- 172.21.15.16:0/140315265 shutdown_connections 2023-11-13T11:02:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.888+0000 7effded68740 1 -- 172.21.15.16:0/140315265 wait complete. 2023-11-13T11:02:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.889+0000 7effded68740 1 -- start start 2023-11-13T11:02:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.889+0000 7effded68740 1 Processor -- start 2023-11-13T11:02:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.890+0000 7effded68740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.890+0000 7effded68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579cea60 0x5578579d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.890+0000 7effded68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579d3350 0x5578579d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.890+0000 7effded68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785790ced0 con 0x5578579d3350 2023-11-13T11:02:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.890+0000 7effded68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578578a9250 con 0x5578579cea60 2023-11-13T11:02:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.890+0000 7effded68740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578578c6c10 con 0x5578579ce680 2023-11-13T11:02:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.890+0000 7effded68740 10 client.0 _ll_register_callbacks cb 0x5578578cad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effda772640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579d3350 0x5578579d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effda772640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579d3350 0x5578579d66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59810/0 (socket says 172.21.15.16:59810) 2023-11-13T11:02:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effda772640 1 -- 172.21.15.16:0/1519447604 learned_addr learned my addr 172.21.15.16:0/1519447604 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effda772640 1 -- 172.21.15.16:0/1519447604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 msgr2=0x5578579d8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effd9770640 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579cea60 0x5578579d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effd9f71640 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effda772640 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579d8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effda772640 1 -- 172.21.15.16:0/1519447604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579cea60 msgr2=0x5578579d2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effda772640 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579cea60 0x5578579d2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effda772640 1 -- 172.21.15.16:0/1519447604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc4001350 con 0x5578579d3350 2023-11-13T11:02:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effd9770640 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579cea60 0x5578579d2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effda772640 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579d3350 0x5578579d66e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7effcc07afa0 tx=0x7effcc07b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43186404e06441c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.891+0000 7effb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effb4ff9640 1 -- 172.21.15.16:0/1519447604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc07fea0 con 0x5578579d3350 2023-11-13T11:02:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effb4ff9640 1 -- 172.21.15.16:0/1519447604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7effcc083310 con 0x5578579d3350 2023-11-13T11:02:30.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effded68740 1 -- 172.21.15.16:0/1519447604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557857935640 con 0x5578579d3350 2023-11-13T11:02:30.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effb4ff9640 1 -- 172.21.15.16:0/1519447604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7effcc0810b0 con 0x5578579d3350 2023-11-13T11:02:30.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effded68740 10 client.6207 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:30.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effded68740 1 -- 172.21.15.16:0/1519447604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55785790b550 con 0x5578579d3350 2023-11-13T11:02:30.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effb4ff9640 1 -- 172.21.15.16:0/1519447604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557857935640 con 0x5578579d3350 2023-11-13T11:02:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effb4ff9640 1 -- 172.21.15.16:0/1519447604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7effcc036030 con 0x5578579d3350 2023-11-13T11:02:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effded68740 10 client.6207 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effded68740 10 client.6207 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effded68740 20 client.6207 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effded68740 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5578579d70f0 0x5578579bd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effded68740 4 client.6207 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.892+0000 7effded68740 1 -- 172.21.15.16:0/1519447604 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578578e0330 con 0x5578579d70f0 2023-11-13T11:02:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.893+0000 7effd9f71640 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5578579d70f0 0x5578579bd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.893+0000 7effd9f71640 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5578579d70f0 0x5578579bd720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee34c709dd40821 server_cookie=363086b67ef70f23 in_seq=0 out_seq=0 2023-11-13T11:02:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.893+0000 7effb4ff9640 10 client.6207 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effb4ff9640 1 -- 172.21.15.16:0/1519447604 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578578e0330 con 0x5578579d70f0 2023-11-13T11:02:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effb4ff9640 10 client.6207 handle_command_reply: tid=0 2023-11-13T11:02:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 client.6207 shutdown 2023-11-13T11:02:30.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 -- 172.21.15.16:0/1519447604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579d3350 msgr2=0x5578579d66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579d3350 0x5578579d66e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7effcc07afa0 tx=0x7effcc07b7d0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 -- 172.21.15.16:0/1519447604 shutdown_connections 2023-11-13T11:02:30.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5578579d70f0 0x5578579bd720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579d3350 0x5578579d66e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579cea60 0x5578579d2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 --2- 172.21.15.16:0/1519447604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579d8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effb4ff9640 0 client.6207 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:30.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 -- 172.21.15.16:0/1519447604 >> 172.21.15.16:0/1519447604 conn(0x5578579a1b60 msgr2=0x5578579f01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:30.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.894+0000 7effded68740 1 -- 172.21.15.16:0/1519447604 shutdown_connections 2023-11-13T11:02:30.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.895+0000 7effded68740 1 -- 172.21.15.16:0/1519447604 wait complete. 2023-11-13T11:02:30.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.895+0000 7effded68740 20 client.6207 trim_cache size 0 max 16384 2023-11-13T11:02:30.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.911+0000 7effded68740 1 Processor -- start 2023-11-13T11:02:30.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.912+0000 7effded68740 1 -- start start 2023-11-13T11:02:30.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.912+0000 7effded68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579ce680 0x5578579d66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.912+0000 7effded68740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579cea60 0x557857a50ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.912+0000 7effded68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579d3350 0x5578579d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.912+0000 7effded68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557857670230 con 0x5578579ce680 2023-11-13T11:02:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.912+0000 7effded68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578579d47d0 con 0x5578579d3350 2023-11-13T11:02:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.912+0000 7effded68740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578579d9520 con 0x5578579cea60 2023-11-13T11:02:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.913+0000 7effd9f71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579ce680 0x5578579d66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.913+0000 7effd9f71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579ce680 0x5578579d66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59818/0 (socket says 172.21.15.16:59818) 2023-11-13T11:02:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.913+0000 7effd9f71640 1 -- 172.21.15.16:0/1815180582 learned_addr learned my addr 172.21.15.16:0/1815180582 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.913+0000 7effd9f71640 1 -- 172.21.15.16:0/1815180582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579cea60 msgr2=0x557857a50ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.913+0000 7effd9f71640 1 --2- 172.21.15.16:0/1815180582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579cea60 0x557857a50ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.913+0000 7effd9f71640 1 -- 172.21.15.16:0/1815180582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579d3350 msgr2=0x5578579d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.913+0000 7effd9f71640 1 --2- 172.21.15.16:0/1815180582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579d3350 0x5578579d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.913+0000 7effd9f71640 1 -- 172.21.15.16:0/1815180582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578578a8ae0 con 0x5578579ce680 2023-11-13T11:02:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.913+0000 7effd9f71640 1 --2- 172.21.15.16:0/1815180582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579ce680 0x5578579d66c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7effd000c9d0 tx=0x7effd0006c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffa543f177baaf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effb7fff640 1 -- 172.21.15.16:0/1815180582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0021790 con 0x5578579ce680 2023-11-13T11:02:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effb7fff640 1 -- 172.21.15.16:0/1815180582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd00218f0 con 0x5578579ce680 2023-11-13T11:02:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effb7fff640 1 -- 172.21.15.16:0/1815180582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0013a70 con 0x5578579ce680 2023-11-13T11:02:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effded68740 1 -- 172.21.15.16:0/1815180582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579ce680 msgr2=0x5578579d66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effded68740 1 --2- 172.21.15.16:0/1815180582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579ce680 0x5578579d66c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7effd000c9d0 tx=0x7effd0006c90 comp rx=0 tx=0).stop 2023-11-13T11:02:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effded68740 1 -- 172.21.15.16:0/1815180582 shutdown_connections 2023-11-13T11:02:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effded68740 1 --2- 172.21.15.16:0/1815180582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579d3350 0x5578579d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effded68740 1 --2- 172.21.15.16:0/1815180582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579cea60 0x557857a50ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effded68740 1 --2- 172.21.15.16:0/1815180582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579ce680 0x5578579d66c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effded68740 1 -- 172.21.15.16:0/1815180582 >> 172.21.15.16:0/1815180582 conn(0x5578579a1b60 msgr2=0x5578579d7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effded68740 1 -- 172.21.15.16:0/1815180582 shutdown_connections 2023-11-13T11:02:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.914+0000 7effded68740 1 -- 172.21.15.16:0/1815180582 wait complete. 2023-11-13T11:02:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.915+0000 7effded68740 1 -- start start 2023-11-13T11:02:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.915+0000 7effded68740 1 Processor -- start 2023-11-13T11:02:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.916+0000 7effded68740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.916+0000 7effded68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.916+0000 7effded68740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579d3350 0x5578579a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.916+0000 7effded68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557857a0d450 con 0x5578579cea60 2023-11-13T11:02:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.916+0000 7effded68740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557857a0d870 con 0x5578579d3350 2023-11-13T11:02:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.916+0000 7effded68740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557857a0a9d0 con 0x5578579ce680 2023-11-13T11:02:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.916+0000 7effded68740 10 client.0 _ll_register_callbacks cb 0x7effc4007c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effd9770640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effd9770640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59828/0 (socket says 172.21.15.16:59828) 2023-11-13T11:02:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effd9770640 1 -- 172.21.15.16:0/3707096483 learned_addr learned my addr 172.21.15.16:0/3707096483 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effd9770640 1 -- 172.21.15.16:0/3707096483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 msgr2=0x5578579bb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effd9770640 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effd9770640 1 -- 172.21.15.16:0/3707096483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579d3350 msgr2=0x5578579a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effda772640 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579d3350 0x5578579a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effd9770640 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579d3350 0x5578579a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effd9770640 1 -- 172.21.15.16:0/3707096483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd000ae40 con 0x5578579cea60 2023-11-13T11:02:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effd9770640 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579cd1a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7effc4007fd0 tx=0x7effc4001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb7c8e4ccf0a5ba server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.917+0000 7effb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effb4ff9640 1 -- 172.21.15.16:0/3707096483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4007a80 con 0x5578579cea60 2023-11-13T11:02:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effb4ff9640 1 -- 172.21.15.16:0/3707096483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7effc400dd60 con 0x5578579cea60 2023-11-13T11:02:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effded68740 1 -- 172.21.15.16:0/3707096483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578579d4910 con 0x5578579cea60 2023-11-13T11:02:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effb4ff9640 1 -- 172.21.15.16:0/3707096483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7effc40035f0 con 0x5578579cea60 2023-11-13T11:02:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effded68740 10 client.6213 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effded68740 1 -- 172.21.15.16:0/3707096483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5578578ce190 con 0x5578579cea60 2023-11-13T11:02:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effb4ff9640 1 -- 172.21.15.16:0/3707096483 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5578579d4910 con 0x5578579cea60 2023-11-13T11:02:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effb4ff9640 1 -- 172.21.15.16:0/3707096483 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7effc4003750 con 0x5578579cea60 2023-11-13T11:02:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effded68740 10 client.6213 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effded68740 10 client.6213 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effded68740 20 client.6213 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effded68740 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5578579a6c60 0x5578579ef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effded68740 4 client.6213 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2023-11-13T11:02:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.918+0000 7effded68740 1 -- 172.21.15.16:0/3707096483 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x7effc400d730 con 0x5578579a6c60 2023-11-13T11:02:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.919+0000 7effd9f71640 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5578579a6c60 0x5578579ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.919+0000 7effd9f71640 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5578579a6c60 0x5578579ef920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48e1893c7aa51b6 server_cookie=8fc4231e6cad2154 in_seq=0 out_seq=0 2023-11-13T11:02:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.919+0000 7effb4ff9640 10 client.6213 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effb4ff9640 1 -- 172.21.15.16:0/3707096483 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x7effc400d730 con 0x5578579a6c60 2023-11-13T11:02:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effb4ff9640 10 client.6213 handle_command_reply: tid=0 2023-11-13T11:02:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 client.6213 shutdown 2023-11-13T11:02:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 -- 172.21.15.16:0/3707096483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 msgr2=0x5578579cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579cd1a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7effc4007fd0 tx=0x7effc4001080 comp rx=0 tx=0).stop 2023-11-13T11:02:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 -- 172.21.15.16:0/3707096483 shutdown_connections 2023-11-13T11:02:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578579d3350 0x5578579a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5578579a6c60 0x5578579ef920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578579cea60 0x5578579cd1a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 --2- 172.21.15.16:0/3707096483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5578579ce680 0x5578579bb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 -- 172.21.15.16:0/3707096483 >> 172.21.15.16:0/3707096483 conn(0x5578579a1b60 msgr2=0x5578579a0430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effb4ff9640 0 client.6213 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 -- 172.21.15.16:0/3707096483 shutdown_connections 2023-11-13T11:02:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 1 -- 172.21.15.16:0/3707096483 wait complete. 2023-11-13T11:02:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.920+0000 7effded68740 20 client.6213 trim_cache size 0 max 16384 2023-11-13T11:02:30.946 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:30.946 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": -2 2023-11-13T11:02:30.946 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 -- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effb806ae40 msgr2=0x7effb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effb806ae40 0x7effb806d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7effd0006fd0 tx=0x7effd0008040 comp rx=0 tx=0).stop 2023-11-13T11:02:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 -- 172.21.15.16:0/4071544738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 msgr2=0x7effd40cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 0x7effd40cb4d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7effcc00d240 tx=0x7effcc004ab0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 -- 172.21.15.16:0/4071544738 shutdown_connections 2023-11-13T11:02:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7effb806ae40 0x7effb806d300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd415a4d0 0x7effd40cb4d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effd4156e30 0x7effd41f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 --2- 172.21.15.16:0/4071544738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7effd40ce210 0x7effd4156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 -- 172.21.15.16:0/4071544738 >> 172.21.15.16:0/4071544738 conn(0x7effd406a190 msgr2=0x7effd40702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 -- 172.21.15.16:0/4071544738 shutdown_connections 2023-11-13T11:02:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:30.921+0000 7effdc1d5640 1 -- 172.21.15.16:0/4071544738 wait complete. 2023-11-13T11:02:30.950 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2023-11-13T11:02:30.951 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2023-11-13T11:02:30.951 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2023-11-13T11:02:30.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.169+0000 7f5b5b59e640 1 Processor -- start 2023-11-13T11:02:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.169+0000 7f5b5b59e640 1 -- start start 2023-11-13T11:02:31.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.171+0000 7f5b5b59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b5c15cfc0 0x7f5b5c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:31.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.171+0000 7f5b5b59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b5c0b30e0 0x7f5b5c0d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:31.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.171+0000 7f5b5b59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5c0b3cf0 con 0x7f5b5c0d2ae0 2023-11-13T11:02:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.171+0000 7f5b5b59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5c0d3480 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.171+0000 7f5b5b59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b3730 con 0x7f5b5c0b30e0 2023-11-13T11:02:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.171+0000 7f5b5a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b5c15cfc0 0x7f5b5c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.171+0000 7f5b5a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b5c15cfc0 0x7f5b5c0cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59844/0 (socket says 172.21.15.16:59844) 2023-11-13T11:02:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.171+0000 7f5b5a59c640 1 -- 172.21.15.16:0/298983297 learned_addr learned my addr 172.21.15.16:0/298983297 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.172+0000 7f5b5ad9d640 1 --2- 172.21.15.16:0/298983297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b5c0b30e0 0x7f5b5c0d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.172+0000 7f5b5a59c640 1 -- 172.21.15.16:0/298983297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b5c0b30e0 msgr2=0x7f5b5c0d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.172+0000 7f5b5a59c640 1 --2- 172.21.15.16:0/298983297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b5c0b30e0 0x7f5b5c0d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.172+0000 7f5b5a59c640 1 -- 172.21.15.16:0/298983297 >> v1:172.21.15.16:6789/0 conn(0x7f5b5c0d2ae0 legacy=0x7f5b5c0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.172+0000 7f5b5a59c640 1 -- 172.21.15.16:0/298983297 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0b3a10 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.173+0000 7f5b5a59c640 1 --2- 172.21.15.16:0/298983297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 0x7f5b5c0cd770 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009870 tx=0x7f5b4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca128ec7e88a1f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.173+0000 7f5b5959a640 1 -- 172.21.15.16:0/298983297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c031640 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.173+0000 7f5b5959a640 1 -- 172.21.15.16:0/298983297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c0317a0 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.173+0000 7f5b5959a640 1 -- 172.21.15.16:0/298983297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c031b30 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.174+0000 7f5b5b59e640 1 -- 172.21.15.16:0/298983297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 msgr2=0x7f5b5c0cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:31.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.174+0000 7f5b5b59e640 1 --2- 172.21.15.16:0/298983297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 0x7f5b5c0cd770 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009870 tx=0x7f5b4c02b8f0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.174+0000 7f5b5b59e640 1 -- 172.21.15.16:0/298983297 shutdown_connections 2023-11-13T11:02:31.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.174+0000 7f5b5b59e640 1 --2- 172.21.15.16:0/298983297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b5c0b30e0 0x7f5b5c0d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.174+0000 7f5b5b59e640 1 --2- 172.21.15.16:0/298983297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 0x7f5b5c0cd770 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.174+0000 7f5b5b59e640 1 -- 172.21.15.16:0/298983297 >> 172.21.15.16:0/298983297 conn(0x7f5b5c08c570 msgr2=0x7f5b5c09be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:31.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.174+0000 7f5b5b59e640 1 -- 172.21.15.16:0/298983297 shutdown_connections 2023-11-13T11:02:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.174+0000 7f5b5b59e640 1 -- 172.21.15.16:0/298983297 wait complete. 2023-11-13T11:02:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.176+0000 7f5b5b59e640 1 Processor -- start 2023-11-13T11:02:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.176+0000 7f5b5b59e640 1 -- start start 2023-11-13T11:02:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.177+0000 7f5b5b59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b5c0b30e0 0x7f5b5c150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.177+0000 7f5b5b59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b5c0d2ae0 0x7f5b5c0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 0x7f5b5c0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c15d430 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5b59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0ca120 con 0x7f5b5c0b30e0 2023-11-13T11:02:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5b59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0c9dd0 con 0x7f5b5c0d2ae0 2023-11-13T11:02:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 0x7f5b5c0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 0x7f5b5c0b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59848/0 (socket says 172.21.15.16:59848) 2023-11-13T11:02:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5ad9d640 1 -- 172.21.15.16:0/4186321090 learned_addr learned my addr 172.21.15.16:0/4186321090 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5a59c640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b5c0b30e0 0x7f5b5c150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5ad9d640 1 -- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b5c0d2ae0 msgr2=0x7f5b5c0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5ad9d640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b5c0d2ae0 0x7f5b5c0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5ad9d640 1 -- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b5c0b30e0 msgr2=0x7f5b5c150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5ad9d640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b5c0b30e0 0x7f5b5c150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.178+0000 7f5b5ad9d640 1 -- 172.21.15.16:0/4186321090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0094d0 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.179+0000 7f5b5ad9d640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 0x7f5b5c0b0ed0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5b54007e60 tx=0x7f5b5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.179+0000 7f5b4affd640 1 -- 172.21.15.16:0/4186321090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54014420 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.179+0000 7f5b4affd640 1 -- 172.21.15.16:0/4186321090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b540145a0 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.179+0000 7f5b4affd640 1 -- 172.21.15.16:0/4186321090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54014950 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.179+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c0c9020 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.179+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c150950 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.180+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c0c9540 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.181+0000 7f5b4affd640 1 -- 172.21.15.16:0/4186321090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5b54014ab0 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.183+0000 7f5b4affd640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b3c06ae40 0x7f5b3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.183+0000 7f5b4affd640 1 -- 172.21.15.16:0/4186321090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f5b5408a0d0 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.184+0000 7f5b5a59c640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b3c06ae40 0x7f5b3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.185+0000 7f5b5a59c640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b3c06ae40 0x7f5b3c06d300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c036000 tx=0x7f5b4c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.188+0000 7f5b4affd640 1 -- 172.21.15.16:0/4186321090 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b54054ff0 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.428+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b5c0ca290 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.430+0000 7f5b4affd640 1 -- 172.21.15.16:0/4186321090 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f5b54058610 con 0x7f5b5c15cfc0 2023-11-13T11:02:31.434 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:31.434 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:01:41.484670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2023-11-13T11:02:31.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b3c06ae40 msgr2=0x7f5b3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:31.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b3c06ae40 0x7f5b3c06d300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c036000 tx=0x7f5b4c03c040 comp rx=0 tx=0).stop 2023-11-13T11:02:31.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 msgr2=0x7f5b5c0b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 0x7f5b5c0b0ed0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5b54007e60 tx=0x7f5b5400e4b0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 shutdown_connections 2023-11-13T11:02:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5b3c06ae40 0x7f5b3c06d300 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b5c15cfc0 0x7f5b5c0b0ed0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b5c0d2ae0 0x7f5b5c0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 --2- 172.21.15.16:0/4186321090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b5c0b30e0 0x7f5b5c150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:31.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.432+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 >> 172.21.15.16:0/4186321090 conn(0x7f5b5c08c570 msgr2=0x7f5b5c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:31.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.433+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 shutdown_connections 2023-11-13T11:02:31.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:31.433+0000 7f5b5b59e640 1 -- 172.21.15.16:0/4186321090 wait complete. 2023-11-13T11:02:31.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2023-11-13T11:02:31.780 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:31.780 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:02:31.780 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:31.797 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-13T11:02:31.798 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2023-11-13T11:02:31.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2023-11-13T11:02:32.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.013+0000 7f6bd64ff640 1 Processor -- start 2023-11-13T11:02:32.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.013+0000 7f6bd64ff640 1 -- start start 2023-11-13T11:02:32.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.014+0000 7f6bd64ff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6bd015ce70 0x7f6bd00cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.014+0000 7f6bd64ff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6bd00d1850 0x7f6bd00c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.015+0000 7f6bd64ff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd00ce0e0 con 0x7f6bd00cdbb0 2023-11-13T11:02:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.015+0000 7f6bd64ff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd00ce3c0 con 0x7f6bd00d1850 2023-11-13T11:02:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.015+0000 7f6bd64ff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd00ce6a0 con 0x7f6bd015ce70 2023-11-13T11:02:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.015+0000 7f6bd5cfe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6bd00d1850 0x7f6bd00c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.015+0000 7f6bd5cfe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6bd00d1850 0x7f6bd00c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59856/0 (socket says 172.21.15.16:59856) 2023-11-13T11:02:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.015+0000 7f6bd5cfe640 1 -- 172.21.15.16:0/2100074051 learned_addr learned my addr 172.21.15.16:0/2100074051 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.015+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/2100074051 >> v2:172.21.15.184:3300/0 conn(0x7f6bd015ce70 0x7f6bd00cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.016+0000 7f6bd5cfe640 1 -- 172.21.15.16:0/2100074051 >> v2:172.21.15.184:3300/0 conn(0x7f6bd015ce70 msgr2=0x7f6bd00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.016+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/2100074051 >> v2:172.21.15.184:3300/0 conn(0x7f6bd015ce70 0x7f6bd00cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.016+0000 7f6bd5cfe640 1 -- 172.21.15.16:0/2100074051 >> v1:172.21.15.184:6789/0 conn(0x7f6bd00cdbb0 legacy=0x7f6bd00d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.016+0000 7f6bd5cfe640 1 -- 172.21.15.16:0/2100074051 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd00ce980 con 0x7f6bd00d1850 2023-11-13T11:02:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.017+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/2100074051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd00d1850 0x7f6bd00c9c30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6bd008c400 tx=0x7f6bc802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22124ad535cdb18 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.017+0000 7f6bc7fff640 1 -- 172.21.15.16:0/2100074051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc802cb20 con 0x7f6bd00d1850 2023-11-13T11:02:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.017+0000 7f6bc7fff640 1 -- 172.21.15.16:0/2100074051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc802cc80 con 0x7f6bd00d1850 2023-11-13T11:02:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.017+0000 7f6bc7fff640 1 -- 172.21.15.16:0/2100074051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80343f0 con 0x7f6bd00d1850 2023-11-13T11:02:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.018+0000 7f6bd64ff640 1 -- 172.21.15.16:0/2100074051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd00d1850 msgr2=0x7f6bd00c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.018+0000 7f6bd64ff640 1 --2- 172.21.15.16:0/2100074051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd00d1850 0x7f6bd00c9c30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6bd008c400 tx=0x7f6bc802bf80 comp rx=0 tx=0).stop 2023-11-13T11:02:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.018+0000 7f6bd64ff640 1 -- 172.21.15.16:0/2100074051 shutdown_connections 2023-11-13T11:02:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.018+0000 7f6bd64ff640 1 --2- 172.21.15.16:0/2100074051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd00d1850 0x7f6bd00c9c30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.018+0000 7f6bd64ff640 1 --2- 172.21.15.16:0/2100074051 >> v2:172.21.15.184:3300/0 conn(0x7f6bd015ce70 0x7f6bd00cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.018+0000 7f6bd64ff640 1 -- 172.21.15.16:0/2100074051 >> 172.21.15.16:0/2100074051 conn(0x7f6bd008c580 msgr2=0x7f6bd009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.018+0000 7f6bd64ff640 1 -- 172.21.15.16:0/2100074051 shutdown_connections 2023-11-13T11:02:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.018+0000 7f6bd64ff640 1 -- 172.21.15.16:0/2100074051 wait complete. 2023-11-13T11:02:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.020+0000 7f6bd64ff640 1 Processor -- start 2023-11-13T11:02:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.020+0000 7f6bd64ff640 1 -- start start 2023-11-13T11:02:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.021+0000 7f6bd64ff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bd00cdbb0 0x7f6bd00a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.021+0000 7f6bd64ff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bd00d1850 0x7f6bd01f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.021+0000 7f6bd64ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd015ce70 0x7f6bd01f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.021+0000 7f6bd64ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00a07b0 con 0x7f6bd015ce70 2023-11-13T11:02:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.021+0000 7f6bd64ff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00b7fa0 con 0x7f6bd00cdbb0 2023-11-13T11:02:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.021+0000 7f6bd64ff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00b8280 con 0x7f6bd00d1850 2023-11-13T11:02:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.021+0000 7f6bd5cfe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd015ce70 0x7f6bd01f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.021+0000 7f6bd5cfe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd015ce70 0x7f6bd01f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59872/0 (socket says 172.21.15.16:59872) 2023-11-13T11:02:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.021+0000 7f6bd5cfe640 1 -- 172.21.15.16:0/3018614063 learned_addr learned my addr 172.21.15.16:0/3018614063 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.022+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bd00cdbb0 0x7f6bd00a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.022+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bd00d1850 0x7f6bd01f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.022+0000 7f6bd5cfe640 1 -- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bd00d1850 msgr2=0x7f6bd01f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.022+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bd00d1850 0x7f6bd01f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.022+0000 7f6bd5cfe640 1 -- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bd00cdbb0 msgr2=0x7f6bd00a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.022+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bd00cdbb0 0x7f6bd00a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.022+0000 7f6bd5cfe640 1 -- 172.21.15.16:0/3018614063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc8009b30 con 0x7f6bd015ce70 2023-11-13T11:02:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.022+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bd00d1850 0x7f6bd01f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.022+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd015ce70 0x7f6bd01f60b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8036040 tx=0x7f6bc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.023+0000 7f6bc5ffb640 1 -- 172.21.15.16:0/3018614063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8009510 con 0x7f6bd015ce70 2023-11-13T11:02:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.023+0000 7f6bc5ffb640 1 -- 172.21.15.16:0/3018614063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc802c920 con 0x7f6bd015ce70 2023-11-13T11:02:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.023+0000 7f6bc5ffb640 1 -- 172.21.15.16:0/3018614063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc803e690 con 0x7f6bd015ce70 2023-11-13T11:02:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.023+0000 7f6bd64ff640 1 -- 172.21.15.16:0/3018614063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd014aca0 con 0x7f6bd015ce70 2023-11-13T11:02:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.023+0000 7f6bd64ff640 1 -- 172.21.15.16:0/3018614063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd014b260 con 0x7f6bd015ce70 2023-11-13T11:02:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.025+0000 7f6bc5ffb640 1 -- 172.21.15.16:0/3018614063 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bc803e7f0 con 0x7f6bd015ce70 2023-11-13T11:02:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.026+0000 7f6bc5ffb640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6bb406ae40 0x7f6bb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.026+0000 7f6bc5ffb640 1 -- 172.21.15.16:0/3018614063 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6bc80ac360 con 0x7f6bd015ce70 2023-11-13T11:02:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.027+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6bb406ae40 0x7f6bb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.027+0000 7f6bda3cb740 1 Processor -- start 2023-11-13T11:02:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.027+0000 7f6bda3cb740 1 -- start start 2023-11-13T11:02:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.028+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6bb406ae40 0x7f6bb406d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009a30 tx=0x7f6bcc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.028+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa5fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.028+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa325e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.028+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa32b20 0x556c8fa62560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.028+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8f9a4ee0 con 0x556c8fa5f940 2023-11-13T11:02:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.028+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8f9a25d0 con 0x556c8fa32b20 2023-11-13T11:02:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.028+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c8f99dec0 con 0x556c8fa382c0 2023-11-13T11:02:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd54fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa5fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd54fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa5fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59876/0 (socket says 172.21.15.16:59876) 2023-11-13T11:02:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd54fd640 1 -- 172.21.15.16:0/3586031460 learned_addr learned my addr 172.21.15.16:0/3586031460 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa32b20 0x556c8fa62560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd54fd640 1 -- 172.21.15.16:0/3586031460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 msgr2=0x556c8fa325e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa325e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd54fd640 1 -- 172.21.15.16:0/3586031460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa32b20 msgr2=0x556c8fa62560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa32b20 0x556c8fa62560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd54fd640 1 -- 172.21.15.16:0/3586031460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c8f960950 con 0x556c8fa5f940 2023-11-13T11:02:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa32b20 0x556c8fa62560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.029+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa5fd20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc011d20 tx=0x7f6bcc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cf6f131bce78dc server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.030+0000 7f6bb2ffd640 1 -- 172.21.15.16:0/3586031460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc011d50 con 0x556c8fa5f940 2023-11-13T11:02:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.030+0000 7f6bb2ffd640 1 -- 172.21.15.16:0/3586031460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc011eb0 con 0x556c8fa5f940 2023-11-13T11:02:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.030+0000 7f6bb2ffd640 1 -- 172.21.15.16:0/3586031460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc021640 con 0x556c8fa5f940 2023-11-13T11:02:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.030+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3586031460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 msgr2=0x556c8fa5fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.030+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa5fd20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc011d20 tx=0x7f6bcc014000 comp rx=0 tx=0).stop 2023-11-13T11:02:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.031+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3586031460 shutdown_connections 2023-11-13T11:02:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.031+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa32b20 0x556c8fa62560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.031+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa325e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.031+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/3586031460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa5fd20 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.031+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3586031460 >> 172.21.15.16:0/3586031460 conn(0x556c8fa34850 msgr2=0x556c8fa36110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.031+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3586031460 shutdown_connections 2023-11-13T11:02:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.031+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3586031460 wait complete. 2023-11-13T11:02:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.032+0000 7f6bda3cb740 1 -- start start 2023-11-13T11:02:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.032+0000 7f6bda3cb740 1 Processor -- start 2023-11-13T11:02:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa382c0 0x556c8fa57cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa57520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa5a020 0x556c8fa55900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8f957ed0 con 0x556c8fa5f940 2023-11-13T11:02:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8f957b70 con 0x556c8fa382c0 2023-11-13T11:02:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c8f701340 con 0x556c8fa5a020 2023-11-13T11:02:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bda3cb740 10 client.0 _ll_register_callbacks cb 0x556c8f95c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bd4cfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa57520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bd4cfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa57520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59878/0 (socket says 172.21.15.16:59878) 2023-11-13T11:02:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.033+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/1433577509 learned_addr learned my addr 172.21.15.16:0/1433577509 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa382c0 0x556c8fa57cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/1433577509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa5a020 msgr2=0x556c8fa55900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa5a020 0x556c8fa55900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa5a020 0x556c8fa55900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/1433577509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa382c0 msgr2=0x556c8fa57cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa382c0 0x556c8fa57cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/1433577509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc0119a0 con 0x556c8fa5f940 2023-11-13T11:02:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa382c0 0x556c8fa57cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa57520 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000f1d0 tx=0x7f6bc000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981eba43b2354cf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6babfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6babfff640 1 -- 172.21.15.16:0/1433577509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc000fec0 con 0x556c8fa5f940 2023-11-13T11:02:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6babfff640 1 -- 172.21.15.16:0/1433577509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6bc00052e0 con 0x556c8fa5f940 2023-11-13T11:02:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.034+0000 7f6bda3cb740 1 -- 172.21.15.16:0/1433577509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c8f9c60f0 con 0x556c8fa5f940 2023-11-13T11:02:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6babfff640 1 -- 172.21.15.16:0/1433577509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bc00054a0 con 0x556c8fa5f940 2023-11-13T11:02:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6bda3cb740 10 client.6234 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6bda3cb740 1 -- 172.21.15.16:0/1433577509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x556c8f99c540 con 0x556c8fa5f940 2023-11-13T11:02:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6babfff640 1 -- 172.21.15.16:0/1433577509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c8f9c60f0 con 0x556c8fa5f940 2023-11-13T11:02:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6babfff640 1 -- 172.21.15.16:0/1433577509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6bc0012070 con 0x556c8fa5f940 2023-11-13T11:02:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6bda3cb740 10 client.6234 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6bda3cb740 10 client.6234 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6bda3cb740 20 client.6234 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x556c8fa55e40 0x556c8fa7da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6bda3cb740 4 client.6234 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6bda3cb740 1 -- 172.21.15.16:0/1433577509 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c8f971230 con 0x556c8fa55e40 2023-11-13T11:02:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.035+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x556c8fa55e40 0x556c8fa7da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.036+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x556c8fa55e40 0x556c8fa7da10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb28e60207949ee server_cookie=6ecc06b66e6b0e26 in_seq=0 out_seq=0 2023-11-13T11:02:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.036+0000 7f6babfff640 10 client.6234 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6babfff640 1 -- 172.21.15.16:0/1433577509 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c8f971230 con 0x556c8fa55e40 2023-11-13T11:02:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6babfff640 10 client.6234 handle_command_reply: tid=0 2023-11-13T11:02:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 client.6234 shutdown 2023-11-13T11:02:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 -- 172.21.15.16:0/1433577509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 msgr2=0x556c8fa57520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa57520 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000f1d0 tx=0x7f6bc000f690 comp rx=0 tx=0).stop 2023-11-13T11:02:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 -- 172.21.15.16:0/1433577509 shutdown_connections 2023-11-13T11:02:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa5a020 0x556c8fa55900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x556c8fa55e40 0x556c8fa7da10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5f940 0x556c8fa57520 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/1433577509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa382c0 0x556c8fa57cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6babfff640 0 client.6234 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 -- 172.21.15.16:0/1433577509 >> 172.21.15.16:0/1433577509 conn(0x556c8fa32b20 msgr2=0x556c8fa7ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 -- 172.21.15.16:0/1433577509 shutdown_connections 2023-11-13T11:02:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 1 -- 172.21.15.16:0/1433577509 wait complete. 2023-11-13T11:02:32.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.037+0000 7f6bda3cb740 20 client.6234 trim_cache size 0 max 16384 2023-11-13T11:02:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.054+0000 7f6bda3cb740 1 Processor -- start 2023-11-13T11:02:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.054+0000 7f6bda3cb740 1 -- start start 2023-11-13T11:02:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.055+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa56220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.055+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.055+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 0x556c8fa96830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.055+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8fa33a70 con 0x556c8fa5a020 2023-11-13T11:02:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.055+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8fa36110 con 0x556c8fa5f940 2023-11-13T11:02:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.055+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c8fa58e60 con 0x556c8fa382c0 2023-11-13T11:02:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.055+0000 7f6bd4cfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.055+0000 7f6bd4cfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59886/0 (socket says 172.21.15.16:59886) 2023-11-13T11:02:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.055+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/3494831713 learned_addr learned my addr 172.21.15.16:0/3494831713 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.056+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/3494831713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 msgr2=0x556c8fa56220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.056+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/3494831713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa56220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.056+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/3494831713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 msgr2=0x556c8fa96830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.056+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/3494831713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 0x556c8fa96830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.056+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/3494831713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c8f939b00 con 0x556c8fa5a020 2023-11-13T11:02:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.056+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/3494831713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56760 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000f6d0 tx=0x7f6bc000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161a1b663f91b774 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.056+0000 7f6bb37fe640 1 -- 172.21.15.16:0/3494831713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00054e0 con 0x556c8fa5a020 2023-11-13T11:02:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.056+0000 7f6bb37fe640 1 -- 172.21.15.16:0/3494831713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc000d070 con 0x556c8fa5a020 2023-11-13T11:02:32.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.056+0000 7f6bb37fe640 1 -- 172.21.15.16:0/3494831713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0003ba0 con 0x556c8fa5a020 2023-11-13T11:02:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.057+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3494831713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 msgr2=0x556c8fa56760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.057+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/3494831713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56760 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000f6d0 tx=0x7f6bc000d000 comp rx=0 tx=0).stop 2023-11-13T11:02:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.057+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3494831713 shutdown_connections 2023-11-13T11:02:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.057+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/3494831713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 0x556c8fa96830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.057+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/3494831713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56760 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.057+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/3494831713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa56220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.057+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3494831713 >> 172.21.15.16:0/3494831713 conn(0x556c8fa32b20 msgr2=0x556c8fa31aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.057+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3494831713 shutdown_connections 2023-11-13T11:02:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.057+0000 7f6bda3cb740 1 -- 172.21.15.16:0/3494831713 wait complete. 2023-11-13T11:02:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.058+0000 7f6bda3cb740 1 -- start start 2023-11-13T11:02:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.058+0000 7f6bda3cb740 1 Processor -- start 2023-11-13T11:02:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa36250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bda3cb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 0x556c8fa64b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8faec620 con 0x556c8fa5a020 2023-11-13T11:02:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8fa646e0 con 0x556c8fa5f940 2023-11-13T11:02:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bda3cb740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c8fae8cf0 con 0x556c8fa382c0 2023-11-13T11:02:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bda3cb740 10 client.0 _ll_register_callbacks cb 0x7f6bc0007d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bd4cfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bd4cfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59902/0 (socket says 172.21.15.16:59902) 2023-11-13T11:02:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.059+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/766968056 learned_addr learned my addr 172.21.15.16:0/766968056 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 0x556c8fa64b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa36250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 msgr2=0x556c8fa36250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa36250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 msgr2=0x556c8fa64b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 0x556c8fa64b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd4cfc640 1 -- 172.21.15.16:0/766968056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c8f99c540 con 0x556c8fa5a020 2023-11-13T11:02:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd5cfe640 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 0x556c8fa64b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa36250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bd4cfc640 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56620 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0014000 tx=0x7f6bc0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfc0375769ed1e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6babfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6babfff640 1 -- 172.21.15.16:0/766968056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0003db0 con 0x556c8fa5a020 2023-11-13T11:02:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6bda3cb740 1 -- 172.21.15.16:0/766968056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bcc011eb0 con 0x556c8fa5a020 2023-11-13T11:02:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.060+0000 7f6babfff640 1 -- 172.21.15.16:0/766968056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6bc0007030 con 0x556c8fa5a020 2023-11-13T11:02:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6babfff640 1 -- 172.21.15.16:0/766968056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bc00071f0 con 0x556c8fa5a020 2023-11-13T11:02:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6bda3cb740 10 client.6240 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6bda3cb740 1 -- 172.21.15.16:0/766968056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x556c8f95f1b0 con 0x556c8fa5a020 2023-11-13T11:02:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6babfff640 1 -- 172.21.15.16:0/766968056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc011eb0 con 0x556c8fa5a020 2023-11-13T11:02:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6babfff640 1 -- 172.21.15.16:0/766968056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6bc0007350 con 0x556c8fa5a020 2023-11-13T11:02:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6bda3cb740 10 client.6240 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6bda3cb740 10 client.6240 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6bda3cb740 20 client.6240 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x556c8fa57be0 0x556c8fa65070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6bda3cb740 4 client.6240 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2023-11-13T11:02:32.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6bda3cb740 1 -- 172.21.15.16:0/766968056 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x556c8f996ea0 con 0x556c8fa57be0 2023-11-13T11:02:32.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.061+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x556c8fa57be0 0x556c8fa65070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.062+0000 7f6bd54fd640 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x556c8fa57be0 0x556c8fa65070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d24752b348700e server_cookie=16d012bb11bdc30 in_seq=0 out_seq=0 2023-11-13T11:02:32.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.062+0000 7f6babfff640 10 client.6240 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.062+0000 7f6babfff640 1 -- 172.21.15.16:0/766968056 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556c8f996ea0 con 0x556c8fa57be0 2023-11-13T11:02:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.062+0000 7f6babfff640 10 client.6240 handle_command_reply: tid=0 2023-11-13T11:02:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.062+0000 7f6bda3cb740 1 client.6240 shutdown 2023-11-13T11:02:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 -- 172.21.15.16:0/766968056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 msgr2=0x556c8fa56620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56620 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0014000 tx=0x7f6bc0001120 comp rx=0 tx=0).stop 2023-11-13T11:02:32.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 -- 172.21.15.16:0/766968056 shutdown_connections 2023-11-13T11:02:32.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556c8fa5f940 0x556c8fa64b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x556c8fa57be0 0x556c8fa65070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8fa5a020 0x556c8fa56620 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 --2- 172.21.15.16:0/766968056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556c8fa382c0 0x556c8fa36250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 -- 172.21.15.16:0/766968056 >> 172.21.15.16:0/766968056 conn(0x556c8fa32b20 msgr2=0x556c8fa2dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6babfff640 0 client.6240 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 -- 172.21.15.16:0/766968056 shutdown_connections 2023-11-13T11:02:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 1 -- 172.21.15.16:0/766968056 wait complete. 2023-11-13T11:02:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.063+0000 7f6bda3cb740 20 client.6240 trim_cache size 0 max 16384 2023-11-13T11:02:32.081 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-11-13T11:02:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a733ba56-624e-496c-818f-72ffc74a2833", 2023-11-13T11:02:32.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-11-13T11:02:32.082 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:32.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 -- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6bb406ae40 msgr2=0x7f6bb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6bb406ae40 0x7f6bb406d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009a30 tx=0x7f6bcc008040 comp rx=0 tx=0).stop 2023-11-13T11:02:32.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 -- 172.21.15.16:0/3018614063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd015ce70 msgr2=0x7f6bd01f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd015ce70 0x7f6bd01f60b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8036040 tx=0x7f6bc8004a40 comp rx=0 tx=0).stop 2023-11-13T11:02:32.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 -- 172.21.15.16:0/3018614063 shutdown_connections 2023-11-13T11:02:32.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6bb406ae40 0x7f6bb406d300 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd015ce70 0x7f6bd01f60b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bd00d1850 0x7f6bd01f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 --2- 172.21.15.16:0/3018614063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bd00cdbb0 0x7f6bd00a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 -- 172.21.15.16:0/3018614063 >> 172.21.15.16:0/3018614063 conn(0x7f6bd008c580 msgr2=0x7f6bd015f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:32.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 -- 172.21.15.16:0/3018614063 shutdown_connections 2023-11-13T11:02:32.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.064+0000 7f6bd64ff640 1 -- 172.21.15.16:0/3018614063 wait complete. 2023-11-13T11:02:32.086 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'a733ba56-624e-496c-818f-72ffc74a2833', 'mode': 'asynchronous'}' 2023-11-13T11:02:32.086 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2023-11-13T11:02:32.086 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2023-11-13T11:02:32.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.300+0000 7f7216e95640 1 Processor -- start 2023-11-13T11:02:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.300+0000 7f7216e95640 1 -- start start 2023-11-13T11:02:32.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.302+0000 7f7216e95640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f721015b190 0x7f721006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.302+0000 7f7216e95640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f721006a190 0x7f721006a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.302+0000 7f7216e95640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7210158c20 con 0x7f72100cb190 2023-11-13T11:02:32.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.302+0000 7f7216e95640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721006ab40 con 0x7f721015b190 2023-11-13T11:02:32.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.302+0000 7f7216e95640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72100b1830 con 0x7f721006a190 2023-11-13T11:02:32.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.302+0000 7f7215692640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f721015b190 0x7f721006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.302+0000 7f7215692640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f721015b190 0x7f721006f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59918/0 (socket says 172.21.15.16:59918) 2023-11-13T11:02:32.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.302+0000 7f7215692640 1 -- 172.21.15.16:0/1277953900 learned_addr learned my addr 172.21.15.16:0/1277953900 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:32.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.302+0000 7f7215e93640 1 --1- 172.21.15.16:0/1277953900 >> v1:172.21.15.16:6789/0 conn(0x7f72100cb190 0x7f72100716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54586/0 (socket says 172.21.15.16:54586) 2023-11-13T11:02:32.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.303+0000 7f7215692640 1 -- 172.21.15.16:0/1277953900 >> v2:172.21.15.184:3300/0 conn(0x7f721006a190 msgr2=0x7f721006a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:32.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.303+0000 7f7215692640 1 --2- 172.21.15.16:0/1277953900 >> v2:172.21.15.184:3300/0 conn(0x7f721006a190 0x7f721006a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.303+0000 7f7215692640 1 -- 172.21.15.16:0/1277953900 >> v1:172.21.15.16:6789/0 conn(0x7f72100cb190 legacy=0x7f72100716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.303+0000 7f7215692640 1 -- 172.21.15.16:0/1277953900 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721015b780 con 0x7f721015b190 2023-11-13T11:02:32.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.304+0000 7f7215692640 1 --2- 172.21.15.16:0/1277953900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721015b190 0x7f721006f970 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f72100b2ba0 tx=0x7f720002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bd0defa6cb5385 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.304+0000 7f7214e91640 1 -- 172.21.15.16:0/1277953900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200031870 con 0x7f721015b190 2023-11-13T11:02:32.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.304+0000 7f7214e91640 1 -- 172.21.15.16:0/1277953900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72000319d0 con 0x7f721015b190 2023-11-13T11:02:32.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.304+0000 7f7214e91640 1 -- 172.21.15.16:0/1277953900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200031d60 con 0x7f721015b190 2023-11-13T11:02:32.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.305+0000 7f7216e95640 1 -- 172.21.15.16:0/1277953900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721015b190 msgr2=0x7f721006f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.305+0000 7f7216e95640 1 --2- 172.21.15.16:0/1277953900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721015b190 0x7f721006f970 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f72100b2ba0 tx=0x7f720002b910 comp rx=0 tx=0).stop 2023-11-13T11:02:32.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.305+0000 7f7216e95640 1 -- 172.21.15.16:0/1277953900 shutdown_connections 2023-11-13T11:02:32.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.305+0000 7f7216e95640 1 --2- 172.21.15.16:0/1277953900 >> v2:172.21.15.184:3300/0 conn(0x7f721006a190 0x7f721006a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.305+0000 7f7216e95640 1 --2- 172.21.15.16:0/1277953900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721015b190 0x7f721006f970 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.305+0000 7f7216e95640 1 -- 172.21.15.16:0/1277953900 >> 172.21.15.16:0/1277953900 conn(0x7f72100b04a0 msgr2=0x7f72100b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.305+0000 7f7216e95640 1 -- 172.21.15.16:0/1277953900 shutdown_connections 2023-11-13T11:02:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.305+0000 7f7216e95640 1 -- 172.21.15.16:0/1277953900 wait complete. 2023-11-13T11:02:32.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.307+0000 7f7216e95640 1 Processor -- start 2023-11-13T11:02:32.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.307+0000 7f7216e95640 1 -- start start 2023-11-13T11:02:32.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.308+0000 7f7216e95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721006a190 0x7f72101f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.308+0000 7f7216e95640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72100cb190 0x7f72101f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.308+0000 7f7216e95640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f721015b190 0x7f72101549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.308+0000 7f7216e95640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210155710 con 0x7f721006a190 2023-11-13T11:02:32.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.308+0000 7f7216e95640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101559f0 con 0x7f72100cb190 2023-11-13T11:02:32.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.308+0000 7f7216e95640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7210155cd0 con 0x7f721015b190 2023-11-13T11:02:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.308+0000 7f7215e93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721006a190 0x7f72101f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7215e93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721006a190 0x7f72101f39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59926/0 (socket says 172.21.15.16:59926) 2023-11-13T11:02:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7215e93640 1 -- 172.21.15.16:0/3873729912 learned_addr learned my addr 172.21.15.16:0/3873729912 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7215e93640 1 -- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f721015b190 msgr2=0x7f72101549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7216694640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f721015b190 0x7f72101549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7215692640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72100cb190 0x7f72101f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7215e93640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f721015b190 0x7f72101549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7215e93640 1 -- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72100cb190 msgr2=0x7f72101f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7215e93640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72100cb190 0x7f72101f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7215e93640 1 -- 172.21.15.16:0/3873729912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000094d0 con 0x7f721006a190 2023-11-13T11:02:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.309+0000 7f7215e93640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721006a190 0x7f72101f39c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c800 tx=0x7f720c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.310+0000 7f72067fc640 1 -- 172.21.15.16:0/3873729912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00a640 con 0x7f721006a190 2023-11-13T11:02:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.310+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7210155fb0 con 0x7f721006a190 2023-11-13T11:02:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.310+0000 7f72067fc640 1 -- 172.21.15.16:0/3873729912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c00a7a0 con 0x7f721006a190 2023-11-13T11:02:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.310+0000 7f72067fc640 1 -- 172.21.15.16:0/3873729912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00ab50 con 0x7f721006a190 2023-11-13T11:02:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.310+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72101511d0 con 0x7f721006a190 2023-11-13T11:02:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.311+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721004e4d0 con 0x7f721006a190 2023-11-13T11:02:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.312+0000 7f72067fc640 1 -- 172.21.15.16:0/3873729912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f720c00acb0 con 0x7f721006a190 2023-11-13T11:02:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.314+0000 7f72067fc640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f71f406ae40 0x7f71f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.314+0000 7f72067fc640 1 -- 172.21.15.16:0/3873729912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f720c088fe0 con 0x7f721006a190 2023-11-13T11:02:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.315+0000 7f7215692640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f71f406ae40 0x7f71f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.316+0000 7f7215692640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f71f406ae40 0x7f71f406d300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7200036000 tx=0x7f720003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:32.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.318+0000 7f72067fc640 1 -- 172.21.15.16:0/3873729912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f720c0579c0 con 0x7f721006a190 2023-11-13T11:02:32.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.553+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f721014ec30 con 0x7f721006a190 2023-11-13T11:02:32.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.554+0000 7f72067fc640 1 -- 172.21.15.16:0/3873729912 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f720c017090 con 0x7f721006a190 2023-11-13T11:02:32.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:32.559 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:01:41.484670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2023-11-13T11:02:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f71f406ae40 msgr2=0x7f71f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f71f406ae40 0x7f71f406d300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7200036000 tx=0x7f720003c040 comp rx=0 tx=0).stop 2023-11-13T11:02:32.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721006a190 msgr2=0x7f72101f39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:32.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721006a190 0x7f72101f39c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c800 tx=0x7f720c00ccc0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 shutdown_connections 2023-11-13T11:02:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f71f406ae40 0x7f71f406d300 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f721015b190 0x7f72101549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72100cb190 0x7f72101f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 --2- 172.21.15.16:0/3873729912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721006a190 0x7f72101f39c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 >> 172.21.15.16:0/3873729912 conn(0x7f72100b04a0 msgr2=0x7f72100cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 shutdown_connections 2023-11-13T11:02:32.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:32.558+0000 7f7216e95640 1 -- 172.21.15.16:0/3873729912 wait complete. 2023-11-13T11:02:32.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2023-11-13T11:02:32.840 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:32.840 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:02:32.840 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:32.860 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-13T11:02:32.860 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2023-11-13T11:02:32.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2023-11-13T11:02:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.081+0000 7f7c3cf0a640 1 Processor -- start 2023-11-13T11:02:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.081+0000 7f7c3cf0a640 1 -- start start 2023-11-13T11:02:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.082+0000 7f7c3cf0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c380d2ed0 0x7f7c38167a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.082+0000 7f7c3cf0a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c380cac40 con 0x7f7c380d1850 2023-11-13T11:02:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.082+0000 7f7c3cf0a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c380ce0e0 con 0x7f7c3815ce70 2023-11-13T11:02:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.082+0000 7f7c3cf0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380ce3c0 con 0x7f7c380d2ed0 2023-11-13T11:02:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.083+0000 7f7c36ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7c380d1850 0x7f7c380d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36408/0 (socket says 172.21.15.16:36408) 2023-11-13T11:02:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.083+0000 7f7c36ffd640 1 -- 172.21.15.16:0/3303340334 learned_addr learned my addr 172.21.15.16:0/3303340334 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.083+0000 7f7c37fff640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c380d2ed0 0x7f7c38167a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.083+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363686241 0 0) 0x7f7c380cac40 con 0x7f7c380d1850 2023-11-13T11:02:33.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.083+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c24003520 con 0x7f7c380d1850 2023-11-13T11:02:33.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1861549026 0 0) 0x7f7c24003520 con 0x7f7c380d1850 2023-11-13T11:02:33.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c380d2ed0 msgr2=0x7f7c38167a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c380d2ed0 0x7f7c38167a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 >> v1:172.21.15.184:6789/0 conn(0x7f7c3815ce70 legacy=0x7f7c380cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380ce6a0 con 0x7f7c380d1850 2023-11-13T11:02:33.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f7c20001670 con 0x7f7c380d1850 2023-11-13T11:02:33.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 >> v1:172.21.15.16:6789/0 conn(0x7f7c380d1850 legacy=0x7f7c380d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c240039a0 0x7f7c24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c240047c0 0x7f7c24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c380d2ed0 0x7f7c24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380ce3c0 con 0x7f7c240047c0 2023-11-13T11:02:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2400ba60 con 0x7f7c240039a0 2023-11-13T11:02:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2400bc20 con 0x7f7c380d2ed0 2023-11-13T11:02:33.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c240047c0 0x7f7c24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c37fff640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c380d2ed0 0x7f7c24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.084+0000 7f7c377fe640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c240039a0 0x7f7c24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.085+0000 7f7c36ffd640 1 -- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c380d2ed0 msgr2=0x7f7c24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.085+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c380d2ed0 0x7f7c24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.085+0000 7f7c36ffd640 1 -- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c240039a0 msgr2=0x7f7c24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.085+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c240039a0 0x7f7c24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.085+0000 7f7c36ffd640 1 -- 172.21.15.16:0/3303340334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c24003520 con 0x7f7c240047c0 2023-11-13T11:02:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.086+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c240047c0 0x7f7c24004c40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000c210 tx=0x7f7c240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461762dd3a5af960 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.086+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20005ad0 con 0x7f7c240047c0 2023-11-13T11:02:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.086+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c20005c30 con 0x7f7c240047c0 2023-11-13T11:02:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.086+0000 7f7c367fc640 1 -- 172.21.15.16:0/3303340334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2002c870 con 0x7f7c240047c0 2023-11-13T11:02:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.087+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/3303340334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c240047c0 msgr2=0x7f7c24004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.087+0000 7f7c3cf0a640 1 --2- 172.21.15.16:0/3303340334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c240047c0 0x7f7c24004c40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000c210 tx=0x7f7c240057f0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.087+0000 7f7c37fff640 1 -- 172.21.15.16:0/3303340334 reap_dead start 2023-11-13T11:02:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.087+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/3303340334 shutdown_connections 2023-11-13T11:02:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.087+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/3303340334 >> 172.21.15.16:0/3303340334 conn(0x7f7c3808c580 msgr2=0x7f7c3809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.087+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/3303340334 shutdown_connections 2023-11-13T11:02:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.087+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/3303340334 wait complete. 2023-11-13T11:02:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.089+0000 7f7c3cf0a640 1 Processor -- start 2023-11-13T11:02:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.089+0000 7f7c3cf0a640 1 -- start start 2023-11-13T11:02:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.090+0000 7f7c3cf0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c240047c0 0x7f7c381f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.090+0000 7f7c3cf0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c3806b9e0 0x7f7c3806be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.090+0000 7f7c3cf0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c38154f40 0x7f7c381553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.090+0000 7f7c3cf0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3815a660 con 0x7f7c38154f40 2023-11-13T11:02:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.090+0000 7f7c3cf0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3806c3a0 con 0x7f7c240047c0 2023-11-13T11:02:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.090+0000 7f7c3cf0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380b85a0 con 0x7f7c3806b9e0 2023-11-13T11:02:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.090+0000 7f7c37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c38154f40 0x7f7c381553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.090+0000 7f7c37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c38154f40 0x7f7c381553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47654/0 (socket says 172.21.15.16:47654) 2023-11-13T11:02:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.090+0000 7f7c37fff640 1 -- 172.21.15.16:0/2416139671 learned_addr learned my addr 172.21.15.16:0/2416139671 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c3806b9e0 0x7f7c3806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c377fe640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c240047c0 0x7f7c381f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c37fff640 1 -- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c3806b9e0 msgr2=0x7f7c3806be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c37fff640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c3806b9e0 0x7f7c3806be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c37fff640 1 -- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c240047c0 msgr2=0x7f7c381f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c37fff640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c240047c0 0x7f7c381f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c37fff640 1 -- 172.21.15.16:0/2416139671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c38155be0 con 0x7f7c38154f40 2023-11-13T11:02:33.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c3806b9e0 0x7f7c3806be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:02:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c37fff640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c38154f40 0x7f7c381553c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7c3806a650 tx=0x7f7c30011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.091+0000 7f7c2ffff640 1 -- 172.21.15.16:0/2416139671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30004440 con 0x7f7c38154f40 2023-11-13T11:02:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.092+0000 7f7c2ffff640 1 -- 172.21.15.16:0/2416139671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c30003610 con 0x7f7c38154f40 2023-11-13T11:02:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.092+0000 7f7c2ffff640 1 -- 172.21.15.16:0/2416139671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c300039c0 con 0x7f7c38154f40 2023-11-13T11:02:33.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.092+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/2416139671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c38155ec0 con 0x7f7c38154f40 2023-11-13T11:02:33.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.092+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/2416139671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c38156480 con 0x7f7c38154f40 2023-11-13T11:02:33.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.094+0000 7f7c2ffff640 1 -- 172.21.15.16:0/2416139671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c30003b20 con 0x7f7c38154f40 2023-11-13T11:02:33.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.095+0000 7f7c2ffff640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7c1806ae70 0x7f7c1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.095+0000 7f7c2ffff640 1 -- 172.21.15.16:0/2416139671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7c30089740 con 0x7f7c38154f40 2023-11-13T11:02:33.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.096+0000 7f7c40ddc740 1 Processor -- start 2023-11-13T11:02:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.096+0000 7f7c40ddc740 1 -- start start 2023-11-13T11:02:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.096+0000 7f7c377fe640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7c1806ae70 0x7f7c1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.097+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.097+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d7069329d0 0x55d70695d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.097+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706937b50 0x55d706935870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.097+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7068f12b0 con 0x55d706955510 2023-11-13T11:02:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.097+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7068a3160 con 0x55d706937b50 2023-11-13T11:02:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.097+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7068a0850 con 0x55d7069329d0 2023-11-13T11:02:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.097+0000 7f7c377fe640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7c1806ae70 0x7f7c1806d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007fd0 tx=0x7f7c28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47668/0 (socket says 172.21.15.16:47668) 2023-11-13T11:02:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c377fe640 1 -- 172.21.15.16:0/1117981485 learned_addr learned my addr 172.21.15.16:0/1117981485 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c37fff640 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706937b50 0x55d706935870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d7069329d0 0x55d70695d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c377fe640 1 -- 172.21.15.16:0/1117981485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d7069329d0 msgr2=0x55d70695d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c377fe640 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d7069329d0 0x55d70695d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c377fe640 1 -- 172.21.15.16:0/1117981485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706937b50 msgr2=0x55d706935870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c377fe640 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706937b50 0x55d706935870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c377fe640 1 -- 172.21.15.16:0/1117981485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70685ec20 con 0x55d706955510 2023-11-13T11:02:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c37fff640 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706937b50 0x55d706935870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.098+0000 7f7c377fe640 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069558f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7c28001d40 tx=0x7f7c28013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa5af745cafaef0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c2d7fa640 1 -- 172.21.15.16:0/1117981485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28014b80 con 0x55d706955510 2023-11-13T11:02:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c2d7fa640 1 -- 172.21.15.16:0/1117981485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c28014ce0 con 0x55d706955510 2023-11-13T11:02:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c2d7fa640 1 -- 172.21.15.16:0/1117981485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28013560 con 0x55d706955510 2023-11-13T11:02:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1117981485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 msgr2=0x55d7069558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069558f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7c28001d40 tx=0x7f7c28013000 comp rx=0 tx=0).stop 2023-11-13T11:02:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1117981485 shutdown_connections 2023-11-13T11:02:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706937b50 0x55d706935870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d7069329d0 0x55d70695d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1117981485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069558f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.099+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1117981485 >> 172.21.15.16:0/1117981485 conn(0x55d7068f7770 msgr2=0x55d70697b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.100+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1117981485 shutdown_connections 2023-11-13T11:02:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.100+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1117981485 wait complete. 2023-11-13T11:02:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.100+0000 7f7c40ddc740 1 -- start start 2023-11-13T11:02:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.101+0000 7f7c40ddc740 1 Processor -- start 2023-11-13T11:02:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706937b50 0x55d70695cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d706963300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70689c140 con 0x55d706955510 2023-11-13T11:02:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d706838670 con 0x55d706956be0 2023-11-13T11:02:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d706855f40 con 0x55d706937b50 2023-11-13T11:02:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c40ddc740 10 client.0 _ll_register_callbacks cb 0x55d706892270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47670/0 (socket says 172.21.15.16:47670) 2023-11-13T11:02:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c36ffd640 1 -- 172.21.15.16:0/3442485431 learned_addr learned my addr 172.21.15.16:0/3442485431 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c377fe640 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706937b50 0x55d70695cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.102+0000 7f7c37fff640 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d706963300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c36ffd640 1 -- 172.21.15.16:0/3442485431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706937b50 msgr2=0x55d70695cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706937b50 0x55d70695cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c36ffd640 1 -- 172.21.15.16:0/3442485431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 msgr2=0x55d706963300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d706963300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c36ffd640 1 -- 172.21.15.16:0/3442485431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2800f040 con 0x55d706955510 2023-11-13T11:02:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069566a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000c330 tx=0x7f7c2002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10a02b94c4c78e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c167fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c167fc640 1 -- 172.21.15.16:0/3442485431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2002ca70 con 0x55d706955510 2023-11-13T11:02:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c167fc640 1 -- 172.21.15.16:0/3442485431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7c2002cbd0 con 0x55d706955510 2023-11-13T11:02:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.103+0000 7f7c40ddc740 1 -- 172.21.15.16:0/3442485431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d706895120 con 0x55d706955510 2023-11-13T11:02:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c167fc640 1 -- 172.21.15.16:0/3442485431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c20005a80 con 0x55d706955510 2023-11-13T11:02:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c40ddc740 10 client.6261 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:33.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c40ddc740 1 -- 172.21.15.16:0/3442485431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55d70689a7c0 con 0x55d706955510 2023-11-13T11:02:33.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c167fc640 1 -- 172.21.15.16:0/3442485431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d706895120 con 0x55d706955510 2023-11-13T11:02:33.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c167fc640 1 -- 172.21.15.16:0/3442485431 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7c20007070 con 0x55d706955510 2023-11-13T11:02:33.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c40ddc740 10 client.6261 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c40ddc740 10 client.6261 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c40ddc740 20 client.6261 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d70698b370 0x55d70698b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c40ddc740 4 client.6261 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:33.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c40ddc740 1 -- 172.21.15.16:0/3442485431 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d706894e40 con 0x55d70698b370 2023-11-13T11:02:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.104+0000 7f7c377fe640 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d70698b370 0x55d70698b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.105+0000 7f7c377fe640 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d70698b370 0x55d70698b780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe19469c628dfa6 server_cookie=9b75af12289e848 in_seq=0 out_seq=0 2023-11-13T11:02:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.105+0000 7f7c167fc640 10 client.6261 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c167fc640 1 -- 172.21.15.16:0/3442485431 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d706894e40 con 0x55d70698b370 2023-11-13T11:02:33.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c167fc640 10 client.6261 handle_command_reply: tid=0 2023-11-13T11:02:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 client.6261 shutdown 2023-11-13T11:02:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 -- 172.21.15.16:0/3442485431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 msgr2=0x55d7069566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069566a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000c330 tx=0x7f7c2002e070 comp rx=0 tx=0).stop 2023-11-13T11:02:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 -- 172.21.15.16:0/3442485431 shutdown_connections 2023-11-13T11:02:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d706963300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d70698b370 0x55d70698b780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706955510 0x55d7069566a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/3442485431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706937b50 0x55d70695cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c167fc640 0 client.6261 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 -- 172.21.15.16:0/3442485431 >> 172.21.15.16:0/3442485431 conn(0x55d7068f7770 msgr2=0x55d706954b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 -- 172.21.15.16:0/3442485431 shutdown_connections 2023-11-13T11:02:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.106+0000 7f7c40ddc740 1 -- 172.21.15.16:0/3442485431 wait complete. 2023-11-13T11:02:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.107+0000 7f7c40ddc740 20 client.6261 trim_cache size 0 max 16384 2023-11-13T11:02:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.123+0000 7f7c40ddc740 1 Processor -- start 2023-11-13T11:02:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.123+0000 7f7c40ddc740 1 -- start start 2023-11-13T11:02:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.124+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d70695d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.124+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706955510 0x55d706935630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.124+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d706935b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.124+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7065ff740 con 0x55d706937b50 2023-11-13T11:02:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.124+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d706954940 con 0x55d706956be0 2023-11-13T11:02:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.124+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d70698e700 con 0x55d706955510 2023-11-13T11:02:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.124+0000 7f7c377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d70695d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.124+0000 7f7c377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d70695d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47684/0 (socket says 172.21.15.16:47684) 2023-11-13T11:02:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.124+0000 7f7c377fe640 1 -- 172.21.15.16:0/2620019254 learned_addr learned my addr 172.21.15.16:0/2620019254 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.125+0000 7f7c377fe640 1 -- 172.21.15.16:0/2620019254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706955510 msgr2=0x55d706935630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.125+0000 7f7c377fe640 1 --2- 172.21.15.16:0/2620019254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706955510 0x55d706935630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.125+0000 7f7c377fe640 1 -- 172.21.15.16:0/2620019254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 msgr2=0x55d706935b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.125+0000 7f7c377fe640 1 --2- 172.21.15.16:0/2620019254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d706935b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.125+0000 7f7c377fe640 1 -- 172.21.15.16:0/2620019254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d706936c30 con 0x55d706937b50 2023-11-13T11:02:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.125+0000 7f7c377fe640 1 --2- 172.21.15.16:0/2620019254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d70695d3e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7c28025040 tx=0x7f7c28014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31aab008b36bf233 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.125+0000 7f7c2dffb640 1 -- 172.21.15.16:0/2620019254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28034d00 con 0x55d706937b50 2023-11-13T11:02:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.125+0000 7f7c2dffb640 1 -- 172.21.15.16:0/2620019254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c28034e60 con 0x55d706937b50 2023-11-13T11:02:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.125+0000 7f7c2dffb640 1 -- 172.21.15.16:0/2620019254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28034460 con 0x55d706937b50 2023-11-13T11:02:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.126+0000 7f7c40ddc740 1 -- 172.21.15.16:0/2620019254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 msgr2=0x55d70695d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.126+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/2620019254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d70695d3e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7c28025040 tx=0x7f7c28014000 comp rx=0 tx=0).stop 2023-11-13T11:02:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.126+0000 7f7c40ddc740 1 -- 172.21.15.16:0/2620019254 shutdown_connections 2023-11-13T11:02:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.126+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/2620019254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d706935b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.126+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/2620019254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706955510 0x55d706935630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.126+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/2620019254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d70695d3e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.126+0000 7f7c40ddc740 1 -- 172.21.15.16:0/2620019254 >> 172.21.15.16:0/2620019254 conn(0x55d7068f7770 msgr2=0x55d7069342c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.126+0000 7f7c40ddc740 1 -- 172.21.15.16:0/2620019254 shutdown_connections 2023-11-13T11:02:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.126+0000 7f7c40ddc740 1 -- 172.21.15.16:0/2620019254 wait complete. 2023-11-13T11:02:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.127+0000 7f7c40ddc740 1 -- start start 2023-11-13T11:02:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.127+0000 7f7c40ddc740 1 Processor -- start 2023-11-13T11:02:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.128+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d706994f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.128+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706955510 0x55d7069e6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.128+0000 7f7c40ddc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d70697b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.128+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7069e68a0 con 0x55d706937b50 2023-11-13T11:02:33.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.128+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7069e6a00 con 0x55d706956be0 2023-11-13T11:02:33.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.128+0000 7f7c40ddc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d706934ac0 con 0x55d706955510 2023-11-13T11:02:33.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.128+0000 7f7c40ddc740 10 client.0 _ll_register_callbacks cb 0x7f7c28003a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.128+0000 7f7c377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d706994f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d706994f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47694/0 (socket says 172.21.15.16:47694) 2023-11-13T11:02:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c377fe640 1 -- 172.21.15.16:0/1554067494 learned_addr learned my addr 172.21.15.16:0/1554067494 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c37fff640 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d70697b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706955510 0x55d7069e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c377fe640 1 -- 172.21.15.16:0/1554067494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706955510 msgr2=0x55d7069e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c377fe640 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706955510 0x55d7069e6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c377fe640 1 -- 172.21.15.16:0/1554067494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 msgr2=0x55d70697b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c377fe640 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d70697b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c377fe640 1 -- 172.21.15.16:0/1554067494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d70685ec20 con 0x55d706937b50 2023-11-13T11:02:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c37fff640 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d70697b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c377fe640 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d706994f50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c28030040 tx=0x7f7c28001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c20c723876857e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.129+0000 7f7c167fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.130+0000 7f7c167fc640 1 -- 172.21.15.16:0/1554067494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28034460 con 0x55d706937b50 2023-11-13T11:02:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.130+0000 7f7c167fc640 1 -- 172.21.15.16:0/1554067494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7c2801c050 con 0x55d706937b50 2023-11-13T11:02:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.130+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1554067494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c28014ce0 con 0x55d706937b50 2023-11-13T11:02:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.130+0000 7f7c167fc640 1 -- 172.21.15.16:0/1554067494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c280345c0 con 0x55d706937b50 2023-11-13T11:02:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.130+0000 7f7c40ddc740 10 client.6267 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.130+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1554067494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55d706837f00 con 0x55d706937b50 2023-11-13T11:02:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.130+0000 7f7c167fc640 1 -- 172.21.15.16:0/1554067494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c28014ce0 con 0x55d706937b50 2023-11-13T11:02:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.130+0000 7f7c167fc640 1 -- 172.21.15.16:0/1554067494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7c28034720 con 0x55d706937b50 2023-11-13T11:02:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.131+0000 7f7c40ddc740 10 client.6267 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.131+0000 7f7c40ddc740 10 client.6267 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.131+0000 7f7c40ddc740 20 client.6267 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.131+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d70695c1b0 0x55d70697c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.131+0000 7f7c40ddc740 4 client.6267 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2023-11-13T11:02:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.131+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1554067494 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55d70659cee0 con 0x55d70695c1b0 2023-11-13T11:02:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.131+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d70695c1b0 0x55d70697c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.132+0000 7f7c36ffd640 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d70695c1b0 0x55d70697c610 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68fb20b8da8c666 server_cookie=47804e2b5317e95e in_seq=0 out_seq=0 2023-11-13T11:02:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.132+0000 7f7c167fc640 10 client.6267 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c167fc640 1 -- 172.21.15.16:0/1554067494 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d70659cee0 con 0x55d70695c1b0 2023-11-13T11:02:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c167fc640 10 client.6267 handle_command_reply: tid=0 2023-11-13T11:02:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c40ddc740 1 client.6267 shutdown 2023-11-13T11:02:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1554067494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 msgr2=0x55d706994f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d706994f50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c28030040 tx=0x7f7c28001510 comp rx=0 tx=0).stop 2023-11-13T11:02:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1554067494 shutdown_connections 2023-11-13T11:02:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d706956be0 0x55d70697b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55d706955510 0x55d7069e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55d70695c1b0 0x55d70697c610 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c40ddc740 1 --2- 172.21.15.16:0/1554067494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d706937b50 0x55d706994f50 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1554067494 >> 172.21.15.16:0/1554067494 conn(0x55d7068f7770 msgr2=0x55d70692fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.140+0000 7f7c167fc640 0 client.6267 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.141+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1554067494 shutdown_connections 2023-11-13T11:02:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.141+0000 7f7c40ddc740 1 -- 172.21.15.16:0/1554067494 wait complete. 2023-11-13T11:02:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.141+0000 7f7c40ddc740 20 client.6267 trim_cache size 0 max 16384 2023-11-13T11:02:33.155 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-11-13T11:02:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "50c81e29-d6b3-48d1-a9ca-caaf3e76c244", 2023-11-13T11:02:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-11-13T11:02:33.156 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7c1806ae70 msgr2=0x7f7c1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7c1806ae70 0x7f7c1806d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007fd0 tx=0x7f7c28009040 comp rx=0 tx=0).stop 2023-11-13T11:02:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/2416139671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c38154f40 msgr2=0x7f7c381553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c38154f40 0x7f7c381553c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7c3806a650 tx=0x7f7c30011920 comp rx=0 tx=0).stop 2023-11-13T11:02:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/2416139671 shutdown_connections 2023-11-13T11:02:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7c1806ae70 0x7f7c1806d330 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c38154f40 0x7f7c381553c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c3806b9e0 0x7f7c3806be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 --2- 172.21.15.16:0/2416139671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c240047c0 0x7f7c381f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/2416139671 >> 172.21.15.16:0/2416139671 conn(0x7f7c3808c580 msgr2=0x7f7c380cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/2416139671 shutdown_connections 2023-11-13T11:02:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.142+0000 7f7c3cf0a640 1 -- 172.21.15.16:0/2416139671 wait complete. 2023-11-13T11:02:33.166 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '50c81e29-d6b3-48d1-a9ca-caaf3e76c244', 'mode': 'asynchronous'}' 2023-11-13T11:02:33.167 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2023-11-13T11:02:33.167 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2023-11-13T11:02:33.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:33.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.367+0000 7f2797fff640 1 Processor -- start 2023-11-13T11:02:33.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.367+0000 7f2797fff640 1 -- start start 2023-11-13T11:02:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.369+0000 7f2797fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f279815b190 0x7f279806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.369+0000 7f2797fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279806a190 0x7f279806a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.369+0000 7f2797fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2798158c20 con 0x7f27980cb190 2023-11-13T11:02:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.369+0000 7f2797fff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279806ab40 con 0x7f279815b190 2023-11-13T11:02:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.369+0000 7f2797fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980b1830 con 0x7f279806a190 2023-11-13T11:02:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.369+0000 7f2796ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f27980cb190 0x7f27980716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36424/0 (socket says 172.21.15.16:36424) 2023-11-13T11:02:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.369+0000 7f2796ffd640 1 -- 172.21.15.16:0/4159897359 learned_addr learned my addr 172.21.15.16:0/4159897359 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.369+0000 7f27977fe640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279806a190 0x7f279806a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.369+0000 7f278e7fc640 1 --2- 172.21.15.16:0/4159897359 >> v2:172.21.15.184:3300/0 conn(0x7f279815b190 0x7f279806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810797344 0 0) 0x7f2798158c20 con 0x7f27980cb190 2023-11-13T11:02:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2784003520 con 0x7f27980cb190 2023-11-13T11:02:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2235984343 0 0) 0x7f2784003520 con 0x7f27980cb190 2023-11-13T11:02:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279806a190 msgr2=0x7f279806a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279806a190 0x7f279806a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 >> v2:172.21.15.184:3300/0 conn(0x7f279815b190 msgr2=0x7f279806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 --2- 172.21.15.16:0/4159897359 >> v2:172.21.15.184:3300/0 conn(0x7f279815b190 0x7f279806f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279815b780 con 0x7f27980cb190 2023-11-13T11:02:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f2788001a10 con 0x7f27980cb190 2023-11-13T11:02:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 >> v1:172.21.15.16:6789/0 conn(0x7f27980cb190 legacy=0x7f27980716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279806a190 0x7f2784003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2784003f40 0x7f27840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784004d00 0x7f2784005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279806ab40 con 0x7f2784003f40 2023-11-13T11:02:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980b1830 con 0x7f2784004d00 2023-11-13T11:02:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.370+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278400ba30 con 0x7f279806a190 2023-11-13T11:02:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.371+0000 7f278e7fc640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2784003f40 0x7f27840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.371+0000 7f2796ffd640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279806a190 0x7f2784003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.371+0000 7f27977fe640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784004d00 0x7f2784005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.371+0000 7f278e7fc640 1 -- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279806a190 msgr2=0x7f2784003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.371+0000 7f278e7fc640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f279806a190 0x7f2784003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.371+0000 7f278e7fc640 1 -- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784004d00 msgr2=0x7f2784005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:33.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.371+0000 7f278e7fc640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784004d00 0x7f2784005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.371+0000 7f278e7fc640 1 -- 172.21.15.16:0/4159897359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2784003520 con 0x7f2784003f40 2023-11-13T11:02:33.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.371+0000 7f27977fe640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784004d00 0x7f2784005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:33.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.372+0000 7f278e7fc640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2784003f40 0x7f27840047c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2780009ac0 tx=0x7f27840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb58845253503af server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.372+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278002c710 con 0x7f2784003f40 2023-11-13T11:02:33.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.372+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278002c870 con 0x7f2784003f40 2023-11-13T11:02:33.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.372+0000 7f27967fc640 1 -- 172.21.15.16:0/4159897359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278002cc00 con 0x7f2784003f40 2023-11-13T11:02:33.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.373+0000 7f2797fff640 1 -- 172.21.15.16:0/4159897359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2784003f40 msgr2=0x7f27840047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.373+0000 7f2797fff640 1 --2- 172.21.15.16:0/4159897359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2784003f40 0x7f27840047c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2780009ac0 tx=0x7f27840057f0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.373+0000 7f27977fe640 1 -- 172.21.15.16:0/4159897359 reap_dead start 2023-11-13T11:02:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.373+0000 7f2797fff640 1 -- 172.21.15.16:0/4159897359 shutdown_connections 2023-11-13T11:02:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.373+0000 7f2797fff640 1 -- 172.21.15.16:0/4159897359 >> 172.21.15.16:0/4159897359 conn(0x7f27980b04a0 msgr2=0x7f27980b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.373+0000 7f2797fff640 1 -- 172.21.15.16:0/4159897359 shutdown_connections 2023-11-13T11:02:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.373+0000 7f2797fff640 1 -- 172.21.15.16:0/4159897359 wait complete. 2023-11-13T11:02:33.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.375+0000 7f2797fff640 1 Processor -- start 2023-11-13T11:02:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.375+0000 7f2797fff640 1 -- start start 2023-11-13T11:02:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.376+0000 7f2797fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784003f40 0x7f27981548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.376+0000 7f2797fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27980c7b20 0x7f27981560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.376+0000 7f2797fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980ce9b0 0x7f27981f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.376+0000 7f2797fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2798154df0 con 0x7f27980c7b20 2023-11-13T11:02:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.376+0000 7f2797fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2798155050 con 0x7f2784003f40 2023-11-13T11:02:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.376+0000 7f2797fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2798155330 con 0x7f27980ce9b0 2023-11-13T11:02:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.376+0000 7f278e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27980c7b20 0x7f27981560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.376+0000 7f278e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27980c7b20 0x7f27981560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47712/0 (socket says 172.21.15.16:47712) 2023-11-13T11:02:33.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.376+0000 7f278e7fc640 1 -- 172.21.15.16:0/2578312310 learned_addr learned my addr 172.21.15.16:0/2578312310 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:33.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f27977fe640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980ce9b0 0x7f27981f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f2796ffd640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784003f40 0x7f27981548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f278e7fc640 1 -- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980ce9b0 msgr2=0x7f27981f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f278e7fc640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980ce9b0 0x7f27981f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f278e7fc640 1 -- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784003f40 msgr2=0x7f27981548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f278e7fc640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784003f40 0x7f27981548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f278e7fc640 1 -- 172.21.15.16:0/2578312310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798155610 con 0x7f27980c7b20 2023-11-13T11:02:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f2796ffd640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784003f40 0x7f27981548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f278e7fc640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27980c7b20 0x7f27981560f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f278002c2b0 tx=0x7f2780004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f278ffff640 1 -- 172.21.15.16:0/2578312310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780003810 con 0x7f27980c7b20 2023-11-13T11:02:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f278ffff640 1 -- 172.21.15.16:0/2578312310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278002de90 con 0x7f27980c7b20 2023-11-13T11:02:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.377+0000 7f278ffff640 1 -- 172.21.15.16:0/2578312310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780003b70 con 0x7f27980c7b20 2023-11-13T11:02:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.378+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27981558f0 con 0x7f27980c7b20 2023-11-13T11:02:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.378+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2798156630 con 0x7f27980c7b20 2023-11-13T11:02:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.379+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279804e4d0 con 0x7f27980c7b20 2023-11-13T11:02:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.380+0000 7f278ffff640 1 -- 172.21.15.16:0/2578312310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2780003cd0 con 0x7f27980c7b20 2023-11-13T11:02:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.381+0000 7f278ffff640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f277806ae70 0x7f277806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.381+0000 7f278ffff640 1 -- 172.21.15.16:0/2578312310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f27800ab670 con 0x7f27980c7b20 2023-11-13T11:02:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.381+0000 7f2796ffd640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f277806ae70 0x7f277806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.383+0000 7f2796ffd640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f277806ae70 0x7f277806d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f27980b3840 tx=0x7f2788001390 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:33.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.385+0000 7f278ffff640 1 -- 172.21.15.16:0/2578312310 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2780076590 con 0x7f27980c7b20 2023-11-13T11:02:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.618+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27980b3100 con 0x7f27980c7b20 2023-11-13T11:02:33.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.619+0000 7f278ffff640 1 -- 172.21.15.16:0/2578312310 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f2780079bb0 con 0x7f27980c7b20 2023-11-13T11:02:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:33.624 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:01:41.484670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:33.624 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2023-11-13T11:02:33.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.622+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f277806ae70 msgr2=0x7f277806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.622+0000 7f2797fff640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f277806ae70 0x7f277806d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f27980b3840 tx=0x7f2788001390 comp rx=0 tx=0).stop 2023-11-13T11:02:33.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.622+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27980c7b20 msgr2=0x7f27981560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:33.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.622+0000 7f2797fff640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27980c7b20 0x7f27981560f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f278002c2b0 tx=0x7f2780004730 comp rx=0 tx=0).stop 2023-11-13T11:02:33.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.622+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 shutdown_connections 2023-11-13T11:02:33.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.623+0000 7f2797fff640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f277806ae70 0x7f277806d330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.623+0000 7f2797fff640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27980ce9b0 0x7f27981f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.623+0000 7f2797fff640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27980c7b20 0x7f27981560f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.623+0000 7f2797fff640 1 --2- 172.21.15.16:0/2578312310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2784003f40 0x7f27981548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.623+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 >> 172.21.15.16:0/2578312310 conn(0x7f27980b04a0 msgr2=0x7f27980cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.623+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 shutdown_connections 2023-11-13T11:02:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:33.623+0000 7f2797fff640 1 -- 172.21.15.16:0/2578312310 wait complete. 2023-11-13T11:02:33.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2023-11-13T11:02:33.904 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:02:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:33.921 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-13T11:02:33.921 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2023-11-13T11:02:33.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2023-11-13T11:02:34.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.154+0000 7fb10ffff640 1 Processor -- start 2023-11-13T11:02:34.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.155+0000 7fb10ffff640 1 -- start start 2023-11-13T11:02:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.156+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb110159bd0 0x7fb11006f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.156+0000 7fb10ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb1100ade40 0x7fb1100b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.157+0000 7fb10ffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1100ccab0 con 0x7fb1100b2290 2023-11-13T11:02:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.157+0000 7fb10ffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1100ccd30 con 0x7fb1100ade40 2023-11-13T11:02:34.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.157+0000 7fb10ffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100cd010 con 0x7fb110159bd0 2023-11-13T11:02:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.157+0000 7fb107fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb1100b2290 0x7fb1100ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36430/0 (socket says 172.21.15.16:36430) 2023-11-13T11:02:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.157+0000 7fb107fff640 1 -- 172.21.15.16:0/935077676 learned_addr learned my addr 172.21.15.16:0/935077676 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.157+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/935077676 >> v2:172.21.15.16:3300/0 conn(0x7fb1100ade40 0x7fb1100b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.157+0000 7fb10effd640 1 --2- 172.21.15.16:0/935077676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb110159bd0 0x7fb11006f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.157+0000 7fb10e7fc640 1 -- 172.21.15.16:0/935077676 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406628063 0 0) 0x7fb1100ccab0 con 0x7fb1100b2290 2023-11-13T11:02:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.157+0000 7fb10e7fc640 1 -- 172.21.15.16:0/935077676 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0fc005730 con 0x7fb1100b2290 2023-11-13T11:02:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.158+0000 7fb10f7fe640 1 -- 172.21.15.16:0/935077676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb110159bd0 msgr2=0x7fb11006f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.158+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/935077676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb110159bd0 0x7fb11006f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.158+0000 7fb10f7fe640 1 -- 172.21.15.16:0/935077676 >> v1:172.21.15.16:6789/0 conn(0x7fb1100b2290 legacy=0x7fb1100ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.158+0000 7fb10f7fe640 1 -- 172.21.15.16:0/935077676 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100cd2f0 con 0x7fb1100ade40 2023-11-13T11:02:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.158+0000 7fb10effd640 1 --2- 172.21.15.16:0/935077676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb110159bd0 0x7fb11006f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:02:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.159+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/935077676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100ade40 0x7fb1100b26f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb108009d90 tx=0x7fb10802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdc98866fd1489c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.159+0000 7fb10e7fc640 1 -- 172.21.15.16:0/935077676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10802cc50 con 0x7fb1100ade40 2023-11-13T11:02:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.159+0000 7fb10e7fc640 1 -- 172.21.15.16:0/935077676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10802cdb0 con 0x7fb1100ade40 2023-11-13T11:02:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.159+0000 7fb10e7fc640 1 -- 172.21.15.16:0/935077676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108034660 con 0x7fb1100ade40 2023-11-13T11:02:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.160+0000 7fb10ffff640 1 -- 172.21.15.16:0/935077676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100ade40 msgr2=0x7fb1100b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.160+0000 7fb10ffff640 1 --2- 172.21.15.16:0/935077676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100ade40 0x7fb1100b26f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb108009d90 tx=0x7fb10802bea0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.160+0000 7fb10ffff640 1 -- 172.21.15.16:0/935077676 shutdown_connections 2023-11-13T11:02:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.160+0000 7fb10ffff640 1 --2- 172.21.15.16:0/935077676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100ade40 0x7fb1100b26f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.160+0000 7fb10ffff640 1 --2- 172.21.15.16:0/935077676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb110159bd0 0x7fb11006f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.160+0000 7fb10ffff640 1 -- 172.21.15.16:0/935077676 >> 172.21.15.16:0/935077676 conn(0x7fb1100b0970 msgr2=0x7fb1100b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:34.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.160+0000 7fb10ffff640 1 -- 172.21.15.16:0/935077676 shutdown_connections 2023-11-13T11:02:34.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.160+0000 7fb10ffff640 1 -- 172.21.15.16:0/935077676 wait complete. 2023-11-13T11:02:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.162+0000 7fb10ffff640 1 Processor -- start 2023-11-13T11:02:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.162+0000 7fb10ffff640 1 -- start start 2023-11-13T11:02:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100ade40 0x7fb11015b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100b2290 0x7fb11014a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb11015ca30 0x7fb1101f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb10ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1101e3130 con 0x7fb1100b2290 2023-11-13T11:02:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb10ffff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb110156610 con 0x7fb11015ca30 2023-11-13T11:02:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb10ffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1101568f0 con 0x7fb1100ade40 2023-11-13T11:02:34.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb107fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100b2290 0x7fb11014a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb107fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100b2290 0x7fb11014a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47716/0 (socket says 172.21.15.16:47716) 2023-11-13T11:02:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb107fff640 1 -- 172.21.15.16:0/156526601 learned_addr learned my addr 172.21.15.16:0/156526601 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.163+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb11015ca30 0x7fb1101f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb10effd640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100ade40 0x7fb11015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb107fff640 1 -- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100ade40 msgr2=0x7fb11015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb107fff640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100ade40 0x7fb11015b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb107fff640 1 -- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb11015ca30 msgr2=0x7fb1101f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb107fff640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb11015ca30 0x7fb1101f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb107fff640 1 -- 172.21.15.16:0/156526601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108009a70 con 0x7fb1100b2290 2023-11-13T11:02:34.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb107fff640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100b2290 0x7fb11014a1e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c500 tx=0x7fb0f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb10cff9640 1 -- 172.21.15.16:0/156526601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8005ce0 con 0x7fb1100b2290 2023-11-13T11:02:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb10cff9640 1 -- 172.21.15.16:0/156526601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f8005e40 con 0x7fb1100b2290 2023-11-13T11:02:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb10ffff640 1 -- 172.21.15.16:0/156526601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1101e2dc0 con 0x7fb1100b2290 2023-11-13T11:02:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb10cff9640 1 -- 172.21.15.16:0/156526601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8007330 con 0x7fb1100b2290 2023-11-13T11:02:34.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.164+0000 7fb10ffff640 1 -- 172.21.15.16:0/156526601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11015cee0 con 0x7fb1100b2290 2023-11-13T11:02:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.167+0000 7fb10cff9640 1 -- 172.21.15.16:0/156526601 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb0f8007550 con 0x7fb1100b2290 2023-11-13T11:02:34.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.168+0000 7fb10cff9640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb0f006ae40 0x7fb0f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.168+0000 7fb10cff9640 1 -- 172.21.15.16:0/156526601 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb0f8088d60 con 0x7fb1100b2290 2023-11-13T11:02:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.168+0000 7fb10effd640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb0f006ae40 0x7fb0f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.169+0000 7fb11846c740 1 Processor -- start 2023-11-13T11:02:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.169+0000 7fb11846c740 1 -- start start 2023-11-13T11:02:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.169+0000 7fb10effd640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb0f006ae40 0x7fb0f006d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb110155f80 tx=0x7fb100008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb11846c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 0x561fc50c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb11846c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50995c0 0x561fc50c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb11846c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb11846c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc505a9b0 con 0x561fc509d120 2023-11-13T11:02:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb11846c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc500ccb0 con 0x561fc50995c0 2023-11-13T11:02:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb11846c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fc500a3a0 con 0x561fc50c95a0 2023-11-13T11:02:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb10f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb10f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47718/0 (socket says 172.21.15.16:47718) 2023-11-13T11:02:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb10f7fe640 1 -- 172.21.15.16:0/631300049 learned_addr learned my addr 172.21.15.16:0/631300049 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb10effd640 1 --2- 172.21.15.16:0/631300049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 0x561fc50c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.170+0000 7fb107fff640 1 --2- 172.21.15.16:0/631300049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50995c0 0x561fc50c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.171+0000 7fb10f7fe640 1 -- 172.21.15.16:0/631300049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 msgr2=0x561fc50c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.171+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/631300049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 0x561fc50c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.171+0000 7fb10f7fe640 1 -- 172.21.15.16:0/631300049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50995c0 msgr2=0x561fc50c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.171+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/631300049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50995c0 0x561fc50c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.171+0000 7fb10f7fe640 1 -- 172.21.15.16:0/631300049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fc4fc85e0 con 0x561fc509d120 2023-11-13T11:02:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.171+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/631300049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50c6d50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb108037040 tx=0x7fb10802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e926426ff91abd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.171+0000 7fb1057fa640 1 -- 172.21.15.16:0/631300049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10803f480 con 0x561fc509d120 2023-11-13T11:02:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.171+0000 7fb1057fa640 1 -- 172.21.15.16:0/631300049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10803f5e0 con 0x561fc509d120 2023-11-13T11:02:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.171+0000 7fb1057fa640 1 -- 172.21.15.16:0/631300049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10803f990 con 0x561fc509d120 2023-11-13T11:02:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.172+0000 7fb11846c740 1 -- 172.21.15.16:0/631300049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 msgr2=0x561fc50c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.172+0000 7fb11846c740 1 --2- 172.21.15.16:0/631300049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50c6d50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb108037040 tx=0x7fb10802e000 comp rx=0 tx=0).stop 2023-11-13T11:02:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.172+0000 7fb11846c740 1 -- 172.21.15.16:0/631300049 shutdown_connections 2023-11-13T11:02:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.172+0000 7fb11846c740 1 --2- 172.21.15.16:0/631300049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50c6d50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.172+0000 7fb11846c740 1 --2- 172.21.15.16:0/631300049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50995c0 0x561fc50c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.172+0000 7fb11846c740 1 --2- 172.21.15.16:0/631300049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 0x561fc50c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.172+0000 7fb11846c740 1 -- 172.21.15.16:0/631300049 >> 172.21.15.16:0/631300049 conn(0x561fc509b830 msgr2=0x561fc50e43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.172+0000 7fb11846c740 1 -- 172.21.15.16:0/631300049 shutdown_connections 2023-11-13T11:02:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.172+0000 7fb11846c740 1 -- 172.21.15.16:0/631300049 wait complete. 2023-11-13T11:02:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.173+0000 7fb11846c740 1 -- start start 2023-11-13T11:02:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.173+0000 7fb11846c740 1 Processor -- start 2023-11-13T11:02:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.174+0000 7fb11846c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.174+0000 7fb11846c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 0x561fc50dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.174+0000 7fb11846c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 0x561fc50b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.174+0000 7fb11846c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc5005c90 con 0x561fc509d120 2023-11-13T11:02:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.174+0000 7fb11846c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc4fa1f00 con 0x561fc50dd8b0 2023-11-13T11:02:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.174+0000 7fb11846c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fc4fbf880 con 0x561fc50c95a0 2023-11-13T11:02:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.174+0000 7fb11846c740 10 client.0 _ll_register_callbacks cb 0x561fc4fc1ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.174+0000 7fb10effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb10effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47734/0 (socket says 172.21.15.16:47734) 2023-11-13T11:02:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb10effd640 1 -- 172.21.15.16:0/3039064367 learned_addr learned my addr 172.21.15.16:0/3039064367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb107fff640 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 0x561fc50dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 0x561fc50b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb10effd640 1 -- 172.21.15.16:0/3039064367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 msgr2=0x561fc50dd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb10effd640 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 0x561fc50dd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb10effd640 1 -- 172.21.15.16:0/3039064367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 msgr2=0x561fc50b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb10effd640 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 0x561fc50b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb10effd640 1 -- 172.21.15.16:0/3039064367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1100cd2f0 con 0x561fc509d120 2023-11-13T11:02:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb10effd640 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50b3420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb10000ed40 tx=0x7fb100004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af22a34b59ab635 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.175+0000 7fb0ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/3039064367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100015bd0 con 0x561fc509d120 2023-11-13T11:02:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb11846c740 1 -- 172.21.15.16:0/3039064367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fc4ffec70 con 0x561fc509d120 2023-11-13T11:02:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/3039064367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb100015d30 con 0x561fc509d120 2023-11-13T11:02:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/3039064367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb100021420 con 0x561fc509d120 2023-11-13T11:02:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb11846c740 10 client.6288 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb11846c740 1 -- 172.21.15.16:0/3039064367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x561fc5004310 con 0x561fc509d120 2023-11-13T11:02:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/3039064367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561fc4ffec70 con 0x561fc509d120 2023-11-13T11:02:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/3039064367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb100021580 con 0x561fc509d120 2023-11-13T11:02:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb11846c740 10 client.6288 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb11846c740 10 client.6288 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb11846c740 20 client.6288 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.176+0000 7fb11846c740 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x561fc50b6170 0x561fc50b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.177+0000 7fb11846c740 4 client.6288 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.177+0000 7fb11846c740 1 -- 172.21.15.16:0/3039064367 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fc4ffe990 con 0x561fc50b6170 2023-11-13T11:02:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.177+0000 7fb107fff640 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x561fc50b6170 0x561fc50b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.177+0000 7fb107fff640 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x561fc50b6170 0x561fc50b6580 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e93b4524c237b4 server_cookie=487a54c20cbd1705 in_seq=0 out_seq=0 2023-11-13T11:02:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.177+0000 7fb0ee7fc640 10 client.6288 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/3039064367 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561fc4ffe990 con 0x561fc50b6170 2023-11-13T11:02:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb0ee7fc640 10 client.6288 handle_command_reply: tid=0 2023-11-13T11:02:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 client.6288 shutdown 2023-11-13T11:02:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 -- 172.21.15.16:0/3039064367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 msgr2=0x561fc50b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50b3420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb10000ed40 tx=0x7fb100004730 comp rx=0 tx=0).stop 2023-11-13T11:02:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 -- 172.21.15.16:0/3039064367 shutdown_connections 2023-11-13T11:02:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 0x561fc50b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50c95a0 0x561fc50dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x561fc50b6170 0x561fc50b6580 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 --2- 172.21.15.16:0/3039064367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50b3420 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb0ee7fc640 0 client.6288 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 -- 172.21.15.16:0/3039064367 >> 172.21.15.16:0/3039064367 conn(0x561fc50995c0 msgr2=0x561fc509dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.178+0000 7fb11846c740 1 -- 172.21.15.16:0/3039064367 shutdown_connections 2023-11-13T11:02:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.179+0000 7fb11846c740 1 -- 172.21.15.16:0/3039064367 wait complete. 2023-11-13T11:02:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.179+0000 7fb11846c740 20 client.6288 trim_cache size 0 max 16384 2023-11-13T11:02:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.195+0000 7fb11846c740 1 Processor -- start 2023-11-13T11:02:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.195+0000 7fb11846c740 1 -- start start 2023-11-13T11:02:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.196+0000 7fb11846c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50e7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.196+0000 7fb11846c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50c95a0 0x561fc50c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.196+0000 7fb11846c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50dd8b0 0x561fc50c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.196+0000 7fb11846c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc4d68ed0 con 0x561fc509d120 2023-11-13T11:02:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.196+0000 7fb11846c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc509b0a0 con 0x561fc50c95a0 2023-11-13T11:02:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.196+0000 7fb11846c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fc50e5810 con 0x561fc50dd8b0 2023-11-13T11:02:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50e7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50e7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47748/0 (socket says 172.21.15.16:47748) 2023-11-13T11:02:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10effd640 1 -- 172.21.15.16:0/3478934665 learned_addr learned my addr 172.21.15.16:0/3478934665 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:34.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/3478934665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50dd8b0 0x561fc50c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10effd640 1 -- 172.21.15.16:0/3478934665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50dd8b0 msgr2=0x561fc50c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10effd640 1 --2- 172.21.15.16:0/3478934665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50dd8b0 0x561fc50c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10effd640 1 -- 172.21.15.16:0/3478934665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50c95a0 msgr2=0x561fc50c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10effd640 1 --2- 172.21.15.16:0/3478934665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50c95a0 0x561fc50c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10effd640 1 -- 172.21.15.16:0/3478934665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fc4fa1790 con 0x561fc509d120 2023-11-13T11:02:34.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.197+0000 7fb10effd640 1 --2- 172.21.15.16:0/3478934665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50e7b40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb100006ec0 tx=0x7fb10001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da57a8f9f8d236b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb105ffb640 1 -- 172.21.15.16:0/3478934665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100015810 con 0x561fc509d120 2023-11-13T11:02:34.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb105ffb640 1 -- 172.21.15.16:0/3478934665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1000219f0 con 0x561fc509d120 2023-11-13T11:02:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb105ffb640 1 -- 172.21.15.16:0/3478934665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100015e10 con 0x561fc509d120 2023-11-13T11:02:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb11846c740 1 -- 172.21.15.16:0/3478934665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 msgr2=0x561fc50e7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb11846c740 1 --2- 172.21.15.16:0/3478934665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50e7b40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb100006ec0 tx=0x7fb10001f000 comp rx=0 tx=0).stop 2023-11-13T11:02:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb11846c740 1 -- 172.21.15.16:0/3478934665 shutdown_connections 2023-11-13T11:02:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb11846c740 1 --2- 172.21.15.16:0/3478934665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc50dd8b0 0x561fc50c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb11846c740 1 --2- 172.21.15.16:0/3478934665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50c95a0 0x561fc50c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb11846c740 1 --2- 172.21.15.16:0/3478934665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc509d120 0x561fc50e7b40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb11846c740 1 -- 172.21.15.16:0/3478934665 >> 172.21.15.16:0/3478934665 conn(0x561fc50995c0 msgr2=0x561fc5099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb11846c740 1 -- 172.21.15.16:0/3478934665 shutdown_connections 2023-11-13T11:02:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.198+0000 7fb11846c740 1 -- 172.21.15.16:0/3478934665 wait complete. 2023-11-13T11:02:34.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.199+0000 7fb11846c740 1 -- start start 2023-11-13T11:02:34.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.199+0000 7fb11846c740 1 Processor -- start 2023-11-13T11:02:34.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.200+0000 7fb11846c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc509d120 0x561fc51387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.200+0000 7fb11846c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc50c95a0 0x561fc5098820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.200+0000 7fb11846c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 0x561fc50e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.200+0000 7fb11846c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc50e1ec0 con 0x561fc50c95a0 2023-11-13T11:02:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.200+0000 7fb11846c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc50ca840 con 0x561fc50dd8b0 2023-11-13T11:02:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.200+0000 7fb11846c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fc50caa90 con 0x561fc509d120 2023-11-13T11:02:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.200+0000 7fb11846c740 10 client.0 _ll_register_callbacks cb 0x7fb100009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb107fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc50c95a0 0x561fc5098820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb107fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc50c95a0 0x561fc5098820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47762/0 (socket says 172.21.15.16:47762) 2023-11-13T11:02:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb107fff640 1 -- 172.21.15.16:0/2000562143 learned_addr learned my addr 172.21.15.16:0/2000562143 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb10effd640 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc509d120 0x561fc51387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb107fff640 1 -- 172.21.15.16:0/2000562143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc509d120 msgr2=0x561fc51387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb107fff640 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc509d120 0x561fc51387e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb10f7fe640 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 0x561fc50e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb107fff640 1 -- 172.21.15.16:0/2000562143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 msgr2=0x561fc50e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb107fff640 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 0x561fc50e6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb107fff640 1 -- 172.21.15.16:0/2000562143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fc5004310 con 0x561fc50c95a0 2023-11-13T11:02:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb107fff640 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc50c95a0 0x561fc5098820 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800cec0 tx=0x7fb0f8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9bce9ff5c96e79 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb0ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/2000562143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8083070 con 0x561fc50c95a0 2023-11-13T11:02:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/2000562143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb0f80541b0 con 0x561fc50c95a0 2023-11-13T11:02:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.201+0000 7fb11846c740 1 -- 172.21.15.16:0/2000562143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb10803f5e0 con 0x561fc50c95a0 2023-11-13T11:02:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/2000562143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0f8054370 con 0x561fc50c95a0 2023-11-13T11:02:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb11846c740 10 client.6294 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb11846c740 1 -- 172.21.15.16:0/2000562143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x561fc4fc6e40 con 0x561fc50c95a0 2023-11-13T11:02:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/2000562143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10803f5e0 con 0x561fc50c95a0 2023-11-13T11:02:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/2000562143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb0f8056f90 con 0x561fc50c95a0 2023-11-13T11:02:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb11846c740 10 client.6294 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb11846c740 10 client.6294 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb11846c740 20 client.6294 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb11846c740 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x561fc50e13d0 0x561fc50c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb11846c740 4 client.6294 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2023-11-13T11:02:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb11846c740 1 -- 172.21.15.16:0/2000562143 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x561fc4d06670 con 0x561fc50e13d0 2023-11-13T11:02:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.202+0000 7fb10effd640 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x561fc50e13d0 0x561fc50c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.203+0000 7fb10effd640 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x561fc50e13d0 0x561fc50c60a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53747ad93a8a2b93 server_cookie=6f854a7871a4eaf8 in_seq=0 out_seq=0 2023-11-13T11:02:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.203+0000 7fb0ee7fc640 10 client.6294 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb0ee7fc640 1 -- 172.21.15.16:0/2000562143 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561fc4d06670 con 0x561fc50e13d0 2023-11-13T11:02:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb0ee7fc640 10 client.6294 handle_command_reply: tid=0 2023-11-13T11:02:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 client.6294 shutdown 2023-11-13T11:02:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 -- 172.21.15.16:0/2000562143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc50c95a0 msgr2=0x561fc5098820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc50c95a0 0x561fc5098820 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800cec0 tx=0x7fb0f8007ea0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 -- 172.21.15.16:0/2000562143 shutdown_connections 2023-11-13T11:02:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561fc50dd8b0 0x561fc50e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x561fc50e13d0 0x561fc50c60a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc50c95a0 0x561fc5098820 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 --2- 172.21.15.16:0/2000562143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561fc509d120 0x561fc51387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 -- 172.21.15.16:0/2000562143 >> 172.21.15.16:0/2000562143 conn(0x561fc50995c0 msgr2=0x561fc5094f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb0ee7fc640 0 client.6294 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 -- 172.21.15.16:0/2000562143 shutdown_connections 2023-11-13T11:02:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 1 -- 172.21.15.16:0/2000562143 wait complete. 2023-11-13T11:02:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.204+0000 7fb11846c740 20 client.6294 trim_cache size 0 max 16384 2023-11-13T11:02:34.221 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-11-13T11:02:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "dfc0f507-4620-48b5-9445-ab890868fbd1", 2023-11-13T11:02:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-11-13T11:02:34.222 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 -- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb0f006ae40 msgr2=0x7fb0f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb0f006ae40 0x7fb0f006d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb110155f80 tx=0x7fb100008040 comp rx=0 tx=0).stop 2023-11-13T11:02:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 -- 172.21.15.16:0/156526601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100b2290 msgr2=0x7fb11014a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100b2290 0x7fb11014a1e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c500 tx=0x7fb0f800c9c0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 -- 172.21.15.16:0/156526601 shutdown_connections 2023-11-13T11:02:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb0f006ae40 0x7fb0f006d300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb11015ca30 0x7fb1101f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1100b2290 0x7fb11014a1e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 --2- 172.21.15.16:0/156526601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1100ade40 0x7fb11015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 -- 172.21.15.16:0/156526601 >> 172.21.15.16:0/156526601 conn(0x7fb1100b0970 msgr2=0x7fb11015a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 -- 172.21.15.16:0/156526601 shutdown_connections 2023-11-13T11:02:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.205+0000 7fb10ffff640 1 -- 172.21.15.16:0/156526601 wait complete. 2023-11-13T11:02:34.231 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'dfc0f507-4620-48b5-9445-ab890868fbd1', 'mode': 'asynchronous'}' 2023-11-13T11:02:34.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:34.232 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2023-11-13T11:02:34.283 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2023-11-13T11:02:34.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:34.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.489+0000 7fc4bac25640 1 Processor -- start 2023-11-13T11:02:34.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.489+0000 7fc4bac25640 1 -- start start 2023-11-13T11:02:34.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.491+0000 7fc4bac25640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4b415b190 0x7fc4b406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.491+0000 7fc4bac25640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc4b406a190 0x7fc4b406a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.491+0000 7fc4bac25640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b4158c20 con 0x7fc4b40cb190 2023-11-13T11:02:34.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.491+0000 7fc4bac25640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b406ab40 con 0x7fc4b415b190 2023-11-13T11:02:34.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.491+0000 7fc4bac25640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b40b1830 con 0x7fc4b406a190 2023-11-13T11:02:34.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.491+0000 7fc4b9422640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4b415b190 0x7fc4b406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.491+0000 7fc4b9422640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4b415b190 0x7fc4b406f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47770/0 (socket says 172.21.15.16:47770) 2023-11-13T11:02:34.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.491+0000 7fc4b9422640 1 -- 172.21.15.16:0/4035322270 learned_addr learned my addr 172.21.15.16:0/4035322270 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.492+0000 7fc4ba424640 1 --2- 172.21.15.16:0/4035322270 >> v2:172.21.15.184:3300/0 conn(0x7fc4b406a190 0x7fc4b406a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.492+0000 7fc4b9422640 1 -- 172.21.15.16:0/4035322270 >> v2:172.21.15.184:3300/0 conn(0x7fc4b406a190 msgr2=0x7fc4b406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.492+0000 7fc4b9422640 1 --2- 172.21.15.16:0/4035322270 >> v2:172.21.15.184:3300/0 conn(0x7fc4b406a190 0x7fc4b406a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.492+0000 7fc4b9422640 1 -- 172.21.15.16:0/4035322270 >> v1:172.21.15.184:6789/0 conn(0x7fc4b40cb190 legacy=0x7fc4b40716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.492+0000 7fc4b9422640 1 -- 172.21.15.16:0/4035322270 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b415b780 con 0x7fc4b415b190 2023-11-13T11:02:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.493+0000 7fc4b9422640 1 --2- 172.21.15.16:0/4035322270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 0x7fc4b406f970 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40b2ba0 tx=0x7fc4a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac88b6f3d5788d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.493+0000 7fc4b8c21640 1 -- 172.21.15.16:0/4035322270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40317e0 con 0x7fc4b415b190 2023-11-13T11:02:34.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.493+0000 7fc4b8c21640 1 -- 172.21.15.16:0/4035322270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a4031940 con 0x7fc4b415b190 2023-11-13T11:02:34.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.493+0000 7fc4b8c21640 1 -- 172.21.15.16:0/4035322270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4031cd0 con 0x7fc4b415b190 2023-11-13T11:02:34.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.494+0000 7fc4bac25640 1 -- 172.21.15.16:0/4035322270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 msgr2=0x7fc4b406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.494+0000 7fc4bac25640 1 --2- 172.21.15.16:0/4035322270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 0x7fc4b406f970 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40b2ba0 tx=0x7fc4a402b880 comp rx=0 tx=0).stop 2023-11-13T11:02:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.494+0000 7fc4bac25640 1 -- 172.21.15.16:0/4035322270 shutdown_connections 2023-11-13T11:02:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.494+0000 7fc4bac25640 1 --2- 172.21.15.16:0/4035322270 >> v2:172.21.15.184:3300/0 conn(0x7fc4b406a190 0x7fc4b406a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.494+0000 7fc4bac25640 1 --2- 172.21.15.16:0/4035322270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 0x7fc4b406f970 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.494+0000 7fc4bac25640 1 -- 172.21.15.16:0/4035322270 >> 172.21.15.16:0/4035322270 conn(0x7fc4b40b04a0 msgr2=0x7fc4b40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.494+0000 7fc4bac25640 1 -- 172.21.15.16:0/4035322270 shutdown_connections 2023-11-13T11:02:34.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.494+0000 7fc4bac25640 1 -- 172.21.15.16:0/4035322270 wait complete. 2023-11-13T11:02:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.496+0000 7fc4bac25640 1 Processor -- start 2023-11-13T11:02:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.496+0000 7fc4bac25640 1 -- start start 2023-11-13T11:02:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.497+0000 7fc4bac25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b406a190 0x7fc4b4148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.497+0000 7fc4bac25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b40cb190 0x7fc4b415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.497+0000 7fc4bac25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 0x7fc4b415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.497+0000 7fc4bac25640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b41f26a0 con 0x7fc4b415b190 2023-11-13T11:02:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.497+0000 7fc4bac25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b414d560 con 0x7fc4b40cb190 2023-11-13T11:02:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.497+0000 7fc4bac25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b414cfa0 con 0x7fc4b406a190 2023-11-13T11:02:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4ba424640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 0x7fc4b415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4ba424640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 0x7fc4b415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47786/0 (socket says 172.21.15.16:47786) 2023-11-13T11:02:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4ba424640 1 -- 172.21.15.16:0/3687931439 learned_addr learned my addr 172.21.15.16:0/3687931439 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4b9c23640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b406a190 0x7fc4b4148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4b9422640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b40cb190 0x7fc4b415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4ba424640 1 -- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b406a190 msgr2=0x7fc4b4148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4ba424640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b406a190 0x7fc4b4148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4ba424640 1 -- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b40cb190 msgr2=0x7fc4b415e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4ba424640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b40cb190 0x7fc4b415e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4ba424640 1 -- 172.21.15.16:0/3687931439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a40094d0 con 0x7fc4b415b190 2023-11-13T11:02:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.498+0000 7fc4ba424640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 0x7fc4b415ec50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007e40 tx=0x7fc4ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.499+0000 7fc4aa7fc640 1 -- 172.21.15.16:0/3687931439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac014620 con 0x7fc4b415b190 2023-11-13T11:02:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.499+0000 7fc4aa7fc640 1 -- 172.21.15.16:0/3687931439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac00ee90 con 0x7fc4b415b190 2023-11-13T11:02:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.499+0000 7fc4aa7fc640 1 -- 172.21.15.16:0/3687931439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac0149e0 con 0x7fc4b415b190 2023-11-13T11:02:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.499+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b414d280 con 0x7fc4b415b190 2023-11-13T11:02:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.499+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b406c290 con 0x7fc4b415b190 2023-11-13T11:02:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.500+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b406be10 con 0x7fc4b415b190 2023-11-13T11:02:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.501+0000 7fc4aa7fc640 1 -- 172.21.15.16:0/3687931439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc4ac014b40 con 0x7fc4b415b190 2023-11-13T11:02:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.502+0000 7fc4aa7fc640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc49806ae40 0x7fc49806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.502+0000 7fc4aa7fc640 1 -- 172.21.15.16:0/3687931439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc4ac08a1c0 con 0x7fc4b415b190 2023-11-13T11:02:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.502+0000 7fc4b9c23640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc49806ae40 0x7fc49806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:34.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.504+0000 7fc4b9c23640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc49806ae40 0x7fc49806d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0004ea0 tx=0x7fc4b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:34.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.507+0000 7fc4aa7fc640 1 -- 172.21.15.16:0/3687931439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc4b406be10 con 0x7fc4b415b190 2023-11-13T11:02:34.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.738+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4b406be10 con 0x7fc4b415b190 2023-11-13T11:02:34.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.739+0000 7fc4aa7fc640 1 -- 172.21.15.16:0/3687931439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fc4b406be10 con 0x7fc4b415b190 2023-11-13T11:02:34.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:34.744 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:01:41.484670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:34.744 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2023-11-13T11:02:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc49806ae40 msgr2=0x7fc49806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc49806ae40 0x7fc49806d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0004ea0 tx=0x7fc4b0009040 comp rx=0 tx=0).stop 2023-11-13T11:02:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 msgr2=0x7fc4b415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 0x7fc4b415ec50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007e40 tx=0x7fc4ac00e4b0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 shutdown_connections 2023-11-13T11:02:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc49806ae40 0x7fc49806d300 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4b415b190 0x7fc4b415ec50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4b40cb190 0x7fc4b415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 --2- 172.21.15.16:0/3687931439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4b406a190 0x7fc4b4148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 >> 172.21.15.16:0/3687931439 conn(0x7fc4b40b04a0 msgr2=0x7fc4b40cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.742+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 shutdown_connections 2023-11-13T11:02:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:34.743+0000 7fc4bac25640 1 -- 172.21.15.16:0/3687931439 wait complete. 2023-11-13T11:02:34.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2023-11-13T11:02:35.053 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:35.053 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:02:35.053 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:35.069 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-13T11:02:35.069 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:02:35.070 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:02:35.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:35.123 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2023-11-13T11:02:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-13T11:02:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-13T11:02:35.208 INFO:teuthology.orchestra.run.smithi016.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280652 s, 37.4 MB/s 2023-11-13T11:02:35.210 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2023-11-13T11:02:35.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.438+0000 7faacd905640 1 Processor -- start 2023-11-13T11:02:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.439+0000 7faacd905640 1 -- start start 2023-11-13T11:02:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.440+0000 7faacd905640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faac815ddd0 0x7faac815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.440+0000 7faacd905640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faac815a610 0x7faac80b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.441+0000 7faacd905640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac806db90 con 0x7faac815d9f0 2023-11-13T11:02:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.441+0000 7faacd905640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac815b070 con 0x7faac815a610 2023-11-13T11:02:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.441+0000 7faacd905640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac815aab0 con 0x7faac815ddd0 2023-11-13T11:02:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.441+0000 7faac77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faac815a610 0x7faac80b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.441+0000 7faac77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faac815a610 0x7faac80b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47792/0 (socket says 172.21.15.16:47792) 2023-11-13T11:02:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.441+0000 7faac77fe640 1 -- 172.21.15.16:0/2079383779 learned_addr learned my addr 172.21.15.16:0/2079383779 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.441+0000 7faac6ffd640 1 --1- 172.21.15.16:0/2079383779 >> v1:172.21.15.16:6789/0 conn(0x7faac815d9f0 0x7faac806d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36436/0 (socket says 172.21.15.16:36436) 2023-11-13T11:02:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.441+0000 7faabffff640 1 --2- 172.21.15.16:0/2079383779 >> v2:172.21.15.184:3300/0 conn(0x7faac815ddd0 0x7faac815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.442+0000 7faac77fe640 1 -- 172.21.15.16:0/2079383779 >> v2:172.21.15.184:3300/0 conn(0x7faac815ddd0 msgr2=0x7faac815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.442+0000 7faac77fe640 1 --2- 172.21.15.16:0/2079383779 >> v2:172.21.15.184:3300/0 conn(0x7faac815ddd0 0x7faac815a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.442+0000 7faac77fe640 1 -- 172.21.15.16:0/2079383779 >> v1:172.21.15.16:6789/0 conn(0x7faac815d9f0 legacy=0x7faac806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.442+0000 7faac77fe640 1 -- 172.21.15.16:0/2079383779 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac815ad90 con 0x7faac815a610 2023-11-13T11:02:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.443+0000 7faac77fe640 1 --2- 172.21.15.16:0/2079383779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815a610 0x7faac80b2630 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7faac80b3d10 tx=0x7faac002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23061e034120dd17 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.443+0000 7faac67fc640 1 -- 172.21.15.16:0/2079383779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac002ca80 con 0x7faac815a610 2023-11-13T11:02:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.443+0000 7faac67fc640 1 -- 172.21.15.16:0/2079383779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac002cbe0 con 0x7faac815a610 2023-11-13T11:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.443+0000 7faac67fc640 1 -- 172.21.15.16:0/2079383779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac00343f0 con 0x7faac815a610 2023-11-13T11:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.444+0000 7faacd905640 1 -- 172.21.15.16:0/2079383779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815a610 msgr2=0x7faac80b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.444+0000 7faacd905640 1 --2- 172.21.15.16:0/2079383779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815a610 0x7faac80b2630 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7faac80b3d10 tx=0x7faac002bee0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.444+0000 7faacd905640 1 -- 172.21.15.16:0/2079383779 shutdown_connections 2023-11-13T11:02:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.444+0000 7faacd905640 1 --2- 172.21.15.16:0/2079383779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815a610 0x7faac80b2630 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.444+0000 7faacd905640 1 --2- 172.21.15.16:0/2079383779 >> v2:172.21.15.184:3300/0 conn(0x7faac815ddd0 0x7faac815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.444+0000 7faacd905640 1 -- 172.21.15.16:0/2079383779 >> 172.21.15.16:0/2079383779 conn(0x7faac806a190 msgr2=0x7faac806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.444+0000 7faacd905640 1 -- 172.21.15.16:0/2079383779 shutdown_connections 2023-11-13T11:02:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.444+0000 7faacd905640 1 -- 172.21.15.16:0/2079383779 wait complete. 2023-11-13T11:02:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.446+0000 7faacd905640 1 Processor -- start 2023-11-13T11:02:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.446+0000 7faacd905640 1 -- start start 2023-11-13T11:02:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.447+0000 7faacd905640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac815a610 0x7faac814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.447+0000 7faacd905640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac815d9f0 0x7faac80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.447+0000 7faacd905640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815ddd0 0x7faac813b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.447+0000 7faacd905640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac813b8b0 con 0x7faac815ddd0 2023-11-13T11:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.447+0000 7faacd905640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac80cbed0 con 0x7faac815a610 2023-11-13T11:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.447+0000 7faacd905640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac814fb80 con 0x7faac815d9f0 2023-11-13T11:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.447+0000 7faac77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815ddd0 0x7faac813b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.447+0000 7faac77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815ddd0 0x7faac813b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47794/0 (socket says 172.21.15.16:47794) 2023-11-13T11:02:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.448+0000 7faac77fe640 1 -- 172.21.15.16:0/2167205090 learned_addr learned my addr 172.21.15.16:0/2167205090 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.448+0000 7faac6ffd640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac815a610 0x7faac814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.448+0000 7faabffff640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac815d9f0 0x7faac80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.448+0000 7faac77fe640 1 -- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac815d9f0 msgr2=0x7faac80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.448+0000 7faac77fe640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac815d9f0 0x7faac80ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.448+0000 7faac77fe640 1 -- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac815a610 msgr2=0x7faac814f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.448+0000 7faac77fe640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac815a610 0x7faac814f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.448+0000 7faac77fe640 1 -- 172.21.15.16:0/2167205090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0009b30 con 0x7faac815ddd0 2023-11-13T11:02:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.448+0000 7faac77fe640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815ddd0 0x7faac813b370 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faac0009e50 tx=0x7faac0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.449+0000 7faac4ff9640 1 -- 172.21.15.16:0/2167205090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0009510 con 0x7faac815ddd0 2023-11-13T11:02:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.449+0000 7faac4ff9640 1 -- 172.21.15.16:0/2167205090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac003e450 con 0x7faac815ddd0 2023-11-13T11:02:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.449+0000 7faac4ff9640 1 -- 172.21.15.16:0/2167205090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac003e800 con 0x7faac815ddd0 2023-11-13T11:02:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.449+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac814fe60 con 0x7faac815ddd0 2023-11-13T11:02:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.449+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac8150420 con 0x7faac815ddd0 2023-11-13T11:02:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.450+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac8067090 con 0x7faac815ddd0 2023-11-13T11:02:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.451+0000 7faac4ff9640 1 -- 172.21.15.16:0/2167205090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faac002cd50 con 0x7faac815ddd0 2023-11-13T11:02:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.452+0000 7faac4ff9640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faaa806ae40 0x7faaa806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.452+0000 7faac4ff9640 1 -- 172.21.15.16:0/2167205090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7faac00ac260 con 0x7faac815ddd0 2023-11-13T11:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.453+0000 7faac6ffd640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faaa806ae40 0x7faaa806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.454+0000 7faac6ffd640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faaa806ae40 0x7faaa806d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faab8000f60 tx=0x7faab8007c80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:35.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.456+0000 7faac4ff9640 1 -- 172.21.15.16:0/2167205090 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faac0077180 con 0x7faac815ddd0 2023-11-13T11:02:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.690+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faac80b1d30 con 0x7faac815ddd0 2023-11-13T11:02:35.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.691+0000 7faac4ff9640 1 -- 172.21.15.16:0/2167205090 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4919 (secure 0 0 0) 0x7faac007a7a0 con 0x7faac815ddd0 2023-11-13T11:02:35.695 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:35.696 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:01:41.484670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:35.696 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2023-11-13T11:02:35.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faaa806ae40 msgr2=0x7faaa806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:35.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faaa806ae40 0x7faaa806d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faab8000f60 tx=0x7faab8007c80 comp rx=0 tx=0).stop 2023-11-13T11:02:35.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815ddd0 msgr2=0x7faac813b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:35.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815ddd0 0x7faac813b370 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faac0009e50 tx=0x7faac0004a40 comp rx=0 tx=0).stop 2023-11-13T11:02:35.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 shutdown_connections 2023-11-13T11:02:35.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faaa806ae40 0x7faaa806d300 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac815ddd0 0x7faac813b370 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac815d9f0 0x7faac80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 --2- 172.21.15.16:0/2167205090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac815a610 0x7faac814f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:35.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 >> 172.21.15.16:0/2167205090 conn(0x7faac806a190 msgr2=0x7faac80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:35.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 shutdown_connections 2023-11-13T11:02:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:35.694+0000 7faacd905640 1 -- 172.21.15.16:0/2167205090 wait complete. 2023-11-13T11:02:35.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2023-11-13T11:02:35.955 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:35.956 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:02:35.956 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:35.972 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-13T11:02:35.972 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2023-11-13T11:02:35.973 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T11:02:35.973 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2023-11-13T11:02:35.974 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:02:35.974 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2023-11-13T11:02:35.974 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T11:02:36.128 INFO:teuthology.orchestra.run.smithi016.stdout:1099511630410 2023-11-13T11:02:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T11:02:36.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2023-11-13T11:02:36.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.418+0000 7f32effff640 1 Processor -- start 2023-11-13T11:02:36.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.418+0000 7f32effff640 1 -- start start 2023-11-13T11:02:36.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.420+0000 7f32effff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f32f015b8e0 0x7f32f006c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.420+0000 7f32effff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32f00ce260 0x7f32f00cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.420+0000 7f32effff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f006b860 con 0x7f32f015bcc0 2023-11-13T11:02:36.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.420+0000 7f32effff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f006bb40 con 0x7f32f00ce260 2023-11-13T11:02:36.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.420+0000 7f32effff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f006be20 con 0x7f32f015b8e0 2023-11-13T11:02:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.421+0000 7f32ee7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32f015bcc0 0x7f32f00cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36444/0 (socket says 172.21.15.16:36444) 2023-11-13T11:02:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.421+0000 7f32ee7fc640 1 -- 172.21.15.16:0/3079971619 learned_addr learned my addr 172.21.15.16:0/3079971619 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.421+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/3079971619 >> v2:172.21.15.16:3300/0 conn(0x7f32f00ce260 0x7f32f00cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.421+0000 7f32eeffd640 1 --2- 172.21.15.16:0/3079971619 >> v2:172.21.15.184:3300/0 conn(0x7f32f015b8e0 0x7f32f006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.421+0000 7f32edffb640 1 -- 172.21.15.16:0/3079971619 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920859884 0 0) 0x7f32f006b860 con 0x7f32f015bcc0 2023-11-13T11:02:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.421+0000 7f32edffb640 1 -- 172.21.15.16:0/3079971619 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d8003520 con 0x7f32f015bcc0 2023-11-13T11:02:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.421+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3079971619 >> v2:172.21.15.184:3300/0 conn(0x7f32f015b8e0 msgr2=0x7f32f006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.421+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/3079971619 >> v2:172.21.15.184:3300/0 conn(0x7f32f015b8e0 0x7f32f006c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.421+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3079971619 >> v1:172.21.15.16:6789/0 conn(0x7f32f015bcc0 legacy=0x7f32f00cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.422+0000 7f32edffb640 1 -- 172.21.15.16:0/3079971619 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 818501900 0 0) 0x7f32d8003520 con 0x7f32f015bcc0 2023-11-13T11:02:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.422+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3079971619 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f006c100 con 0x7f32f00ce260 2023-11-13T11:02:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.423+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/3079971619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 0x7f32f00cbae0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f32f0071d70 tx=0x7f32e802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8637c917583e502 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.423+0000 7f32edffb640 1 -- 172.21.15.16:0/3079971619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4001670 con 0x7f32f00ce260 2023-11-13T11:02:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.423+0000 7f32edffb640 1 -- 172.21.15.16:0/3079971619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e802c8c0 con 0x7f32f00ce260 2023-11-13T11:02:36.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.423+0000 7f32edffb640 1 -- 172.21.15.16:0/3079971619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e802cc50 con 0x7f32f00ce260 2023-11-13T11:02:36.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.424+0000 7f32effff640 1 -- 172.21.15.16:0/3079971619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 msgr2=0x7f32f00cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.424+0000 7f32effff640 1 --2- 172.21.15.16:0/3079971619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 0x7f32f00cbae0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f32f0071d70 tx=0x7f32e802bd90 comp rx=0 tx=0).stop 2023-11-13T11:02:36.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.424+0000 7f32effff640 1 -- 172.21.15.16:0/3079971619 shutdown_connections 2023-11-13T11:02:36.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.424+0000 7f32effff640 1 --2- 172.21.15.16:0/3079971619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 0x7f32f00cbae0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.424+0000 7f32effff640 1 --2- 172.21.15.16:0/3079971619 >> v2:172.21.15.184:3300/0 conn(0x7f32f015b8e0 0x7f32f006c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.424+0000 7f32effff640 1 -- 172.21.15.16:0/3079971619 >> 172.21.15.16:0/3079971619 conn(0x7f32f00b0970 msgr2=0x7f32f00b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.424+0000 7f32effff640 1 -- 172.21.15.16:0/3079971619 shutdown_connections 2023-11-13T11:02:36.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.424+0000 7f32effff640 1 -- 172.21.15.16:0/3079971619 wait complete. 2023-11-13T11:02:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.426+0000 7f32effff640 1 Processor -- start 2023-11-13T11:02:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.426+0000 7f32effff640 1 -- start start 2023-11-13T11:02:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.427+0000 7f32effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 0x7f32f0148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.427+0000 7f32effff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 0x7f32f0146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.427+0000 7f32effff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32f01f05c0 0x7f32f01f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.427+0000 7f32effff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0146fa0 con 0x7f32f00ce260 2023-11-13T11:02:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.427+0000 7f32effff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0147280 con 0x7f32f015b8e0 2023-11-13T11:02:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.427+0000 7f32effff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0147560 con 0x7f32f01f05c0 2023-11-13T11:02:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.427+0000 7f32eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 0x7f32f0148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.427+0000 7f32eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 0x7f32f0148040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47808/0 (socket says 172.21.15.16:47808) 2023-11-13T11:02:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.427+0000 7f32eeffd640 1 -- 172.21.15.16:0/508558216 learned_addr learned my addr 172.21.15.16:0/508558216 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 0x7f32f0146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32eeffd640 1 -- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32f01f05c0 msgr2=0x7f32f01f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32eeffd640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32f01f05c0 0x7f32f01f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32eeffd640 1 -- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 msgr2=0x7f32f0146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32eeffd640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 0x7f32f0146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32eeffd640 1 -- 172.21.15.16:0/508558216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8009a70 con 0x7f32f00ce260 2023-11-13T11:02:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32eeffd640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 0x7f32f0148040 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9b0 tx=0x7f32e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32d77fe640 1 -- 172.21.15.16:0/508558216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0009670 con 0x7f32f00ce260 2023-11-13T11:02:36.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32d77fe640 1 -- 172.21.15.16:0/508558216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e000c9a0 con 0x7f32f00ce260 2023-11-13T11:02:36.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32d77fe640 1 -- 172.21.15.16:0/508558216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e000cd50 con 0x7f32f00ce260 2023-11-13T11:02:36.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32effff640 1 -- 172.21.15.16:0/508558216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f0147840 con 0x7f32f00ce260 2023-11-13T11:02:36.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.428+0000 7f32effff640 1 -- 172.21.15.16:0/508558216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f01462c0 con 0x7f32f00ce260 2023-11-13T11:02:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.431+0000 7f32d77fe640 1 -- 172.21.15.16:0/508558216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32e0013470 con 0x7f32f00ce260 2023-11-13T11:02:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.432+0000 7f32d77fe640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32dc06ae40 0x7f32dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.433+0000 7f32d77fe640 1 -- 172.21.15.16:0/508558216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f32e0089ae0 con 0x7f32f00ce260 2023-11-13T11:02:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.433+0000 7f32f8374740 1 Processor -- start 2023-11-13T11:02:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.433+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32dc06ae40 0x7f32dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.433+0000 7f32f8374740 1 -- start start 2023-11-13T11:02:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.434+0000 7f32f8374740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3d200 0x5619b5a1f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.434+0000 7f32f8374740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3ad20 0x5619b5a1f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.434+0000 7f32f8374740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a1c910 0x5619b59f3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.434+0000 7f32f8374740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b5962d20 con 0x5619b5a3ad20 2023-11-13T11:02:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.434+0000 7f32f8374740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b5960410 con 0x5619b5a3d200 2023-11-13T11:02:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.434+0000 7f32f8374740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619b595bd00 con 0x5619b5a1c910 2023-11-13T11:02:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32dc06ae40 0x7f32dc06d300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f32e4001380 tx=0x7f32e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3ad20 0x5619b5a1f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3ad20 0x5619b5a1f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47818/0 (socket says 172.21.15.16:47818) 2023-11-13T11:02:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ee7fc640 1 -- 172.21.15.16:0/2145188158 learned_addr learned my addr 172.21.15.16:0/2145188158 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32eeffd640 1 --2- 172.21.15.16:0/2145188158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3d200 0x5619b5a1f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/2145188158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a1c910 0x5619b59f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ee7fc640 1 -- 172.21.15.16:0/2145188158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a1c910 msgr2=0x5619b59f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/2145188158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a1c910 0x5619b59f3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ee7fc640 1 -- 172.21.15.16:0/2145188158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3d200 msgr2=0x5619b5a1f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/2145188158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3d200 0x5619b5a1f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.435+0000 7f32ee7fc640 1 -- 172.21.15.16:0/2145188158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619b59154f0 con 0x5619b5a3ad20 2023-11-13T11:02:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.436+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/2145188158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3ad20 0x5619b5a1f820 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005b60 tx=0x7f32e4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ce6d2e3b8b658 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.436+0000 7f32d4ff9640 1 -- 172.21.15.16:0/2145188158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e400dbb0 con 0x5619b5a3ad20 2023-11-13T11:02:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.436+0000 7f32d4ff9640 1 -- 172.21.15.16:0/2145188158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e400dd10 con 0x5619b5a3ad20 2023-11-13T11:02:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.436+0000 7f32d4ff9640 1 -- 172.21.15.16:0/2145188158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4020650 con 0x5619b5a3ad20 2023-11-13T11:02:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.436+0000 7f32f8374740 1 -- 172.21.15.16:0/2145188158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3ad20 msgr2=0x5619b5a1f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.436+0000 7f32f8374740 1 --2- 172.21.15.16:0/2145188158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3ad20 0x5619b5a1f820 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005b60 tx=0x7f32e4010000 comp rx=0 tx=0).stop 2023-11-13T11:02:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.437+0000 7f32f8374740 1 -- 172.21.15.16:0/2145188158 shutdown_connections 2023-11-13T11:02:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.437+0000 7f32f8374740 1 --2- 172.21.15.16:0/2145188158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a1c910 0x5619b59f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.437+0000 7f32f8374740 1 --2- 172.21.15.16:0/2145188158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3ad20 0x5619b5a1f820 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.437+0000 7f32f8374740 1 --2- 172.21.15.16:0/2145188158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3d200 0x5619b5a1f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.437+0000 7f32f8374740 1 -- 172.21.15.16:0/2145188158 >> 172.21.15.16:0/2145188158 conn(0x5619b59f1760 msgr2=0x5619b5a3a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.437+0000 7f32f8374740 1 -- 172.21.15.16:0/2145188158 shutdown_connections 2023-11-13T11:02:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.437+0000 7f32f8374740 1 -- 172.21.15.16:0/2145188158 wait complete. 2023-11-13T11:02:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.437+0000 7f32f8374740 1 -- start start 2023-11-13T11:02:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.438+0000 7f32f8374740 1 Processor -- start 2023-11-13T11:02:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32f8374740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3ad20 0x5619b5a206d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32f8374740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a06770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32f8374740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 0x5619b5a1b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32f8374740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b5915fa0 con 0x5619b5a3d200 2023-11-13T11:02:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32f8374740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b5915c40 con 0x5619b5a1b500 2023-11-13T11:02:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32f8374740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619b5654f90 con 0x5619b5a3ad20 2023-11-13T11:02:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32f8374740 10 client.0 _ll_register_callbacks cb 0x5619b591a370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a06770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a06770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47832/0 (socket says 172.21.15.16:47832) 2023-11-13T11:02:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32ee7fc640 1 -- 172.21.15.16:0/59579535 learned_addr learned my addr 172.21.15.16:0/59579535 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 0x5619b5a1b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.439+0000 7f32eeffd640 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3ad20 0x5619b5a206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32ee7fc640 1 -- 172.21.15.16:0/59579535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3ad20 msgr2=0x5619b5a206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3ad20 0x5619b5a206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32ee7fc640 1 -- 172.21.15.16:0/59579535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 msgr2=0x5619b5a1b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 0x5619b5a1b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32ee7fc640 1 -- 172.21.15.16:0/59579535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e40058d0 con 0x5619b5a3d200 2023-11-13T11:02:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a06770 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f32e40074a0 tx=0x7f32e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea038f67f76d6e3d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32cdffb640 1 -- 172.21.15.16:0/59579535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4027070 con 0x5619b5a3d200 2023-11-13T11:02:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32f8374740 1 -- 172.21.15.16:0/59579535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619b5954ce0 con 0x5619b5a3d200 2023-11-13T11:02:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.440+0000 7f32cdffb640 1 -- 172.21.15.16:0/59579535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f32e4022280 con 0x5619b5a3d200 2023-11-13T11:02:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32cdffb640 1 -- 172.21.15.16:0/59579535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32e40227a0 con 0x5619b5a3d200 2023-11-13T11:02:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32f8374740 10 client.6324 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32f8374740 1 -- 172.21.15.16:0/59579535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5619b595a380 con 0x5619b5a3d200 2023-11-13T11:02:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32cdffb640 1 -- 172.21.15.16:0/59579535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619b5954ce0 con 0x5619b5a3d200 2023-11-13T11:02:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32cdffb640 1 -- 172.21.15.16:0/59579535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32e4022900 con 0x5619b5a3d200 2023-11-13T11:02:36.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32f8374740 10 client.6324 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32f8374740 10 client.6324 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32f8374740 20 client.6324 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32f8374740 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5619b5a45ec0 0x5619b5a462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32f8374740 4 client.6324 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:36.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32f8374740 1 -- 172.21.15.16:0/59579535 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619b5954a00 con 0x5619b5a45ec0 2023-11-13T11:02:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.441+0000 7f32eeffd640 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5619b5a45ec0 0x5619b5a462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.442+0000 7f32eeffd640 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5619b5a45ec0 0x5619b5a462d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33e1d68f90142c7f server_cookie=c837c541ea1328f1 in_seq=0 out_seq=0 2023-11-13T11:02:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.442+0000 7f32cdffb640 10 client.6324 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32cdffb640 1 -- 172.21.15.16:0/59579535 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619b5954a00 con 0x5619b5a45ec0 2023-11-13T11:02:36.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32cdffb640 10 client.6324 handle_command_reply: tid=0 2023-11-13T11:02:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 client.6324 shutdown 2023-11-13T11:02:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 -- 172.21.15.16:0/59579535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 msgr2=0x5619b5a06770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a06770 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f32e40074a0 tx=0x7f32e4012000 comp rx=0 tx=0).stop 2023-11-13T11:02:36.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 -- 172.21.15.16:0/59579535 shutdown_connections 2023-11-13T11:02:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 0x5619b5a1b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5619b5a45ec0 0x5619b5a462d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a06770 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 --2- 172.21.15.16:0/59579535 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3ad20 0x5619b5a206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32cdffb640 0 client.6324 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 -- 172.21.15.16:0/59579535 >> 172.21.15.16:0/59579535 conn(0x5619b5a1c910 msgr2=0x5619b59eec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 -- 172.21.15.16:0/59579535 shutdown_connections 2023-11-13T11:02:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 1 -- 172.21.15.16:0/59579535 wait complete. 2023-11-13T11:02:36.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.443+0000 7f32f8374740 20 client.6324 trim_cache size 0 max 16384 2023-11-13T11:02:36.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.456+0000 7f32f8374740 1 Processor -- start 2023-11-13T11:02:36.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.456+0000 7f32f8374740 1 -- start start 2023-11-13T11:02:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.457+0000 7f32f8374740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a1b500 0x5619b5a16410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.457+0000 7f32f8374740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3ad20 0x5619b5a16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.457+0000 7f32f8374740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3d200 0x5619b59ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.457+0000 7f32f8374740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b595aea0 con 0x5619b5a1b500 2023-11-13T11:02:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.457+0000 7f32f8374740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b5a1f300 con 0x5619b5a3ad20 2023-11-13T11:02:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.457+0000 7f32f8374740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619b5a1b0b0 con 0x5619b5a3d200 2023-11-13T11:02:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a1b500 0x5619b5a16410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a1b500 0x5619b5a16410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47838/0 (socket says 172.21.15.16:47838) 2023-11-13T11:02:36.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32eeffd640 1 -- 172.21.15.16:0/4288226565 learned_addr learned my addr 172.21.15.16:0/4288226565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3ad20 0x5619b5a16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3d200 0x5619b59ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32eeffd640 1 -- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3d200 msgr2=0x5619b59ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32eeffd640 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3d200 0x5619b59ee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32eeffd640 1 -- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3ad20 msgr2=0x5619b5a16950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32eeffd640 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3ad20 0x5619b5a16950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32eeffd640 1 -- 172.21.15.16:0/4288226565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619b58f7cf0 con 0x5619b5a1b500 2023-11-13T11:02:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32ee7fc640 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3ad20 0x5619b5a16950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.458+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3d200 0x5619b59ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32eeffd640 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a1b500 0x5619b5a16410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f32e0087d90 tx=0x7f32e0087cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e5dc13dc119f10 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32d57fa640 1 -- 172.21.15.16:0/4288226565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0087f00 con 0x5619b5a1b500 2023-11-13T11:02:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32d57fa640 1 -- 172.21.15.16:0/4288226565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e00569d0 con 0x5619b5a1b500 2023-11-13T11:02:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32d57fa640 1 -- 172.21.15.16:0/4288226565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e00604a0 con 0x5619b5a1b500 2023-11-13T11:02:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32f8374740 1 -- 172.21.15.16:0/4288226565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a1b500 msgr2=0x5619b5a16410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32f8374740 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a1b500 0x5619b5a16410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f32e0087d90 tx=0x7f32e0087cc0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32f8374740 1 -- 172.21.15.16:0/4288226565 shutdown_connections 2023-11-13T11:02:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32f8374740 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3d200 0x5619b59ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32f8374740 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a3ad20 0x5619b5a16950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32f8374740 1 --2- 172.21.15.16:0/4288226565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a1b500 0x5619b5a16410 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.459+0000 7f32f8374740 1 -- 172.21.15.16:0/4288226565 >> 172.21.15.16:0/4288226565 conn(0x5619b5a1c910 msgr2=0x5619b5a20550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.460+0000 7f32f8374740 1 -- 172.21.15.16:0/4288226565 shutdown_connections 2023-11-13T11:02:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.460+0000 7f32f8374740 1 -- 172.21.15.16:0/4288226565 wait complete. 2023-11-13T11:02:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.460+0000 7f32f8374740 1 -- start start 2023-11-13T11:02:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.460+0000 7f32f8374740 1 Processor -- start 2023-11-13T11:02:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.461+0000 7f32f8374740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 0x5619b5aa8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32f8374740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3ad20 0x5619b5a16410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32f8374740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32f8374740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b5a3a990 con 0x5619b5a3d200 2023-11-13T11:02:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32f8374740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b59eecf0 con 0x5619b5a1b500 2023-11-13T11:02:36.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32f8374740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619b5a3a500 con 0x5619b5a3ad20 2023-11-13T11:02:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32f8374740 10 client.0 _ll_register_callbacks cb 0x7f32e400df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a16950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47846/0 (socket says 172.21.15.16:47846) 2023-11-13T11:02:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32ef7fe640 1 -- 172.21.15.16:0/2057387619 learned_addr learned my addr 172.21.15.16:0/2057387619 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32eeffd640 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 0x5619b5aa8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32ef7fe640 1 -- 172.21.15.16:0/2057387619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3ad20 msgr2=0x5619b5a16410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3ad20 0x5619b5a16410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32ef7fe640 1 -- 172.21.15.16:0/2057387619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 msgr2=0x5619b5aa8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 0x5619b5aa8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.462+0000 7f32ef7fe640 1 -- 172.21.15.16:0/2057387619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f01462c0 con 0x5619b5a3d200 2023-11-13T11:02:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.463+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a16950 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f32e802c3a0 tx=0x7f32e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b9d859ab6ef0fb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.463+0000 7f32cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.463+0000 7f32cdffb640 1 -- 172.21.15.16:0/2057387619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e803fb90 con 0x5619b5a3d200 2023-11-13T11:02:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.463+0000 7f32cdffb640 1 -- 172.21.15.16:0/2057387619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f32e802ce10 con 0x5619b5a3d200 2023-11-13T11:02:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.463+0000 7f32f8374740 1 -- 172.21.15.16:0/2057387619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32e400dd10 con 0x5619b5a3d200 2023-11-13T11:02:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.463+0000 7f32cdffb640 1 -- 172.21.15.16:0/2057387619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32e803fcf0 con 0x5619b5a3d200 2023-11-13T11:02:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.463+0000 7f32f8374740 10 client.6330 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.463+0000 7f32f8374740 1 -- 172.21.15.16:0/2057387619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5619b591d2e0 con 0x5619b5a3d200 2023-11-13T11:02:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.464+0000 7f32cdffb640 1 -- 172.21.15.16:0/2057387619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e400dd10 con 0x5619b5a3d200 2023-11-13T11:02:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.464+0000 7f32cdffb640 1 -- 172.21.15.16:0/2057387619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32e8003990 con 0x5619b5a3d200 2023-11-13T11:02:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.464+0000 7f32f8374740 10 client.6330 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.464+0000 7f32f8374740 10 client.6330 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.464+0000 7f32f8374740 20 client.6330 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.464+0000 7f32f8374740 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5619b5a3c690 0x5619b5a04d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.464+0000 7f32f8374740 4 client.6330 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-11-13T11:02:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.464+0000 7f32f8374740 1 -- 172.21.15.16:0/2057387619 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5619b565c7e0 con 0x5619b5a3c690 2023-11-13T11:02:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.464+0000 7f32eeffd640 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5619b5a3c690 0x5619b5a04d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32eeffd640 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5619b5a3c690 0x5619b5a04d40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23e77d5f56128296 server_cookie=6cdc0f3c936a14e5 in_seq=0 out_seq=0 2023-11-13T11:02:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32cdffb640 10 client.6330 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32cdffb640 1 -- 172.21.15.16:0/2057387619 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5619b565c7e0 con 0x5619b5a3c690 2023-11-13T11:02:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32cdffb640 10 client.6330 handle_command_reply: tid=0 2023-11-13T11:02:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 client.6330 shutdown 2023-11-13T11:02:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 -- 172.21.15.16:0/2057387619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 msgr2=0x5619b5a16950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a16950 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f32e802c3a0 tx=0x7f32e8004a40 comp rx=0 tx=0).stop 2023-11-13T11:02:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 -- 172.21.15.16:0/2057387619 shutdown_connections 2023-11-13T11:02:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5619b5a3c690 0x5619b5a04d40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619b5a3d200 0x5619b5a16950 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5619b5a3ad20 0x5619b5a16410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 --2- 172.21.15.16:0/2057387619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5619b5a1b500 0x5619b5aa8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 -- 172.21.15.16:0/2057387619 >> 172.21.15.16:0/2057387619 conn(0x5619b5a1c910 msgr2=0x5619b59efcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32cdffb640 0 client.6330 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:36.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 -- 172.21.15.16:0/2057387619 shutdown_connections 2023-11-13T11:02:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.465+0000 7f32f8374740 1 -- 172.21.15.16:0/2057387619 wait complete. 2023-11-13T11:02:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32f8374740 20 client.6330 trim_cache size 0 max 16384 2023-11-13T11:02:36.487 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2023-11-13T11:02:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 -- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32dc06ae40 msgr2=0x7f32dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32dc06ae40 0x7f32dc06d300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f32e4001380 tx=0x7f32e4009040 comp rx=0 tx=0).stop 2023-11-13T11:02:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 -- 172.21.15.16:0/508558216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 msgr2=0x7f32f0148040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 0x7f32f0148040 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9b0 tx=0x7f32e000ee70 comp rx=0 tx=0).stop 2023-11-13T11:02:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 -- 172.21.15.16:0/508558216 shutdown_connections 2023-11-13T11:02:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f32dc06ae40 0x7f32dc06d300 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32f01f05c0 0x7f32f01f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32f015b8e0 0x7f32f0146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 --2- 172.21.15.16:0/508558216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f00ce260 0x7f32f0148040 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 -- 172.21.15.16:0/508558216 >> 172.21.15.16:0/508558216 conn(0x7f32f00b0970 msgr2=0x7f32f015e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 -- 172.21.15.16:0/508558216 shutdown_connections 2023-11-13T11:02:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.466+0000 7f32effff640 1 -- 172.21.15.16:0/508558216 wait complete. 2023-11-13T11:02:36.492 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4a.00000000 parent 2023-11-13T11:02:36.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.593+0000 7f731944cf40 1 Processor -- start 2023-11-13T11:02:36.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.593+0000 7f731944cf40 1 -- start start 2023-11-13T11:02:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f731944cf40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a45780360 0x555a45780740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f731944cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a457893e0 0x555a456e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f731944cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555a4578ef00 0x555a456e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f731944cf40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45669da0 con 0x555a45780360 2023-11-13T11:02:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f731944cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45665690 con 0x555a457893e0 2023-11-13T11:02:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f731944cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a4561cbc0 con 0x555a4578ef00 2023-11-13T11:02:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f73166b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a45780360 0x555a45780740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f73166b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a45780360 0x555a45780740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47858/0 (socket says 172.21.15.16:47858) 2023-11-13T11:02:36.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f73166b5640 1 -- 172.21.15.16:0/1033482330 learned_addr learned my addr 172.21.15.16:0/1033482330 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f73166b5640 1 -- 172.21.15.16:0/1033482330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555a4578ef00 msgr2=0x555a456e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f73166b5640 1 --2- 172.21.15.16:0/1033482330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555a4578ef00 0x555a456e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f73166b5640 1 -- 172.21.15.16:0/1033482330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a457893e0 msgr2=0x555a456e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f73166b5640 1 --2- 172.21.15.16:0/1033482330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a457893e0 0x555a456e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.594+0000 7f73166b5640 1 -- 172.21.15.16:0/1033482330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a455fe830 con 0x555a45780360 2023-11-13T11:02:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.595+0000 7f73166b5640 1 --2- 172.21.15.16:0/1033482330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a45780360 0x555a45780740 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7308009800 tx=0x7f7308005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cbdb55682a8040 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.595+0000 7f7314eb2640 1 -- 172.21.15.16:0/1033482330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730800f050 con 0x555a45780360 2023-11-13T11:02:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.595+0000 7f7314eb2640 1 -- 172.21.15.16:0/1033482330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7308010040 con 0x555a45780360 2023-11-13T11:02:36.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.595+0000 7f7314eb2640 1 -- 172.21.15.16:0/1033482330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308008af0 con 0x555a45780360 2023-11-13T11:02:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.596+0000 7f731944cf40 1 -- 172.21.15.16:0/1033482330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a45780360 msgr2=0x555a45780740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.596+0000 7f731944cf40 1 --2- 172.21.15.16:0/1033482330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a45780360 0x555a45780740 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7308009800 tx=0x7f7308005a00 comp rx=0 tx=0).stop 2023-11-13T11:02:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.596+0000 7f731944cf40 1 -- 172.21.15.16:0/1033482330 shutdown_connections 2023-11-13T11:02:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.596+0000 7f731944cf40 1 --2- 172.21.15.16:0/1033482330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555a4578ef00 0x555a456e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.596+0000 7f731944cf40 1 --2- 172.21.15.16:0/1033482330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a457893e0 0x555a456e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.596+0000 7f731944cf40 1 --2- 172.21.15.16:0/1033482330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a45780360 0x555a45780740 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.596+0000 7f731944cf40 1 -- 172.21.15.16:0/1033482330 >> 172.21.15.16:0/1033482330 conn(0x555a457044c0 msgr2=0x555a456b5750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.596+0000 7f731944cf40 1 -- 172.21.15.16:0/1033482330 shutdown_connections 2023-11-13T11:02:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.596+0000 7f731944cf40 1 -- 172.21.15.16:0/1033482330 wait complete. 2023-11-13T11:02:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.597+0000 7f731944cf40 1 Processor -- start 2023-11-13T11:02:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.597+0000 7f731944cf40 1 -- start start 2023-11-13T11:02:36.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f731944cf40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a45780360 0x555a456b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f731944cf40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555a457893e0 0x555a45775c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f731944cf40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4578ef00 0x555a45779b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f731944cf40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a455ff9c0 con 0x555a4578ef00 2023-11-13T11:02:36.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f731944cf40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a455f3520 con 0x555a45780360 2023-11-13T11:02:36.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f731944cf40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a45776c40 con 0x555a457893e0 2023-11-13T11:02:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f7316eb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4578ef00 0x555a45779b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f7316eb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4578ef00 0x555a45779b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47864/0 (socket says 172.21.15.16:47864) 2023-11-13T11:02:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f7316eb6640 1 -- 172.21.15.16:0/2929273231 learned_addr learned my addr 172.21.15.16:0/2929273231 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f7315eb4640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555a457893e0 0x555a45775c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f73166b5640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a45780360 0x555a456b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.598+0000 7f7316eb6640 1 -- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555a457893e0 msgr2=0x555a45775c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f7316eb6640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555a457893e0 0x555a45775c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f7316eb6640 1 -- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a45780360 msgr2=0x555a456b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f7316eb6640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555a45780360 0x555a456b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f7316eb6640 1 -- 172.21.15.16:0/2929273231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308009430 con 0x555a4578ef00 2023-11-13T11:02:36.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f7315eb4640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x555a457893e0 0x555a45775c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:36.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f7316eb6640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4578ef00 0x555a45779b40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f73100012e0 tx=0x7f731000d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f73077fe640 1 -- 172.21.15.16:0/2929273231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731000dec0 con 0x555a4578ef00 2023-11-13T11:02:36.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f73077fe640 1 -- 172.21.15.16:0/2929273231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7310013480 con 0x555a4578ef00 2023-11-13T11:02:36.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555a455fe290 con 0x555a4578ef00 2023-11-13T11:02:36.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f73077fe640 1 -- 172.21.15.16:0/2929273231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310013830 con 0x555a4578ef00 2023-11-13T11:02:36.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.599+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a45779410 con 0x555a4578ef00 2023-11-13T11:02:36.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.601+0000 7f73077fe640 1 -- 172.21.15.16:0/2929273231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f731001c3b0 con 0x555a4578ef00 2023-11-13T11:02:36.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.602+0000 7f73077fe640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f72f8069820 0x7f72f806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.602+0000 7f73077fe640 1 -- 172.21.15.16:0/2929273231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7310087a40 con 0x555a4578ef00 2023-11-13T11:02:36.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.603+0000 7f731944cf40 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x555a45776190 0x555a4577f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.603+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 --> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] -- osd_op(unknown.0.0:1 8.24 8:26e73f49:::10000000a4a.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57) v8 -- 0x555a4577fb60 con 0x555a45776190 2023-11-13T11:02:36.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.603+0000 7f73166b5640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f72f8069820 0x7f72f806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.603+0000 7f7315eb4640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x555a45776190 0x555a4577f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.604+0000 7f7315eb4640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x555a45776190 0x555a4577f620 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.604+0000 7f73166b5640 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f72f8069820 0x7f72f806bce0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7308017040 tx=0x7f7308014000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f7315eb4640 1 -- 172.21.15.16:0/2929273231 <== osd.2 v2:172.21.15.16:6808/3899450571 1 ==== osd_op_reply(1 10000000a4a.00000000 [rmxattr] v57'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f730c04ad70 con 0x555a45776190 2023-11-13T11:02:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x555a45776190 msgr2=0x555a4577f620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f731944cf40 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x555a45776190 0x555a4577f620 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f72f8069820 msgr2=0x7f72f806bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f731944cf40 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f72f8069820 0x7f72f806bce0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7308017040 tx=0x7f7308014000 comp rx=0 tx=0).stop 2023-11-13T11:02:36.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4578ef00 msgr2=0x555a45779b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f731944cf40 1 --2- 172.21.15.16:0/2929273231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4578ef00 0x555a45779b40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f73100012e0 tx=0x7f731000d530 comp rx=0 tx=0).stop 2023-11-13T11:02:36.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f7316eb6640 1 -- 172.21.15.16:0/2929273231 reap_dead start 2023-11-13T11:02:36.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 shutdown_connections 2023-11-13T11:02:36.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.606+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 >> 172.21.15.16:0/2929273231 conn(0x555a457044c0 msgr2=0x555a45780740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.607+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 shutdown_connections 2023-11-13T11:02:36.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.607+0000 7f731944cf40 1 -- 172.21.15.16:0/2929273231 wait complete. 2023-11-13T11:02:36.619 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2023-11-13T11:02:36.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2023-11-13T11:02:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.817+0000 7fad8be61640 1 Processor -- start 2023-11-13T11:02:36.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.817+0000 7fad8be61640 1 -- start start 2023-11-13T11:02:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.819+0000 7fad8be61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad840b30b0 0x7fad840d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.819+0000 7fad8be61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad840ceca0 0x7fad840d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.819+0000 7fad8be61640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad840b41b0 con 0x7fad840b3490 2023-11-13T11:02:36.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.819+0000 7fad8be61640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad840d16d0 con 0x7fad840b30b0 2023-11-13T11:02:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.819+0000 7fad8be61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840d19b0 con 0x7fad840ceca0 2023-11-13T11:02:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.819+0000 7fad89bfd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad840b30b0 0x7fad840d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.819+0000 7fad89bfd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad840b30b0 0x7fad840d3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47872/0 (socket says 172.21.15.16:47872) 2023-11-13T11:02:36.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.819+0000 7fad89bfd640 1 -- 172.21.15.16:0/2828033527 learned_addr learned my addr 172.21.15.16:0/2828033527 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.819+0000 7fad893fc640 1 --2- 172.21.15.16:0/2828033527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad840ceca0 0x7fad840d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.820+0000 7fad89bfd640 1 -- 172.21.15.16:0/2828033527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad840ceca0 msgr2=0x7fad840d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.820+0000 7fad89bfd640 1 --2- 172.21.15.16:0/2828033527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad840ceca0 0x7fad840d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.820+0000 7fad89bfd640 1 -- 172.21.15.16:0/2828033527 >> v1:172.21.15.16:6789/0 conn(0x7fad840b3490 legacy=0x7fad840cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.820+0000 7fad88bfb640 1 -- 172.21.15.16:0/2828033527 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880959917 0 0) 0x7fad840b41b0 con 0x7fad840b3490 2023-11-13T11:02:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.820+0000 7fad89bfd640 1 -- 172.21.15.16:0/2828033527 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840d1c90 con 0x7fad840b30b0 2023-11-13T11:02:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.821+0000 7fad89bfd640 1 --2- 172.21.15.16:0/2828033527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 0x7fad840d3810 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fad80009870 tx=0x7fad8002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377f2b571df0c85f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.821+0000 7fad88bfb640 1 -- 172.21.15.16:0/2828033527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80030840 con 0x7fad840b30b0 2023-11-13T11:02:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.821+0000 7fad88bfb640 1 -- 172.21.15.16:0/2828033527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad800309a0 con 0x7fad840b30b0 2023-11-13T11:02:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.821+0000 7fad88bfb640 1 -- 172.21.15.16:0/2828033527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80030d30 con 0x7fad840b30b0 2023-11-13T11:02:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.822+0000 7fad8be61640 1 -- 172.21.15.16:0/2828033527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 msgr2=0x7fad840d3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.822+0000 7fad8be61640 1 --2- 172.21.15.16:0/2828033527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 0x7fad840d3810 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fad80009870 tx=0x7fad8002b980 comp rx=0 tx=0).stop 2023-11-13T11:02:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.822+0000 7fad8be61640 1 -- 172.21.15.16:0/2828033527 shutdown_connections 2023-11-13T11:02:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.822+0000 7fad8be61640 1 --2- 172.21.15.16:0/2828033527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad840ceca0 0x7fad840d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.822+0000 7fad8be61640 1 --2- 172.21.15.16:0/2828033527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 0x7fad840d3810 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.822+0000 7fad8be61640 1 -- 172.21.15.16:0/2828033527 >> 172.21.15.16:0/2828033527 conn(0x7fad840d2d20 msgr2=0x7fad840d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.822+0000 7fad8be61640 1 -- 172.21.15.16:0/2828033527 shutdown_connections 2023-11-13T11:02:36.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.822+0000 7fad8be61640 1 -- 172.21.15.16:0/2828033527 wait complete. 2023-11-13T11:02:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.824+0000 7fad8be61640 1 Processor -- start 2023-11-13T11:02:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.824+0000 7fad8be61640 1 -- start start 2023-11-13T11:02:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad8be61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 0x7fad8406b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad8be61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad840b3490 0x7fad841f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad8be61640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad840ceca0 0x7fad8415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad8be61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8406b9d0 con 0x7fad840b30b0 2023-11-13T11:02:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad8be61640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad841f60e0 con 0x7fad840ceca0 2023-11-13T11:02:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad8be61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad841f6ab0 con 0x7fad840b3490 2023-11-13T11:02:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad89bfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 0x7fad8406b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad89bfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 0x7fad8406b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47880/0 (socket says 172.21.15.16:47880) 2023-11-13T11:02:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad89bfd640 1 -- 172.21.15.16:0/3798290180 learned_addr learned my addr 172.21.15.16:0/3798290180 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad89bfd640 1 -- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad840b3490 msgr2=0x7fad841f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad89bfd640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad840b3490 0x7fad841f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad89bfd640 1 -- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad840ceca0 msgr2=0x7fad8415d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad89bfd640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad840ceca0 0x7fad8415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.825+0000 7fad89bfd640 1 -- 172.21.15.16:0/3798290180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad800094d0 con 0x7fad840b30b0 2023-11-13T11:02:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.826+0000 7fad89bfd640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 0x7fad8406b490 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fad80000c00 tx=0x7fad800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.826+0000 7fad767fc640 1 -- 172.21.15.16:0/3798290180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8003e070 con 0x7fad840b30b0 2023-11-13T11:02:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.826+0000 7fad767fc640 1 -- 172.21.15.16:0/3798290180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad80004030 con 0x7fad840b30b0 2023-11-13T11:02:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.826+0000 7fad8be61640 1 -- 172.21.15.16:0/3798290180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad840a01f0 con 0x7fad840b30b0 2023-11-13T11:02:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.826+0000 7fad767fc640 1 -- 172.21.15.16:0/3798290180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad800043a0 con 0x7fad840b30b0 2023-11-13T11:02:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.826+0000 7fad8be61640 1 -- 172.21.15.16:0/3798290180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad840a07b0 con 0x7fad840b30b0 2023-11-13T11:02:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.829+0000 7fad767fc640 1 -- 172.21.15.16:0/3798290180 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad80042070 con 0x7fad840b30b0 2023-11-13T11:02:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.830+0000 7fad767fc640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad6806ae40 0x7fad6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.830+0000 7fad767fc640 1 -- 172.21.15.16:0/3798290180 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fad800ab6c0 con 0x7fad840b30b0 2023-11-13T11:02:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.830+0000 7fad893fc640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad6806ae40 0x7fad6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.830+0000 7fad8e9fc740 1 Processor -- start 2023-11-13T11:02:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.831+0000 7fad8e9fc740 1 -- start start 2023-11-13T11:02:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.831+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf2ed70 0x5624daf2f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.831+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daefed90 0x5624daf2bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.831+0000 7fad893fc640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad6806ae40 0x7fad6806d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad8409ffb0 tx=0x7fad78008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf028f0 0x5624daf2c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624daebfd80 con 0x5624daf028f0 2023-11-13T11:02:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624dae72180 con 0x5624daf2ed70 2023-11-13T11:02:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624dae6f870 con 0x5624daefed90 2023-11-13T11:02:36.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8a3fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf028f0 0x5624daf2c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8a3fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf028f0 0x5624daf2c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47890/0 (socket says 172.21.15.16:47890) 2023-11-13T11:02:36.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8a3fe640 1 -- 172.21.15.16:0/532190901 learned_addr learned my addr 172.21.15.16:0/532190901 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad89bfd640 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf2ed70 0x5624daf2f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad893fc640 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daefed90 0x5624daf2bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8a3fe640 1 -- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daefed90 msgr2=0x5624daf2bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8a3fe640 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daefed90 0x5624daf2bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8a3fe640 1 -- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf2ed70 msgr2=0x5624daf2f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8a3fe640 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf2ed70 0x5624daf2f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad8a3fe640 1 -- 172.21.15.16:0/532190901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624dae2dab0 con 0x5624daf028f0 2023-11-13T11:02:36.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad89bfd640 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf2ed70 0x5624daf2f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:36.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.832+0000 7fad893fc640 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daefed90 0x5624daf2bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:36.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.833+0000 7fad8a3fe640 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf028f0 0x5624daf2c520 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fad7c001ce0 tx=0x7fad7c0021a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f750e3d7e1ff832 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.833+0000 7fad677fe640 1 -- 172.21.15.16:0/532190901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c003da0 con 0x5624daf028f0 2023-11-13T11:02:36.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.833+0000 7fad677fe640 1 -- 172.21.15.16:0/532190901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c00c8a0 con 0x5624daf028f0 2023-11-13T11:02:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.833+0000 7fad677fe640 1 -- 172.21.15.16:0/532190901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00cc30 con 0x5624daf028f0 2023-11-13T11:02:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.834+0000 7fad8e9fc740 1 -- 172.21.15.16:0/532190901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf028f0 msgr2=0x5624daf2c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.834+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf028f0 0x5624daf2c520 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fad7c001ce0 tx=0x7fad7c0021a0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.834+0000 7fad8e9fc740 1 -- 172.21.15.16:0/532190901 shutdown_connections 2023-11-13T11:02:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.834+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf028f0 0x5624daf2c520 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.834+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daefed90 0x5624daf2bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.834+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/532190901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf2ed70 0x5624daf2f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.834+0000 7fad8e9fc740 1 -- 172.21.15.16:0/532190901 >> 172.21.15.16:0/532190901 conn(0x5624daf01000 msgr2=0x5624daf49b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.834+0000 7fad8e9fc740 1 -- 172.21.15.16:0/532190901 shutdown_connections 2023-11-13T11:02:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.834+0000 7fad8e9fc740 1 -- 172.21.15.16:0/532190901 wait complete. 2023-11-13T11:02:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.835+0000 7fad8e9fc740 1 -- start start 2023-11-13T11:02:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.835+0000 7fad8e9fc740 1 Processor -- start 2023-11-13T11:02:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf24a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf2ed70 0x5624daf27910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf298c0 0x5624daf2faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624dae6b160 con 0x5624daf2ed70 2023-11-13T11:02:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624dae07290 con 0x5624daf298c0 2023-11-13T11:02:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624dae24c10 con 0x5624daf028f0 2023-11-13T11:02:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad8e9fc740 10 client.0 _ll_register_callbacks cb 0x5624dae27270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:36.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad893fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf2ed70 0x5624daf27910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad893fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf2ed70 0x5624daf27910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47896/0 (socket says 172.21.15.16:47896) 2023-11-13T11:02:36.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad893fc640 1 -- 172.21.15.16:0/876428635 learned_addr learned my addr 172.21.15.16:0/876428635 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad8a3fe640 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf298c0 0x5624daf2faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.836+0000 7fad89bfd640 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf24a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad893fc640 1 -- 172.21.15.16:0/876428635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 msgr2=0x5624daf24a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad893fc640 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf24a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad893fc640 1 -- 172.21.15.16:0/876428635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf298c0 msgr2=0x5624daf2faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad893fc640 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf298c0 0x5624daf2faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad893fc640 1 -- 172.21.15.16:0/876428635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c001940 con 0x5624daf2ed70 2023-11-13T11:02:36.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad8a3fe640 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf298c0 0x5624daf2faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:36.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad893fc640 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf2ed70 0x5624daf27910 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fad840c77d0 tx=0x7fad78006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07d06465e5c540b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:36.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad64ff9640 1 -- 172.21.15.16:0/876428635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78023070 con 0x5624daf2ed70 2023-11-13T11:02:36.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad64ff9640 1 -- 172.21.15.16:0/876428635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fad78004020 con 0x5624daf2ed70 2023-11-13T11:02:36.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.837+0000 7fad8e9fc740 1 -- 172.21.15.16:0/876428635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624dae64140 con 0x5624daf2ed70 2023-11-13T11:02:36.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad64ff9640 1 -- 172.21.15.16:0/876428635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad780041e0 con 0x5624daf2ed70 2023-11-13T11:02:36.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad8e9fc740 10 client.6354 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:36.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad8e9fc740 1 -- 172.21.15.16:0/876428635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5624dae697e0 con 0x5624daf2ed70 2023-11-13T11:02:36.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad64ff9640 1 -- 172.21.15.16:0/876428635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624dae64140 con 0x5624daf2ed70 2023-11-13T11:02:36.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad64ff9640 1 -- 172.21.15.16:0/876428635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fad78026020 con 0x5624daf2ed70 2023-11-13T11:02:36.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad8e9fc740 10 client.6354 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:36.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad8e9fc740 10 client.6354 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:36.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad8e9fc740 20 client.6354 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5624daf26640 0x5624daf26a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad8e9fc740 4 client.6354 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad8e9fc740 1 -- 172.21.15.16:0/876428635 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624dae63e60 con 0x5624daf26640 2023-11-13T11:02:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.838+0000 7fad89bfd640 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5624daf26640 0x5624daf26a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.839+0000 7fad89bfd640 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5624daf26640 0x5624daf26a50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96799707b7712635 server_cookie=79118171744e43bd in_seq=0 out_seq=0 2023-11-13T11:02:36.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.839+0000 7fad64ff9640 10 client.6354 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad64ff9640 1 -- 172.21.15.16:0/876428635 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5624dae63e60 con 0x5624daf26640 2023-11-13T11:02:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad64ff9640 10 client.6354 handle_command_reply: tid=0 2023-11-13T11:02:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 client.6354 shutdown 2023-11-13T11:02:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 -- 172.21.15.16:0/876428635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf2ed70 msgr2=0x5624daf27910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf2ed70 0x5624daf27910 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fad840c77d0 tx=0x7fad78006e10 comp rx=0 tx=0).stop 2023-11-13T11:02:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 -- 172.21.15.16:0/876428635 shutdown_connections 2023-11-13T11:02:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf298c0 0x5624daf2faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5624daf26640 0x5624daf26a50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf2ed70 0x5624daf27910 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/876428635 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf24a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 -- 172.21.15.16:0/876428635 >> 172.21.15.16:0/876428635 conn(0x5624daefed90 msgr2=0x5624daf32a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad64ff9640 0 client.6354 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:36.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 -- 172.21.15.16:0/876428635 shutdown_connections 2023-11-13T11:02:36.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 1 -- 172.21.15.16:0/876428635 wait complete. 2023-11-13T11:02:36.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.840+0000 7fad8e9fc740 20 client.6354 trim_cache size 0 max 16384 2023-11-13T11:02:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.857+0000 7fad8e9fc740 1 Processor -- start 2023-11-13T11:02:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.857+0000 7fad8e9fc740 1 -- start start 2023-11-13T11:02:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.858+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf49c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.858+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624daefe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.858+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf6b660 0x5624daf256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.858+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624dab64390 con 0x5624daf298c0 2023-11-13T11:02:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.858+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624daf2f380 con 0x5624daf6b660 2023-11-13T11:02:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.858+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624daf19720 con 0x5624daf028f0 2023-11-13T11:02:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.858+0000 7fad893fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624daefe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.858+0000 7fad893fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624daefe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47908/0 (socket says 172.21.15.16:47908) 2023-11-13T11:02:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.858+0000 7fad893fc640 1 -- 172.21.15.16:0/3953283156 learned_addr learned my addr 172.21.15.16:0/3953283156 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad89bfd640 1 --2- 172.21.15.16:0/3953283156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf49c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad893fc640 1 -- 172.21.15.16:0/3953283156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 msgr2=0x5624daf49c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad893fc640 1 --2- 172.21.15.16:0/3953283156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf49c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad893fc640 1 -- 172.21.15.16:0/3953283156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf6b660 msgr2=0x5624daf256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad893fc640 1 --2- 172.21.15.16:0/3953283156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf6b660 0x5624daf256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad893fc640 1 -- 172.21.15.16:0/3953283156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624dae06b20 con 0x5624daf298c0 2023-11-13T11:02:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad89bfd640 1 --2- 172.21.15.16:0/3953283156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf49c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad893fc640 1 --2- 172.21.15.16:0/3953283156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624daefe000 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fad78000c00 tx=0x7fad7801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e7df58e452e004 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad67fff640 1 -- 172.21.15.16:0/3953283156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78023040 con 0x5624daf298c0 2023-11-13T11:02:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad67fff640 1 -- 172.21.15.16:0/3953283156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7801a050 con 0x5624daf298c0 2023-11-13T11:02:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.859+0000 7fad67fff640 1 -- 172.21.15.16:0/3953283156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78014c90 con 0x5624daf298c0 2023-11-13T11:02:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.860+0000 7fad8e9fc740 1 -- 172.21.15.16:0/3953283156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 msgr2=0x5624daefe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.860+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/3953283156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624daefe000 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fad78000c00 tx=0x7fad7801d000 comp rx=0 tx=0).stop 2023-11-13T11:02:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.860+0000 7fad8e9fc740 1 -- 172.21.15.16:0/3953283156 shutdown_connections 2023-11-13T11:02:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.860+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/3953283156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf6b660 0x5624daf256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.860+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/3953283156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624daefe000 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.860+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/3953283156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf49c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.860+0000 7fad8e9fc740 1 -- 172.21.15.16:0/3953283156 >> 172.21.15.16:0/3953283156 conn(0x5624daefed90 msgr2=0x5624daf23a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.860+0000 7fad8e9fc740 1 -- 172.21.15.16:0/3953283156 shutdown_connections 2023-11-13T11:02:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.860+0000 7fad8e9fc740 1 -- 172.21.15.16:0/3953283156 wait complete. 2023-11-13T11:02:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.861+0000 7fad8e9fc740 1 -- start start 2023-11-13T11:02:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.861+0000 7fad8e9fc740 1 Processor -- start 2023-11-13T11:02:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf27b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624dafbd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad8e9fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf6b660 0x5624daefa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624dafbe330 con 0x5624daf298c0 2023-11-13T11:02:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624daf49d30 con 0x5624daf6b660 2023-11-13T11:02:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad8e9fc740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624daf28050 con 0x5624daf028f0 2023-11-13T11:02:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad8e9fc740 10 client.0 _ll_register_callbacks cb 0x7fad78009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad893fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624dafbd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad893fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624dafbd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47922/0 (socket says 172.21.15.16:47922) 2023-11-13T11:02:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.862+0000 7fad893fc640 1 -- 172.21.15.16:0/2709103420 learned_addr learned my addr 172.21.15.16:0/2709103420 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad89bfd640 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf27b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad893fc640 1 -- 172.21.15.16:0/2709103420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 msgr2=0x5624daf27b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad893fc640 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf27b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad893fc640 1 -- 172.21.15.16:0/2709103420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf6b660 msgr2=0x5624daefa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad893fc640 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf6b660 0x5624daefa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad893fc640 1 -- 172.21.15.16:0/2709103420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624dae697e0 con 0x5624daf298c0 2023-11-13T11:02:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad893fc640 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624dafbd5e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fad78000c00 tx=0x7fad78012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378540b33ca14aa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad64ff9640 1 -- 172.21.15.16:0/2709103420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78023040 con 0x5624daf298c0 2023-11-13T11:02:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad64ff9640 1 -- 172.21.15.16:0/2709103420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fad78006310 con 0x5624daf298c0 2023-11-13T11:02:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.863+0000 7fad8e9fc740 1 -- 172.21.15.16:0/2709103420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad7c00c8a0 con 0x5624daf298c0 2023-11-13T11:02:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad64ff9640 1 -- 172.21.15.16:0/2709103420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad780064d0 con 0x5624daf298c0 2023-11-13T11:02:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad8e9fc740 10 client.6360 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad8e9fc740 1 -- 172.21.15.16:0/2709103420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5624dae2c310 con 0x5624daf298c0 2023-11-13T11:02:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad64ff9640 1 -- 172.21.15.16:0/2709103420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c00c8a0 con 0x5624daf298c0 2023-11-13T11:02:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad64ff9640 1 -- 172.21.15.16:0/2709103420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fad78026070 con 0x5624daf298c0 2023-11-13T11:02:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad8e9fc740 10 client.6360 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad8e9fc740 10 client.6360 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad8e9fc740 20 client.6360 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5624daf62be0 0x5624daefa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad8e9fc740 4 client.6360 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2023-11-13T11:02:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad8e9fc740 1 -- 172.21.15.16:0/2709103420 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x5624dab6bbe0 con 0x5624daf62be0 2023-11-13T11:02:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.864+0000 7fad89bfd640 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5624daf62be0 0x5624daefa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.865+0000 7fad89bfd640 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5624daf62be0 0x5624daefa8f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3978610fb677fce0 server_cookie=6189a5b921993ec5 in_seq=0 out_seq=0 2023-11-13T11:02:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.865+0000 7fad64ff9640 10 client.6360 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.865+0000 7fad64ff9640 1 -- 172.21.15.16:0/2709103420 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5624dab6bbe0 con 0x5624daf62be0 2023-11-13T11:02:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.865+0000 7fad64ff9640 10 client.6360 handle_command_reply: tid=0 2023-11-13T11:02:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.865+0000 7fad8e9fc740 1 client.6360 shutdown 2023-11-13T11:02:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 -- 172.21.15.16:0/2709103420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 msgr2=0x5624dafbd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624dafbd5e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fad78000c00 tx=0x7fad78012000 comp rx=0 tx=0).stop 2023-11-13T11:02:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 -- 172.21.15.16:0/2709103420 shutdown_connections 2023-11-13T11:02:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5624daf6b660 0x5624daefa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5624daf62be0 0x5624daefa8f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624daf298c0 0x5624dafbd5e0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 --2- 172.21.15.16:0/2709103420 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5624daf028f0 0x5624daf27b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 -- 172.21.15.16:0/2709103420 >> 172.21.15.16:0/2709103420 conn(0x5624daefed90 msgr2=0x5624daf23a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad64ff9640 0 client.6360 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:36.885 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-11-13T11:02:36.866+0000 7f0f71a64640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4a [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v777 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6183=0-4194304@1} caps={6183=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6183 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55fd3fc66100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a4a:[<0x10000000001/new_file_0 v777>,<0x10000000000/ceph-qa-suite v116>,<0x1/scrub_test_path v83>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-11-13T11:02:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 -- 172.21.15.16:0/2709103420 shutdown_connections 2023-11-13T11:02:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 1 -- 172.21.15.16:0/2709103420 wait complete. 2023-11-13T11:02:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.866+0000 7fad8e9fc740 20 client.6360 trim_cache size 0 max 16384 2023-11-13T11:02:36.886 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:36.886 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-11-13T11:02:36.887 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "aee84d2a-1ddd-4ab5-af16-53585a3b1580", 2023-11-13T11:02:36.887 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-11-13T11:02:36.887 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:36.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 -- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad6806ae40 msgr2=0x7fad6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad6806ae40 0x7fad6806d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad8409ffb0 tx=0x7fad78008040 comp rx=0 tx=0).stop 2023-11-13T11:02:36.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 -- 172.21.15.16:0/3798290180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 msgr2=0x7fad8406b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:36.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 0x7fad8406b490 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fad80000c00 tx=0x7fad800046c0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 -- 172.21.15.16:0/3798290180 shutdown_connections 2023-11-13T11:02:36.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fad6806ae40 0x7fad6806d300 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad840ceca0 0x7fad8415d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad840b3490 0x7fad841f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 --2- 172.21.15.16:0/3798290180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad840b30b0 0x7fad8406b490 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:36.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 -- 172.21.15.16:0/3798290180 >> 172.21.15.16:0/3798290180 conn(0x7fad840d2d20 msgr2=0x7fad840d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:36.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 -- 172.21.15.16:0/3798290180 shutdown_connections 2023-11-13T11:02:36.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:36.867+0000 7fad8be61640 1 -- 172.21.15.16:0/3798290180 wait complete. 2023-11-13T11:02:36.891 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': 'aee84d2a-1ddd-4ab5-af16-53585a3b1580', 'mode': 'asynchronous'}' 2023-11-13T11:02:36.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-13T11:02:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.118+0000 7f7bb986d640 1 Processor -- start 2023-11-13T11:02:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.119+0000 7f7bb986d640 1 -- start start 2023-11-13T11:02:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.120+0000 7f7bb986d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7bb415b5c0 0x7f7bb4164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.120+0000 7f7bb986d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.120+0000 7f7bb986d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb4165b40 con 0x7f7bb4161640 2023-11-13T11:02:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.120+0000 7f7bb986d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb41652a0 con 0x7f7bb415b5c0 2023-11-13T11:02:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.120+0000 7f7bb986d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4165580 con 0x7f7bb40b2b40 2023-11-13T11:02:37.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.121+0000 7f7bb906c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.121+0000 7f7bb906c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb40ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:35106/0 (socket says 172.21.15.16:35106) 2023-11-13T11:02:37.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.121+0000 7f7bb906c640 1 -- 172.21.15.16:0/4228836503 learned_addr learned my addr 172.21.15.16:0/4228836503 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.121+0000 7f7bb886b640 1 --2- 172.21.15.16:0/4228836503 >> v2:172.21.15.184:3300/0 conn(0x7f7bb415b5c0 0x7f7bb4164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.122+0000 7f7bb906c640 1 -- 172.21.15.16:0/4228836503 >> v2:172.21.15.184:3300/0 conn(0x7f7bb415b5c0 msgr2=0x7f7bb4164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.122+0000 7f7bb906c640 1 --2- 172.21.15.16:0/4228836503 >> v2:172.21.15.184:3300/0 conn(0x7f7bb415b5c0 0x7f7bb4164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.122+0000 7f7bb906c640 1 -- 172.21.15.16:0/4228836503 >> v1:172.21.15.184:6789/0 conn(0x7f7bb4161640 legacy=0x7f7bb40b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.122+0000 7f7bb906c640 1 -- 172.21.15.16:0/4228836503 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4165860 con 0x7f7bb40b2b40 2023-11-13T11:02:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.123+0000 7f7bb906c640 1 --2- 172.21.15.16:0/4228836503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb40ad520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009e50 tx=0x7f7bac02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e6c6122410a22ed server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.124+0000 7f7bb37fe640 1 -- 172.21.15.16:0/4228836503 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac02cb00 con 0x7f7bb40b2b40 2023-11-13T11:02:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.124+0000 7f7bb37fe640 1 -- 172.21.15.16:0/4228836503 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac02cc60 con 0x7f7bb40b2b40 2023-11-13T11:02:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.124+0000 7f7bb37fe640 1 -- 172.21.15.16:0/4228836503 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0343f0 con 0x7f7bb40b2b40 2023-11-13T11:02:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.124+0000 7f7bb986d640 1 -- 172.21.15.16:0/4228836503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 msgr2=0x7f7bb40ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.124+0000 7f7bb986d640 1 --2- 172.21.15.16:0/4228836503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb40ad520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009e50 tx=0x7f7bac02bf60 comp rx=0 tx=0).stop 2023-11-13T11:02:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.124+0000 7f7bb986d640 1 -- 172.21.15.16:0/4228836503 shutdown_connections 2023-11-13T11:02:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.124+0000 7f7bb986d640 1 --2- 172.21.15.16:0/4228836503 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb40ad520 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.124+0000 7f7bb986d640 1 --2- 172.21.15.16:0/4228836503 >> v2:172.21.15.184:3300/0 conn(0x7f7bb415b5c0 0x7f7bb4164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.124+0000 7f7bb986d640 1 -- 172.21.15.16:0/4228836503 >> 172.21.15.16:0/4228836503 conn(0x7f7bb40b0970 msgr2=0x7f7bb40b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.125+0000 7f7bb986d640 1 -- 172.21.15.16:0/4228836503 shutdown_connections 2023-11-13T11:02:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.125+0000 7f7bb986d640 1 -- 172.21.15.16:0/4228836503 wait complete. 2023-11-13T11:02:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.126+0000 7f7bb986d640 1 Processor -- start 2023-11-13T11:02:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.127+0000 7f7bb986d640 1 -- start start 2023-11-13T11:02:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.127+0000 7f7bb986d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb41f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.127+0000 7f7bb986d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb415b5c0 0x7f7bb41ebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb986d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4161640 0x7f7bb4160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb986d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb41f6e70 con 0x7f7bb4161640 2023-11-13T11:02:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb986d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb41eac10 con 0x7f7bb415b5c0 2023-11-13T11:02:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb986d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb41eaef0 con 0x7f7bb40b2b40 2023-11-13T11:02:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb906c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4161640 0x7f7bb4160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb906c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4161640 0x7f7bb4160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47936/0 (socket says 172.21.15.16:47936) 2023-11-13T11:02:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb906c640 1 -- 172.21.15.16:0/675576930 learned_addr learned my addr 172.21.15.16:0/675576930 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb886b640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb41f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb906c640 1 -- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 msgr2=0x7f7bb41f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb906c640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb41f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb906c640 1 -- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb415b5c0 msgr2=0x7f7bb41ebcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb906c640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb415b5c0 0x7f7bb41ebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb906c640 1 -- 172.21.15.16:0/675576930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac009b30 con 0x7f7bb4161640 2023-11-13T11:02:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.128+0000 7f7bb886b640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb41f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.129+0000 7f7bb906c640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4161640 0x7f7bb4160200 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009e50 tx=0x7f7bac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:37.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.129+0000 7f7bb17fa640 1 -- 172.21.15.16:0/675576930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac009510 con 0x7f7bb4161640 2023-11-13T11:02:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.129+0000 7f7bb17fa640 1 -- 172.21.15.16:0/675576930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac03e450 con 0x7f7bb4161640 2023-11-13T11:02:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.129+0000 7f7bb17fa640 1 -- 172.21.15.16:0/675576930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac03e800 con 0x7f7bb4161640 2023-11-13T11:02:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.129+0000 7f7bb986d640 1 -- 172.21.15.16:0/675576930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb41eb1d0 con 0x7f7bb4161640 2023-11-13T11:02:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.129+0000 7f7bb986d640 1 -- 172.21.15.16:0/675576930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb41eb790 con 0x7f7bb4161640 2023-11-13T11:02:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.132+0000 7f7bb17fa640 1 -- 172.21.15.16:0/675576930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bac03e960 con 0x7f7bb4161640 2023-11-13T11:02:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.133+0000 7f7bb17fa640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b9806ae40 0x7f7b9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.133+0000 7f7bb17fa640 1 -- 172.21.15.16:0/675576930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7bac0ac540 con 0x7f7bb4161640 2023-11-13T11:02:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.134+0000 7f7bb886b640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b9806ae40 0x7f7b9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.134+0000 7f7bbd665740 1 Processor -- start 2023-11-13T11:02:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.134+0000 7f7bbd665740 1 -- start start 2023-11-13T11:02:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bb886b640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b9806ae40 0x7f7b9806d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009ac0 tx=0x7f7ba8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81270 0x5562efc54210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 0x5562efc84000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc54750 0x5562efc84540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bbd665740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562efbc6e90 con 0x5562efc81270 2023-11-13T11:02:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bbd665740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562efbc4580 con 0x5562efc54750 2023-11-13T11:02:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bbd665740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562efbbfe70 con 0x5562efc81650 2023-11-13T11:02:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bb886b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81270 0x5562efc54210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bb886b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81270 0x5562efc54210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47946/0 (socket says 172.21.15.16:47946) 2023-11-13T11:02:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.135+0000 7f7bb886b640 1 -- 172.21.15.16:0/2821109485 learned_addr learned my addr 172.21.15.16:0/2821109485 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.136+0000 7f7bb3fff640 1 --2- 172.21.15.16:0/2821109485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 0x5562efc84000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.136+0000 7f7bb906c640 1 --2- 172.21.15.16:0/2821109485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc54750 0x5562efc84540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.136+0000 7f7bb886b640 1 -- 172.21.15.16:0/2821109485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 msgr2=0x5562efc84000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.136+0000 7f7bb886b640 1 --2- 172.21.15.16:0/2821109485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 0x5562efc84000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.136+0000 7f7bb886b640 1 -- 172.21.15.16:0/2821109485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc54750 msgr2=0x5562efc84540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.136+0000 7f7bb886b640 1 --2- 172.21.15.16:0/2821109485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc54750 0x5562efc84540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.136+0000 7f7bb886b640 1 -- 172.21.15.16:0/2821109485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562efb8a020 con 0x5562efc81270 2023-11-13T11:02:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.136+0000 7f7bb886b640 1 --2- 172.21.15.16:0/2821109485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81270 0x5562efc54210 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8010b90 tx=0x7f7ba8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef110dbb54f8b20f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7b9e7fc640 1 -- 172.21.15.16:0/2821109485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8021440 con 0x5562efc81270 2023-11-13T11:02:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7b9e7fc640 1 -- 172.21.15.16:0/2821109485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba801a070 con 0x5562efc81270 2023-11-13T11:02:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7b9e7fc640 1 -- 172.21.15.16:0/2821109485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80217a0 con 0x5562efc81270 2023-11-13T11:02:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7bbd665740 1 -- 172.21.15.16:0/2821109485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81270 msgr2=0x5562efc54210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7bbd665740 1 --2- 172.21.15.16:0/2821109485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81270 0x5562efc54210 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8010b90 tx=0x7f7ba8013000 comp rx=0 tx=0).stop 2023-11-13T11:02:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7bbd665740 1 -- 172.21.15.16:0/2821109485 shutdown_connections 2023-11-13T11:02:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7bbd665740 1 --2- 172.21.15.16:0/2821109485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc54750 0x5562efc84540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7bbd665740 1 --2- 172.21.15.16:0/2821109485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 0x5562efc84000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7bbd665740 1 --2- 172.21.15.16:0/2821109485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81270 0x5562efc54210 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7bbd665740 1 -- 172.21.15.16:0/2821109485 >> 172.21.15.16:0/2821109485 conn(0x5562efc56480 msgr2=0x5562efc9f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7bbd665740 1 -- 172.21.15.16:0/2821109485 shutdown_connections 2023-11-13T11:02:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.137+0000 7f7bbd665740 1 -- 172.21.15.16:0/2821109485 wait complete. 2023-11-13T11:02:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.138+0000 7f7bbd665740 1 -- start start 2023-11-13T11:02:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.138+0000 7f7bbd665740 1 Processor -- start 2023-11-13T11:02:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.139+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc71600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.139+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.139+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc86ca0 0x5562efc87780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.139+0000 7f7bbd665740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562efb5c480 con 0x5562efc81650 2023-11-13T11:02:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.139+0000 7f7bbd665740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562efb79dc0 con 0x5562efc81270 2023-11-13T11:02:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.139+0000 7f7bbd665740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562ef8b8f90 con 0x5562efc86ca0 2023-11-13T11:02:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bbd665740 10 client.0 _ll_register_callbacks cb 0x5562ef8d1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47960/0 (socket says 172.21.15.16:47960) 2023-11-13T11:02:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb3fff640 1 -- 172.21.15.16:0/1728372230 learned_addr learned my addr 172.21.15.16:0/1728372230 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb886b640 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb906c640 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc86ca0 0x5562efc87780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb3fff640 1 -- 172.21.15.16:0/1728372230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc86ca0 msgr2=0x5562efc87780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb3fff640 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc86ca0 0x5562efc87780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb3fff640 1 -- 172.21.15.16:0/1728372230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 msgr2=0x5562efc71600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb3fff640 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc71600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb3fff640 1 -- 172.21.15.16:0/1728372230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba800e050 con 0x5562efc81650 2023-11-13T11:02:37.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.140+0000 7f7bb886b640 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc71600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:37.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.141+0000 7f7bb3fff640 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84ff0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0001dc0 tx=0x7f7ba000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e164095d5b4596 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:37.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.141+0000 7f7b937fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:37.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.141+0000 7f7b937fe640 1 -- 172.21.15.16:0/1728372230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000ee80 con 0x5562efc81650 2023-11-13T11:02:37.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.141+0000 7f7b937fe640 1 -- 172.21.15.16:0/1728372230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7ba00050d0 con 0x5562efc81650 2023-11-13T11:02:37.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.141+0000 7f7bbd665740 1 -- 172.21.15.16:0/1728372230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562efbb8e50 con 0x5562efc81650 2023-11-13T11:02:37.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.141+0000 7f7b937fe640 1 -- 172.21.15.16:0/1728372230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba0005290 con 0x5562efc81650 2023-11-13T11:02:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.141+0000 7f7bbd665740 10 client.6369 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.141+0000 7f7bbd665740 1 -- 172.21.15.16:0/1728372230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5562efbbe4f0 con 0x5562efc81650 2023-11-13T11:02:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.141+0000 7f7b937fe640 1 -- 172.21.15.16:0/1728372230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5562efbb8e50 con 0x5562efc81650 2023-11-13T11:02:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.142+0000 7f7b937fe640 1 -- 172.21.15.16:0/1728372230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7ba00035f0 con 0x5562efc81650 2023-11-13T11:02:37.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.142+0000 7f7bbd665740 10 client.6369 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.142+0000 7f7bbd665740 10 client.6369 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.142+0000 7f7bbd665740 20 client.6369 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.142+0000 7f7bbd665740 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5562efcaeb80 0x5562efcaef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.142+0000 7f7bbd665740 4 client.6369 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.142+0000 7f7bbd665740 1 -- 172.21.15.16:0/1728372230 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562efbb8b70 con 0x5562efcaeb80 2023-11-13T11:02:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.142+0000 7f7bb886b640 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5562efcaeb80 0x5562efcaef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.143+0000 7f7b937fe640 10 client.6369 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.142+0000 7f7bb886b640 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5562efcaeb80 0x5562efcaef90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=844d2c1cbbe3a8fd server_cookie=7bc328ba2442fc9b in_seq=0 out_seq=0 2023-11-13T11:02:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.143+0000 7f7b937fe640 1 -- 172.21.15.16:0/1728372230 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5562efbb8b70 con 0x5562efcaeb80 2023-11-13T11:02:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.143+0000 7f7b937fe640 10 client.6369 handle_command_reply: tid=0 2023-11-13T11:02:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.143+0000 7f7bbd665740 1 client.6369 shutdown 2023-11-13T11:02:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.143+0000 7f7bbd665740 1 -- 172.21.15.16:0/1728372230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 msgr2=0x5562efc84ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.143+0000 7f7bbd665740 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84ff0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0001dc0 tx=0x7f7ba000e530 comp rx=0 tx=0).stop 2023-11-13T11:02:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.143+0000 7f7bbd665740 1 -- 172.21.15.16:0/1728372230 shutdown_connections 2023-11-13T11:02:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.143+0000 7f7bbd665740 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc86ca0 0x5562efc87780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.144+0000 7f7bbd665740 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5562efcaeb80 0x5562efcaef90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.144+0000 7f7bbd665740 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84ff0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.144+0000 7f7bbd665740 1 --2- 172.21.15.16:0/1728372230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc71600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.144+0000 7f7b937fe640 0 client.6369 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.144+0000 7f7bbd665740 1 -- 172.21.15.16:0/1728372230 >> 172.21.15.16:0/1728372230 conn(0x5562efc54750 msgr2=0x5562efc53970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.144+0000 7f7bbd665740 1 -- 172.21.15.16:0/1728372230 shutdown_connections 2023-11-13T11:02:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.144+0000 7f7bbd665740 1 -- 172.21.15.16:0/1728372230 wait complete. 2023-11-13T11:02:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.144+0000 7f7bbd665740 20 client.6369 trim_cache size 0 max 16384 2023-11-13T11:02:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.160+0000 7f7bbd665740 1 Processor -- start 2023-11-13T11:02:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.160+0000 7f7bbd665740 1 -- start start 2023-11-13T11:02:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.161+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc83a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.161+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 0x5562efc870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.161+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc86ca0 0x5562efc87610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.161+0000 7f7bbd665740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562efc7f350 con 0x5562efc86ca0 2023-11-13T11:02:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.161+0000 7f7bbd665740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562efcb1b80 con 0x5562efc81270 2023-11-13T11:02:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.161+0000 7f7bbd665740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562efc7c790 con 0x5562efc81650 2023-11-13T11:02:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb906c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc86ca0 0x5562efc87610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb906c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc86ca0 0x5562efc87610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47962/0 (socket says 172.21.15.16:47962) 2023-11-13T11:02:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb906c640 1 -- 172.21.15.16:0/2552211608 learned_addr learned my addr 172.21.15.16:0/2552211608 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb3fff640 1 --2- 172.21.15.16:0/2552211608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 0x5562efc870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb906c640 1 -- 172.21.15.16:0/2552211608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 msgr2=0x5562efc870d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb886b640 1 --2- 172.21.15.16:0/2552211608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc83a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb906c640 1 --2- 172.21.15.16:0/2552211608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 0x5562efc870d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb906c640 1 -- 172.21.15.16:0/2552211608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 msgr2=0x5562efc83a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb906c640 1 --2- 172.21.15.16:0/2552211608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc83a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb906c640 1 -- 172.21.15.16:0/2552211608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562efb5bd10 con 0x5562efc86ca0 2023-11-13T11:02:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.162+0000 7f7bb906c640 1 --2- 172.21.15.16:0/2552211608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc86ca0 0x5562efc87610 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7bac02bfa0 tx=0x7f7bac03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e08208c172a53e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.163+0000 7f7b9effd640 1 -- 172.21.15.16:0/2552211608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac07b090 con 0x5562efc86ca0 2023-11-13T11:02:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.163+0000 7f7b9effd640 1 -- 172.21.15.16:0/2552211608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac07f1e0 con 0x5562efc86ca0 2023-11-13T11:02:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.163+0000 7f7b9effd640 1 -- 172.21.15.16:0/2552211608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac083390 con 0x5562efc86ca0 2023-11-13T11:02:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.163+0000 7f7bbd665740 1 -- 172.21.15.16:0/2552211608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc86ca0 msgr2=0x5562efc87610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.163+0000 7f7bbd665740 1 --2- 172.21.15.16:0/2552211608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc86ca0 0x5562efc87610 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7bac02bfa0 tx=0x7f7bac03c000 comp rx=0 tx=0).stop 2023-11-13T11:02:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.163+0000 7f7bbd665740 1 -- 172.21.15.16:0/2552211608 shutdown_connections 2023-11-13T11:02:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.163+0000 7f7bbd665740 1 --2- 172.21.15.16:0/2552211608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc86ca0 0x5562efc87610 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.163+0000 7f7bbd665740 1 --2- 172.21.15.16:0/2552211608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc81650 0x5562efc870d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.163+0000 7f7bbd665740 1 --2- 172.21.15.16:0/2552211608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc83a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.164+0000 7f7bbd665740 1 -- 172.21.15.16:0/2552211608 >> 172.21.15.16:0/2552211608 conn(0x5562efc54750 msgr2=0x5562efc4f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.164+0000 7f7bbd665740 1 -- 172.21.15.16:0/2552211608 shutdown_connections 2023-11-13T11:02:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.164+0000 7f7bbd665740 1 -- 172.21.15.16:0/2552211608 wait complete. 2023-11-13T11:02:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.164+0000 7f7bbd665740 1 -- start start 2023-11-13T11:02:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.164+0000 7f7bbd665740 1 Processor -- start 2023-11-13T11:02:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.165+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc84430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.165+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.165+0000 7f7bbd665740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc86ca0 0x5562efc7f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.165+0000 7f7bbd665740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562efcadd60 con 0x5562efc81650 2023-11-13T11:02:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.165+0000 7f7bbd665740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562efc6c280 con 0x5562efc81270 2023-11-13T11:02:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.165+0000 7f7bbd665740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562efc6b730 con 0x5562efc86ca0 2023-11-13T11:02:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bbd665740 10 client.0 _ll_register_callbacks cb 0x7f7ba8009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47976/0 (socket says 172.21.15.16:47976) 2023-11-13T11:02:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bb3fff640 1 -- 172.21.15.16:0/3949462753 learned_addr learned my addr 172.21.15.16:0/3949462753 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bb3fff640 1 -- 172.21.15.16:0/3949462753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc86ca0 msgr2=0x5562efc7f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bb3fff640 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc86ca0 0x5562efc7f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bb3fff640 1 -- 172.21.15.16:0/3949462753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 msgr2=0x5562efc84430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bb3fff640 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc84430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bb3fff640 1 -- 172.21.15.16:0/3949462753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb41eb790 con 0x5562efc81650 2023-11-13T11:02:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7bb3fff640 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84970 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000ea0 tx=0x7f7ba0001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8673d18a21e3e92 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.166+0000 7f7b937fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.167+0000 7f7b937fe640 1 -- 172.21.15.16:0/3949462753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000b040 con 0x5562efc81650 2023-11-13T11:02:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.167+0000 7f7b937fe640 1 -- 172.21.15.16:0/3949462753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7ba000ee80 con 0x5562efc81650 2023-11-13T11:02:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.167+0000 7f7bbd665740 1 -- 172.21.15.16:0/3949462753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ba801a070 con 0x5562efc81650 2023-11-13T11:02:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.167+0000 7f7b937fe640 1 -- 172.21.15.16:0/3949462753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba0001920 con 0x5562efc81650 2023-11-13T11:02:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.167+0000 7f7bbd665740 10 client.6375 fetch_fsmap learned FSMap version 28 2023-11-13T11:02:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.167+0000 7f7bbd665740 1 -- 172.21.15.16:0/3949462753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5562efb81300 con 0x5562efc81650 2023-11-13T11:02:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.167+0000 7f7b937fe640 1 -- 172.21.15.16:0/3949462753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba801a070 con 0x5562efc81650 2023-11-13T11:02:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.167+0000 7f7b937fe640 1 -- 172.21.15.16:0/3949462753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7ba001c070 con 0x5562efc81650 2023-11-13T11:02:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.168+0000 7f7bbd665740 10 client.6375 fetch_fsmap finished waiting for FSMap version 28 2023-11-13T11:02:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.168+0000 7f7bbd665740 10 client.6375 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.168+0000 7f7bbd665740 20 client.6375 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.168+0000 7f7bbd665740 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5562efcad630 0x5562efc7fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.168+0000 7f7bbd665740 4 client.6375 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-13T11:02:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.168+0000 7f7bbd665740 1 -- 172.21.15.16:0/3949462753 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5562ef8c07e0 con 0x5562efcad630 2023-11-13T11:02:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.168+0000 7f7bb886b640 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5562efcad630 0x5562efc7fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.168+0000 7f7bb886b640 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5562efcad630 0x5562efc7fc80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1257aae8a9ccb52c server_cookie=e46b7ae086f622d5 in_seq=0 out_seq=0 2023-11-13T11:02:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.168+0000 7f7b937fe640 10 client.6375 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7b937fe640 1 -- 172.21.15.16:0/3949462753 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+696 (crc 0 0 0) 0x5562ef8c07e0 con 0x5562efcad630 2023-11-13T11:02:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7b937fe640 10 client.6375 handle_command_reply: tid=0 2023-11-13T11:02:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 client.6375 shutdown 2023-11-13T11:02:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 -- 172.21.15.16:0/3949462753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 msgr2=0x5562efc84970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84970 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000ea0 tx=0x7f7ba0001080 comp rx=0 tx=0).stop 2023-11-13T11:02:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 -- 172.21.15.16:0/3949462753 shutdown_connections 2023-11-13T11:02:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5562efc86ca0 0x5562efc7f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5562efcad630 0x5562efc7fc80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562efc81650 0x5562efc84970 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 --2- 172.21.15.16:0/3949462753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5562efc81270 0x5562efc84430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 -- 172.21.15.16:0/3949462753 >> 172.21.15.16:0/3949462753 conn(0x5562efc54750 msgr2=0x5562efc870d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7b937fe640 0 client.6375 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 -- 172.21.15.16:0/3949462753 shutdown_connections 2023-11-13T11:02:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 1 -- 172.21.15.16:0/3949462753 wait complete. 2023-11-13T11:02:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.169+0000 7f7bbd665740 20 client.6375 trim_cache size 0 max 16384 2023-11-13T11:02:37.185 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:37.186 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-13T11:02:37.186 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2023-11-13T11:02:37.186 INFO:teuthology.orchestra.run.smithi016.stdout: "50c81e29-d6b3-48d1-a9ca-caaf3e76c244": { 2023-11-13T11:02:37.186 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2023-11-13T11:02:37.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "50c81e29-d6b3-48d1-a9ca-caaf3e76c244", 2023-11-13T11:02:37.187 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "" 2023-11-13T11:02:37.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:02:37.187 INFO:teuthology.orchestra.run.smithi016.stdout: "aee84d2a-1ddd-4ab5-af16-53585a3b1580": { 2023-11-13T11:02:37.187 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2023-11-13T11:02:37.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "aee84d2a-1ddd-4ab5-af16-53585a3b1580", 2023-11-13T11:02:37.188 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "" 2023-11-13T11:02:37.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:02:37.188 INFO:teuthology.orchestra.run.smithi016.stdout: "dfc0f507-4620-48b5-9445-ab890868fbd1": { 2023-11-13T11:02:37.188 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2023-11-13T11:02:37.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "dfc0f507-4620-48b5-9445-ab890868fbd1", 2023-11-13T11:02:37.189 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "" 2023-11-13T11:02:37.189 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:02:37.189 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:02:37.189 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 -- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b9806ae40 msgr2=0x7f7b9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b9806ae40 0x7f7b9806d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009ac0 tx=0x7f7ba8008040 comp rx=0 tx=0).stop 2023-11-13T11:02:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 -- 172.21.15.16:0/675576930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4161640 msgr2=0x7f7bb4160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4161640 0x7f7bb4160200 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009e50 tx=0x7f7bac004a40 comp rx=0 tx=0).stop 2023-11-13T11:02:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 -- 172.21.15.16:0/675576930 shutdown_connections 2023-11-13T11:02:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b9806ae40 0x7f7b9806d300 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb4161640 0x7f7bb4160200 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb415b5c0 0x7f7bb41ebcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 --2- 172.21.15.16:0/675576930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7bb40b2b40 0x7f7bb41f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 -- 172.21.15.16:0/675576930 >> 172.21.15.16:0/675576930 conn(0x7f7bb40b0970 msgr2=0x7f7bb40b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 -- 172.21.15.16:0/675576930 shutdown_connections 2023-11-13T11:02:37.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:37.170+0000 7f7bb986d640 1 -- 172.21.15.16:0/675576930 wait complete. 2023-11-13T11:02:39.958 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-13T11:02:39.953+0000 7fe1578d7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-13T11:02:42.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-13T11:02:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.429+0000 7f46ce5a7640 1 Processor -- start 2023-11-13T11:02:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.430+0000 7f46ce5a7640 1 -- start start 2023-11-13T11:02:42.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.431+0000 7f46ce5a7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c815ce70 0x7f46c80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.431+0000 7f46ce5a7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46c80d1850 0x7f46c80c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.431+0000 7f46ce5a7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80ce0e0 con 0x7f46c80cdbb0 2023-11-13T11:02:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.431+0000 7f46ce5a7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c80ce3c0 con 0x7f46c80d1850 2023-11-13T11:02:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.431+0000 7f46ce5a7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80ce6a0 con 0x7f46c815ce70 2023-11-13T11:02:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.432+0000 7f46cdda6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46c80d1850 0x7f46c80c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.432+0000 7f46cdda6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46c80d1850 0x7f46c80c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47990/0 (socket says 172.21.15.16:47990) 2023-11-13T11:02:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.432+0000 7f46cdda6640 1 -- 172.21.15.16:0/4223429692 learned_addr learned my addr 172.21.15.16:0/4223429692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.432+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/4223429692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c815ce70 0x7f46c80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.432+0000 7f46cdda6640 1 -- 172.21.15.16:0/4223429692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c815ce70 msgr2=0x7f46c80cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.432+0000 7f46cdda6640 1 --2- 172.21.15.16:0/4223429692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c815ce70 0x7f46c80cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.433+0000 7f46cdda6640 1 -- 172.21.15.16:0/4223429692 >> v1:172.21.15.184:6789/0 conn(0x7f46c80cdbb0 legacy=0x7f46c80d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.433+0000 7f46cdda6640 1 -- 172.21.15.16:0/4223429692 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c80ce980 con 0x7f46c80d1850 2023-11-13T11:02:42.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.434+0000 7f46cdda6640 1 --2- 172.21.15.16:0/4223429692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 0x7f46c80c9c30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f46c00098a0 tx=0x7f46c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e97ffc2dfd17adf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.434+0000 7f46c7fff640 1 -- 172.21.15.16:0/4223429692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0030840 con 0x7f46c80d1850 2023-11-13T11:02:42.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46c7fff640 1 -- 172.21.15.16:0/4223429692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c00309a0 con 0x7f46c80d1850 2023-11-13T11:02:42.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46c7fff640 1 -- 172.21.15.16:0/4223429692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0030d30 con 0x7f46c80d1850 2023-11-13T11:02:42.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4223429692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 msgr2=0x7f46c80c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46ce5a7640 1 --2- 172.21.15.16:0/4223429692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 0x7f46c80c9c30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f46c00098a0 tx=0x7f46c002b980 comp rx=0 tx=0).stop 2023-11-13T11:02:42.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4223429692 shutdown_connections 2023-11-13T11:02:42.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46ce5a7640 1 --2- 172.21.15.16:0/4223429692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 0x7f46c80c9c30 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46ce5a7640 1 --2- 172.21.15.16:0/4223429692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c815ce70 0x7f46c80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4223429692 >> 172.21.15.16:0/4223429692 conn(0x7f46c808c580 msgr2=0x7f46c809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4223429692 shutdown_connections 2023-11-13T11:02:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.435+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4223429692 wait complete. 2023-11-13T11:02:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.438+0000 7f46ce5a7640 1 Processor -- start 2023-11-13T11:02:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.438+0000 7f46ce5a7640 1 -- start start 2023-11-13T11:02:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.439+0000 7f46ce5a7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c80cdbb0 0x7f46c81eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.439+0000 7f46ce5a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 0x7f46c81568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.439+0000 7f46ce5a7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c815ce70 0x7f46c80b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.439+0000 7f46ce5a7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c81f39f0 con 0x7f46c80d1850 2023-11-13T11:02:42.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.439+0000 7f46ce5a7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80b6b60 con 0x7f46c815ce70 2023-11-13T11:02:42.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.439+0000 7f46ce5a7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80b6e40 con 0x7f46c80cdbb0 2023-11-13T11:02:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46ccda4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 0x7f46c81568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46ccda4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 0x7f46c81568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47996/0 (socket says 172.21.15.16:47996) 2023-11-13T11:02:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46ccda4640 1 -- 172.21.15.16:0/4133721297 learned_addr learned my addr 172.21.15.16:0/4133721297 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46cdda6640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c815ce70 0x7f46c80b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c80cdbb0 0x7f46c81eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46ccda4640 1 -- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c80cdbb0 msgr2=0x7f46c81eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46ccda4640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c80cdbb0 0x7f46c81eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46ccda4640 1 -- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c815ce70 msgr2=0x7f46c80b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46ccda4640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c815ce70 0x7f46c80b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46ccda4640 1 -- 172.21.15.16:0/4133721297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00094d0 con 0x7f46c80d1850 2023-11-13T11:02:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.440+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c80cdbb0 0x7f46c81eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:02:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.441+0000 7f46ccda4640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 0x7f46c81568c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c400 tx=0x7f46b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.441+0000 7f46c5ffb640 1 -- 172.21.15.16:0/4133721297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40052d0 con 0x7f46c80d1850 2023-11-13T11:02:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.441+0000 7f46c5ffb640 1 -- 172.21.15.16:0/4133721297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b4005430 con 0x7f46c80d1850 2023-11-13T11:02:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.441+0000 7f46c5ffb640 1 -- 172.21.15.16:0/4133721297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4003830 con 0x7f46c80d1850 2023-11-13T11:02:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.441+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4133721297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c80b7120 con 0x7f46c80d1850 2023-11-13T11:02:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.441+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4133721297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c80b76e0 con 0x7f46c80d1850 2023-11-13T11:02:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.444+0000 7f46c5ffb640 1 -- 172.21.15.16:0/4133721297 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f46b4003990 con 0x7f46c80d1850 2023-11-13T11:02:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.446+0000 7f46c5ffb640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f46ac06ae40 0x7f46ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.446+0000 7f46c5ffb640 1 -- 172.21.15.16:0/4133721297 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f46b4088d30 con 0x7f46c80d1850 2023-11-13T11:02:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.446+0000 7f46d239e740 1 Processor -- start 2023-11-13T11:02:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.446+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f46ac06ae40 0x7f46ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.447+0000 7f46d239e740 1 -- start start 2023-11-13T11:02:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46d239e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be270 0x563fe3291210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46d239e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be650 0x563fe32c1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46d239e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe3291750 0x563fe32c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46d239e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe3203e90 con 0x563fe3291750 2023-11-13T11:02:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46d239e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe3201580 con 0x563fe32be650 2023-11-13T11:02:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46d239e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fe31fce70 con 0x563fe32be270 2023-11-13T11:02:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f46ac06ae40 0x7f46ac06d300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009a80 tx=0x7f46bc0039d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cdda6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe3291750 0x563fe32c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cdda6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe3291750 0x563fe32c1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48008/0 (socket says 172.21.15.16:48008) 2023-11-13T11:02:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cdda6640 1 -- 172.21.15.16:0/218071598 learned_addr learned my addr 172.21.15.16:0/218071598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be270 0x563fe3291210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46ccda4640 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be650 0x563fe32c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cdda6640 1 -- 172.21.15.16:0/218071598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be270 msgr2=0x563fe3291210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cdda6640 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be270 0x563fe3291210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cdda6640 1 -- 172.21.15.16:0/218071598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be650 msgr2=0x563fe32c1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cdda6640 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be650 0x563fe32c1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46cdda6640 1 -- 172.21.15.16:0/218071598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fe31c7020 con 0x563fe3291750 2023-11-13T11:02:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.448+0000 7f46ccda4640 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be650 0x563fe32c1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.449+0000 7f46cdda6640 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe3291750 0x563fe32c1540 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099c0 tx=0x7f46c0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d9532d28636d96 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.449+0000 7f46aaffd640 1 -- 172.21.15.16:0/218071598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0030840 con 0x563fe3291750 2023-11-13T11:02:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.449+0000 7f46aaffd640 1 -- 172.21.15.16:0/218071598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0030d10 con 0x563fe3291750 2023-11-13T11:02:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.449+0000 7f46aaffd640 1 -- 172.21.15.16:0/218071598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0039650 con 0x563fe3291750 2023-11-13T11:02:42.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.450+0000 7f46d239e740 1 -- 172.21.15.16:0/218071598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe3291750 msgr2=0x563fe32c1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.450+0000 7f46d239e740 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe3291750 0x563fe32c1540 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f46c00099c0 tx=0x7f46c0030000 comp rx=0 tx=0).stop 2023-11-13T11:02:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.450+0000 7f46d239e740 1 -- 172.21.15.16:0/218071598 shutdown_connections 2023-11-13T11:02:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.450+0000 7f46d239e740 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe3291750 0x563fe32c1540 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.450+0000 7f46d239e740 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be650 0x563fe32c1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.450+0000 7f46d239e740 1 --2- 172.21.15.16:0/218071598 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be270 0x563fe3291210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.450+0000 7f46d239e740 1 -- 172.21.15.16:0/218071598 >> 172.21.15.16:0/218071598 conn(0x563fe3293480 msgr2=0x563fe32dc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.450+0000 7f46d239e740 1 -- 172.21.15.16:0/218071598 shutdown_connections 2023-11-13T11:02:42.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.450+0000 7f46d239e740 1 -- 172.21.15.16:0/218071598 wait complete. 2023-11-13T11:02:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.451+0000 7f46d239e740 1 -- start start 2023-11-13T11:02:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.451+0000 7f46d239e740 1 Processor -- start 2023-11-13T11:02:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46d239e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 0x563fe32c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46d239e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46d239e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe32c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46d239e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe3199480 con 0x563fe32c6650 2023-11-13T11:02:42.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46d239e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe31b6dc0 con 0x563fe32be270 2023-11-13T11:02:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46d239e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fe2ef5f90 con 0x563fe32be650 2023-11-13T11:02:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46d239e740 10 client.0 _ll_register_callbacks cb 0x563fe2f0ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46cdda6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe32c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46cdda6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe32c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48012/0 (socket says 172.21.15.16:48012) 2023-11-13T11:02:42.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.452+0000 7f46cdda6640 1 -- 172.21.15.16:0/3777386488 learned_addr learned my addr 172.21.15.16:0/3777386488 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46ccda4640 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 0x563fe32c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46cdda6640 1 -- 172.21.15.16:0/3777386488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 msgr2=0x563fe32c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46cdda6640 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46cdda6640 1 -- 172.21.15.16:0/3777386488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 msgr2=0x563fe32c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46cdda6640 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 0x563fe32c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46cdda6640 1 -- 172.21.15.16:0/3777386488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c80ce980 con 0x563fe32c6650 2023-11-13T11:02:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46ccda4640 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:02:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46cdda6640 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe32c6ad0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f46c002bf90 tx=0x7f46c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d672c60ebbd794 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:42.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.453+0000 7f46a3fff640 1 -- 172.21.15.16:0/3777386488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0030840 con 0x563fe32c6650 2023-11-13T11:02:42.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46d239e740 1 -- 172.21.15.16:0/3777386488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fe31f5e50 con 0x563fe32c6650 2023-11-13T11:02:42.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46a3fff640 1 -- 172.21.15.16:0/3777386488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f46c003e040 con 0x563fe32c6650 2023-11-13T11:02:42.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46a3fff640 1 -- 172.21.15.16:0/3777386488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46c0030e80 con 0x563fe32c6650 2023-11-13T11:02:42.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46d239e740 10 client.6387 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:42.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46d239e740 1 -- 172.21.15.16:0/3777386488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x563fe31fb4f0 con 0x563fe32c6650 2023-11-13T11:02:42.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46a3fff640 1 -- 172.21.15.16:0/3777386488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563fe31f5e50 con 0x563fe32c6650 2023-11-13T11:02:42.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46a3fff640 1 -- 172.21.15.16:0/3777386488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f46c0042070 con 0x563fe32c6650 2023-11-13T11:02:42.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46d239e740 10 client.6387 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:42.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46d239e740 10 client.6387 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46d239e740 20 client.6387 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46d239e740 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x563fe32f7130 0x563fe32f9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46d239e740 4 client.6387 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.454+0000 7f46d239e740 1 -- 172.21.15.16:0/3777386488 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fe31f5b70 con 0x563fe32f7130 2023-11-13T11:02:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.455+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x563fe32f7130 0x563fe32f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.455+0000 7f46a3fff640 10 client.6387 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.455+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x563fe32f7130 0x563fe32f9520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=790e5324d2b1bc1d server_cookie=9cb0173c66cd1941 in_seq=0 out_seq=0 2023-11-13T11:02:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46a3fff640 1 -- 172.21.15.16:0/3777386488 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fe31f5b70 con 0x563fe32f7130 2023-11-13T11:02:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46a3fff640 10 client.6387 handle_command_reply: tid=0 2023-11-13T11:02:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 client.6387 shutdown 2023-11-13T11:02:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 -- 172.21.15.16:0/3777386488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 msgr2=0x563fe32c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe32c6ad0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f46c002bf90 tx=0x7f46c0004730 comp rx=0 tx=0).stop 2023-11-13T11:02:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 -- 172.21.15.16:0/3777386488 shutdown_connections 2023-11-13T11:02:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x563fe32f7130 0x563fe32f9520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe32c6ad0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46a3fff640 0 client.6387 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 --2- 172.21.15.16:0/3777386488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 0x563fe32c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 -- 172.21.15.16:0/3777386488 >> 172.21.15.16:0/3777386488 conn(0x563fe3291750 msgr2=0x563fe3290970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.456+0000 7f46d239e740 1 -- 172.21.15.16:0/3777386488 shutdown_connections 2023-11-13T11:02:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.457+0000 7f46d239e740 1 -- 172.21.15.16:0/3777386488 wait complete. 2023-11-13T11:02:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.457+0000 7f46d239e740 20 client.6387 trim_cache size 0 max 16384 2023-11-13T11:02:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.473+0000 7f46d239e740 1 Processor -- start 2023-11-13T11:02:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.473+0000 7f46d239e740 1 -- start start 2023-11-13T11:02:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.474+0000 7f46d239e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32be270 0x563fe32eb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.474+0000 7f46d239e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32de630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.474+0000 7f46d239e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32c6650 0x563fe32c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.474+0000 7f46d239e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe32df110 con 0x563fe32be270 2023-11-13T11:02:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.474+0000 7f46d239e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe3290950 con 0x563fe32c6650 2023-11-13T11:02:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.474+0000 7f46d239e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fe3183100 con 0x563fe32be650 2023-11-13T11:02:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cd5a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32be270 0x563fe32eb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cd5a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32be270 0x563fe32eb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48026/0 (socket says 172.21.15.16:48026) 2023-11-13T11:02:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cd5a5640 1 -- 172.21.15.16:0/2739249479 learned_addr learned my addr 172.21.15.16:0/2739249479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cdda6640 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32c6650 0x563fe32c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46ccda4640 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cd5a5640 1 -- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 msgr2=0x563fe32de630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32de630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cd5a5640 1 -- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32c6650 msgr2=0x563fe32c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32c6650 0x563fe32c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cd5a5640 1 -- 172.21.15.16:0/2739249479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fe3198d10 con 0x563fe32be270 2023-11-13T11:02:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46ccda4640 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.475+0000 7f46cdda6640 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32c6650 0x563fe32c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.476+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32be270 0x563fe32eb3b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009890 tx=0x7f46bc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56f379d3b9e2144 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.476+0000 7f46ab7fe640 1 -- 172.21.15.16:0/2739249479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc016890 con 0x563fe32be270 2023-11-13T11:02:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.476+0000 7f46ab7fe640 1 -- 172.21.15.16:0/2739249479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc0169f0 con 0x563fe32be270 2023-11-13T11:02:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.476+0000 7f46ab7fe640 1 -- 172.21.15.16:0/2739249479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc016d60 con 0x563fe32be270 2023-11-13T11:02:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.477+0000 7f46d239e740 1 -- 172.21.15.16:0/2739249479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32be270 msgr2=0x563fe32eb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.477+0000 7f46d239e740 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32be270 0x563fe32eb3b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009890 tx=0x7f46bc015000 comp rx=0 tx=0).stop 2023-11-13T11:02:42.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.477+0000 7f46d239e740 1 -- 172.21.15.16:0/2739249479 shutdown_connections 2023-11-13T11:02:42.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.477+0000 7f46d239e740 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32c6650 0x563fe32c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.477+0000 7f46d239e740 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe32de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.477+0000 7f46d239e740 1 --2- 172.21.15.16:0/2739249479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32be270 0x563fe32eb3b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.477+0000 7f46d239e740 1 -- 172.21.15.16:0/2739249479 >> 172.21.15.16:0/2739249479 conn(0x563fe3291750 msgr2=0x563fe328c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.477+0000 7f46d239e740 1 -- 172.21.15.16:0/2739249479 shutdown_connections 2023-11-13T11:02:42.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.477+0000 7f46d239e740 1 -- 172.21.15.16:0/2739249479 wait complete. 2023-11-13T11:02:42.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.478+0000 7f46d239e740 1 -- start start 2023-11-13T11:02:42.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.478+0000 7f46d239e740 1 Processor -- start 2023-11-13T11:02:42.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46d239e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 0x563fe32bcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46d239e740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe3346310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46d239e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe3346850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46d239e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe33471d0 con 0x563fe32c6650 2023-11-13T11:02:42.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46d239e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe3295230 con 0x563fe32be270 2023-11-13T11:02:42.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46d239e740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fe3294be0 con 0x563fe32be650 2023-11-13T11:02:42.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46d239e740 10 client.0 _ll_register_callbacks cb 0x7f46c00096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:42.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46cdda6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe3346850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46cdda6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe3346850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48036/0 (socket says 172.21.15.16:48036) 2023-11-13T11:02:42.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.479+0000 7f46cdda6640 1 -- 172.21.15.16:0/2316982694 learned_addr learned my addr 172.21.15.16:0/2316982694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46cdda6640 1 -- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 msgr2=0x563fe3346310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 0x563fe32bcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46ccda4640 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe3346310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46cdda6640 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe3346310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46cdda6640 1 -- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 msgr2=0x563fe32bcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46cdda6640 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 0x563fe32bcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46cdda6640 1 -- 172.21.15.16:0/2316982694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46bc01e040 con 0x563fe32c6650 2023-11-13T11:02:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46ccda4640 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe3346310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 0x563fe32bcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46cdda6640 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe3346850 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f46c00090b0 tx=0x7f46c0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b5e4c6b0a65dd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46a3fff640 1 -- 172.21.15.16:0/2316982694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0039650 con 0x563fe32c6650 2023-11-13T11:02:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46a3fff640 1 -- 172.21.15.16:0/2316982694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f46c003e040 con 0x563fe32c6650 2023-11-13T11:02:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.480+0000 7f46d239e740 1 -- 172.21.15.16:0/2316982694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46c0030d10 con 0x563fe32c6650 2023-11-13T11:02:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46a3fff640 1 -- 172.21.15.16:0/2316982694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46c00397b0 con 0x563fe32c6650 2023-11-13T11:02:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46d239e740 10 client.6393 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46d239e740 1 -- 172.21.15.16:0/2316982694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x563fe31be300 con 0x563fe32c6650 2023-11-13T11:02:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46a3fff640 1 -- 172.21.15.16:0/2316982694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0030d10 con 0x563fe32c6650 2023-11-13T11:02:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46a3fff640 1 -- 172.21.15.16:0/2316982694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f46c0039910 con 0x563fe32c6650 2023-11-13T11:02:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46d239e740 10 client.6393 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46d239e740 10 client.6393 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46d239e740 20 client.6393 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46d239e740 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x563fe3346d90 0x563fe32ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46d239e740 4 client.6393 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-13T11:02:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46d239e740 1 -- 172.21.15.16:0/2316982694 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563fe2efd7e0 con 0x563fe3346d90 2023-11-13T11:02:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.481+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x563fe3346d90 0x563fe32ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.482+0000 7f46cd5a5640 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x563fe3346d90 0x563fe32ac040 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55be219794279ad7 server_cookie=f6079e010411fdca in_seq=0 out_seq=0 2023-11-13T11:02:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.482+0000 7f46a3fff640 10 client.6393 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.482+0000 7f46a3fff640 1 -- 172.21.15.16:0/2316982694 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563fe2efd7e0 con 0x563fe3346d90 2023-11-13T11:02:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.482+0000 7f46a3fff640 10 client.6393 handle_command_reply: tid=0 2023-11-13T11:02:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.482+0000 7f46d239e740 1 client.6393 shutdown 2023-11-13T11:02:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 -- 172.21.15.16:0/2316982694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 msgr2=0x563fe3346850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe3346850 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f46c00090b0 tx=0x7f46c0008030 comp rx=0 tx=0).stop 2023-11-13T11:02:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 -- 172.21.15.16:0/2316982694 shutdown_connections 2023-11-13T11:02:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x563fe3346d90 0x563fe32ac040 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fe32c6650 0x563fe3346850 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563fe32be650 0x563fe3346310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46a3fff640 0 client.6393 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 --2- 172.21.15.16:0/2316982694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fe32be270 0x563fe32bcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 -- 172.21.15.16:0/2316982694 >> 172.21.15.16:0/2316982694 conn(0x563fe3291750 msgr2=0x563fe3296ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 -- 172.21.15.16:0/2316982694 shutdown_connections 2023-11-13T11:02:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 1 -- 172.21.15.16:0/2316982694 wait complete. 2023-11-13T11:02:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.483+0000 7f46d239e740 20 client.6393 trim_cache size 0 max 16384 2023-11-13T11:02:42.502 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:42.502 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2023-11-13T11:02:42.502 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2023-11-13T11:02:42.503 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f46ac06ae40 msgr2=0x7f46ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f46ac06ae40 0x7f46ac06d300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009a80 tx=0x7f46bc0039d0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4133721297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 msgr2=0x7f46c81568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 0x7f46c81568c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c400 tx=0x7f46b400c8c0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4133721297 shutdown_connections 2023-11-13T11:02:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f46ac06ae40 0x7f46ac06d300 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46c815ce70 0x7f46c80b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d1850 0x7f46c81568c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 --2- 172.21.15.16:0/4133721297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f46c80cdbb0 0x7f46c81eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4133721297 >> 172.21.15.16:0/4133721297 conn(0x7f46c808c580 msgr2=0x7f46c809bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4133721297 shutdown_connections 2023-11-13T11:02:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.484+0000 7f46ce5a7640 1 -- 172.21.15.16:0/4133721297 wait complete. 2023-11-13T11:02:42.506 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-13T11:02:42.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2023-11-13T11:02:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.729+0000 7f2f70da9640 1 Processor -- start 2023-11-13T11:02:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.729+0000 7f2f70da9640 1 -- start start 2023-11-13T11:02:42.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.731+0000 7f2f70da9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c15cf90 0x7f2f6c0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.731+0000 7f2f70da9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f6c0b3b90 0x7f2f6c070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.731+0000 7f2f70da9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c0b2e00 con 0x7f2f6c06c3b0 2023-11-13T11:02:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.731+0000 7f2f70da9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c0b21f0 con 0x7f2f6c0b3b90 2023-11-13T11:02:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.731+0000 7f2f70da9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0b24d0 con 0x7f2f6c15cf90 2023-11-13T11:02:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.731+0000 7f2f69d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f6c06c3b0 0x7f2f6c0b3480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36462/0 (socket says 172.21.15.16:36462) 2023-11-13T11:02:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.731+0000 7f2f69d9b640 1 -- 172.21.15.16:0/2705287486 learned_addr learned my addr 172.21.15.16:0/2705287486 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.731+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/2705287486 >> v2:172.21.15.16:3300/0 conn(0x7f2f6c0b3b90 0x7f2f6c070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.732+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/2705287486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c15cf90 0x7f2f6c0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.732+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/2705287486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c15cf90 msgr2=0x7f2f6c0b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.732+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/2705287486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c15cf90 0x7f2f6c0b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.732+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/2705287486 >> v1:172.21.15.16:6789/0 conn(0x7f2f6c06c3b0 legacy=0x7f2f6c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.732+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/2705287486 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c0b27b0 con 0x7f2f6c0b3b90 2023-11-13T11:02:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.732+0000 7f2f6959a640 1 -- 172.21.15.16:0/2705287486 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633703227 0 0) 0x7f2f6c0b2e00 con 0x7f2f6c06c3b0 2023-11-13T11:02:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.733+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/2705287486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c0b3b90 0x7f2f6c070440 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009d90 tx=0x7f2f6402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1d1a7e70fbd449 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f6959a640 1 -- 172.21.15.16:0/2705287486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6402cc50 con 0x7f2f6c0b3b90 2023-11-13T11:02:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f6959a640 1 -- 172.21.15.16:0/2705287486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6402cdb0 con 0x7f2f6c0b3b90 2023-11-13T11:02:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f6959a640 1 -- 172.21.15.16:0/2705287486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64034660 con 0x7f2f6c0b3b90 2023-11-13T11:02:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f70da9640 1 -- 172.21.15.16:0/2705287486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c0b3b90 msgr2=0x7f2f6c070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f70da9640 1 --2- 172.21.15.16:0/2705287486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c0b3b90 0x7f2f6c070440 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009d90 tx=0x7f2f6402bea0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f70da9640 1 -- 172.21.15.16:0/2705287486 shutdown_connections 2023-11-13T11:02:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f70da9640 1 --2- 172.21.15.16:0/2705287486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c0b3b90 0x7f2f6c070440 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f70da9640 1 --2- 172.21.15.16:0/2705287486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c15cf90 0x7f2f6c0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f70da9640 1 -- 172.21.15.16:0/2705287486 >> 172.21.15.16:0/2705287486 conn(0x7f2f6c06a190 msgr2=0x7f2f6c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.734+0000 7f2f70da9640 1 -- 172.21.15.16:0/2705287486 shutdown_connections 2023-11-13T11:02:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.735+0000 7f2f70da9640 1 -- 172.21.15.16:0/2705287486 wait complete. 2023-11-13T11:02:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.736+0000 7f2f70da9640 1 Processor -- start 2023-11-13T11:02:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.737+0000 7f2f70da9640 1 -- start start 2023-11-13T11:02:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.737+0000 7f2f70da9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f6c06c3b0 0x7f2f6c1e8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.737+0000 7f2f70da9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c0b3b90 0x7f2f6c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f70da9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c15cf90 0x7f2f6c145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f70da9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c1e5d80 con 0x7f2f6c15cf90 2023-11-13T11:02:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f70da9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c160320 con 0x7f2f6c06c3b0 2023-11-13T11:02:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f70da9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c157060 con 0x7f2f6c0b3b90 2023-11-13T11:02:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c15cf90 0x7f2f6c145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c15cf90 0x7f2f6c145f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48054/0 (socket says 172.21.15.16:48054) 2023-11-13T11:02:42.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/333164080 learned_addr learned my addr 172.21.15.16:0/333164080 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f69d9b640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c0b3b90 0x7f2f6c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f6c06c3b0 0x7f2f6c1e8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c0b3b90 msgr2=0x7f2f6c0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c0b3b90 0x7f2f6c0acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f6c06c3b0 msgr2=0x7f2f6c1e8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f6c06c3b0 0x7f2f6c1e8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.738+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/333164080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64009a70 con 0x7f2f6c15cf90 2023-11-13T11:02:42.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.739+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f6c06c3b0 0x7f2f6c1e8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:42.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.739+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c15cf90 0x7f2f6c145f80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009eb0 tx=0x7f2f64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.739+0000 7f2f5affd640 1 -- 172.21.15.16:0/333164080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64040650 con 0x7f2f6c15cf90 2023-11-13T11:02:42.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.739+0000 7f2f5affd640 1 -- 172.21.15.16:0/333164080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f640407d0 con 0x7f2f6c15cf90 2023-11-13T11:02:42.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.739+0000 7f2f5affd640 1 -- 172.21.15.16:0/333164080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64040b80 con 0x7f2f6c15cf90 2023-11-13T11:02:42.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.739+0000 7f2f70da9640 1 -- 172.21.15.16:0/333164080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c1e5700 con 0x7f2f6c15cf90 2023-11-13T11:02:42.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.739+0000 7f2f70da9640 1 -- 172.21.15.16:0/333164080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c1e59a0 con 0x7f2f6c15cf90 2023-11-13T11:02:42.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.741+0000 7f2f5affd640 1 -- 172.21.15.16:0/333164080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f64040ce0 con 0x7f2f6c15cf90 2023-11-13T11:02:42.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.742+0000 7f2f5affd640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2f4c06ae40 0x7f2f4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.742+0000 7f2f5affd640 1 -- 172.21.15.16:0/333164080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f2f640ac5e0 con 0x7f2f6c15cf90 2023-11-13T11:02:42.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.743+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2f4c06ae40 0x7f2f4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.743+0000 7f2f73a1b740 1 Processor -- start 2023-11-13T11:02:42.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.743+0000 7f2f73a1b740 1 -- start start 2023-11-13T11:02:42.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.744+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2f4c06ae40 0x7f2f4c06d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006fd0 tx=0x7f2f5c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.744+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 0x56152977c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.744+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152974c310 0x561529779560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.744+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x561529779aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.744+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615296bfd20 con 0x56152974fe70 2023-11-13T11:02:42.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.744+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615296bd410 con 0x56152977c2f0 2023-11-13T11:02:42.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.744+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615296b8d00 con 0x56152974c310 2023-11-13T11:02:42.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x561529779aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x561529779aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48060/0 (socket says 172.21.15.16:48060) 2023-11-13T11:02:42.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/2556371030 learned_addr learned my addr 172.21.15.16:0/2556371030 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/2556371030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 0x56152977c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f69d9b640 1 --2- 172.21.15.16:0/2556371030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152974c310 0x561529779560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/2556371030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152974c310 msgr2=0x561529779560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/2556371030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152974c310 0x561529779560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/2556371030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 msgr2=0x56152977c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/2556371030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 0x56152977c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/2556371030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615296724f0 con 0x56152974fe70 2023-11-13T11:02:42.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/2556371030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x561529779aa0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f640776c0 tx=0x7f2f6407ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d978867e89ad0ba0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.745+0000 7f2f4bfff640 1 -- 172.21.15.16:0/2556371030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6407f120 con 0x56152974fe70 2023-11-13T11:02:42.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f4bfff640 1 -- 172.21.15.16:0/2556371030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6407b1c0 con 0x56152974fe70 2023-11-13T11:02:42.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f4bfff640 1 -- 172.21.15.16:0/2556371030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6407f310 con 0x56152974fe70 2023-11-13T11:02:42.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2556371030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 msgr2=0x561529779aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/2556371030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x561529779aa0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f640776c0 tx=0x7f2f6407ab20 comp rx=0 tx=0).stop 2023-11-13T11:02:42.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2556371030 shutdown_connections 2023-11-13T11:02:42.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/2556371030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x561529779aa0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/2556371030 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152974c310 0x561529779560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/2556371030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 0x56152977c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2556371030 >> 172.21.15.16:0/2556371030 conn(0x56152974e580 msgr2=0x561529797100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2556371030 shutdown_connections 2023-11-13T11:02:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.746+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2556371030 wait complete. 2023-11-13T11:02:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.747+0000 7f2f73a1b740 1 -- start start 2023-11-13T11:02:42.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.747+0000 7f2f73a1b740 1 Processor -- start 2023-11-13T11:02:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.748+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.748+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 0x5615297a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5615297a7860 0x5615297a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561529672fa0 con 0x56152974fe70 2023-11-13T11:02:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561529672c40 con 0x56152977c2f0 2023-11-13T11:02:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615293b1f90 con 0x5615297a7860 2023-11-13T11:02:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f73a1b740 10 client.0 _ll_register_callbacks cb 0x561529677370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48070/0 (socket says 172.21.15.16:48070) 2023-11-13T11:02:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6a59c640 1 -- 172.21.15.16:0/3186783486 learned_addr learned my addr 172.21.15.16:0/3186783486 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f69d9b640 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 0x5615297a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5615297a7860 0x5615297a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6a59c640 1 -- 172.21.15.16:0/3186783486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5615297a7860 msgr2=0x5615297a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5615297a7860 0x5615297a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6a59c640 1 -- 172.21.15.16:0/3186783486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 msgr2=0x5615297a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 0x5615297a7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6a59c640 1 -- 172.21.15.16:0/3186783486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c1e59a0 con 0x56152974fe70 2023-11-13T11:02:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.749+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5615297a7860 0x5615297a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.750+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977c6d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00ea20 tx=0x7f2f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce6301405278472 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.750+0000 7f2f497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.750+0000 7f2f497fa640 1 -- 172.21.15.16:0/3186783486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00eee0 con 0x56152974fe70 2023-11-13T11:02:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.750+0000 7f2f497fa640 1 -- 172.21.15.16:0/3186783486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f2f5c013430 con 0x56152974fe70 2023-11-13T11:02:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.750+0000 7f2f73a1b740 1 -- 172.21.15.16:0/3186783486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615296b1ce0 con 0x56152974fe70 2023-11-13T11:02:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f497fa640 1 -- 172.21.15.16:0/3186783486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f5c009e50 con 0x56152974fe70 2023-11-13T11:02:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f73a1b740 10 client.6408 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f73a1b740 1 -- 172.21.15.16:0/3186783486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5615296b7380 con 0x56152974fe70 2023-11-13T11:02:42.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f497fa640 1 -- 172.21.15.16:0/3186783486 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5615296b1ce0 con 0x56152974fe70 2023-11-13T11:02:42.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f497fa640 1 -- 172.21.15.16:0/3186783486 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2f5c0135f0 con 0x56152974fe70 2023-11-13T11:02:42.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f73a1b740 10 client.6408 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:42.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f73a1b740 10 client.6408 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:42.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f73a1b740 20 client.6408 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5615297b3110 0x5615297b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f73a1b740 4 client.6408 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f73a1b740 1 -- 172.21.15.16:0/3186783486 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615296b1a00 con 0x5615297b3110 2023-11-13T11:02:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.751+0000 7f2f69d9b640 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5615297b3110 0x5615297b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.752+0000 7f2f69d9b640 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5615297b3110 0x5615297b5500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae28894d15fcd717 server_cookie=9e27005fac79a985 in_seq=0 out_seq=0 2023-11-13T11:02:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.752+0000 7f2f497fa640 10 client.6408 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f497fa640 1 -- 172.21.15.16:0/3186783486 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615296b1a00 con 0x5615297b3110 2023-11-13T11:02:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f497fa640 10 client.6408 handle_command_reply: tid=0 2023-11-13T11:02:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 client.6408 shutdown 2023-11-13T11:02:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 -- 172.21.15.16:0/3186783486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 msgr2=0x56152977c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977c6d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00ea20 tx=0x7f2f5c0046c0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 -- 172.21.15.16:0/3186783486 shutdown_connections 2023-11-13T11:02:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5615297a7860 0x5615297a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152977c2f0 0x5615297a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5615297b3110 0x5615297b5500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/3186783486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977c6d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 -- 172.21.15.16:0/3186783486 >> 172.21.15.16:0/3186783486 conn(0x56152974c310 msgr2=0x56152979a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f497fa640 0 client.6408 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 -- 172.21.15.16:0/3186783486 shutdown_connections 2023-11-13T11:02:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.753+0000 7f2f73a1b740 1 -- 172.21.15.16:0/3186783486 wait complete. 2023-11-13T11:02:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.754+0000 7f2f73a1b740 20 client.6408 trim_cache size 0 max 16384 2023-11-13T11:02:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.766+0000 7f2f73a1b740 1 Processor -- start 2023-11-13T11:02:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.767+0000 7f2f73a1b740 1 -- start start 2023-11-13T11:02:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.767+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.767+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152977c2f0 0x56152977f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.767+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615297a7860 0x5615297b9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.767+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615297514e0 con 0x56152974fe70 2023-11-13T11:02:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.767+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615297a6730 con 0x5615297a7860 2023-11-13T11:02:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.767+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x561529794670 con 0x56152977c2f0 2023-11-13T11:02:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48074/0 (socket says 172.21.15.16:48074) 2023-11-13T11:02:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f6a59c640 1 -- 172.21.15.16:0/2467003313 learned_addr learned my addr 172.21.15.16:0/2467003313 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/2467003313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615297a7860 0x5615297b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f69d9b640 1 --2- 172.21.15.16:0/2467003313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152977c2f0 0x56152977f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f6a59c640 1 -- 172.21.15.16:0/2467003313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152977c2f0 msgr2=0x56152977f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/2467003313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152977c2f0 0x56152977f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f6a59c640 1 -- 172.21.15.16:0/2467003313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615297a7860 msgr2=0x5615297b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/2467003313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615297a7860 0x5615297b9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.768+0000 7f2f6a59c640 1 -- 172.21.15.16:0/2467003313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561529654cf0 con 0x56152974fe70 2023-11-13T11:02:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.769+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/2467003313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977ef90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c013d80 tx=0x7f2f5c018030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23e310b391581ca server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.769+0000 7f2f58ff9640 1 -- 172.21.15.16:0/2467003313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00eee0 con 0x56152974fe70 2023-11-13T11:02:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.769+0000 7f2f58ff9640 1 -- 172.21.15.16:0/2467003313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c01e040 con 0x56152974fe70 2023-11-13T11:02:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.769+0000 7f2f58ff9640 1 -- 172.21.15.16:0/2467003313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0043d0 con 0x56152974fe70 2023-11-13T11:02:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.770+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2467003313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 msgr2=0x56152977ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.770+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/2467003313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977ef90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c013d80 tx=0x7f2f5c018030 comp rx=0 tx=0).stop 2023-11-13T11:02:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.770+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2467003313 shutdown_connections 2023-11-13T11:02:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.770+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/2467003313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5615297a7860 0x5615297b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.770+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/2467003313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152977c2f0 0x56152977f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.770+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/2467003313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56152974fe70 0x56152977ef90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.770+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2467003313 >> 172.21.15.16:0/2467003313 conn(0x56152974c310 msgr2=0x56152974b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.770+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2467003313 shutdown_connections 2023-11-13T11:02:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.770+0000 7f2f73a1b740 1 -- 172.21.15.16:0/2467003313 wait complete. 2023-11-13T11:02:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.771+0000 7f2f73a1b740 1 -- start start 2023-11-13T11:02:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.771+0000 7f2f73a1b740 1 Processor -- start 2023-11-13T11:02:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152974fe70 0x5615297b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152977c2f0 0x5615297fc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f73a1b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615297a7860 0x56152977ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615297794d0 con 0x5615297a7860 2023-11-13T11:02:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561529779160 con 0x56152974fe70 2023-11-13T11:02:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f73a1b740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615297ac8b0 con 0x56152977c2f0 2023-11-13T11:02:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f73a1b740 10 client.0 _ll_register_callbacks cb 0x7f2f5c009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615297a7860 0x56152977ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615297a7860 0x56152977ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48078/0 (socket says 172.21.15.16:48078) 2023-11-13T11:02:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/4099038274 learned_addr learned my addr 172.21.15.16:0/4099038274 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/4099038274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152977c2f0 msgr2=0x5615297fc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152977c2f0 0x5615297fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152974fe70 0x5615297b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/4099038274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152974fe70 msgr2=0x5615297b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152974fe70 0x5615297b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.772+0000 7f2f6ad9d640 1 -- 172.21.15.16:0/4099038274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615296b7380 con 0x5615297a7860 2023-11-13T11:02:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.773+0000 7f2f6ad9d640 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615297a7860 0x56152977ef90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009e80 tx=0x7f2f640ac040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5693b2f29b453100 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.773+0000 7f2f497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.773+0000 7f2f497fa640 1 -- 172.21.15.16:0/4099038274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6407b620 con 0x5615297a7860 2023-11-13T11:02:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.773+0000 7f2f497fa640 1 -- 172.21.15.16:0/4099038274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f2f640a9040 con 0x5615297a7860 2023-11-13T11:02:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.773+0000 7f2f73a1b740 1 -- 172.21.15.16:0/4099038274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f6407b1c0 con 0x5615297a7860 2023-11-13T11:02:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.773+0000 7f2f497fa640 1 -- 172.21.15.16:0/4099038274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f6407f120 con 0x5615297a7860 2023-11-13T11:02:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.773+0000 7f2f73a1b740 10 client.6414 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.773+0000 7f2f73a1b740 1 -- 172.21.15.16:0/4099038274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x56152967a2e0 con 0x5615297a7860 2023-11-13T11:02:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.773+0000 7f2f497fa640 1 -- 172.21.15.16:0/4099038274 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6407b1c0 con 0x5615297a7860 2023-11-13T11:02:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.774+0000 7f2f497fa640 1 -- 172.21.15.16:0/4099038274 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2f64050020 con 0x5615297a7860 2023-11-13T11:02:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.774+0000 7f2f73a1b740 10 client.6414 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.774+0000 7f2f73a1b740 10 client.6414 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.774+0000 7f2f73a1b740 20 client.6414 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.774+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x56152977f4d0 0x561529802580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.774+0000 7f2f73a1b740 4 client.6414 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-11-13T11:02:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.774+0000 7f2f73a1b740 1 -- 172.21.15.16:0/4099038274 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615293b97e0 con 0x56152977f4d0 2023-11-13T11:02:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.774+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x56152977f4d0 0x561529802580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f6a59c640 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x56152977f4d0 0x561529802580 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbfe1e7e61125a17 server_cookie=51f2dff378cfb18b in_seq=0 out_seq=0 2023-11-13T11:02:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f497fa640 10 client.6414 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f497fa640 1 -- 172.21.15.16:0/4099038274 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+167 (crc 0 0 0) 0x5615293b97e0 con 0x56152977f4d0 2023-11-13T11:02:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f497fa640 10 client.6414 handle_command_reply: tid=0 2023-11-13T11:02:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 client.6414 shutdown 2023-11-13T11:02:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 -- 172.21.15.16:0/4099038274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615297a7860 msgr2=0x56152977ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615297a7860 0x56152977ef90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009e80 tx=0x7f2f640ac040 comp rx=0 tx=0).stop 2023-11-13T11:02:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 -- 172.21.15.16:0/4099038274 shutdown_connections 2023-11-13T11:02:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x56152977f4d0 0x561529802580 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615297a7860 0x56152977ef90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56152977c2f0 0x5615297fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 --2- 172.21.15.16:0/4099038274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56152974fe70 0x5615297b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 -- 172.21.15.16:0/4099038274 >> 172.21.15.16:0/4099038274 conn(0x56152974c310 msgr2=0x561529752230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f497fa640 0 client.6414 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 -- 172.21.15.16:0/4099038274 shutdown_connections 2023-11-13T11:02:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.775+0000 7f2f73a1b740 1 -- 172.21.15.16:0/4099038274 wait complete. 2023-11-13T11:02:42.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f73a1b740 20 client.6414 trim_cache size 0 max 16384 2023-11-13T11:02:42.796 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-13T11:02:42.796 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:02:42.796 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2023-11-13T11:02:42.796 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 909192629, 2023-11-13T11:02:42.797 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511630410, 2023-11-13T11:02:42.797 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2023-11-13T11:02:42.797 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:02:42.797 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-13T11:02:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 -- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2f4c06ae40 msgr2=0x7f2f4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2f4c06ae40 0x7f2f4c06d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006fd0 tx=0x7f2f5c008040 comp rx=0 tx=0).stop 2023-11-13T11:02:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 -- 172.21.15.16:0/333164080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c15cf90 msgr2=0x7f2f6c145f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c15cf90 0x7f2f6c145f80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009eb0 tx=0x7f2f64004a40 comp rx=0 tx=0).stop 2023-11-13T11:02:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 -- 172.21.15.16:0/333164080 shutdown_connections 2023-11-13T11:02:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2f4c06ae40 0x7f2f4c06d300 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6c15cf90 0x7f2f6c145f80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2f6c0b3b90 0x7f2f6c0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 --2- 172.21.15.16:0/333164080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f6c06c3b0 0x7f2f6c1e8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 -- 172.21.15.16:0/333164080 >> 172.21.15.16:0/333164080 conn(0x7f2f6c06a190 msgr2=0x7f2f6c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 -- 172.21.15.16:0/333164080 shutdown_connections 2023-11-13T11:02:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:42.776+0000 7f2f70da9640 1 -- 172.21.15.16:0/333164080 wait complete. 2023-11-13T11:02:42.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.016+0000 7f726567e640 1 Processor -- start 2023-11-13T11:02:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.016+0000 7f726567e640 1 -- start start 2023-11-13T11:02:43.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f726567e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f726567e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72600716b0 0x7f726006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f726567e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7260158c20 con 0x7f726006a190 2023-11-13T11:02:43.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f726567e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72600b1830 con 0x7f72600716b0 2023-11-13T11:02:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f726567e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726015b780 con 0x7f72600cb190 2023-11-13T11:02:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f7257fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72600716b0 0x7f726006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f7264e7d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f726006a190 0x7f72600b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39944/0 (socket says 172.21.15.16:39944) 2023-11-13T11:02:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f7264e7d640 1 -- 172.21.15.16:0/859205879 learned_addr learned my addr 172.21.15.16:0/859205879 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f7257fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72600716b0 0x7f726006f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38590/0 (socket says 172.21.15.16:38590) 2023-11-13T11:02:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f725ffff640 1 --2- 172.21.15.16:0/859205879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f7257fff640 1 -- 172.21.15.16:0/859205879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 msgr2=0x7f726015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.018+0000 7f7257fff640 1 --2- 172.21.15.16:0/859205879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.019+0000 7f7257fff640 1 -- 172.21.15.16:0/859205879 >> v1:172.21.15.16:6789/0 conn(0x7f726006a190 legacy=0x7f72600b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.019+0000 7f7257fff640 1 -- 172.21.15.16:0/859205879 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726015ba60 con 0x7f72600716b0 2023-11-13T11:02:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.019+0000 7f725f7fe640 1 -- 172.21.15.16:0/859205879 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647263270 0 0) 0x7f7260158c20 con 0x7f726006a190 2023-11-13T11:02:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.019+0000 7f725ffff640 1 --2- 172.21.15.16:0/859205879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.020+0000 7f7257fff640 1 --2- 172.21.15.16:0/859205879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 0x7f726006f970 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f72480097a0 tx=0x7f724802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fe9c345b89e5d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.020+0000 7f725f7fe640 1 -- 172.21.15.16:0/859205879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724802d040 con 0x7f72600716b0 2023-11-13T11:02:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.020+0000 7f725f7fe640 1 -- 172.21.15.16:0/859205879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7248031810 con 0x7f72600716b0 2023-11-13T11:02:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.020+0000 7f725f7fe640 1 -- 172.21.15.16:0/859205879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248031ba0 con 0x7f72600716b0 2023-11-13T11:02:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.021+0000 7f726567e640 1 -- 172.21.15.16:0/859205879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 msgr2=0x7f726006f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.021+0000 7f726567e640 1 --2- 172.21.15.16:0/859205879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 0x7f726006f970 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f72480097a0 tx=0x7f724802b8b0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.021+0000 7f726567e640 1 -- 172.21.15.16:0/859205879 shutdown_connections 2023-11-13T11:02:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.021+0000 7f726567e640 1 --2- 172.21.15.16:0/859205879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 0x7f726006f970 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.021+0000 7f726567e640 1 --2- 172.21.15.16:0/859205879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.021+0000 7f726567e640 1 -- 172.21.15.16:0/859205879 >> 172.21.15.16:0/859205879 conn(0x7f72600b04a0 msgr2=0x7f72600b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:43.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.021+0000 7f726567e640 1 -- 172.21.15.16:0/859205879 shutdown_connections 2023-11-13T11:02:43.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.021+0000 7f726567e640 1 -- 172.21.15.16:0/859205879 wait complete. 2023-11-13T11:02:43.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.023+0000 7f726567e640 1 Processor -- start 2023-11-13T11:02:43.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.023+0000 7f726567e640 1 -- start start 2023-11-13T11:02:43.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.024+0000 7f726567e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f726006a190 0x7f726014ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.024+0000 7f726567e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 0x7f726006c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.024+0000 7f726567e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.024+0000 7f726567e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72600c7c50 con 0x7f72600716b0 2023-11-13T11:02:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.024+0000 7f726567e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72601f69b0 con 0x7f726006a190 2023-11-13T11:02:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.024+0000 7f726567e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72601f6c90 con 0x7f72600cb190 2023-11-13T11:02:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.024+0000 7f7257fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 0x7f726006c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.024+0000 7f7257fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 0x7f726006c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38604/0 (socket says 172.21.15.16:38604) 2023-11-13T11:02:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.024+0000 7f7257fff640 1 -- 172.21.15.16:0/3221289476 learned_addr learned my addr 172.21.15.16:0/3221289476 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f7264e7d640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f725ffff640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f726006a190 0x7f726014ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f7257fff640 1 -- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 msgr2=0x7f726015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f7257fff640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f7257fff640 1 -- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f726006a190 msgr2=0x7f726014ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f7257fff640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f726006a190 0x7f726014ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f7257fff640 1 -- 172.21.15.16:0/3221289476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7248009400 con 0x7f72600716b0 2023-11-13T11:02:43.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f725ffff640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f726006a190 0x7f726014ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:43.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f7264e7d640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f7257fff640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 0x7f726006c530 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7248000c00 tx=0x7f72480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f725d7fa640 1 -- 172.21.15.16:0/3221289476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724803bae0 con 0x7f72600716b0 2023-11-13T11:02:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f725d7fa640 1 -- 172.21.15.16:0/3221289476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724803bc40 con 0x7f72600716b0 2023-11-13T11:02:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f725d7fa640 1 -- 172.21.15.16:0/3221289476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248004030 con 0x7f72600716b0 2023-11-13T11:02:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.025+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72601f6f70 con 0x7f72600716b0 2023-11-13T11:02:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.026+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726006ca70 con 0x7f72600716b0 2023-11-13T11:02:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.027+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72600c7e00 con 0x7f72600716b0 2023-11-13T11:02:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.027+0000 7f725d7fa640 1 -- 172.21.15.16:0/3221289476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7248043070 con 0x7f72600716b0 2023-11-13T11:02:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.028+0000 7f725d7fa640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f724006ae70 0x7f724006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.028+0000 7f725d7fa640 1 -- 172.21.15.16:0/3221289476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f72480ab710 con 0x7f72600716b0 2023-11-13T11:02:43.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.029+0000 7f725ffff640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f724006ae70 0x7f724006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.030+0000 7f725ffff640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f724006ae70 0x7f724006d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7250006fd0 tx=0x7f7250008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.033+0000 7f725d7fa640 1 -- 172.21.15.16:0/3221289476 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72600c7e00 con 0x7f72600716b0 2023-11-13T11:02:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.271+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72600c7e00 con 0x7f72600716b0 2023-11-13T11:02:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.272+0000 7f725d7fa640 1 -- 172.21.15.16:0/3221289476 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f72600c7e00 con 0x7f72600716b0 2023-11-13T11:02:43.278 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:43.278 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:39.012251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-11-13T11:02:43.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.275+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f724006ae70 msgr2=0x7f724006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.275+0000 7f726567e640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f724006ae70 0x7f724006d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7250006fd0 tx=0x7f7250008040 comp rx=0 tx=0).stop 2023-11-13T11:02:43.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.275+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 msgr2=0x7f726006c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.275+0000 7f726567e640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 0x7f726006c530 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7248000c00 tx=0x7f72480046c0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.276+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 shutdown_connections 2023-11-13T11:02:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.276+0000 7f726567e640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f724006ae70 0x7f724006d330 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.276+0000 7f726567e640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72600cb190 0x7f726015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.276+0000 7f726567e640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72600716b0 0x7f726006c530 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.276+0000 7f726567e640 1 --2- 172.21.15.16:0/3221289476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f726006a190 0x7f726014ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.276+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 >> 172.21.15.16:0/3221289476 conn(0x7f72600b04a0 msgr2=0x7f72600ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.276+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 shutdown_connections 2023-11-13T11:02:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.276+0000 7f726567e640 1 -- 172.21.15.16:0/3221289476 wait complete. 2023-11-13T11:02:43.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 909192629 2023-11-13T11:02:43.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.509+0000 7fa5471ff640 1 Processor -- start 2023-11-13T11:02:43.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.510+0000 7fa5471ff640 1 -- start start 2023-11-13T11:02:43.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.511+0000 7fa5471ff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa54015bcc0 0x7fa5400cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.511+0000 7fa5471ff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400ce090 0x7fa5400cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.512+0000 7fa5471ff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54006d010 con 0x7fa54015b8e0 2023-11-13T11:02:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.512+0000 7fa5471ff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5400ce510 con 0x7fa54015bcc0 2023-11-13T11:02:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.512+0000 7fa5471ff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54006b860 con 0x7fa5400ce090 2023-11-13T11:02:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.512+0000 7fa5461fd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa54015b8e0 0x7fa54006c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39954/0 (socket says 172.21.15.16:39954) 2023-11-13T11:02:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.512+0000 7fa5461fd640 1 -- 172.21.15.16:0/2309974367 learned_addr learned my addr 172.21.15.16:0/2309974367 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.512+0000 7fa5459fc640 1 --2- 172.21.15.16:0/2309974367 >> v2:172.21.15.184:3300/0 conn(0x7fa54015bcc0 0x7fa5400cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.512+0000 7fa5469fe640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400ce090 0x7fa5400cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.512+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970220170 0 0) 0x7fa54006d010 con 0x7fa54015b8e0 2023-11-13T11:02:43.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.512+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa52c003520 con 0x7fa54015b8e0 2023-11-13T11:02:43.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1387024159 0 0) 0x7fa52c003520 con 0x7fa54015b8e0 2023-11-13T11:02:43.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400ce090 msgr2=0x7fa5400cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400ce090 0x7fa5400cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 >> v2:172.21.15.184:3300/0 conn(0x7fa54015bcc0 msgr2=0x7fa5400cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 --2- 172.21.15.16:0/2309974367 >> v2:172.21.15.184:3300/0 conn(0x7fa54015bcc0 0x7fa5400cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54006bb40 con 0x7fa54015b8e0 2023-11-13T11:02:43.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fa53c001670 con 0x7fa54015b8e0 2023-11-13T11:02:43.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 >> v1:172.21.15.16:6789/0 conn(0x7fa54015b8e0 legacy=0x7fa54006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c003970 0x7fa52c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400ce090 0x7fa52c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa52c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400ce510 con 0x7fa52c004d00 2023-11-13T11:02:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54006b860 con 0x7fa52c003970 2023-11-13T11:02:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c00b980 con 0x7fa5400ce090 2023-11-13T11:02:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.513+0000 7fa5469fe640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa52c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.514+0000 7fa5459fc640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400ce090 0x7fa52c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.514+0000 7fa5461fd640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c003970 0x7fa52c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.514+0000 7fa5469fe640 1 -- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400ce090 msgr2=0x7fa52c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.514+0000 7fa5469fe640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400ce090 0x7fa52c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.514+0000 7fa5469fe640 1 -- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c003970 msgr2=0x7fa52c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.514+0000 7fa5469fe640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c003970 0x7fa52c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.514+0000 7fa5469fe640 1 -- 172.21.15.16:0/2309974367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c003520 con 0x7fa52c004d00 2023-11-13T11:02:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.514+0000 7fa5459fc640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa5400ce090 0x7fa52c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.514+0000 7fa5461fd640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c003970 0x7fa52c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:43.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.515+0000 7fa5469fe640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa52c005180 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa538009fc0 tx=0x7fa52c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c2a9d5112f6c58 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.515+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53802cbf0 con 0x7fa52c004d00 2023-11-13T11:02:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.516+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53800f040 con 0x7fa52c004d00 2023-11-13T11:02:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.516+0000 7fa5451fb640 1 -- 172.21.15.16:0/2309974367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538034440 con 0x7fa52c004d00 2023-11-13T11:02:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.516+0000 7fa5471ff640 1 -- 172.21.15.16:0/2309974367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 msgr2=0x7fa52c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.516+0000 7fa5471ff640 1 --2- 172.21.15.16:0/2309974367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa52c005180 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa538009fc0 tx=0x7fa52c0057f0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.516+0000 7fa5469fe640 1 -- 172.21.15.16:0/2309974367 reap_dead start 2023-11-13T11:02:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.516+0000 7fa5471ff640 1 -- 172.21.15.16:0/2309974367 shutdown_connections 2023-11-13T11:02:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.516+0000 7fa5471ff640 1 -- 172.21.15.16:0/2309974367 >> 172.21.15.16:0/2309974367 conn(0x7fa5400b0970 msgr2=0x7fa5400b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.516+0000 7fa5471ff640 1 -- 172.21.15.16:0/2309974367 shutdown_connections 2023-11-13T11:02:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.516+0000 7fa5471ff640 1 -- 172.21.15.16:0/2309974367 wait complete. 2023-11-13T11:02:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.518+0000 7fa5471ff640 1 Processor -- start 2023-11-13T11:02:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.518+0000 7fa5471ff640 1 -- start start 2023-11-13T11:02:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.519+0000 7fa5471ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa54013e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.519+0000 7fa5471ff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540152550 0x7fa5401416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.519+0000 7fa5471ff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa540152f20 0x7fa54015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.519+0000 7fa5471ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5401412c0 con 0x7fa52c004d00 2023-11-13T11:02:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.519+0000 7fa5471ff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa540153420 con 0x7fa540152550 2023-11-13T11:02:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.519+0000 7fa5471ff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa540153700 con 0x7fa540152f20 2023-11-13T11:02:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5461fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa54013e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5461fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa54013e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38630/0 (socket says 172.21.15.16:38630) 2023-11-13T11:02:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5461fd640 1 -- 172.21.15.16:0/608274130 learned_addr learned my addr 172.21.15.16:0/608274130 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5461fd640 1 -- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa540152f20 msgr2=0x7fa54015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5461fd640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa540152f20 0x7fa54015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5461fd640 1 -- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540152550 msgr2=0x7fa5401416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5459fc640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540152550 0x7fa5401416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5461fd640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540152550 0x7fa5401416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5461fd640 1 -- 172.21.15.16:0/608274130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5401539e0 con 0x7fa52c004d00 2023-11-13T11:02:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.520+0000 7fa5461fd640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa54013e440 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00d740 tx=0x7fa53c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.521+0000 7fa536ffd640 1 -- 172.21.15.16:0/608274130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0074e0 con 0x7fa52c004d00 2023-11-13T11:02:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.521+0000 7fa536ffd640 1 -- 172.21.15.16:0/608274130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53c007640 con 0x7fa52c004d00 2023-11-13T11:02:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.521+0000 7fa536ffd640 1 -- 172.21.15.16:0/608274130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0049e0 con 0x7fa52c004d00 2023-11-13T11:02:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.521+0000 7fa5471ff640 1 -- 172.21.15.16:0/608274130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa540140ee0 con 0x7fa52c004d00 2023-11-13T11:02:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.521+0000 7fa5471ff640 1 -- 172.21.15.16:0/608274130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa540141c20 con 0x7fa52c004d00 2023-11-13T11:02:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.523+0000 7fa536ffd640 1 -- 172.21.15.16:0/608274130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa53c004b40 con 0x7fa52c004d00 2023-11-13T11:02:43.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.524+0000 7fa536ffd640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa52406ae40 0x7fa52406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.524+0000 7fa536ffd640 1 -- 172.21.15.16:0/608274130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fa53c088f50 con 0x7fa52c004d00 2023-11-13T11:02:43.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.525+0000 7fa5459fc640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa52406ae40 0x7fa52406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.525+0000 7fa54aff2740 1 Processor -- start 2023-11-13T11:02:43.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.525+0000 7fa54aff2740 1 -- start start 2023-11-13T11:02:43.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa5459fc640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa52406ae40 0x7fa52406d300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa540155c30 tx=0x7fa530008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 0x55773395d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x55773398cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773395d660 0x55773398d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa54aff2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773391dae0 con 0x55773398a560 2023-11-13T11:02:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa54aff2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577338cfe40 con 0x55773395d660 2023-11-13T11:02:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa54aff2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577338cd530 con 0x55773398a180 2023-11-13T11:02:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa5459fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x55773398cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa5459fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x55773398cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38638/0 (socket says 172.21.15.16:38638) 2023-11-13T11:02:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.526+0000 7fa5459fc640 1 -- 172.21.15.16:0/2849495631 learned_addr learned my addr 172.21.15.16:0/2849495631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.527+0000 7fa5469fe640 1 --2- 172.21.15.16:0/2849495631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773395d660 0x55773398d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.527+0000 7fa5461fd640 1 --2- 172.21.15.16:0/2849495631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 0x55773395d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.527+0000 7fa5459fc640 1 -- 172.21.15.16:0/2849495631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 msgr2=0x55773395d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.527+0000 7fa5459fc640 1 --2- 172.21.15.16:0/2849495631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 0x55773395d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.527+0000 7fa5459fc640 1 -- 172.21.15.16:0/2849495631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773395d660 msgr2=0x55773398d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.527+0000 7fa5459fc640 1 --2- 172.21.15.16:0/2849495631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773395d660 0x55773398d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.527+0000 7fa5459fc640 1 -- 172.21.15.16:0/2849495631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55773388b8f0 con 0x55773398a560 2023-11-13T11:02:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.527+0000 7fa5459fc640 1 --2- 172.21.15.16:0/2849495631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x55773398cf10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa530011df0 tx=0x7fa530014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451c31f167bcbc server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.527+0000 7fa523fff640 1 -- 172.21.15.16:0/2849495631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530011e20 con 0x55773398a560 2023-11-13T11:02:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa523fff640 1 -- 172.21.15.16:0/2849495631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa530021440 con 0x55773398a560 2023-11-13T11:02:43.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa523fff640 1 -- 172.21.15.16:0/2849495631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300217f0 con 0x55773398a560 2023-11-13T11:02:43.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa54aff2740 1 -- 172.21.15.16:0/2849495631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 msgr2=0x55773398cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2849495631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x55773398cf10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa530011df0 tx=0x7fa530014000 comp rx=0 tx=0).stop 2023-11-13T11:02:43.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa54aff2740 1 -- 172.21.15.16:0/2849495631 shutdown_connections 2023-11-13T11:02:43.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2849495631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773395d660 0x55773398d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2849495631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x55773398cf10 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2849495631 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 0x55773395d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa54aff2740 1 -- 172.21.15.16:0/2849495631 >> 172.21.15.16:0/2849495631 conn(0x55773395f390 msgr2=0x5577339a7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:43.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa54aff2740 1 -- 172.21.15.16:0/2849495631 shutdown_connections 2023-11-13T11:02:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.528+0000 7fa54aff2740 1 -- 172.21.15.16:0/2849495631 wait complete. 2023-11-13T11:02:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.529+0000 7fa54aff2740 1 -- start start 2023-11-13T11:02:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.529+0000 7fa54aff2740 1 Processor -- start 2023-11-13T11:02:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.530+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773398a180 0x5577339b8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.530+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a560 0x55773398c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.530+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557733985730 0x557733991960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.530+0000 7fa54aff2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577338c8e20 con 0x557733985730 2023-11-13T11:02:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.530+0000 7fa54aff2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557733865200 con 0x55773398a180 2023-11-13T11:02:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.530+0000 7fa54aff2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557733882a90 con 0x55773398a560 2023-11-13T11:02:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.530+0000 7fa54aff2740 10 client.0 _ll_register_callbacks cb 0x5577335dace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5469fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557733985730 0x557733991960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5469fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557733985730 0x557733991960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38648/0 (socket says 172.21.15.16:38648) 2023-11-13T11:02:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5469fe640 1 -- 172.21.15.16:0/45128682 learned_addr learned my addr 172.21.15.16:0/45128682 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5461fd640 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773398a180 0x5577339b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5469fe640 1 -- 172.21.15.16:0/45128682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a560 msgr2=0x55773398c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5469fe640 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a560 0x55773398c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5469fe640 1 -- 172.21.15.16:0/45128682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773398a180 msgr2=0x5577339b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5469fe640 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773398a180 0x5577339b8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5469fe640 1 -- 172.21.15.16:0/45128682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa530011aa0 con 0x557733985730 2023-11-13T11:02:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5469fe640 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557733985730 0x557733991960 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa53802c660 tx=0x7fa538009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934bdabce4de5444 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.531+0000 7fa5217fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:43.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa5217fa640 1 -- 172.21.15.16:0/45128682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538003da0 con 0x557733985730 2023-11-13T11:02:43.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa54aff2740 1 -- 172.21.15.16:0/45128682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577338c1e00 con 0x557733985730 2023-11-13T11:02:43.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa5217fa640 1 -- 172.21.15.16:0/45128682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fa538004030 con 0x557733985730 2023-11-13T11:02:43.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa5217fa640 1 -- 172.21.15.16:0/45128682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5380041f0 con 0x557733985730 2023-11-13T11:02:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa54aff2740 10 client.6435 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa54aff2740 1 -- 172.21.15.16:0/45128682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5577338c74a0 con 0x557733985730 2023-11-13T11:02:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa5217fa640 1 -- 172.21.15.16:0/45128682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577338c1e00 con 0x557733985730 2023-11-13T11:02:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa5217fa640 1 -- 172.21.15.16:0/45128682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa538004350 con 0x557733985730 2023-11-13T11:02:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa54aff2740 10 client.6435 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:43.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa54aff2740 10 client.6435 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T11:02:43.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa54aff2740 20 client.6435 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:43.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa54aff2740 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x557733985b90 0x5577339c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa54aff2740 4 client.6435 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:43.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.532+0000 7fa54aff2740 1 -- 172.21.15.16:0/45128682 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577338c1b20 con 0x557733985b90 2023-11-13T11:02:43.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.533+0000 7fa5461fd640 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x557733985b90 0x5577339c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.533+0000 7fa5461fd640 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x557733985b90 0x5577339c5810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d222e7749d5830 server_cookie=bc308f733f74a1d in_seq=0 out_seq=0 2023-11-13T11:02:43.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.533+0000 7fa5217fa640 10 client.6435 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:43.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa5217fa640 1 -- 172.21.15.16:0/45128682 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577338c1b20 con 0x557733985b90 2023-11-13T11:02:43.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa5217fa640 10 client.6435 handle_command_reply: tid=0 2023-11-13T11:02:43.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 client.6435 shutdown 2023-11-13T11:02:43.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 -- 172.21.15.16:0/45128682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557733985730 msgr2=0x557733991960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557733985730 0x557733991960 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa53802c660 tx=0x7fa538009750 comp rx=0 tx=0).stop 2023-11-13T11:02:43.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 -- 172.21.15.16:0/45128682 shutdown_connections 2023-11-13T11:02:43.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x557733985b90 0x5577339c5810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557733985730 0x557733991960 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a560 0x55773398c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 --2- 172.21.15.16:0/45128682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773398a180 0x5577339b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa5217fa640 0 client.6435 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:43.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 -- 172.21.15.16:0/45128682 >> 172.21.15.16:0/45128682 conn(0x55773395d660 msgr2=0x557733961140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:43.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.534+0000 7fa54aff2740 1 -- 172.21.15.16:0/45128682 shutdown_connections 2023-11-13T11:02:43.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.535+0000 7fa54aff2740 1 -- 172.21.15.16:0/45128682 wait complete. 2023-11-13T11:02:43.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.535+0000 7fa54aff2740 20 client.6435 trim_cache size 0 max 16384 2023-11-13T11:02:43.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.548+0000 7fa54aff2740 1 Processor -- start 2023-11-13T11:02:43.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.548+0000 7fa54aff2740 1 -- start start 2023-11-13T11:02:43.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557733985730 0x5577339b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a180 0x5577339ab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773398a560 0x5577339ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa54aff2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577335c2090 con 0x55773398a180 2023-11-13T11:02:43.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa54aff2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577339b9240 con 0x55773398a560 2023-11-13T11:02:43.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa54aff2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577339bad40 con 0x557733985730 2023-11-13T11:02:43.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa5459fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a180 0x5577339ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa5459fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a180 0x5577339ab420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38654/0 (socket says 172.21.15.16:38654) 2023-11-13T11:02:43.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa5459fc640 1 -- 172.21.15.16:0/1127043639 learned_addr learned my addr 172.21.15.16:0/1127043639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:43.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa5459fc640 1 -- 172.21.15.16:0/1127043639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557733985730 msgr2=0x5577339b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa5459fc640 1 --2- 172.21.15.16:0/1127043639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557733985730 0x5577339b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa5459fc640 1 -- 172.21.15.16:0/1127043639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773398a560 msgr2=0x5577339ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa5459fc640 1 --2- 172.21.15.16:0/1127043639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773398a560 0x5577339ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.549+0000 7fa5459fc640 1 -- 172.21.15.16:0/1127043639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557733864a90 con 0x55773398a180 2023-11-13T11:02:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.550+0000 7fa5459fc640 1 --2- 172.21.15.16:0/1127043639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a180 0x5577339ab420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa530008010 tx=0x7fa5300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3778a635c5ee69cf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.550+0000 7fa534ff9640 1 -- 172.21.15.16:0/1127043639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530009770 con 0x55773398a180 2023-11-13T11:02:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.550+0000 7fa534ff9640 1 -- 172.21.15.16:0/1127043639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa530011e20 con 0x55773398a180 2023-11-13T11:02:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.550+0000 7fa534ff9640 1 -- 172.21.15.16:0/1127043639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300217f0 con 0x55773398a180 2023-11-13T11:02:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 -- 172.21.15.16:0/1127043639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a180 msgr2=0x5577339ab420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 --2- 172.21.15.16:0/1127043639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a180 0x5577339ab420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa530008010 tx=0x7fa5300046c0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 -- 172.21.15.16:0/1127043639 shutdown_connections 2023-11-13T11:02:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 --2- 172.21.15.16:0/1127043639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55773398a560 0x5577339ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 --2- 172.21.15.16:0/1127043639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a180 0x5577339ab420 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 --2- 172.21.15.16:0/1127043639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557733985730 0x5577339b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 -- 172.21.15.16:0/1127043639 >> 172.21.15.16:0/1127043639 conn(0x55773395d660 msgr2=0x557733990480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:43.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 -- 172.21.15.16:0/1127043639 shutdown_connections 2023-11-13T11:02:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 -- 172.21.15.16:0/1127043639 wait complete. 2023-11-13T11:02:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.551+0000 7fa54aff2740 1 -- start start 2023-11-13T11:02:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.552+0000 7fa54aff2740 1 Processor -- start 2023-11-13T11:02:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557733985730 0x5577339b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 0x557733a161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa54aff2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x5577339ab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa54aff2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577339aa940 con 0x55773398a560 2023-11-13T11:02:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa54aff2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557733a16d40 con 0x557733985730 2023-11-13T11:02:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa54aff2740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577339a9e20 con 0x55773398a180 2023-11-13T11:02:43.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa54aff2740 10 client.0 _ll_register_callbacks cb 0x7fa5300118b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:43.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5469fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x5577339ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5469fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x5577339ab420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38668/0 (socket says 172.21.15.16:38668) 2023-11-13T11:02:43.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5469fe640 1 -- 172.21.15.16:0/2997851116 learned_addr learned my addr 172.21.15.16:0/2997851116 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:43.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5461fd640 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557733985730 0x5577339b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5459fc640 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 0x557733a161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5469fe640 1 -- 172.21.15.16:0/2997851116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 msgr2=0x557733a161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5469fe640 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 0x557733a161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5469fe640 1 -- 172.21.15.16:0/2997851116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557733985730 msgr2=0x5577339b9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5469fe640 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557733985730 0x5577339b9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.553+0000 7fa5469fe640 1 -- 172.21.15.16:0/2997851116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55773388b8f0 con 0x55773398a560 2023-11-13T11:02:43.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.554+0000 7fa5461fd640 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557733985730 0x5577339b9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:43.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.554+0000 7fa5469fe640 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x5577339ab420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa538000f80 tx=0x7fa53802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4e4e8c6cd65fd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.554+0000 7fa5217fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:43.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.554+0000 7fa5217fa640 1 -- 172.21.15.16:0/2997851116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538004360 con 0x55773398a560 2023-11-13T11:02:43.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.554+0000 7fa5217fa640 1 -- 172.21.15.16:0/2997851116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fa538034440 con 0x55773398a560 2023-11-13T11:02:43.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.554+0000 7fa54aff2740 1 -- 172.21.15.16:0/2997851116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa530021440 con 0x55773398a560 2023-11-13T11:02:43.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.554+0000 7fa5217fa640 1 -- 172.21.15.16:0/2997851116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5380044c0 con 0x55773398a560 2023-11-13T11:02:43.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.554+0000 7fa54aff2740 10 client.6441 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:43.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.554+0000 7fa54aff2740 1 -- 172.21.15.16:0/2997851116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55773388a150 con 0x55773398a560 2023-11-13T11:02:43.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.555+0000 7fa5217fa640 1 -- 172.21.15.16:0/2997851116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa530021440 con 0x55773398a560 2023-11-13T11:02:43.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.555+0000 7fa5217fa640 1 -- 172.21.15.16:0/2997851116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa538034600 con 0x55773398a560 2023-11-13T11:02:43.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.555+0000 7fa54aff2740 10 client.6441 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:43.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.555+0000 7fa54aff2740 10 client.6441 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T11:02:43.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.555+0000 7fa54aff2740 20 client.6441 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:43.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.555+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5577339b7f30 0x5577339ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.555+0000 7fa54aff2740 4 client.6441 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 909192629}] 2023-11-13T11:02:43.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.555+0000 7fa54aff2740 1 -- 172.21.15.16:0/2997851116 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "damage rm", "damage_id": 909192629}) v1 -- 0x5577335c98e0 con 0x5577339b7f30 2023-11-13T11:02:43.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.555+0000 7fa5461fd640 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5577339b7f30 0x5577339ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa5217fa640 10 client.6441 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:43.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa5461fd640 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5577339b7f30 0x5577339ab960 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab9523cd6a713f18 server_cookie=20172877a0677d42 in_seq=0 out_seq=0 2023-11-13T11:02:43.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa5217fa640 1 -- 172.21.15.16:0/2997851116 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x5577335c98e0 con 0x5577339b7f30 2023-11-13T11:02:43.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa5217fa640 10 client.6441 handle_command_reply: tid=0 2023-11-13T11:02:43.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 client.6441 shutdown 2023-11-13T11:02:43.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 -- 172.21.15.16:0/2997851116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 msgr2=0x5577339ab420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x5577339ab420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa538000f80 tx=0x7fa53802e000 comp rx=0 tx=0).stop 2023-11-13T11:02:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 -- 172.21.15.16:0/2997851116 shutdown_connections 2023-11-13T11:02:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5577339b7f30 0x5577339ab960 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55773398a560 0x5577339ab420 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55773398a180 0x557733a161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 --2- 172.21.15.16:0/2997851116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557733985730 0x5577339b9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 -- 172.21.15.16:0/2997851116 >> 172.21.15.16:0/2997851116 conn(0x55773395d660 msgr2=0x557733990480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa5217fa640 0 client.6441 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 -- 172.21.15.16:0/2997851116 shutdown_connections 2023-11-13T11:02:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.556+0000 7fa54aff2740 1 -- 172.21.15.16:0/2997851116 wait complete. 2023-11-13T11:02:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa54aff2740 20 client.6441 trim_cache size 0 max 16384 2023-11-13T11:02:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 -- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa52406ae40 msgr2=0x7fa52406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa52406ae40 0x7fa52406d300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa540155c30 tx=0x7fa530008040 comp rx=0 tx=0).stop 2023-11-13T11:02:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 -- 172.21.15.16:0/608274130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 msgr2=0x7fa54013e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa54013e440 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00d740 tx=0x7fa53c00dc00 comp rx=0 tx=0).stop 2023-11-13T11:02:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 -- 172.21.15.16:0/608274130 shutdown_connections 2023-11-13T11:02:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa52406ae40 0x7fa52406d300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa540152f20 0x7fa54015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa540152550 0x7fa5401416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 --2- 172.21.15.16:0/608274130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c004d00 0x7fa54013e440 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 -- 172.21.15.16:0/608274130 >> 172.21.15.16:0/608274130 conn(0x7fa5400b0970 msgr2=0x7fa5400b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 -- 172.21.15.16:0/608274130 shutdown_connections 2023-11-13T11:02:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.557+0000 7fa5471ff640 1 -- 172.21.15.16:0/608274130 wait complete. 2023-11-13T11:02:43.584 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2023-11-13T11:02:43.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.789+0000 7fb138be9640 1 Processor -- start 2023-11-13T11:02:43.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.789+0000 7fb138be9640 1 -- start start 2023-11-13T11:02:43.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.791+0000 7fb138be9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb13415cfc0 0x7fb1340cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.791+0000 7fb138be9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb1340d2ae0 0x7fb1340d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.791+0000 7fb138be9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1340b3cf0 con 0x7fb1340b29d0 2023-11-13T11:02:43.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.791+0000 7fb138be9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1340b3550 con 0x7fb13415cfc0 2023-11-13T11:02:43.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.791+0000 7fb138be9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1340d3480 con 0x7fb1340d2ae0 2023-11-13T11:02:43.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.791+0000 7fb1337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb13415cfc0 0x7fb1340cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.791+0000 7fb133fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb1340b29d0 0x7fb1340b2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39958/0 (socket says 172.21.15.16:39958) 2023-11-13T11:02:43.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.791+0000 7fb1337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb13415cfc0 0x7fb1340cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38682/0 (socket says 172.21.15.16:38682) 2023-11-13T11:02:43.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.791+0000 7fb133fff640 1 -- 172.21.15.16:0/3561105921 learned_addr learned my addr 172.21.15.16:0/3561105921 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:43.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.792+0000 7fb12bfff640 1 --2- 172.21.15.16:0/3561105921 >> v2:172.21.15.184:3300/0 conn(0x7fb1340d2ae0 0x7fb1340d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.792+0000 7fb1337fe640 1 -- 172.21.15.16:0/3561105921 >> v2:172.21.15.184:3300/0 conn(0x7fb1340d2ae0 msgr2=0x7fb1340d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:43.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.792+0000 7fb1337fe640 1 --2- 172.21.15.16:0/3561105921 >> v2:172.21.15.184:3300/0 conn(0x7fb1340d2ae0 0x7fb1340d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.792+0000 7fb1337fe640 1 -- 172.21.15.16:0/3561105921 >> v1:172.21.15.16:6789/0 conn(0x7fb1340b29d0 legacy=0x7fb1340b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.792+0000 7fb132ffd640 1 -- 172.21.15.16:0/3561105921 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932777736 0 0) 0x7fb1340b3cf0 con 0x7fb1340b29d0 2023-11-13T11:02:43.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.792+0000 7fb1337fe640 1 -- 172.21.15.16:0/3561105921 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340b3730 con 0x7fb13415cfc0 2023-11-13T11:02:43.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.794+0000 7fb1337fe640 1 --2- 172.21.15.16:0/3561105921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb13415cfc0 0x7fb1340cd770 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb13408ce30 tx=0x7fb12402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8444fd4589bed31 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.794+0000 7fb132ffd640 1 -- 172.21.15.16:0/3561105921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12402e070 con 0x7fb13415cfc0 2023-11-13T11:02:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.794+0000 7fb132ffd640 1 -- 172.21.15.16:0/3561105921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1240315a0 con 0x7fb13415cfc0 2023-11-13T11:02:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.794+0000 7fb132ffd640 1 -- 172.21.15.16:0/3561105921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124031930 con 0x7fb13415cfc0 2023-11-13T11:02:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.795+0000 7fb138be9640 1 -- 172.21.15.16:0/3561105921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb13415cfc0 msgr2=0x7fb1340cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.795+0000 7fb138be9640 1 --2- 172.21.15.16:0/3561105921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb13415cfc0 0x7fb1340cd770 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb13408ce30 tx=0x7fb12402b840 comp rx=0 tx=0).stop 2023-11-13T11:02:43.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.795+0000 7fb138be9640 1 -- 172.21.15.16:0/3561105921 shutdown_connections 2023-11-13T11:02:43.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.795+0000 7fb138be9640 1 --2- 172.21.15.16:0/3561105921 >> v2:172.21.15.184:3300/0 conn(0x7fb1340d2ae0 0x7fb1340d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.795+0000 7fb138be9640 1 --2- 172.21.15.16:0/3561105921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb13415cfc0 0x7fb1340cd770 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.795+0000 7fb138be9640 1 -- 172.21.15.16:0/3561105921 >> 172.21.15.16:0/3561105921 conn(0x7fb13408c570 msgr2=0x7fb13409be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:43.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.795+0000 7fb138be9640 1 -- 172.21.15.16:0/3561105921 shutdown_connections 2023-11-13T11:02:43.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.795+0000 7fb138be9640 1 -- 172.21.15.16:0/3561105921 wait complete. 2023-11-13T11:02:43.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.797+0000 7fb138be9640 1 Processor -- start 2023-11-13T11:02:43.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.797+0000 7fb138be9640 1 -- start start 2023-11-13T11:02:43.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.798+0000 7fb138be9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1340b29d0 0x7fb1341efff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.798+0000 7fb138be9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1340d2ae0 0x7fb1340c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.798+0000 7fb138be9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb13415cfc0 0x7fb1340b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.798+0000 7fb138be9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13415e500 con 0x7fb1340b29d0 2023-11-13T11:02:43.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.798+0000 7fb138be9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340c8ff0 con 0x7fb13415cfc0 2023-11-13T11:02:43.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.798+0000 7fb138be9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340c92d0 con 0x7fb1340d2ae0 2023-11-13T11:02:43.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.798+0000 7fb1337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1340b29d0 0x7fb1341efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.798+0000 7fb1337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1340b29d0 0x7fb1341efff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38694/0 (socket says 172.21.15.16:38694) 2023-11-13T11:02:43.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.798+0000 7fb1337fe640 1 -- 172.21.15.16:0/3213037125 learned_addr learned my addr 172.21.15.16:0/3213037125 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:43.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb1337fe640 1 -- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1340d2ae0 msgr2=0x7fb1340c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb1337fe640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1340d2ae0 0x7fb1340c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb1337fe640 1 -- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb13415cfc0 msgr2=0x7fb1340b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:43.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb1337fe640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb13415cfc0 0x7fb1340b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:43.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb1337fe640 1 -- 172.21.15.16:0/3213037125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124009400 con 0x7fb1340b29d0 2023-11-13T11:02:43.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb1337fe640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1340b29d0 0x7fb1341efff0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb124009890 tx=0x7fb1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb130ff9640 1 -- 172.21.15.16:0/3213037125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124032840 con 0x7fb1340b29d0 2023-11-13T11:02:43.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb130ff9640 1 -- 172.21.15.16:0/3213037125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1240329a0 con 0x7fb1340b29d0 2023-11-13T11:02:43.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb130ff9640 1 -- 172.21.15.16:0/3213037125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124032d50 con 0x7fb1340b29d0 2023-11-13T11:02:43.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.799+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13415e7c0 con 0x7fb1340b29d0 2023-11-13T11:02:43.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.800+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1340c8a70 con 0x7fb1340b29d0 2023-11-13T11:02:43.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.801+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13415e9b0 con 0x7fb1340b29d0 2023-11-13T11:02:43.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.802+0000 7fb130ff9640 1 -- 172.21.15.16:0/3213037125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb12403b3f0 con 0x7fb1340b29d0 2023-11-13T11:02:43.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.802+0000 7fb130ff9640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb11406ae40 0x7fb11406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:43.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.803+0000 7fb130ff9640 1 -- 172.21.15.16:0/3213037125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb1240ab7e0 con 0x7fb1340b29d0 2023-11-13T11:02:43.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.803+0000 7fb12bfff640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb11406ae40 0x7fb11406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:43.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.805+0000 7fb12bfff640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb11406ae40 0x7fb11406d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb13415da10 tx=0x7fb11c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:43.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:43.807+0000 7fb130ff9640 1 -- 172.21.15.16:0/3213037125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb124076670 con 0x7fb1340b29d0 2023-11-13T11:02:44.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.039+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb13404e4d0 con 0x7fb1340b29d0 2023-11-13T11:02:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.041+0000 7fb130ff9640 1 -- 172.21.15.16:0/3213037125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fb124079c90 con 0x7fb1340b29d0 2023-11-13T11:02:44.046 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:44.046 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:39.012251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:44.047 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-11-13T11:02:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.043+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb11406ae40 msgr2=0x7fb11406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:44.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.043+0000 7fb138be9640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb11406ae40 0x7fb11406d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb13415da10 tx=0x7fb11c008040 comp rx=0 tx=0).stop 2023-11-13T11:02:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.043+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1340b29d0 msgr2=0x7fb1341efff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.043+0000 7fb138be9640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1340b29d0 0x7fb1341efff0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb124009890 tx=0x7fb1240046c0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.044+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 shutdown_connections 2023-11-13T11:02:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.044+0000 7fb138be9640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb11406ae40 0x7fb11406d300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.044+0000 7fb138be9640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb13415cfc0 0x7fb1340b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.044+0000 7fb138be9640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1340d2ae0 0x7fb1340c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.044+0000 7fb138be9640 1 --2- 172.21.15.16:0/3213037125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1340b29d0 0x7fb1341efff0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.044+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 >> 172.21.15.16:0/3213037125 conn(0x7fb13408c570 msgr2=0x7fb13406af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.044+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 shutdown_connections 2023-11-13T11:02:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.044+0000 7fb138be9640 1 -- 172.21.15.16:0/3213037125 wait complete. 2023-11-13T11:02:44.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2023-11-13T11:02:44.299 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:44.300 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:02:44.300 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:44.315 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-13T11:02:44.315 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2023-11-13T11:02:44.316 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2023-11-13T11:02:44.316 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2023-11-13T11:02:44.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:44.317 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2023-11-13T11:02:44.381 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2023-11-13T11:02:44.381 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T11:02:44.382 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2023-11-13T11:02:44.382 DEBUG:teuthology.orchestra.run.smithi016:> import json 2023-11-13T11:02:44.382 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2023-11-13T11:02:44.383 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:02:44.383 DEBUG:teuthology.orchestra.run.smithi016:> try: 2023-11-13T11:02:44.383 DEBUG:teuthology.orchestra.run.smithi016:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2023-11-13T11:02:44.384 DEBUG:teuthology.orchestra.run.smithi016:> except OSError as e: 2023-11-13T11:02:44.384 DEBUG:teuthology.orchestra.run.smithi016:> sys.exit(e.errno) 2023-11-13T11:02:44.384 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:02:44.385 DEBUG:teuthology.orchestra.run.smithi016:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-13T11:02:44.385 DEBUG:teuthology.orchestra.run.smithi016:> print(json.dumps( 2023-11-13T11:02:44.385 DEBUG:teuthology.orchestra.run.smithi016:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-13T11:02:44.386 DEBUG:teuthology.orchestra.run.smithi016:> indent=2)) 2023-11-13T11:02:44.386 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T11:02:44.540 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "st_mode": 16877, 2023-11-13T11:02:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: "st_ino": 1099511627777, 2023-11-13T11:02:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "st_dev": 41, 2023-11-13T11:02:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "st_nlink": 16, 2023-11-13T11:02:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "st_uid": 1000, 2023-11-13T11:02:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "st_gid": 1301, 2023-11-13T11:02:44.541 INFO:teuthology.orchestra.run.smithi016.stdout: "st_size": 9986206, 2023-11-13T11:02:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "st_atime": 1699873313.629523, 2023-11-13T11:02:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "st_mtime": 1699873355.1755176, 2023-11-13T11:02:44.542 INFO:teuthology.orchestra.run.smithi016.stdout: "st_ctime": 1699873355.1755176 2023-11-13T11:02:44.542 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T11:02:44.555 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2023-11-13T11:02:44.555 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2023-11-13T11:02:44.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:44.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.825+0000 7f80e0432640 1 Processor -- start 2023-11-13T11:02:44.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.825+0000 7f80e0432640 1 -- start start 2023-11-13T11:02:44.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.826+0000 7f80e0432640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d80ccc60 0x7f80d8071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:44.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.826+0000 7f80e0432640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f80d8071980 0x7f80d806fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:44.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.826+0000 7f80e0432640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d80cd0e0 con 0x7f80d8155410 2023-11-13T11:02:44.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.826+0000 7f80e0432640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d8071e10 con 0x7f80d8071980 2023-11-13T11:02:44.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.826+0000 7f80e0432640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d815b500 con 0x7f80d80ccc60 2023-11-13T11:02:44.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.827+0000 7f80de9cf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f80d8071980 0x7f80d806fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:44.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.827+0000 7f80dd9cd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d80ccc60 0x7f80d8071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:44.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.827+0000 7f80dd9cd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d80ccc60 0x7f80d8071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:57804/0 (socket says 172.21.15.16:57804) 2023-11-13T11:02:44.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.827+0000 7f80dd9cd640 1 -- 172.21.15.16:0/897323570 learned_addr learned my addr 172.21.15.16:0/897323570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:44.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.828+0000 7f80dd9cd640 1 -- 172.21.15.16:0/897323570 >> v2:172.21.15.184:3300/0 conn(0x7f80d8071980 msgr2=0x7f80d806fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:44.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.828+0000 7f80dd9cd640 1 --2- 172.21.15.16:0/897323570 >> v2:172.21.15.184:3300/0 conn(0x7f80d8071980 0x7f80d806fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.828+0000 7f80dd9cd640 1 -- 172.21.15.16:0/897323570 >> v1:172.21.15.184:6789/0 conn(0x7f80d8155410 legacy=0x7f80d80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:44.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.828+0000 7f80dd9cd640 1 -- 172.21.15.16:0/897323570 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d815b7e0 con 0x7f80d80ccc60 2023-11-13T11:02:44.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.828+0000 7f80de9cf640 1 --2- 172.21.15.16:0/897323570 >> v2:172.21.15.184:3300/0 conn(0x7f80d8071980 0x7f80d806fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T11:02:44.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.829+0000 7f80dd9cd640 1 --2- 172.21.15.16:0/897323570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d80ccc60 0x7f80d8071440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009870 tx=0x7f80c802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be911b5b56f48b9d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:44.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.830+0000 7f80dd1cc640 1 -- 172.21.15.16:0/897323570 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8030840 con 0x7f80d80ccc60 2023-11-13T11:02:44.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.830+0000 7f80dd1cc640 1 -- 172.21.15.16:0/897323570 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c80309a0 con 0x7f80d80ccc60 2023-11-13T11:02:44.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.830+0000 7f80dd1cc640 1 -- 172.21.15.16:0/897323570 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8030d30 con 0x7f80d80ccc60 2023-11-13T11:02:44.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.830+0000 7f80e0432640 1 -- 172.21.15.16:0/897323570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d80ccc60 msgr2=0x7f80d8071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.830+0000 7f80e0432640 1 --2- 172.21.15.16:0/897323570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d80ccc60 0x7f80d8071440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009870 tx=0x7f80c802b980 comp rx=0 tx=0).stop 2023-11-13T11:02:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.831+0000 7f80e0432640 1 -- 172.21.15.16:0/897323570 shutdown_connections 2023-11-13T11:02:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.831+0000 7f80e0432640 1 --2- 172.21.15.16:0/897323570 >> v2:172.21.15.184:3300/0 conn(0x7f80d8071980 0x7f80d806fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.831+0000 7f80e0432640 1 --2- 172.21.15.16:0/897323570 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d80ccc60 0x7f80d8071440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.831+0000 7f80e0432640 1 -- 172.21.15.16:0/897323570 >> 172.21.15.16:0/897323570 conn(0x7f80d806a190 msgr2=0x7f80d806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.831+0000 7f80e0432640 1 -- 172.21.15.16:0/897323570 shutdown_connections 2023-11-13T11:02:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.831+0000 7f80e0432640 1 -- 172.21.15.16:0/897323570 wait complete. 2023-11-13T11:02:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.833+0000 7f80e0432640 1 Processor -- start 2023-11-13T11:02:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.833+0000 7f80e0432640 1 -- start start 2023-11-13T11:02:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80e0432640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d8071980 0x7f80d813cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80e0432640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d80ccc60 0x7f80d8153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80e0432640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d8155410 0x7f80d813fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80e0432640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8152a00 con 0x7f80d8155410 2023-11-13T11:02:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80e0432640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8152ce0 con 0x7f80d80ccc60 2023-11-13T11:02:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80e0432640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8152fc0 con 0x7f80d8071980 2023-11-13T11:02:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80de9cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d8155410 0x7f80d813fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80de9cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d8155410 0x7f80d813fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38708/0 (socket says 172.21.15.16:38708) 2023-11-13T11:02:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80de9cf640 1 -- 172.21.15.16:0/4104676425 learned_addr learned my addr 172.21.15.16:0/4104676425 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.834+0000 7f80de1ce640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d8071980 0x7f80d813cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80dd9cd640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d80ccc60 0x7f80d8153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80de9cf640 1 -- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d8071980 msgr2=0x7f80d813cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80de9cf640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d8071980 0x7f80d813cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80de9cf640 1 -- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d80ccc60 msgr2=0x7f80d8153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80de9cf640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d80ccc60 0x7f80d8153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80de9cf640 1 -- 172.21.15.16:0/4104676425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d0009a70 con 0x7f80d8155410 2023-11-13T11:02:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80de1ce640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d8071980 0x7f80d813cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:02:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80de9cf640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d8155410 0x7f80d813fcf0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f80d000efc0 tx=0x7f80d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80ceffd640 1 -- 172.21.15.16:0/4104676425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0009d70 con 0x7f80d8155410 2023-11-13T11:02:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80ceffd640 1 -- 172.21.15.16:0/4104676425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d000f040 con 0x7f80d8155410 2023-11-13T11:02:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80ceffd640 1 -- 172.21.15.16:0/4104676425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0003990 con 0x7f80d8155410 2023-11-13T11:02:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.835+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c80094d0 con 0x7f80d8155410 2023-11-13T11:02:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.836+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d8153580 con 0x7f80d8155410 2023-11-13T11:02:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.837+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d81537a0 con 0x7f80d8155410 2023-11-13T11:02:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.838+0000 7f80ceffd640 1 -- 172.21.15.16:0/4104676425 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80d00144a0 con 0x7f80d8155410 2023-11-13T11:02:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.839+0000 7f80ceffd640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80bc06ae40 0x7f80bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.839+0000 7f80ceffd640 1 -- 172.21.15.16:0/4104676425 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f80d008a370 con 0x7f80d8155410 2023-11-13T11:02:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.840+0000 7f80de1ce640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80bc06ae40 0x7f80bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.841+0000 7f80de1ce640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80bc06ae40 0x7f80bc06d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f80d4007fd0 tx=0x7f80d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:44.845+0000 7f80ceffd640 1 -- 172.21.15.16:0/4104676425 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80d0055330 con 0x7f80d8155410 2023-11-13T11:02:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.076+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80d80cbfa0 con 0x7f80d8155410 2023-11-13T11:02:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.077+0000 7f80ceffd640 1 -- 172.21.15.16:0/4104676425 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f80d0058950 con 0x7f80d8155410 2023-11-13T11:02:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:45.083 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:39.012251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-11-13T11:02:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80bc06ae40 msgr2=0x7f80bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80bc06ae40 0x7f80bc06d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f80d4007fd0 tx=0x7f80d4009040 comp rx=0 tx=0).stop 2023-11-13T11:02:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d8155410 msgr2=0x7f80d813fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d8155410 0x7f80d813fcf0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f80d000efc0 tx=0x7f80d0009f90 comp rx=0 tx=0).stop 2023-11-13T11:02:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 shutdown_connections 2023-11-13T11:02:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80bc06ae40 0x7f80bc06d300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80d8155410 0x7f80d813fcf0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80d80ccc60 0x7f80d8153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 --2- 172.21.15.16:0/4104676425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80d8071980 0x7f80d813cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 >> 172.21.15.16:0/4104676425 conn(0x7f80d806a190 msgr2=0x7f80d80ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 shutdown_connections 2023-11-13T11:02:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.080+0000 7f80e0432640 1 -- 172.21.15.16:0/4104676425 wait complete. 2023-11-13T11:02:45.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2023-11-13T11:02:45.330 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": -2 2023-11-13T11:02:45.331 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:45.347 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2023-11-13T11:02:45.347 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2023-11-13T11:02:45.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2023-11-13T11:02:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.574+0000 7f7758885640 1 Processor -- start 2023-11-13T11:02:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.574+0000 7f7758885640 1 -- start start 2023-11-13T11:02:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.575+0000 7f7758885640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f775415b8e0 0x7f775406c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.575+0000 7f7758885640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f77540ce260 0x7f77540cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.575+0000 7f7758885640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775406b860 con 0x7f775415bcc0 2023-11-13T11:02:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.575+0000 7f7758885640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f775406bb40 con 0x7f775415b8e0 2023-11-13T11:02:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.575+0000 7f7758885640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f775406be20 con 0x7f77540ce260 2023-11-13T11:02:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.576+0000 7f77537fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f775415b8e0 0x7f775406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.576+0000 7f77537fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f775415b8e0 0x7f775406c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38724/0 (socket says 172.21.15.16:38724) 2023-11-13T11:02:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.576+0000 7f77537fe640 1 -- 172.21.15.16:0/643240002 learned_addr learned my addr 172.21.15.16:0/643240002 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.576+0000 7f7753fff640 1 --2- 172.21.15.16:0/643240002 >> v2:172.21.15.184:3300/0 conn(0x7f77540ce260 0x7f77540cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.576+0000 7f77537fe640 1 -- 172.21.15.16:0/643240002 >> v2:172.21.15.184:3300/0 conn(0x7f77540ce260 msgr2=0x7f77540cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.576+0000 7f77537fe640 1 --2- 172.21.15.16:0/643240002 >> v2:172.21.15.184:3300/0 conn(0x7f77540ce260 0x7f77540cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.576+0000 7f77537fe640 1 -- 172.21.15.16:0/643240002 >> v1:172.21.15.16:6789/0 conn(0x7f775415bcc0 legacy=0x7f77540cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.576+0000 7f77537fe640 1 -- 172.21.15.16:0/643240002 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775406c100 con 0x7f775415b8e0 2023-11-13T11:02:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.578+0000 7f77537fe640 1 --2- 172.21.15.16:0/643240002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f775415b8e0 0x7f775406c900 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f77540cc830 tx=0x7f774402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111536fecb46b5e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.578+0000 7f77527fc640 1 -- 172.21.15.16:0/643240002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774402cb10 con 0x7f775415b8e0 2023-11-13T11:02:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.578+0000 7f77527fc640 1 -- 172.21.15.16:0/643240002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774402cc70 con 0x7f775415b8e0 2023-11-13T11:02:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.578+0000 7f77527fc640 1 -- 172.21.15.16:0/643240002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77440343f0 con 0x7f775415b8e0 2023-11-13T11:02:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.578+0000 7f7758885640 1 -- 172.21.15.16:0/643240002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f775415b8e0 msgr2=0x7f775406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.578+0000 7f7758885640 1 --2- 172.21.15.16:0/643240002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f775415b8e0 0x7f775406c900 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f77540cc830 tx=0x7f774402bf70 comp rx=0 tx=0).stop 2023-11-13T11:02:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.579+0000 7f7758885640 1 -- 172.21.15.16:0/643240002 shutdown_connections 2023-11-13T11:02:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.579+0000 7f7758885640 1 --2- 172.21.15.16:0/643240002 >> v2:172.21.15.184:3300/0 conn(0x7f77540ce260 0x7f77540cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.579+0000 7f7758885640 1 --2- 172.21.15.16:0/643240002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f775415b8e0 0x7f775406c900 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.579+0000 7f7758885640 1 -- 172.21.15.16:0/643240002 >> 172.21.15.16:0/643240002 conn(0x7f77540b0970 msgr2=0x7f77540b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.579+0000 7f7758885640 1 -- 172.21.15.16:0/643240002 shutdown_connections 2023-11-13T11:02:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.579+0000 7f7758885640 1 -- 172.21.15.16:0/643240002 wait complete. 2023-11-13T11:02:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.581+0000 7f7758885640 1 Processor -- start 2023-11-13T11:02:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.581+0000 7f7758885640 1 -- start start 2023-11-13T11:02:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.582+0000 7f7758885640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540ce260 0x7f77540af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.582+0000 7f7758885640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f775415b8e0 0x7f77540acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.582+0000 7f7758885640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77541f3080 0x7f77541f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.582+0000 7f7758885640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77541f5b20 con 0x7f77540ce260 2023-11-13T11:02:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.582+0000 7f7758885640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77541f5e00 con 0x7f77541f3080 2023-11-13T11:02:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.582+0000 7f7758885640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77541f60e0 con 0x7f775415b8e0 2023-11-13T11:02:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.582+0000 7f77537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540ce260 0x7f77540af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.582+0000 7f77537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540ce260 0x7f77540af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38736/0 (socket says 172.21.15.16:38736) 2023-11-13T11:02:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.582+0000 7f77537fe640 1 -- 172.21.15.16:0/438813587 learned_addr learned my addr 172.21.15.16:0/438813587 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f77537fe640 1 -- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f775415b8e0 msgr2=0x7f77540acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f77537fe640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f775415b8e0 0x7f77540acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f77537fe640 1 -- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77541f3080 msgr2=0x7f77541f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f77537fe640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77541f3080 0x7f77541f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f77537fe640 1 -- 172.21.15.16:0/438813587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744009b30 con 0x7f77540ce260 2023-11-13T11:02:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f77537fe640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540ce260 0x7f77540af6c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7744000f80 tx=0x7f7744004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f774bfff640 1 -- 172.21.15.16:0/438813587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744009510 con 0x7f77540ce260 2023-11-13T11:02:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f774bfff640 1 -- 172.21.15.16:0/438813587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774403e450 con 0x7f77540ce260 2023-11-13T11:02:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f774bfff640 1 -- 172.21.15.16:0/438813587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774403e800 con 0x7f77540ce260 2023-11-13T11:02:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.583+0000 7f7758885640 1 -- 172.21.15.16:0/438813587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77541f63c0 con 0x7f77540ce260 2023-11-13T11:02:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.584+0000 7f7758885640 1 -- 172.21.15.16:0/438813587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77540afc00 con 0x7f77540ce260 2023-11-13T11:02:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.586+0000 7f774bfff640 1 -- 172.21.15.16:0/438813587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f774403e960 con 0x7f77540ce260 2023-11-13T11:02:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.587+0000 7f774bfff640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f773406ae40 0x7f773406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.587+0000 7f774bfff640 1 -- 172.21.15.16:0/438813587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f77440ac690 con 0x7f77540ce260 2023-11-13T11:02:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.587+0000 7f775c67f740 1 Processor -- start 2023-11-13T11:02:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.588+0000 7f775c67f740 1 -- start start 2023-11-13T11:02:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.588+0000 7f7752ffd640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f773406ae40 0x7f773406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.588+0000 7f775c67f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b88a0 0x5633c008b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.588+0000 7f775c67f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 0x5633c00bb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.588+0000 7f775c67f740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c008bd80 0x5633c00bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.588+0000 7f775c67f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c004c530 con 0x5633c00b88a0 2023-11-13T11:02:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.588+0000 7f775c67f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bfffdef0 con 0x5633c00b8c80 2023-11-13T11:02:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.588+0000 7f775c67f740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633bfffb5e0 con 0x5633c008bd80 2023-11-13T11:02:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f77537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b88a0 0x5633c008b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f77537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b88a0 0x5633c008b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38750/0 (socket says 172.21.15.16:38750) 2023-11-13T11:02:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f77537fe640 1 -- 172.21.15.16:0/3886709379 learned_addr learned my addr 172.21.15.16:0/3886709379 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f7752ffd640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f773406ae40 0x7f773406d300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f773c000f60 tx=0x7f773c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f77537fe640 1 -- 172.21.15.16:0/3886709379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c008bd80 msgr2=0x5633c00bbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f7752ffd640 1 --2- 172.21.15.16:0/3886709379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 0x5633c00bb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f77537fe640 1 --2- 172.21.15.16:0/3886709379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c008bd80 0x5633c00bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f77537fe640 1 -- 172.21.15.16:0/3886709379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 msgr2=0x5633c00bb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f77537fe640 1 --2- 172.21.15.16:0/3886709379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 0x5633c00bb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f77537fe640 1 -- 172.21.15.16:0/3886709379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633bffbcd40 con 0x5633c00b88a0 2023-11-13T11:02:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.589+0000 7f7752ffd640 1 --2- 172.21.15.16:0/3886709379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 0x5633c00bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.590+0000 7f77537fe640 1 --2- 172.21.15.16:0/3886709379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b88a0 0x5633c008b840 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f77440776e0 tx=0x7f774403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c782e5c5a83fdf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.590+0000 7f77497fa640 1 -- 172.21.15.16:0/3886709379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774407f330 con 0x5633c00b88a0 2023-11-13T11:02:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.590+0000 7f77497fa640 1 -- 172.21.15.16:0/3886709379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77440834e0 con 0x5633c00b88a0 2023-11-13T11:02:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.590+0000 7f77497fa640 1 -- 172.21.15.16:0/3886709379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744079320 con 0x5633c00b88a0 2023-11-13T11:02:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.590+0000 7f775c67f740 1 -- 172.21.15.16:0/3886709379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b88a0 msgr2=0x5633c008b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.590+0000 7f775c67f740 1 --2- 172.21.15.16:0/3886709379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b88a0 0x5633c008b840 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f77440776e0 tx=0x7f774403c000 comp rx=0 tx=0).stop 2023-11-13T11:02:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.591+0000 7f775c67f740 1 -- 172.21.15.16:0/3886709379 shutdown_connections 2023-11-13T11:02:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.591+0000 7f775c67f740 1 --2- 172.21.15.16:0/3886709379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c008bd80 0x5633c00bbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.591+0000 7f775c67f740 1 --2- 172.21.15.16:0/3886709379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 0x5633c00bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.591+0000 7f775c67f740 1 --2- 172.21.15.16:0/3886709379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b88a0 0x5633c008b840 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.591+0000 7f775c67f740 1 -- 172.21.15.16:0/3886709379 >> 172.21.15.16:0/3886709379 conn(0x5633c008dab0 msgr2=0x5633c00d6630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.591+0000 7f775c67f740 1 -- 172.21.15.16:0/3886709379 shutdown_connections 2023-11-13T11:02:45.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.591+0000 7f775c67f740 1 -- 172.21.15.16:0/3886709379 wait complete. 2023-11-13T11:02:45.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.591+0000 7f775c67f740 1 -- start start 2023-11-13T11:02:45.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.592+0000 7f775c67f740 1 Processor -- start 2023-11-13T11:02:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f775c67f740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00b88a0 0x5633c00b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f775c67f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 0x5633c00d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f775c67f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00d9b40 0x5633c00d9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f775c67f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bfff6ed0 con 0x5633c00d9b40 2023-11-13T11:02:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f775c67f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bff93250 con 0x5633c00b8c80 2023-11-13T11:02:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f775c67f740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633bffb0c10 con 0x5633c00b88a0 2023-11-13T11:02:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f775c67f740 10 client.0 _ll_register_callbacks cb 0x5633bffb4d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f7753fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00d9b40 0x5633c00d9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f7753fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00d9b40 0x5633c00d9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38758/0 (socket says 172.21.15.16:38758) 2023-11-13T11:02:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f7753fff640 1 -- 172.21.15.16:0/614382126 learned_addr learned my addr 172.21.15.16:0/614382126 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.593+0000 7f7752ffd640 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 0x5633c00d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f77537fe640 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00b88a0 0x5633c00b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f7753fff640 1 -- 172.21.15.16:0/614382126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00b88a0 msgr2=0x5633c00b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f7753fff640 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00b88a0 0x5633c00b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f7753fff640 1 -- 172.21.15.16:0/614382126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 msgr2=0x5633c00d58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f7753fff640 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 0x5633c00d58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f7753fff640 1 -- 172.21.15.16:0/614382126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77540afc00 con 0x5633c00d9b40 2023-11-13T11:02:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f7753fff640 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00d9b40 0x5633c00d9fa0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f774c007f30 tx=0x7f774c00d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d66aa60a8e0a8c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f77327fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f77327fc640 1 -- 172.21.15.16:0/614382126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c013480 con 0x5633c00d9b40 2023-11-13T11:02:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f77327fc640 1 -- 172.21.15.16:0/614382126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f774c0135e0 con 0x5633c00d9b40 2023-11-13T11:02:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.594+0000 7f775c67f740 1 -- 172.21.15.16:0/614382126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633c001f640 con 0x5633c00d9b40 2023-11-13T11:02:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f77327fc640 1 -- 172.21.15.16:0/614382126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f774c0137a0 con 0x5633c00d9b40 2023-11-13T11:02:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f775c67f740 10 client.6468 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f775c67f740 1 -- 172.21.15.16:0/614382126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5633bfff5550 con 0x5633c00d9b40 2023-11-13T11:02:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f77327fc640 1 -- 172.21.15.16:0/614382126 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633c001f640 con 0x5633c00d9b40 2023-11-13T11:02:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f77327fc640 1 -- 172.21.15.16:0/614382126 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f774c013900 con 0x5633c00d9b40 2023-11-13T11:02:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f775c67f740 10 client.6468 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f775c67f740 10 client.6468 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f775c67f740 20 client.6468 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f775c67f740 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633c00bdb40 0x5633c00bdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f775c67f740 4 client.6468 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f775c67f740 1 -- 172.21.15.16:0/614382126 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633bffca330 con 0x5633c00bdb40 2023-11-13T11:02:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.595+0000 7f77537fe640 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633c00bdb40 0x5633c00bdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.596+0000 7f77537fe640 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633c00bdb40 0x5633c00bdf50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b090a9c690b4692e server_cookie=79352eecc0649c15 in_seq=0 out_seq=0 2023-11-13T11:02:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.596+0000 7f77327fc640 10 client.6468 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.596+0000 7f77327fc640 1 -- 172.21.15.16:0/614382126 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5633bffca330 con 0x5633c00bdb40 2023-11-13T11:02:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.596+0000 7f77327fc640 10 client.6468 handle_command_reply: tid=0 2023-11-13T11:02:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 client.6468 shutdown 2023-11-13T11:02:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 -- 172.21.15.16:0/614382126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00d9b40 msgr2=0x5633c00d9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00d9b40 0x5633c00d9fa0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f774c007f30 tx=0x7f774c00d5f0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 -- 172.21.15.16:0/614382126 shutdown_connections 2023-11-13T11:02:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633c00bdb40 0x5633c00bdf50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00d9b40 0x5633c00d9fa0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b8c80 0x5633c00d58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 --2- 172.21.15.16:0/614382126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00b88a0 0x5633c00b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f77327fc640 0 client.6468 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 -- 172.21.15.16:0/614382126 >> 172.21.15.16:0/614382126 conn(0x5633c008bd80 msgr2=0x5633c008af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 -- 172.21.15.16:0/614382126 shutdown_connections 2023-11-13T11:02:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 1 -- 172.21.15.16:0/614382126 wait complete. 2023-11-13T11:02:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.597+0000 7f775c67f740 20 client.6468 trim_cache size 0 max 16384 2023-11-13T11:02:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.614+0000 7f775c67f740 1 Processor -- start 2023-11-13T11:02:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.614+0000 7f775c67f740 1 -- start start 2023-11-13T11:02:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.615+0000 7f775c67f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 0x5633c013ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.615+0000 7f775c67f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c00bb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.615+0000 7f775c67f740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 0x5633c00bbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.615+0000 7f775c67f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bfd5a230 con 0x5633c00b8c80 2023-11-13T11:02:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.615+0000 7f775c67f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c00d0530 con 0x5633c00b88a0 2023-11-13T11:02:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.615+0000 7f775c67f740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633c00d0a10 con 0x5633c00d9b40 2023-11-13T11:02:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.615+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c00bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.615+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c00bb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38772/0 (socket says 172.21.15.16:38772) 2023-11-13T11:02:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.615+0000 7f7752ffd640 1 -- 172.21.15.16:0/3674443910 learned_addr learned my addr 172.21.15.16:0/3674443910 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f77537fe640 1 --2- 172.21.15.16:0/3674443910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 0x5633c013ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7753fff640 1 --2- 172.21.15.16:0/3674443910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 0x5633c00bbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7752ffd640 1 -- 172.21.15.16:0/3674443910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 msgr2=0x5633c00bbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7752ffd640 1 --2- 172.21.15.16:0/3674443910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 0x5633c00bbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7752ffd640 1 -- 172.21.15.16:0/3674443910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 msgr2=0x5633c013ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7752ffd640 1 --2- 172.21.15.16:0/3674443910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 0x5633c013ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7752ffd640 1 -- 172.21.15.16:0/3674443910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633bff92ae0 con 0x5633c00b8c80 2023-11-13T11:02:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7752ffd640 1 --2- 172.21.15.16:0/3674443910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c00bb620 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f773c0146a0 tx=0x7f773c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab185cd8eca3987 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7749ffb640 1 -- 172.21.15.16:0/3674443910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c005320 con 0x5633c00b8c80 2023-11-13T11:02:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7749ffb640 1 -- 172.21.15.16:0/3674443910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c005480 con 0x5633c00b8c80 2023-11-13T11:02:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.616+0000 7f7749ffb640 1 -- 172.21.15.16:0/3674443910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c0037f0 con 0x5633c00b8c80 2023-11-13T11:02:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.617+0000 7f775c67f740 1 -- 172.21.15.16:0/3674443910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 msgr2=0x5633c00bb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.617+0000 7f775c67f740 1 --2- 172.21.15.16:0/3674443910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c00bb620 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f773c0146a0 tx=0x7f773c011000 comp rx=0 tx=0).stop 2023-11-13T11:02:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.617+0000 7f775c67f740 1 -- 172.21.15.16:0/3674443910 shutdown_connections 2023-11-13T11:02:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.617+0000 7f775c67f740 1 --2- 172.21.15.16:0/3674443910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 0x5633c00bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.617+0000 7f775c67f740 1 --2- 172.21.15.16:0/3674443910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c00bb620 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.617+0000 7f775c67f740 1 --2- 172.21.15.16:0/3674443910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 0x5633c013ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.617+0000 7f775c67f740 1 -- 172.21.15.16:0/3674443910 >> 172.21.15.16:0/3674443910 conn(0x5633c008bd80 msgr2=0x5633c00d9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.617+0000 7f775c67f740 1 -- 172.21.15.16:0/3674443910 shutdown_connections 2023-11-13T11:02:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.618+0000 7f775c67f740 1 -- 172.21.15.16:0/3674443910 wait complete. 2023-11-13T11:02:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.618+0000 7f775c67f740 1 -- start start 2023-11-13T11:02:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.618+0000 7f775c67f740 1 Processor -- start 2023-11-13T11:02:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.619+0000 7f775c67f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 0x5633c01374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.619+0000 7f775c67f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c0137a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f775c67f740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 0x5633c0138670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f775c67f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c00d3eb0 con 0x5633c00b8c80 2023-11-13T11:02:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f775c67f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633c0139590 con 0x5633c00b88a0 2023-11-13T11:02:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f775c67f740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633c00d4020 con 0x5633c00d9b40 2023-11-13T11:02:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f775c67f740 10 client.0 _ll_register_callbacks cb 0x7f774402cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c0137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c0137a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38788/0 (socket says 172.21.15.16:38788) 2023-11-13T11:02:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f7752ffd640 1 -- 172.21.15.16:0/488120222 learned_addr learned my addr 172.21.15.16:0/488120222 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f77537fe640 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 0x5633c01374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f7753fff640 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 0x5633c0138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f7752ffd640 1 -- 172.21.15.16:0/488120222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 msgr2=0x5633c0138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f7752ffd640 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 0x5633c0138670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f7752ffd640 1 -- 172.21.15.16:0/488120222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 msgr2=0x5633c01374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f7752ffd640 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 0x5633c01374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f7752ffd640 1 -- 172.21.15.16:0/488120222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c01c040 con 0x5633c00b8c80 2023-11-13T11:02:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.620+0000 7f77537fe640 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 0x5633c01374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.621+0000 7f7752ffd640 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c0137a30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f773c016010 tx=0x7f773c0010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafbe4ff73044de server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.621+0000 7f77327fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.621+0000 7f77327fc640 1 -- 172.21.15.16:0/488120222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c003c30 con 0x5633c00b8c80 2023-11-13T11:02:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.621+0000 7f77327fc640 1 -- 172.21.15.16:0/488120222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f773c003d90 con 0x5633c00b8c80 2023-11-13T11:02:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.621+0000 7f775c67f740 1 -- 172.21.15.16:0/488120222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77440834e0 con 0x5633c00b8c80 2023-11-13T11:02:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.621+0000 7f77327fc640 1 -- 172.21.15.16:0/488120222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f773c013570 con 0x5633c00b8c80 2023-11-13T11:02:45.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.621+0000 7f775c67f740 10 client.6474 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:45.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.621+0000 7f775c67f740 1 -- 172.21.15.16:0/488120222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5633bffb8190 con 0x5633c00b8c80 2023-11-13T11:02:45.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.622+0000 7f77327fc640 1 -- 172.21.15.16:0/488120222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77440834e0 con 0x5633c00b8c80 2023-11-13T11:02:45.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.622+0000 7f77327fc640 1 -- 172.21.15.16:0/488120222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f773c0136d0 con 0x5633c00b8c80 2023-11-13T11:02:45.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.622+0000 7f775c67f740 10 client.6474 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.622+0000 7f775c67f740 10 client.6474 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.622+0000 7f775c67f740 20 client.6474 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.622+0000 7f775c67f740 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633c00bc160 0x5633c0138bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.622+0000 7f775c67f740 4 client.6474 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2023-11-13T11:02:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.622+0000 7f775c67f740 1 -- 172.21.15.16:0/488120222 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x5633bffefeb0 con 0x5633c00bc160 2023-11-13T11:02:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.622+0000 7f77537fe640 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633c00bc160 0x5633c0138bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f77537fe640 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633c00bc160 0x5633c0138bb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1df5738b434ea11 server_cookie=f26803e64c13deea in_seq=0 out_seq=0 2023-11-13T11:02:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f77327fc640 10 client.6474 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f77327fc640 1 -- 172.21.15.16:0/488120222 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x5633bffefeb0 con 0x5633c00bc160 2023-11-13T11:02:45.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f77327fc640 10 client.6474 handle_command_reply: tid=0 2023-11-13T11:02:45.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 client.6474 shutdown 2023-11-13T11:02:45.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 -- 172.21.15.16:0/488120222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 msgr2=0x5633c0137a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c0137a30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f773c016010 tx=0x7f773c0010f0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 -- 172.21.15.16:0/488120222 shutdown_connections 2023-11-13T11:02:45.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633c00d9b40 0x5633c0138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633c00bc160 0x5633c0138bb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633c00b8c80 0x5633c0137a30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 --2- 172.21.15.16:0/488120222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633c00b88a0 0x5633c01374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f77327fc640 0 client.6474 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 -- 172.21.15.16:0/488120222 >> 172.21.15.16:0/488120222 conn(0x5633c008bd80 msgr2=0x5633c0086ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 -- 172.21.15.16:0/488120222 shutdown_connections 2023-11-13T11:02:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.623+0000 7f775c67f740 1 -- 172.21.15.16:0/488120222 wait complete. 2023-11-13T11:02:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f775c67f740 20 client.6474 trim_cache size 0 max 16384 2023-11-13T11:02:45.643 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": -2 2023-11-13T11:02:45.643 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 -- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f773406ae40 msgr2=0x7f773406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f773406ae40 0x7f773406d300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f773c000f60 tx=0x7f773c0079b0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 -- 172.21.15.16:0/438813587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540ce260 msgr2=0x7f77540af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540ce260 0x7f77540af6c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7744000f80 tx=0x7f7744004a40 comp rx=0 tx=0).stop 2023-11-13T11:02:45.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 -- 172.21.15.16:0/438813587 shutdown_connections 2023-11-13T11:02:45.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f773406ae40 0x7f773406d300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77541f3080 0x7f77541f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f775415b8e0 0x7f77540acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 --2- 172.21.15.16:0/438813587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540ce260 0x7f77540af6c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 -- 172.21.15.16:0/438813587 >> 172.21.15.16:0/438813587 conn(0x7f77540b0970 msgr2=0x7f77540cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 -- 172.21.15.16:0/438813587 shutdown_connections 2023-11-13T11:02:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.624+0000 7f7758885640 1 -- 172.21.15.16:0/438813587 wait complete. 2023-11-13T11:02:45.647 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2023-11-13T11:02:45.648 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2023-11-13T11:02:45.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2023-11-13T11:02:45.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.856+0000 7fab4fd41640 1 Processor -- start 2023-11-13T11:02:45.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.857+0000 7fab4fd41640 1 -- start start 2023-11-13T11:02:45.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.858+0000 7fab4fd41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab48144570 0x7fab4815f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.858+0000 7fab4fd41640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fab480ce550 0x7fab480ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.858+0000 7fab4fd41640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab4806bb30 0x7fab4806bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.858+0000 7fab4fd41640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab480cef20 con 0x7fab48144570 2023-11-13T11:02:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.858+0000 7fab4fd41640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab48144950 con 0x7fab480ce550 2023-11-13T11:02:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.858+0000 7fab4fd41640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4815f740 con 0x7fab4806bb30 2023-11-13T11:02:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.859+0000 7fab4dadd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab48144570 0x7fab4815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.859+0000 7fab4dadd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab48144570 0x7fab4815f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38790/0 (socket says 172.21.15.16:38790) 2023-11-13T11:02:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.859+0000 7fab4dadd640 1 -- 172.21.15.16:0/2719682046 learned_addr learned my addr 172.21.15.16:0/2719682046 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.859+0000 7fab4d2dc640 1 --2- 172.21.15.16:0/2719682046 >> v2:172.21.15.184:3300/0 conn(0x7fab480ce550 0x7fab480ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.859+0000 7fab4dadd640 1 -- 172.21.15.16:0/2719682046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab4806bb30 msgr2=0x7fab4806bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.859+0000 7fab4dadd640 1 --2- 172.21.15.16:0/2719682046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab4806bb30 0x7fab4806bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.859+0000 7fab4dadd640 1 -- 172.21.15.16:0/2719682046 >> v2:172.21.15.184:3300/0 conn(0x7fab480ce550 msgr2=0x7fab480ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.859+0000 7fab4dadd640 1 --2- 172.21.15.16:0/2719682046 >> v2:172.21.15.184:3300/0 conn(0x7fab480ce550 0x7fab480ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.859+0000 7fab4dadd640 1 -- 172.21.15.16:0/2719682046 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480ce300 con 0x7fab48144570 2023-11-13T11:02:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.860+0000 7fab4dadd640 1 --2- 172.21.15.16:0/2719682046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 0x7fab4815f200 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fab4806d430 tx=0x7fab4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554e8d0cdae3cadc server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.861+0000 7fab4cadb640 1 -- 172.21.15.16:0/2719682046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44031870 con 0x7fab48144570 2023-11-13T11:02:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.861+0000 7fab4cadb640 1 -- 172.21.15.16:0/2719682046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab440319d0 con 0x7fab48144570 2023-11-13T11:02:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.861+0000 7fab4cadb640 1 -- 172.21.15.16:0/2719682046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44031d60 con 0x7fab48144570 2023-11-13T11:02:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.861+0000 7fab4fd41640 1 -- 172.21.15.16:0/2719682046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 msgr2=0x7fab4815f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.861+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2719682046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 0x7fab4815f200 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fab4806d430 tx=0x7fab4402b910 comp rx=0 tx=0).stop 2023-11-13T11:02:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.862+0000 7fab4fd41640 1 -- 172.21.15.16:0/2719682046 shutdown_connections 2023-11-13T11:02:45.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.862+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2719682046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab4806bb30 0x7fab4806bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.862+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2719682046 >> v2:172.21.15.184:3300/0 conn(0x7fab480ce550 0x7fab480ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.862+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2719682046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 0x7fab4815f200 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.862+0000 7fab4fd41640 1 -- 172.21.15.16:0/2719682046 >> 172.21.15.16:0/2719682046 conn(0x7fab4806a190 msgr2=0x7fab4806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.862+0000 7fab4fd41640 1 -- 172.21.15.16:0/2719682046 shutdown_connections 2023-11-13T11:02:45.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.862+0000 7fab4fd41640 1 -- 172.21.15.16:0/2719682046 wait complete. 2023-11-13T11:02:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.863+0000 7fab4fd41640 1 Processor -- start 2023-11-13T11:02:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.864+0000 7fab4fd41640 1 -- start start 2023-11-13T11:02:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.864+0000 7fab4fd41640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4806bb30 0x7fab4815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.864+0000 7fab4fd41640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab480ce550 0x7fab4814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.864+0000 7fab4fd41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 0x7fab48142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.864+0000 7fab4fd41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4814f670 con 0x7fab48144570 2023-11-13T11:02:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.864+0000 7fab4fd41640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48151360 con 0x7fab4806bb30 2023-11-13T11:02:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.864+0000 7fab4fd41640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4813ad00 con 0x7fab480ce550 2023-11-13T11:02:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.865+0000 7fab4e2de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 0x7fab48142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.865+0000 7fab4e2de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 0x7fab48142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38794/0 (socket says 172.21.15.16:38794) 2023-11-13T11:02:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.865+0000 7fab4e2de640 1 -- 172.21.15.16:0/2270754948 learned_addr learned my addr 172.21.15.16:0/2270754948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.865+0000 7fab4e2de640 1 -- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab480ce550 msgr2=0x7fab4814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.865+0000 7fab4e2de640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab480ce550 0x7fab4814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.865+0000 7fab4e2de640 1 -- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4806bb30 msgr2=0x7fab4815e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.865+0000 7fab4e2de640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4806bb30 0x7fab4815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.865+0000 7fab4e2de640 1 -- 172.21.15.16:0/2270754948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440094d0 con 0x7fab48144570 2023-11-13T11:02:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.866+0000 7fab4e2de640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 0x7fab48142060 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fab4000b9a0 tx=0x7fab4000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.866+0000 7fab3e7fc640 1 -- 172.21.15.16:0/2270754948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000cba0 con 0x7fab48144570 2023-11-13T11:02:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.866+0000 7fab3e7fc640 1 -- 172.21.15.16:0/2270754948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4000cd20 con 0x7fab48144570 2023-11-13T11:02:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.866+0000 7fab4fd41640 1 -- 172.21.15.16:0/2270754948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4813a7c0 con 0x7fab48144570 2023-11-13T11:02:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.866+0000 7fab3e7fc640 1 -- 172.21.15.16:0/2270754948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40014680 con 0x7fab48144570 2023-11-13T11:02:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.866+0000 7fab4fd41640 1 -- 172.21.15.16:0/2270754948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4814efb0 con 0x7fab48144570 2023-11-13T11:02:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.868+0000 7fab3e7fc640 1 -- 172.21.15.16:0/2270754948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab4001d3b0 con 0x7fab48144570 2023-11-13T11:02:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.869+0000 7fab3e7fc640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab2c06ae40 0x7fab2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.869+0000 7fab3e7fc640 1 -- 172.21.15.16:0/2270754948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fab40089740 con 0x7fab48144570 2023-11-13T11:02:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.870+0000 7fab529b4740 1 Processor -- start 2023-11-13T11:02:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.870+0000 7fab4dadd640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab2c06ae40 0x7fab2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.870+0000 7fab529b4740 1 -- start start 2023-11-13T11:02:45.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab529b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a345e0 0x55f068a349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab529b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a38630 0x55f068a3a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab529b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a64370 0x55f068a647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab529b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0689a6ee0 con 0x55f068a345e0 2023-11-13T11:02:45.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab529b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0689a45d0 con 0x55f068a64370 2023-11-13T11:02:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab529b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f06899fec0 con 0x55f068a38630 2023-11-13T11:02:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab4dadd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a345e0 0x55f068a349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab4dadd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a345e0 0x55f068a349c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38802/0 (socket says 172.21.15.16:38802) 2023-11-13T11:02:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab4dadd640 1 -- 172.21.15.16:0/3921166459 learned_addr learned my addr 172.21.15.16:0/3921166459 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.871+0000 7fab4dadd640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab2c06ae40 0x7fab2c06d300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fab44036000 tx=0x7fab44030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab4d2dc640 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a38630 0x55f068a3a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab4e2de640 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a64370 0x55f068a647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab4dadd640 1 -- 172.21.15.16:0/3921166459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a38630 msgr2=0x55f068a3a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab4dadd640 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a38630 0x55f068a3a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab4dadd640 1 -- 172.21.15.16:0/3921166459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a64370 msgr2=0x55f068a647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab4dadd640 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a64370 0x55f068a647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab4dadd640 1 -- 172.21.15.16:0/3921166459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f068962950 con 0x55f068a345e0 2023-11-13T11:02:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab4e2de640 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a64370 0x55f068a647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab4dadd640 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a345e0 0x55f068a349c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fab44009930 tx=0x7fab44008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e84ba0c9d948ad8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.872+0000 7fab2b7fe640 1 -- 172.21.15.16:0/3921166459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4403aba0 con 0x55f068a345e0 2023-11-13T11:02:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab2b7fe640 1 -- 172.21.15.16:0/3921166459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4404a070 con 0x55f068a345e0 2023-11-13T11:02:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab2b7fe640 1 -- 172.21.15.16:0/3921166459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4404f470 con 0x55f068a345e0 2023-11-13T11:02:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab529b4740 1 -- 172.21.15.16:0/3921166459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a345e0 msgr2=0x55f068a349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab529b4740 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a345e0 0x55f068a349c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fab44009930 tx=0x7fab44008030 comp rx=0 tx=0).stop 2023-11-13T11:02:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab529b4740 1 -- 172.21.15.16:0/3921166459 shutdown_connections 2023-11-13T11:02:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab529b4740 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a64370 0x55f068a647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab529b4740 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a38630 0x55f068a3a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab529b4740 1 --2- 172.21.15.16:0/3921166459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a345e0 0x55f068a349c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab529b4740 1 -- 172.21.15.16:0/3921166459 >> 172.21.15.16:0/3921166459 conn(0x55f068a36660 msgr2=0x55f068a37f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab529b4740 1 -- 172.21.15.16:0/3921166459 shutdown_connections 2023-11-13T11:02:45.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.873+0000 7fab529b4740 1 -- 172.21.15.16:0/3921166459 wait complete. 2023-11-13T11:02:45.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.874+0000 7fab529b4740 1 -- start start 2023-11-13T11:02:45.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.874+0000 7fab529b4740 1 Processor -- start 2023-11-13T11:02:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.875+0000 7fab529b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 0x55f068a51a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.875+0000 7fab529b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a68290 0x55f068a53450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.875+0000 7fab529b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 0x55f068a53990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.875+0000 7fab529b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068959ed0 con 0x55f068a68290 2023-11-13T11:02:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.875+0000 7fab529b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068959b70 con 0x55f068a51540 2023-11-13T11:02:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.875+0000 7fab529b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f068703340 con 0x55f068a64370 2023-11-13T11:02:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.875+0000 7fab529b4740 10 client.0 _ll_register_callbacks cb 0x55f06895e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4d2dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a68290 0x55f068a53450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4d2dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a68290 0x55f068a53450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38808/0 (socket says 172.21.15.16:38808) 2023-11-13T11:02:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4d2dc640 1 -- 172.21.15.16:0/2933970728 learned_addr learned my addr 172.21.15.16:0/2933970728 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4e2de640 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 0x55f068a53990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4dadd640 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 0x55f068a51a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4d2dc640 1 -- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 msgr2=0x55f068a51a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4d2dc640 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 0x55f068a51a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4d2dc640 1 -- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 msgr2=0x55f068a53990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4d2dc640 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 0x55f068a53990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4d2dc640 1 -- 172.21.15.16:0/2933970728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab44040070 con 0x55f068a68290 2023-11-13T11:02:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4e2de640 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 0x55f068a53990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.876+0000 7fab4dadd640 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 0x55f068a51a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:45.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab4d2dc640 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a68290 0x55f068a53450 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e280 tx=0x7fab3800e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9daedacc0eb06d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:45.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab28ff9640 1 -- 172.21.15.16:0/2933970728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38013660 con 0x55f068a68290 2023-11-13T11:02:45.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab28ff9640 1 -- 172.21.15.16:0/2933970728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fab380137c0 con 0x55f068a68290 2023-11-13T11:02:45.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab529b4740 1 -- 172.21.15.16:0/2933970728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0689c80f0 con 0x55f068a68290 2023-11-13T11:02:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab28ff9640 1 -- 172.21.15.16:0/2933970728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab38013980 con 0x55f068a68290 2023-11-13T11:02:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab529b4740 10 client.6486 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab529b4740 1 -- 172.21.15.16:0/2933970728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55f06899e540 con 0x55f068a68290 2023-11-13T11:02:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab28ff9640 1 -- 172.21.15.16:0/2933970728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f0689c80f0 con 0x55f068a68290 2023-11-13T11:02:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.877+0000 7fab28ff9640 1 -- 172.21.15.16:0/2933970728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab38011070 con 0x55f068a68290 2023-11-13T11:02:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.878+0000 7fab529b4740 10 client.6486 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.878+0000 7fab529b4740 10 client.6486 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.878+0000 7fab529b4740 20 client.6486 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.878+0000 7fab529b4740 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55f068a54a90 0x55f068a54e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.878+0000 7fab529b4740 4 client.6486 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.878+0000 7fab529b4740 1 -- 172.21.15.16:0/2933970728 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f068973230 con 0x55f068a54a90 2023-11-13T11:02:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.878+0000 7fab4dadd640 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55f068a54a90 0x55f068a54e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.878+0000 7fab4dadd640 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55f068a54a90 0x55f068a54e70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a4854f6fc7eb10 server_cookie=e1abbe039de47121 in_seq=0 out_seq=0 2023-11-13T11:02:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.878+0000 7fab28ff9640 10 client.6486 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab28ff9640 1 -- 172.21.15.16:0/2933970728 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f068973230 con 0x55f068a54a90 2023-11-13T11:02:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab28ff9640 10 client.6486 handle_command_reply: tid=0 2023-11-13T11:02:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab529b4740 1 client.6486 shutdown 2023-11-13T11:02:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab529b4740 1 -- 172.21.15.16:0/2933970728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a68290 msgr2=0x55f068a53450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab529b4740 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a68290 0x55f068a53450 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e280 tx=0x7fab3800e740 comp rx=0 tx=0).stop 2023-11-13T11:02:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab529b4740 1 -- 172.21.15.16:0/2933970728 shutdown_connections 2023-11-13T11:02:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab529b4740 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 0x55f068a53990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab529b4740 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55f068a54a90 0x55f068a54e70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab529b4740 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a68290 0x55f068a53450 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab529b4740 1 --2- 172.21.15.16:0/2933970728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 0x55f068a51a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.879+0000 7fab28ff9640 0 client.6486 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.880+0000 7fab529b4740 1 -- 172.21.15.16:0/2933970728 >> 172.21.15.16:0/2933970728 conn(0x55f068a345e0 msgr2=0x55f068a3a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.880+0000 7fab529b4740 1 -- 172.21.15.16:0/2933970728 shutdown_connections 2023-11-13T11:02:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.880+0000 7fab529b4740 1 -- 172.21.15.16:0/2933970728 wait complete. 2023-11-13T11:02:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.880+0000 7fab529b4740 20 client.6486 trim_cache size 0 max 16384 2023-11-13T11:02:45.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.896+0000 7fab529b4740 1 Processor -- start 2023-11-13T11:02:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.896+0000 7fab529b4740 1 -- start start 2023-11-13T11:02:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.897+0000 7fab529b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a51540 0x55f068a33b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.897+0000 7fab529b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 0x55f068a3a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.897+0000 7fab529b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a68290 0x55f068a3a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.897+0000 7fab529b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068a52e10 con 0x55f068a51540 2023-11-13T11:02:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.897+0000 7fab529b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068a80a50 con 0x55f068a68290 2023-11-13T11:02:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.897+0000 7fab529b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f068a56030 con 0x55f068a64370 2023-11-13T11:02:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4dadd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a51540 0x55f068a33b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4dadd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a51540 0x55f068a33b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38812/0 (socket says 172.21.15.16:38812) 2023-11-13T11:02:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4dadd640 1 -- 172.21.15.16:0/3570753538 learned_addr learned my addr 172.21.15.16:0/3570753538 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4d2dc640 1 --2- 172.21.15.16:0/3570753538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 0x55f068a3a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4e2de640 1 --2- 172.21.15.16:0/3570753538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a68290 0x55f068a3a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4dadd640 1 -- 172.21.15.16:0/3570753538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 msgr2=0x55f068a3a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4dadd640 1 --2- 172.21.15.16:0/3570753538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 0x55f068a3a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4dadd640 1 -- 172.21.15.16:0/3570753538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a68290 msgr2=0x55f068a3a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4dadd640 1 --2- 172.21.15.16:0/3570753538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a68290 0x55f068a3a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4dadd640 1 -- 172.21.15.16:0/3570753538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f06893bb00 con 0x55f068a51540 2023-11-13T11:02:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.898+0000 7fab4dadd640 1 --2- 172.21.15.16:0/3570753538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a51540 0x55f068a33b50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x55f068a62040 tx=0x7fab44031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e593cb8cfe261a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab2bfff640 1 -- 172.21.15.16:0/3570753538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44004500 con 0x55f068a51540 2023-11-13T11:02:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab2bfff640 1 -- 172.21.15.16:0/3570753538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4403a920 con 0x55f068a51540 2023-11-13T11:02:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab2bfff640 1 -- 172.21.15.16:0/3570753538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4403ac90 con 0x55f068a51540 2023-11-13T11:02:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab529b4740 1 -- 172.21.15.16:0/3570753538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a51540 msgr2=0x55f068a33b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab529b4740 1 --2- 172.21.15.16:0/3570753538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a51540 0x55f068a33b50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x55f068a62040 tx=0x7fab44031000 comp rx=0 tx=0).stop 2023-11-13T11:02:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab529b4740 1 -- 172.21.15.16:0/3570753538 shutdown_connections 2023-11-13T11:02:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab529b4740 1 --2- 172.21.15.16:0/3570753538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a68290 0x55f068a3a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab529b4740 1 --2- 172.21.15.16:0/3570753538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a64370 0x55f068a3a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab529b4740 1 --2- 172.21.15.16:0/3570753538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a51540 0x55f068a33b50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.899+0000 7fab529b4740 1 -- 172.21.15.16:0/3570753538 >> 172.21.15.16:0/3570753538 conn(0x55f068a345e0 msgr2=0x55f068a618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.900+0000 7fab529b4740 1 -- 172.21.15.16:0/3570753538 shutdown_connections 2023-11-13T11:02:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.900+0000 7fab529b4740 1 -- 172.21.15.16:0/3570753538 wait complete. 2023-11-13T11:02:45.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.900+0000 7fab529b4740 1 -- start start 2023-11-13T11:02:45.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.900+0000 7fab529b4740 1 Processor -- start 2023-11-13T11:02:45.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.901+0000 7fab529b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 0x55f068a53390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.901+0000 7fab529b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a64370 0x55f068a55610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.901+0000 7fab529b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a68290 0x55f068a55b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.901+0000 7fab529b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068a51de0 con 0x55f068a64370 2023-11-13T11:02:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.901+0000 7fab529b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f068a66760 con 0x55f068a51540 2023-11-13T11:02:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.901+0000 7fab529b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f068a66bc0 con 0x55f068a68290 2023-11-13T11:02:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.901+0000 7fab529b4740 10 client.0 _ll_register_callbacks cb 0x7fab3800c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab4d2dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a64370 0x55f068a55610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab4d2dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a64370 0x55f068a55610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38814/0 (socket says 172.21.15.16:38814) 2023-11-13T11:02:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab4d2dc640 1 -- 172.21.15.16:0/532331738 learned_addr learned my addr 172.21.15.16:0/532331738 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab4d2dc640 1 -- 172.21.15.16:0/532331738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a68290 msgr2=0x55f068a55b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab4d2dc640 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a68290 0x55f068a55b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab4d2dc640 1 -- 172.21.15.16:0/532331738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 msgr2=0x55f068a53390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:45.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab4d2dc640 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 0x55f068a53390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab4d2dc640 1 -- 172.21.15.16:0/532331738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f068962950 con 0x55f068a64370 2023-11-13T11:02:45.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab4d2dc640 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a64370 0x55f068a55610 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fab38008010 tx=0x7fab38011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384a4433c65f1012 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.902+0000 7fab28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab28ff9640 1 -- 172.21.15.16:0/532331738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38014bc0 con 0x55f068a64370 2023-11-13T11:02:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab28ff9640 1 -- 172.21.15.16:0/532331738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fab3800ec40 con 0x55f068a64370 2023-11-13T11:02:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab529b4740 1 -- 172.21.15.16:0/532331738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab4404a070 con 0x55f068a64370 2023-11-13T11:02:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab28ff9640 1 -- 172.21.15.16:0/532331738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab38013e90 con 0x55f068a64370 2023-11-13T11:02:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab529b4740 10 client.6492 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab529b4740 1 -- 172.21.15.16:0/532331738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55f0689611b0 con 0x55f068a64370 2023-11-13T11:02:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab28ff9640 1 -- 172.21.15.16:0/532331738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4404a070 con 0x55f068a64370 2023-11-13T11:02:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab28ff9640 1 -- 172.21.15.16:0/532331738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab3802d060 con 0x55f068a64370 2023-11-13T11:02:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab529b4740 10 client.6492 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab529b4740 10 client.6492 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab529b4740 20 client.6492 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab529b4740 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55f068ad0290 0x55f068aed8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab529b4740 4 client.6492 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2023-11-13T11:02:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.903+0000 7fab529b4740 1 -- 172.21.15.16:0/532331738 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55f068998ea0 con 0x55f068ad0290 2023-11-13T11:02:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.904+0000 7fab4dadd640 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55f068ad0290 0x55f068aed8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.904+0000 7fab4dadd640 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55f068ad0290 0x55f068aed8c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7744afa89ffabdf server_cookie=9135c46b7d102a11 in_seq=0 out_seq=0 2023-11-13T11:02:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.904+0000 7fab28ff9640 10 client.6492 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab28ff9640 1 -- 172.21.15.16:0/532331738 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f068998ea0 con 0x55f068ad0290 2023-11-13T11:02:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab28ff9640 10 client.6492 handle_command_reply: tid=0 2023-11-13T11:02:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 client.6492 shutdown 2023-11-13T11:02:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 -- 172.21.15.16:0/532331738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a64370 msgr2=0x55f068a55610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a64370 0x55f068a55610 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fab38008010 tx=0x7fab38011000 comp rx=0 tx=0).stop 2023-11-13T11:02:45.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 -- 172.21.15.16:0/532331738 shutdown_connections 2023-11-13T11:02:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55f068a68290 0x55f068a55b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55f068ad0290 0x55f068aed8c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f068a64370 0x55f068a55610 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 --2- 172.21.15.16:0/532331738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f068a51540 0x55f068a53390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab28ff9640 0 client.6492 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 -- 172.21.15.16:0/532331738 >> 172.21.15.16:0/532331738 conn(0x55f068a345e0 msgr2=0x55f068aa2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 -- 172.21.15.16:0/532331738 shutdown_connections 2023-11-13T11:02:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.905+0000 7fab529b4740 1 -- 172.21.15.16:0/532331738 wait complete. 2023-11-13T11:02:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab529b4740 20 client.6492 trim_cache size 0 max 16384 2023-11-13T11:02:45.925 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-11-13T11:02:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a972539a-2a6d-4456-9dec-feefd7c6b2f0", 2023-11-13T11:02:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-11-13T11:02:45.925 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 -- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab2c06ae40 msgr2=0x7fab2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab2c06ae40 0x7fab2c06d300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fab44036000 tx=0x7fab44030000 comp rx=0 tx=0).stop 2023-11-13T11:02:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 -- 172.21.15.16:0/2270754948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 msgr2=0x7fab48142060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 0x7fab48142060 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fab4000b9a0 tx=0x7fab4000be60 comp rx=0 tx=0).stop 2023-11-13T11:02:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 -- 172.21.15.16:0/2270754948 shutdown_connections 2023-11-13T11:02:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fab2c06ae40 0x7fab2c06d300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab48144570 0x7fab48142060 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab480ce550 0x7fab4814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 --2- 172.21.15.16:0/2270754948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab4806bb30 0x7fab4815e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 -- 172.21.15.16:0/2270754948 >> 172.21.15.16:0/2270754948 conn(0x7fab4806a190 msgr2=0x7fab48157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 -- 172.21.15.16:0/2270754948 shutdown_connections 2023-11-13T11:02:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:45.906+0000 7fab4fd41640 1 -- 172.21.15.16:0/2270754948 wait complete. 2023-11-13T11:02:45.930 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'a972539a-2a6d-4456-9dec-feefd7c6b2f0', 'mode': 'asynchronous'}' 2023-11-13T11:02:45.930 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2023-11-13T11:02:45.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2023-11-13T11:02:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.140+0000 7f4e710df640 1 Processor -- start 2023-11-13T11:02:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.140+0000 7f4e710df640 1 -- start start 2023-11-13T11:02:46.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.142+0000 7f4e710df640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4e6c15bcc0 0x7f4e6c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.142+0000 7f4e710df640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e6c0ce090 0x7f4e6c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.142+0000 7f4e710df640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c06d010 con 0x7f4e6c15b8e0 2023-11-13T11:02:46.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.142+0000 7f4e710df640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0ce540 con 0x7f4e6c0ce090 2023-11-13T11:02:46.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.142+0000 7f4e710df640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c06b860 con 0x7f4e6c15bcc0 2023-11-13T11:02:46.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.143+0000 7f4e708de640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e6c0ce090 0x7f4e6c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.143+0000 7f4e708de640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e6c0ce090 0x7f4e6c0cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38822/0 (socket says 172.21.15.16:38822) 2023-11-13T11:02:46.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.143+0000 7f4e708de640 1 -- 172.21.15.16:0/2009050657 learned_addr learned my addr 172.21.15.16:0/2009050657 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.143+0000 7f4e708de640 1 -- 172.21.15.16:0/2009050657 >> v2:172.21.15.184:3300/0 conn(0x7f4e6c15bcc0 msgr2=0x7f4e6c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.143+0000 7f4e6b7fe640 1 --2- 172.21.15.16:0/2009050657 >> v2:172.21.15.184:3300/0 conn(0x7f4e6c15bcc0 0x7f4e6c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.143+0000 7f4e708de640 1 --2- 172.21.15.16:0/2009050657 >> v2:172.21.15.184:3300/0 conn(0x7f4e6c15bcc0 0x7f4e6c0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.143+0000 7f4e708de640 1 -- 172.21.15.16:0/2009050657 >> v1:172.21.15.16:6789/0 conn(0x7f4e6c15b8e0 legacy=0x7f4e6c06c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.143+0000 7f4e708de640 1 -- 172.21.15.16:0/2009050657 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c06bb40 con 0x7f4e6c0ce090 2023-11-13T11:02:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.143+0000 7f4e6b7fe640 1 --2- 172.21.15.16:0/2009050657 >> v2:172.21.15.184:3300/0 conn(0x7f4e6c15bcc0 0x7f4e6c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.144+0000 7f4e708de640 1 --2- 172.21.15.16:0/2009050657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0ce090 0x7f4e6c0cbae0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c071d20 tx=0x7f4e6402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c564a4af8f7920d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.145+0000 7f4e6affd640 1 -- 172.21.15.16:0/2009050657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6402cb10 con 0x7f4e6c0ce090 2023-11-13T11:02:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.145+0000 7f4e6affd640 1 -- 172.21.15.16:0/2009050657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6402cc70 con 0x7f4e6c0ce090 2023-11-13T11:02:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.145+0000 7f4e6affd640 1 -- 172.21.15.16:0/2009050657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e640343f0 con 0x7f4e6c0ce090 2023-11-13T11:02:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.145+0000 7f4e710df640 1 -- 172.21.15.16:0/2009050657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0ce090 msgr2=0x7f4e6c0cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.145+0000 7f4e710df640 1 --2- 172.21.15.16:0/2009050657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0ce090 0x7f4e6c0cbae0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c071d20 tx=0x7f4e6402bf70 comp rx=0 tx=0).stop 2023-11-13T11:02:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.145+0000 7f4e710df640 1 -- 172.21.15.16:0/2009050657 shutdown_connections 2023-11-13T11:02:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.145+0000 7f4e710df640 1 --2- 172.21.15.16:0/2009050657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0ce090 0x7f4e6c0cbae0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.145+0000 7f4e710df640 1 --2- 172.21.15.16:0/2009050657 >> v2:172.21.15.184:3300/0 conn(0x7f4e6c15bcc0 0x7f4e6c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.145+0000 7f4e710df640 1 -- 172.21.15.16:0/2009050657 >> 172.21.15.16:0/2009050657 conn(0x7f4e6c0b0970 msgr2=0x7f4e6c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.146+0000 7f4e710df640 1 -- 172.21.15.16:0/2009050657 shutdown_connections 2023-11-13T11:02:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.146+0000 7f4e710df640 1 -- 172.21.15.16:0/2009050657 wait complete. 2023-11-13T11:02:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.147+0000 7f4e710df640 1 Processor -- start 2023-11-13T11:02:46.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.148+0000 7f4e710df640 1 -- start start 2023-11-13T11:02:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e710df640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e6c0ce090 0x7f4e6c1f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e710df640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e6c15b8e0 0x7f4e6c1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e710df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0acb50 0x7f4e6c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e710df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c06f940 con 0x7f4e6c0acb50 2023-11-13T11:02:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e710df640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c06fb90 con 0x7f4e6c0ce090 2023-11-13T11:02:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e710df640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c06fe70 con 0x7f4e6c15b8e0 2023-11-13T11:02:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e708de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0acb50 0x7f4e6c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e708de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0acb50 0x7f4e6c14bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38832/0 (socket says 172.21.15.16:38832) 2023-11-13T11:02:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e708de640 1 -- 172.21.15.16:0/1405631893 learned_addr learned my addr 172.21.15.16:0/1405631893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e708de640 1 -- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e6c15b8e0 msgr2=0x7f4e6c1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e708de640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e6c15b8e0 0x7f4e6c1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e708de640 1 -- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e6c0ce090 msgr2=0x7f4e6c1f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e708de640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e6c0ce090 0x7f4e6c1f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.149+0000 7f4e708de640 1 -- 172.21.15.16:0/1405631893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e64009b30 con 0x7f4e6c0acb50 2023-11-13T11:02:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.150+0000 7f4e708de640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0acb50 0x7f4e6c14bd60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4e64036040 tx=0x7f4e64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.150+0000 7f4e68ff9640 1 -- 172.21.15.16:0/1405631893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6403e450 con 0x7f4e6c0acb50 2023-11-13T11:02:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.150+0000 7f4e68ff9640 1 -- 172.21.15.16:0/1405631893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6403e5d0 con 0x7f4e6c0acb50 2023-11-13T11:02:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.150+0000 7f4e68ff9640 1 -- 172.21.15.16:0/1405631893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6403e980 con 0x7f4e6c0acb50 2023-11-13T11:02:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.150+0000 7f4e710df640 1 -- 172.21.15.16:0/1405631893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c070150 con 0x7f4e6c0acb50 2023-11-13T11:02:46.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.150+0000 7f4e710df640 1 -- 172.21.15.16:0/1405631893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c070710 con 0x7f4e6c0acb50 2023-11-13T11:02:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.154+0000 7f4e68ff9640 1 -- 172.21.15.16:0/1405631893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e6403eae0 con 0x7f4e6c0acb50 2023-11-13T11:02:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.154+0000 7f4e68ff9640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e5806ae40 0x7f4e5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.155+0000 7f4e68ff9640 1 -- 172.21.15.16:0/1405631893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4e640ac380 con 0x7f4e6c0acb50 2023-11-13T11:02:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.155+0000 7f4e74fb4740 1 Processor -- start 2023-11-13T11:02:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.155+0000 7f4e74fb4740 1 -- start start 2023-11-13T11:02:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.155+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e5806ae40 0x7f4e5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.156+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628fa28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.156+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f7f9d0 0x558628faa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.156+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628f82870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.156+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558628f3e2b0 con 0x558628fa2510 2023-11-13T11:02:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.156+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558628ef0160 con 0x558628f84b50 2023-11-13T11:02:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.156+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x558628eed850 con 0x558628f7f9d0 2023-11-13T11:02:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.156+0000 7f4e6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628fa28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.156+0000 7f4e6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628fa28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38842/0 (socket says 172.21.15.16:38842) 2023-11-13T11:02:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.156+0000 7f4e6bfff640 1 -- 172.21.15.16:0/2577023332 learned_addr learned my addr 172.21.15.16:0/2577023332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e5806ae40 0x7f4e5806d300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c000f60 tx=0x7f4e5c0079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e6b7fe640 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f7f9d0 0x558628faa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e708de640 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628f82870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e6bfff640 1 -- 172.21.15.16:0/2577023332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f7f9d0 msgr2=0x558628faa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f7f9d0 0x558628faa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e6bfff640 1 -- 172.21.15.16:0/2577023332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 msgr2=0x558628f82870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628f82870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e6bfff640 1 -- 172.21.15.16:0/2577023332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558628eabc20 con 0x558628fa2510 2023-11-13T11:02:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e6b7fe640 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f7f9d0 0x558628faa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628fa28f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0166d0 tx=0x7f4e5c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268c84950c773ce9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e527fc640 1 -- 172.21.15.16:0/2577023332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c01b070 con 0x558628fa2510 2023-11-13T11:02:46.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e527fc640 1 -- 172.21.15.16:0/2577023332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c0050d0 con 0x558628fa2510 2023-11-13T11:02:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.157+0000 7f4e527fc640 1 -- 172.21.15.16:0/2577023332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c005460 con 0x558628fa2510 2023-11-13T11:02:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.158+0000 7f4e74fb4740 1 -- 172.21.15.16:0/2577023332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 msgr2=0x558628fa28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.158+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628fa28f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0166d0 tx=0x7f4e5c00f000 comp rx=0 tx=0).stop 2023-11-13T11:02:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.158+0000 7f4e74fb4740 1 -- 172.21.15.16:0/2577023332 shutdown_connections 2023-11-13T11:02:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.158+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628f82870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.158+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f7f9d0 0x558628faa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.158+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/2577023332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628fa28f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.158+0000 7f4e74fb4740 1 -- 172.21.15.16:0/2577023332 >> 172.21.15.16:0/2577023332 conn(0x558628f44770 msgr2=0x558628fc8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.158+0000 7f4e74fb4740 1 -- 172.21.15.16:0/2577023332 shutdown_connections 2023-11-13T11:02:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.158+0000 7f4e74fb4740 1 -- 172.21.15.16:0/2577023332 wait complete. 2023-11-13T11:02:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.159+0000 7f4e74fb4740 1 -- start start 2023-11-13T11:02:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.159+0000 7f4e74fb4740 1 Processor -- start 2023-11-13T11:02:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.160+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f84b50 0x558628fc8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.160+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628fa2510 0x558628fafcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.160+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x558628fc9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.160+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558628ee9140 con 0x558628fb0210 2023-11-13T11:02:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.160+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558628e85670 con 0x558628fa2510 2023-11-13T11:02:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.160+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x558628ea2f40 con 0x558628f84b50 2023-11-13T11:02:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.160+0000 7f4e74fb4740 10 client.0 _ll_register_callbacks cb 0x558628edf270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e708de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x558628fc9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e708de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x558628fc9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38858/0 (socket says 172.21.15.16:38858) 2023-11-13T11:02:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e708de640 1 -- 172.21.15.16:0/1303463997 learned_addr learned my addr 172.21.15.16:0/1303463997 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e6b7fe640 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628fa2510 0x558628fafcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f84b50 0x558628fc8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e708de640 1 -- 172.21.15.16:0/1303463997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f84b50 msgr2=0x558628fc8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e708de640 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f84b50 0x558628fc8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e708de640 1 -- 172.21.15.16:0/1303463997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628fa2510 msgr2=0x558628fafcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e708de640 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628fa2510 0x558628fafcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e708de640 1 -- 172.21.15.16:0/1303463997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c06d010 con 0x558628fb0210 2023-11-13T11:02:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f84b50 0x558628fc8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e708de640 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x558628fc9640 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4e6407ae60 tx=0x7f4e6407a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf97d14991fe2f1c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.161+0000 7f4e4affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e4affd640 1 -- 172.21.15.16:0/1303463997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6407f020 con 0x558628fb0210 2023-11-13T11:02:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e4affd640 1 -- 172.21.15.16:0/1303463997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4e640831d0 con 0x558628fb0210 2023-11-13T11:02:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1303463997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558628ee2120 con 0x558628fb0210 2023-11-13T11:02:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e4affd640 1 -- 172.21.15.16:0/1303463997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e64079270 con 0x558628fb0210 2023-11-13T11:02:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e74fb4740 10 client.6507 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1303463997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x558628ee77c0 con 0x558628fb0210 2023-11-13T11:02:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e4affd640 1 -- 172.21.15.16:0/1303463997 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558628ee2120 con 0x558628fb0210 2023-11-13T11:02:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e4affd640 1 -- 172.21.15.16:0/1303463997 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e6403c070 con 0x558628fb0210 2023-11-13T11:02:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e74fb4740 10 client.6507 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e74fb4740 10 client.6507 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e74fb4740 20 client.6507 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x558628fcb8b0 0x558628fcbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e74fb4740 4 client.6507 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.162+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1303463997 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558628ee1e40 con 0x558628fcb8b0 2023-11-13T11:02:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.163+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x558628fcb8b0 0x558628fcbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.163+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x558628fcb8b0 0x558628fcbcc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba8bdf84ffd277fc server_cookie=af9fc85b07b93ca4 in_seq=0 out_seq=0 2023-11-13T11:02:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.163+0000 7f4e4affd640 10 client.6507 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e4affd640 1 -- 172.21.15.16:0/1303463997 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558628ee1e40 con 0x558628fcb8b0 2023-11-13T11:02:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e4affd640 10 client.6507 handle_command_reply: tid=0 2023-11-13T11:02:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 client.6507 shutdown 2023-11-13T11:02:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1303463997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 msgr2=0x558628fc9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x558628fc9640 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4e6407ae60 tx=0x7f4e6407a8f0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1303463997 shutdown_connections 2023-11-13T11:02:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x558628fcb8b0 0x558628fcbcc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x558628fc9640 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628fa2510 0x558628fafcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1303463997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628f84b50 0x558628fc8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1303463997 >> 172.21.15.16:0/1303463997 conn(0x558628f44770 msgr2=0x558628fa9910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e4affd640 0 client.6507 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.164+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1303463997 shutdown_connections 2023-11-13T11:02:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.165+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1303463997 wait complete. 2023-11-13T11:02:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.165+0000 7f4e74fb4740 20 client.6507 trim_cache size 0 max 16384 2023-11-13T11:02:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.181+0000 7f4e74fb4740 1 Processor -- start 2023-11-13T11:02:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.181+0000 7f4e74fb4740 1 -- start start 2023-11-13T11:02:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.182+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628f9f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.182+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628f9f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.182+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fb0210 0x558628fadb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.182+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558628c4c740 con 0x558628fa2510 2023-11-13T11:02:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.182+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558628f9f540 con 0x558628f84b50 2023-11-13T11:02:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.182+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x558628fcc630 con 0x558628fb0210 2023-11-13T11:02:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.182+0000 7f4e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628f9f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.182+0000 7f4e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628f9f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38862/0 (socket says 172.21.15.16:38862) 2023-11-13T11:02:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.182+0000 7f4e6b7fe640 1 -- 172.21.15.16:0/1909497133 learned_addr learned my addr 172.21.15.16:0/1909497133 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/1909497133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628f9f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e708de640 1 --2- 172.21.15.16:0/1909497133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fb0210 0x558628fadb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e6b7fe640 1 -- 172.21.15.16:0/1909497133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fb0210 msgr2=0x558628fadb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e6b7fe640 1 --2- 172.21.15.16:0/1909497133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fb0210 0x558628fadb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e6b7fe640 1 -- 172.21.15.16:0/1909497133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 msgr2=0x558628f9f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e6b7fe640 1 --2- 172.21.15.16:0/1909497133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628f9f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e6b7fe640 1 -- 172.21.15.16:0/1909497133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558628e84f00 con 0x558628fa2510 2023-11-13T11:02:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e6b7fe640 1 --2- 172.21.15.16:0/1909497133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628f9f000 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000c820 tx=0x7f4e6000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab067b4dc0fb0985 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e52ffd640 1 -- 172.21.15.16:0/1909497133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6000ec20 con 0x558628fa2510 2023-11-13T11:02:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e52ffd640 1 -- 172.21.15.16:0/1909497133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6000ed80 con 0x558628fa2510 2023-11-13T11:02:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.183+0000 7f4e52ffd640 1 -- 172.21.15.16:0/1909497133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60012670 con 0x558628fa2510 2023-11-13T11:02:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.184+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1909497133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 msgr2=0x558628f9f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.184+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1909497133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628f9f000 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000c820 tx=0x7f4e6000cce0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.184+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1909497133 shutdown_connections 2023-11-13T11:02:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.184+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1909497133 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fb0210 0x558628fadb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.184+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1909497133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fa2510 0x558628f9f000 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.184+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/1909497133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628f9f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.184+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1909497133 >> 172.21.15.16:0/1909497133 conn(0x558628f44770 msgr2=0x558628fc8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.184+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1909497133 shutdown_connections 2023-11-13T11:02:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.184+0000 7f4e74fb4740 1 -- 172.21.15.16:0/1909497133 wait complete. 2023-11-13T11:02:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.185+0000 7f4e74fb4740 1 -- start start 2023-11-13T11:02:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.185+0000 7f4e74fb4740 1 Processor -- start 2023-11-13T11:02:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628fda5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fa2510 0x558628fdaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e74fb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x55862903b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558628feb6d0 con 0x558628fb0210 2023-11-13T11:02:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862903c0d0 con 0x558628f84b50 2023-11-13T11:02:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e74fb4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x558628fdb1d0 con 0x558628fa2510 2023-11-13T11:02:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e74fb4740 10 client.0 _ll_register_callbacks cb 0x7f4e5c016440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e708de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x55862903b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e708de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x55862903b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38868/0 (socket says 172.21.15.16:38868) 2023-11-13T11:02:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.186+0000 7f4e708de640 1 -- 172.21.15.16:0/4031525353 learned_addr learned my addr 172.21.15.16:0/4031525353 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628fda5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e6b7fe640 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fa2510 0x558628fdaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e708de640 1 -- 172.21.15.16:0/4031525353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fa2510 msgr2=0x558628fdaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e708de640 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fa2510 0x558628fdaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e708de640 1 -- 172.21.15.16:0/4031525353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 msgr2=0x558628fda5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e708de640 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628fda5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e708de640 1 -- 172.21.15.16:0/4031525353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6000c480 con 0x558628fb0210 2023-11-13T11:02:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628fda5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e708de640 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x55862903b1d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4e6407afb0 tx=0x7f4e640aa560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd43e93cf613bad server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e4affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e4affd640 1 -- 172.21.15.16:0/4031525353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64079270 con 0x558628fb0210 2023-11-13T11:02:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e4affd640 1 -- 172.21.15.16:0/4031525353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4e640831d0 con 0x558628fb0210 2023-11-13T11:02:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.187+0000 7f4e74fb4740 1 -- 172.21.15.16:0/4031525353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e5c0050d0 con 0x558628fb0210 2023-11-13T11:02:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e4affd640 1 -- 172.21.15.16:0/4031525353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e6407f020 con 0x558628fb0210 2023-11-13T11:02:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e74fb4740 10 client.6513 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e74fb4740 1 -- 172.21.15.16:0/4031525353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x558628eaa480 con 0x558628fb0210 2023-11-13T11:02:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e4affd640 1 -- 172.21.15.16:0/4031525353 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6409e020 con 0x558628fb0210 2023-11-13T11:02:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e4affd640 1 -- 172.21.15.16:0/4031525353 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e6403c040 con 0x558628fb0210 2023-11-13T11:02:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e74fb4740 10 client.6513 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e74fb4740 10 client.6513 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e74fb4740 20 client.6513 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x558628fa4330 0x55862903b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e74fb4740 4 client.6513 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2023-11-13T11:02:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e74fb4740 1 -- 172.21.15.16:0/4031525353 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x558628be9ee0 con 0x558628fa4330 2023-11-13T11:02:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.188+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x558628fa4330 0x55862903b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.189+0000 7f4e6bfff640 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x558628fa4330 0x55862903b710 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a41ea7df222fee server_cookie=59ad866537f02c6d in_seq=0 out_seq=0 2023-11-13T11:02:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.189+0000 7f4e4affd640 10 client.6513 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.189+0000 7f4e4affd640 1 -- 172.21.15.16:0/4031525353 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558628be9ee0 con 0x558628fa4330 2023-11-13T11:02:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.189+0000 7f4e4affd640 10 client.6513 handle_command_reply: tid=0 2023-11-13T11:02:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 client.6513 shutdown 2023-11-13T11:02:46.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 -- 172.21.15.16:0/4031525353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 msgr2=0x55862903b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x55862903b1d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4e6407afb0 tx=0x7f4e640aa560 comp rx=0 tx=0).stop 2023-11-13T11:02:46.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 -- 172.21.15.16:0/4031525353 shutdown_connections 2023-11-13T11:02:46.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x558628fa4330 0x55862903b710 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558628fb0210 0x55862903b1d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x558628fa2510 0x558628fdaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 --2- 172.21.15.16:0/4031525353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558628f84b50 0x558628fda5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e4affd640 0 client.6513 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:46.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 -- 172.21.15.16:0/4031525353 >> 172.21.15.16:0/4031525353 conn(0x558628f44770 msgr2=0x558628fa28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 -- 172.21.15.16:0/4031525353 shutdown_connections 2023-11-13T11:02:46.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 1 -- 172.21.15.16:0/4031525353 wait complete. 2023-11-13T11:02:46.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.190+0000 7f4e74fb4740 20 client.6513 trim_cache size 0 max 16384 2023-11-13T11:02:46.210 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:46.211 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-11-13T11:02:46.211 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "460529c5-65ae-4c8f-947c-5cfe7fdcd447", 2023-11-13T11:02:46.211 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-11-13T11:02:46.211 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:46.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 -- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e5806ae40 msgr2=0x7f4e5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e5806ae40 0x7f4e5806d300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c000f60 tx=0x7f4e5c0079d0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 -- 172.21.15.16:0/1405631893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0acb50 msgr2=0x7f4e6c14bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0acb50 0x7f4e6c14bd60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4e64036040 tx=0x7f4e64004a40 comp rx=0 tx=0).stop 2023-11-13T11:02:46.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 -- 172.21.15.16:0/1405631893 shutdown_connections 2023-11-13T11:02:46.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e5806ae40 0x7f4e5806d300 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e6c0acb50 0x7f4e6c14bd60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e6c15b8e0 0x7f4e6c1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 --2- 172.21.15.16:0/1405631893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e6c0ce090 0x7f4e6c1f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 -- 172.21.15.16:0/1405631893 >> 172.21.15.16:0/1405631893 conn(0x7f4e6c0b0970 msgr2=0x7f4e6c15c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 -- 172.21.15.16:0/1405631893 shutdown_connections 2023-11-13T11:02:46.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.191+0000 7f4e710df640 1 -- 172.21.15.16:0/1405631893 wait complete. 2023-11-13T11:02:46.215 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '460529c5-65ae-4c8f-947c-5cfe7fdcd447', 'mode': 'asynchronous'}' 2023-11-13T11:02:46.216 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2023-11-13T11:02:46.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2023-11-13T11:02:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.433+0000 7fcf049c4640 1 Processor -- start 2023-11-13T11:02:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.433+0000 7fcf049c4640 1 -- start start 2023-11-13T11:02:46.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.435+0000 7fcf049c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf000b2290 0x7fcf000b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.435+0000 7fcf049c4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf000ad730 0x7fcf000adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.435+0000 7fcf049c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000ccab0 con 0x7fcf00159bd0 2023-11-13T11:02:46.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.435+0000 7fcf049c4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf000ae100 con 0x7fcf000ad730 2023-11-13T11:02:46.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.435+0000 7fcf049c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000ccd30 con 0x7fcf000b2290 2023-11-13T11:02:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.435+0000 7fceff59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf000ad730 0x7fcf000adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.435+0000 7fceff59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf000ad730 0x7fcf000adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38876/0 (socket says 172.21.15.16:38876) 2023-11-13T11:02:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.435+0000 7fceff59e640 1 -- 172.21.15.16:0/2076769216 learned_addr learned my addr 172.21.15.16:0/2076769216 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.436+0000 7fcefe59c640 1 --2- 172.21.15.16:0/2076769216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf000b2290 0x7fcf000b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.436+0000 7fceff59e640 1 -- 172.21.15.16:0/2076769216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf000b2290 msgr2=0x7fcf000b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.436+0000 7fceff59e640 1 --2- 172.21.15.16:0/2076769216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf000b2290 0x7fcf000b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.436+0000 7fceff59e640 1 -- 172.21.15.16:0/2076769216 >> v1:172.21.15.184:6789/0 conn(0x7fcf00159bd0 legacy=0x7fcf0006f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.436+0000 7fceff59e640 1 -- 172.21.15.16:0/2076769216 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf000cd010 con 0x7fcf000ad730 2023-11-13T11:02:46.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.437+0000 7fceff59e640 1 --2- 172.21.15.16:0/2076769216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 0x7fcf000adbc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009e50 tx=0x7fcef802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9895d826c8eed855 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.437+0000 7fcefdd9b640 1 -- 172.21.15.16:0/2076769216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef802cb00 con 0x7fcf000ad730 2023-11-13T11:02:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.437+0000 7fcefdd9b640 1 -- 172.21.15.16:0/2076769216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef802cc60 con 0x7fcf000ad730 2023-11-13T11:02:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.437+0000 7fcefdd9b640 1 -- 172.21.15.16:0/2076769216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80343f0 con 0x7fcf000ad730 2023-11-13T11:02:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.438+0000 7fcf049c4640 1 -- 172.21.15.16:0/2076769216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 msgr2=0x7fcf000adbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.438+0000 7fcf049c4640 1 --2- 172.21.15.16:0/2076769216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 0x7fcf000adbc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009e50 tx=0x7fcef802bf60 comp rx=0 tx=0).stop 2023-11-13T11:02:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.438+0000 7fcf049c4640 1 -- 172.21.15.16:0/2076769216 shutdown_connections 2023-11-13T11:02:46.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.438+0000 7fcf049c4640 1 --2- 172.21.15.16:0/2076769216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 0x7fcf000adbc0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.438+0000 7fcf049c4640 1 --2- 172.21.15.16:0/2076769216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf000b2290 0x7fcf000b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.438+0000 7fcf049c4640 1 -- 172.21.15.16:0/2076769216 >> 172.21.15.16:0/2076769216 conn(0x7fcf000b0970 msgr2=0x7fcf000b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.438+0000 7fcf049c4640 1 -- 172.21.15.16:0/2076769216 shutdown_connections 2023-11-13T11:02:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.438+0000 7fcf049c4640 1 -- 172.21.15.16:0/2076769216 wait complete. 2023-11-13T11:02:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.440+0000 7fcf049c4640 1 Processor -- start 2023-11-13T11:02:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.440+0000 7fcf049c4640 1 -- start start 2023-11-13T11:02:46.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.441+0000 7fcf049c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 0x7fcf001f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.441+0000 7fcf049c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf000b2290 0x7fcf0015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.441+0000 7fcf049c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf000af750 0x7fcf000afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.441+0000 7fcf049c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b0140 con 0x7fcf000ad730 2023-11-13T11:02:46.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.441+0000 7fcf049c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf00152aa0 con 0x7fcf000b2290 2023-11-13T11:02:46.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.441+0000 7fcf049c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf00157640 con 0x7fcf000af750 2023-11-13T11:02:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.441+0000 7fcefed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 0x7fcf001f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.441+0000 7fcefed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 0x7fcf001f2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38890/0 (socket says 172.21.15.16:38890) 2023-11-13T11:02:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.441+0000 7fcefed9d640 1 -- 172.21.15.16:0/2151632682 learned_addr learned my addr 172.21.15.16:0/2151632682 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcefe59c640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf000b2290 0x7fcf0015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcefed9d640 1 -- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf000af750 msgr2=0x7fcf000afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcefed9d640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf000af750 0x7fcf000afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcefed9d640 1 -- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf000b2290 msgr2=0x7fcf0015ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcefed9d640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf000b2290 0x7fcf0015ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcefed9d640 1 -- 172.21.15.16:0/2151632682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8009b30 con 0x7fcf000ad730 2023-11-13T11:02:46.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcefed9d640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 0x7fcf001f2cc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c400 tx=0x7fcef000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcef77fe640 1 -- 172.21.15.16:0/2151632682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00052d0 con 0x7fcf000ad730 2023-11-13T11:02:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcef77fe640 1 -- 172.21.15.16:0/2151632682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0005430 con 0x7fcf000ad730 2023-11-13T11:02:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcef77fe640 1 -- 172.21.15.16:0/2151632682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0003830 con 0x7fcf000ad730 2023-11-13T11:02:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.442+0000 7fcf049c4640 1 -- 172.21.15.16:0/2151632682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf001f3200 con 0x7fcf000ad730 2023-11-13T11:02:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.443+0000 7fcf049c4640 1 -- 172.21.15.16:0/2151632682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf00156d10 con 0x7fcf000ad730 2023-11-13T11:02:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.445+0000 7fcef77fe640 1 -- 172.21.15.16:0/2151632682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcef0003990 con 0x7fcf000ad730 2023-11-13T11:02:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.446+0000 7fcef77fe640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcee006ae40 0x7fcee006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.446+0000 7fcef77fe640 1 -- 172.21.15.16:0/2151632682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fcef0088d90 con 0x7fcf000ad730 2023-11-13T11:02:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.446+0000 7fcf07e31740 1 Processor -- start 2023-11-13T11:02:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.447+0000 7fcf07e31740 1 -- start start 2023-11-13T11:02:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.447+0000 7fcefe59c640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcee006ae40 0x7fcee006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.447+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 0x5627fcd35980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.447+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd055c0 0x5627fcd32810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.447+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd32d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.447+0000 7fcf07e31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fccc69b0 con 0x5627fcd055c0 2023-11-13T11:02:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.447+0000 7fcf07e31740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fcc78cb0 con 0x5627fcd09120 2023-11-13T11:02:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.447+0000 7fcf07e31740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627fcc763a0 con 0x5627fcd355a0 2023-11-13T11:02:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd055c0 0x5627fcd32810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd055c0 0x5627fcd32810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38896/0 (socket says 172.21.15.16:38896) 2023-11-13T11:02:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefe59c640 1 -- 172.21.15.16:0/2342676040 learned_addr learned my addr 172.21.15.16:0/2342676040 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefed9d640 1 --2- 172.21.15.16:0/2342676040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 0x5627fcd35980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fceff59e640 1 --2- 172.21.15.16:0/2342676040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd32d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefe59c640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcee006ae40 0x7fcee006d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcf000b02c0 tx=0x7fcee8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefe59c640 1 -- 172.21.15.16:0/2342676040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 msgr2=0x5627fcd35980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefe59c640 1 --2- 172.21.15.16:0/2342676040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 0x5627fcd35980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefe59c640 1 -- 172.21.15.16:0/2342676040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 msgr2=0x5627fcd32d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefe59c640 1 --2- 172.21.15.16:0/2342676040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd32d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.448+0000 7fcefe59c640 1 -- 172.21.15.16:0/2342676040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627fcc345e0 con 0x5627fcd055c0 2023-11-13T11:02:46.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.449+0000 7fcefe59c640 1 --2- 172.21.15.16:0/2342676040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd055c0 0x5627fcd32810 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e870 tx=0x7fcee8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9e7c0c7d4e1973 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.449+0000 7fcef4ff9640 1 -- 172.21.15.16:0/2342676040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8015640 con 0x5627fcd055c0 2023-11-13T11:02:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.449+0000 7fcef4ff9640 1 -- 172.21.15.16:0/2342676040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee80157a0 con 0x5627fcd055c0 2023-11-13T11:02:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.449+0000 7fcef4ff9640 1 -- 172.21.15.16:0/2342676040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8015b50 con 0x5627fcd055c0 2023-11-13T11:02:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.450+0000 7fcf07e31740 1 -- 172.21.15.16:0/2342676040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd055c0 msgr2=0x5627fcd32810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.450+0000 7fcf07e31740 1 --2- 172.21.15.16:0/2342676040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd055c0 0x5627fcd32810 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e870 tx=0x7fcee8011000 comp rx=0 tx=0).stop 2023-11-13T11:02:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.450+0000 7fcf07e31740 1 -- 172.21.15.16:0/2342676040 shutdown_connections 2023-11-13T11:02:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.450+0000 7fcf07e31740 1 --2- 172.21.15.16:0/2342676040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd32d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.450+0000 7fcf07e31740 1 --2- 172.21.15.16:0/2342676040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd055c0 0x5627fcd32810 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.450+0000 7fcf07e31740 1 --2- 172.21.15.16:0/2342676040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 0x5627fcd35980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.450+0000 7fcf07e31740 1 -- 172.21.15.16:0/2342676040 >> 172.21.15.16:0/2342676040 conn(0x5627fcd07830 msgr2=0x5627fcd503b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.450+0000 7fcf07e31740 1 -- 172.21.15.16:0/2342676040 shutdown_connections 2023-11-13T11:02:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.450+0000 7fcf07e31740 1 -- 172.21.15.16:0/2342676040 wait complete. 2023-11-13T11:02:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.451+0000 7fcf07e31740 1 -- start start 2023-11-13T11:02:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.451+0000 7fcf07e31740 1 Processor -- start 2023-11-13T11:02:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 0x5627fcd2d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd63f00 0x5627fcd2b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fcf07e31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fcc71c90 con 0x5627fcd63f00 2023-11-13T11:02:46.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fcf07e31740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fcc0df00 con 0x5627fcd09120 2023-11-13T11:02:46.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fcf07e31740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627fcc2b880 con 0x5627fcd355a0 2023-11-13T11:02:46.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fcf07e31740 10 client.0 _ll_register_callbacks cb 0x5627fcc2dee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:46.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fceff59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd63f00 0x5627fcd2b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fceff59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd63f00 0x5627fcd2b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38898/0 (socket says 172.21.15.16:38898) 2023-11-13T11:02:46.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.452+0000 7fceff59e640 1 -- 172.21.15.16:0/3203130582 learned_addr learned my addr 172.21.15.16:0/3203130582 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fcefed9d640 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fcefe59c640 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 0x5627fcd2d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fceff59e640 1 -- 172.21.15.16:0/3203130582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 msgr2=0x5627fcd2d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fceff59e640 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 0x5627fcd2d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fceff59e640 1 -- 172.21.15.16:0/3203130582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 msgr2=0x5627fcd2ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fceff59e640 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fceff59e640 1 -- 172.21.15.16:0/3203130582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee8009e20 con 0x5627fcd63f00 2023-11-13T11:02:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fcefed9d640 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fceff59e640 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd63f00 0x5627fcd2b5f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009e50 tx=0x7fcef8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678d054e187c2502 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fceddffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fceddffb640 1 -- 172.21.15.16:0/3203130582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8034bf0 con 0x5627fcd63f00 2023-11-13T11:02:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fceddffb640 1 -- 172.21.15.16:0/3203130582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fcef8034d50 con 0x5627fcd63f00 2023-11-13T11:02:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.453+0000 7fcf07e31740 1 -- 172.21.15.16:0/3203130582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627fcc6ac70 con 0x5627fcd63f00 2023-11-13T11:02:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fceddffb640 1 -- 172.21.15.16:0/3203130582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcef803e430 con 0x5627fcd63f00 2023-11-13T11:02:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fcf07e31740 10 client.6525 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fcf07e31740 1 -- 172.21.15.16:0/3203130582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5627fcc70310 con 0x5627fcd63f00 2023-11-13T11:02:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fceddffb640 1 -- 172.21.15.16:0/3203130582 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627fcc6ac70 con 0x5627fcd63f00 2023-11-13T11:02:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fceddffb640 1 -- 172.21.15.16:0/3203130582 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcef803c040 con 0x5627fcd63f00 2023-11-13T11:02:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fcf07e31740 10 client.6525 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fcf07e31740 10 client.6525 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fcf07e31740 20 client.6525 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fcf07e31740 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5627fcd370e0 0x5627fcd374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fcf07e31740 4 client.6525 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fcf07e31740 1 -- 172.21.15.16:0/3203130582 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627fcc6a990 con 0x5627fcd370e0 2023-11-13T11:02:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.454+0000 7fcefed9d640 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5627fcd370e0 0x5627fcd374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.455+0000 7fcefed9d640 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5627fcd370e0 0x5627fcd374c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2760ece5fb66561 server_cookie=8223cbb4ef9890b5 in_seq=0 out_seq=0 2023-11-13T11:02:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.455+0000 7fceddffb640 10 client.6525 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.455+0000 7fceddffb640 1 -- 172.21.15.16:0/3203130582 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5627fcc6a990 con 0x5627fcd370e0 2023-11-13T11:02:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fceddffb640 10 client.6525 handle_command_reply: tid=0 2023-11-13T11:02:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 client.6525 shutdown 2023-11-13T11:02:46.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 -- 172.21.15.16:0/3203130582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd63f00 msgr2=0x5627fcd2b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd63f00 0x5627fcd2b5f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009e50 tx=0x7fcef8004a40 comp rx=0 tx=0).stop 2023-11-13T11:02:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 -- 172.21.15.16:0/3203130582 shutdown_connections 2023-11-13T11:02:46.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5627fcd370e0 0x5627fcd374c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd63f00 0x5627fcd2b5f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd355a0 0x5627fcd2d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 --2- 172.21.15.16:0/3203130582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fceddffb640 0 client.6525 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:46.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 -- 172.21.15.16:0/3203130582 >> 172.21.15.16:0/3203130582 conn(0x5627fcd055c0 msgr2=0x5627fcd39120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 -- 172.21.15.16:0/3203130582 shutdown_connections 2023-11-13T11:02:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 1 -- 172.21.15.16:0/3203130582 wait complete. 2023-11-13T11:02:46.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.456+0000 7fcf07e31740 20 client.6525 trim_cache size 0 max 16384 2023-11-13T11:02:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.473+0000 7fcf07e31740 1 Processor -- start 2023-11-13T11:02:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.473+0000 7fcf07e31740 1 -- start start 2023-11-13T11:02:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.474+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd09120 0x5627fcd2d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.474+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd355a0 0x5627fcd600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.474+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd63f00 0x5627fcd538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.474+0000 7fcf07e31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fc9d4ed0 con 0x5627fcd09120 2023-11-13T11:02:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.474+0000 7fcf07e31740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fcd73ce0 con 0x5627fcd355a0 2023-11-13T11:02:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.474+0000 7fcf07e31740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627fcd321f0 con 0x5627fcd63f00 2023-11-13T11:02:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.474+0000 7fcefed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd09120 0x5627fcd2d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.474+0000 7fcefed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd09120 0x5627fcd2d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38902/0 (socket says 172.21.15.16:38902) 2023-11-13T11:02:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.474+0000 7fcefed9d640 1 -- 172.21.15.16:0/237929304 learned_addr learned my addr 172.21.15.16:0/237929304 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcefed9d640 1 -- 172.21.15.16:0/237929304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd63f00 msgr2=0x5627fcd538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcefed9d640 1 --2- 172.21.15.16:0/237929304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd63f00 0x5627fcd538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcefe59c640 1 --2- 172.21.15.16:0/237929304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd355a0 0x5627fcd600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcefed9d640 1 -- 172.21.15.16:0/237929304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd355a0 msgr2=0x5627fcd600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcefed9d640 1 --2- 172.21.15.16:0/237929304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd355a0 0x5627fcd600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcefed9d640 1 -- 172.21.15.16:0/237929304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627fcc0d790 con 0x5627fcd09120 2023-11-13T11:02:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcefed9d640 1 --2- 172.21.15.16:0/237929304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd09120 0x5627fcd2d560 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcef0057870 tx=0x7fcef0001220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6958ee8275c36d87 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcef57fa640 1 -- 172.21.15.16:0/237929304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef006f1c0 con 0x5627fcd09120 2023-11-13T11:02:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcef57fa640 1 -- 172.21.15.16:0/237929304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef00871b0 con 0x5627fcd09120 2023-11-13T11:02:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.475+0000 7fcef57fa640 1 -- 172.21.15.16:0/237929304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0053ff0 con 0x5627fcd09120 2023-11-13T11:02:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.476+0000 7fcf07e31740 1 -- 172.21.15.16:0/237929304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd09120 msgr2=0x5627fcd2d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.476+0000 7fcf07e31740 1 --2- 172.21.15.16:0/237929304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd09120 0x5627fcd2d560 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcef0057870 tx=0x7fcef0001220 comp rx=0 tx=0).stop 2023-11-13T11:02:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.476+0000 7fcf07e31740 1 -- 172.21.15.16:0/237929304 shutdown_connections 2023-11-13T11:02:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.476+0000 7fcf07e31740 1 --2- 172.21.15.16:0/237929304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd63f00 0x5627fcd538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.476+0000 7fcf07e31740 1 --2- 172.21.15.16:0/237929304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd355a0 0x5627fcd600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.476+0000 7fcf07e31740 1 --2- 172.21.15.16:0/237929304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd09120 0x5627fcd2d560 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.476+0000 7fcf07e31740 1 -- 172.21.15.16:0/237929304 >> 172.21.15.16:0/237929304 conn(0x5627fcd055c0 msgr2=0x5627fcd733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.476+0000 7fcf07e31740 1 -- 172.21.15.16:0/237929304 shutdown_connections 2023-11-13T11:02:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.476+0000 7fcf07e31740 1 -- 172.21.15.16:0/237929304 wait complete. 2023-11-13T11:02:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.477+0000 7fcf07e31740 1 -- start start 2023-11-13T11:02:46.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.477+0000 7fcf07e31740 1 Processor -- start 2023-11-13T11:02:46.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd355a0 0x5627fcd0a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcf07e31740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd63f00 0x5627fcd630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcf07e31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fcd36f20 con 0x5627fcd355a0 2023-11-13T11:02:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcf07e31740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fcd0b2f0 con 0x5627fcd09120 2023-11-13T11:02:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcf07e31740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627fcd29300 con 0x5627fcd63f00 2023-11-13T11:02:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcf07e31740 10 client.0 _ll_register_callbacks cb 0x7fcef8009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcefe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd355a0 0x5627fcd0a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcefe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd355a0 0x5627fcd0a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38916/0 (socket says 172.21.15.16:38916) 2023-11-13T11:02:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fcefe59c640 1 -- 172.21.15.16:0/1402209517 learned_addr learned my addr 172.21.15.16:0/1402209517 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.478+0000 7fceff59e640 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd63f00 0x5627fcd630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fcefed9d640 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fcefe59c640 1 -- 172.21.15.16:0/1402209517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd63f00 msgr2=0x5627fcd630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fcefe59c640 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd63f00 0x5627fcd630b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fcefe59c640 1 -- 172.21.15.16:0/1402209517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 msgr2=0x5627fcd2c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fcefe59c640 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fcefe59c640 1 -- 172.21.15.16:0/1402209517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf00156d10 con 0x5627fcd355a0 2023-11-13T11:02:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fcefed9d640 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fcefe59c640 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd355a0 0x5627fcd0a8e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcee8006fd0 tx=0x7fcee8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dc9d856e63eec9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fceddffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fceddffb640 1 -- 172.21.15.16:0/1402209517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee802b680 con 0x5627fcd355a0 2023-11-13T11:02:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fceddffb640 1 -- 172.21.15.16:0/1402209517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fcee802b7e0 con 0x5627fcd355a0 2023-11-13T11:02:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.479+0000 7fcf07e31740 1 -- 172.21.15.16:0/1402209517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcee80157a0 con 0x5627fcd355a0 2023-11-13T11:02:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fceddffb640 1 -- 172.21.15.16:0/1402209517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee802b9a0 con 0x5627fcd355a0 2023-11-13T11:02:46.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fcf07e31740 10 client.6531 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:46.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fcf07e31740 1 -- 172.21.15.16:0/1402209517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5627fcc32e40 con 0x5627fcd355a0 2023-11-13T11:02:46.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fceddffb640 1 -- 172.21.15.16:0/1402209517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee80157a0 con 0x5627fcd355a0 2023-11-13T11:02:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fceddffb640 1 -- 172.21.15.16:0/1402209517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcee802bb00 con 0x5627fcd355a0 2023-11-13T11:02:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fcf07e31740 10 client.6531 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fcf07e31740 10 client.6531 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:46.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fcf07e31740 20 client.6531 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:46.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fcf07e31740 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5627fcd635f0 0x5627fcd38ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fcf07e31740 4 client.6531 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2023-11-13T11:02:46.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fcf07e31740 1 -- 172.21.15.16:0/1402209517 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x5627fc972670 con 0x5627fcd635f0 2023-11-13T11:02:46.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.480+0000 7fcefed9d640 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5627fcd635f0 0x5627fcd38ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.481+0000 7fcefed9d640 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5627fcd635f0 0x5627fcd38ef0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d506904defdb33b server_cookie=f9eb4766238fe99f in_seq=0 out_seq=0 2023-11-13T11:02:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.481+0000 7fceddffb640 10 client.6531 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.481+0000 7fceddffb640 1 -- 172.21.15.16:0/1402209517 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5627fc972670 con 0x5627fcd635f0 2023-11-13T11:02:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.481+0000 7fceddffb640 10 client.6531 handle_command_reply: tid=0 2023-11-13T11:02:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.481+0000 7fcf07e31740 1 client.6531 shutdown 2023-11-13T11:02:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 -- 172.21.15.16:0/1402209517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd355a0 msgr2=0x5627fcd0a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd355a0 0x5627fcd0a8e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcee8006fd0 tx=0x7fcee8004730 comp rx=0 tx=0).stop 2023-11-13T11:02:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 -- 172.21.15.16:0/1402209517 shutdown_connections 2023-11-13T11:02:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5627fcd63f00 0x5627fcd630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5627fcd635f0 0x5627fcd38ef0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fcd355a0 0x5627fcd0a8e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 --2- 172.21.15.16:0/1402209517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5627fcd09120 0x5627fcd2c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fceddffb640 0 client.6531 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:46.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 -- 172.21.15.16:0/1402209517 >> 172.21.15.16:0/1402209517 conn(0x5627fcd055c0 msgr2=0x5627fcd538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 -- 172.21.15.16:0/1402209517 shutdown_connections 2023-11-13T11:02:46.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 1 -- 172.21.15.16:0/1402209517 wait complete. 2023-11-13T11:02:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.482+0000 7fcf07e31740 20 client.6531 trim_cache size 0 max 16384 2023-11-13T11:02:46.523 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:46.524 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-11-13T11:02:46.524 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "5d366c0e-d91d-4687-8876-d8984fe2832f", 2023-11-13T11:02:46.525 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-11-13T11:02:46.525 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:46.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 -- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcee006ae40 msgr2=0x7fcee006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcee006ae40 0x7fcee006d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcf000b02c0 tx=0x7fcee8008040 comp rx=0 tx=0).stop 2023-11-13T11:02:46.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 -- 172.21.15.16:0/2151632682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 msgr2=0x7fcf001f2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 0x7fcf001f2cc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c400 tx=0x7fcef000c8c0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 -- 172.21.15.16:0/2151632682 shutdown_connections 2023-11-13T11:02:46.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcee006ae40 0x7fcee006d300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcf000af750 0x7fcf000afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf000b2290 0x7fcf0015ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 --2- 172.21.15.16:0/2151632682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf000ad730 0x7fcf001f2cc0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 -- 172.21.15.16:0/2151632682 >> 172.21.15.16:0/2151632682 conn(0x7fcf000b0970 msgr2=0x7fcf000b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 -- 172.21.15.16:0/2151632682 shutdown_connections 2023-11-13T11:02:46.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.483+0000 7fcf049c4640 1 -- 172.21.15.16:0/2151632682 wait complete. 2023-11-13T11:02:46.530 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '5d366c0e-d91d-4687-8876-d8984fe2832f', 'mode': 'asynchronous'}' 2023-11-13T11:02:46.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:46.531 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2023-11-13T11:02:46.596 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2023-11-13T11:02:46.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.817+0000 7f74f759e640 1 Processor -- start 2023-11-13T11:02:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.818+0000 7f74f759e640 1 -- start start 2023-11-13T11:02:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.819+0000 7f74f759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f815aa90 0x7f74f806a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.819+0000 7f74f759e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f74f806a6d0 0x7f74f8070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.819+0000 7f74f759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f806d680 con 0x7f74f80708a0 2023-11-13T11:02:46.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.820+0000 7f74f759e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f806ab50 con 0x7f74f806a6d0 2023-11-13T11:02:46.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.820+0000 7f74f759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f8070d30 con 0x7f74f815aa90 2023-11-13T11:02:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.820+0000 7f74f6d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f74f80708a0 0x7f74f80b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39996/0 (socket says 172.21.15.16:39996) 2023-11-13T11:02:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.820+0000 7f74f6d9d640 1 -- 172.21.15.16:0/521156753 learned_addr learned my addr 172.21.15.16:0/521156753 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.820+0000 7f74f659c640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f815aa90 0x7f74f806a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.820+0000 7f74effff640 1 --2- 172.21.15.16:0/521156753 >> v2:172.21.15.184:3300/0 conn(0x7f74f806a6d0 0x7f74f8070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.820+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612008877 0 0) 0x7f74f806d680 con 0x7f74f80708a0 2023-11-13T11:02:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.820+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e4003520 con 0x7f74f80708a0 2023-11-13T11:02:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3761952977 0 0) 0x7f74e4003520 con 0x7f74f80708a0 2023-11-13T11:02:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f815aa90 msgr2=0x7f74f806a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f815aa90 0x7f74f806a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 >> v2:172.21.15.184:3300/0 conn(0x7f74f806a6d0 msgr2=0x7f74f8070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 --2- 172.21.15.16:0/521156753 >> v2:172.21.15.184:3300/0 conn(0x7f74f806a6d0 0x7f74f8070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f815d630 con 0x7f74f80708a0 2023-11-13T11:02:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f74f00019f0 con 0x7f74f80708a0 2023-11-13T11:02:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 >> v1:172.21.15.16:6789/0 conn(0x7f74f80708a0 legacy=0x7f74f80b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f815aa90 0x7f74e4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4003f40 0x7f74e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e4004d00 0x7f74e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f806ab50 con 0x7f74e4004d00 2023-11-13T11:02:46.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f8070d30 con 0x7f74e4003f40 2023-11-13T11:02:46.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e400ba30 con 0x7f74f815aa90 2023-11-13T11:02:46.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.821+0000 7f74f6d9d640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e4004d00 0x7f74e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.822+0000 7f74f659c640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f815aa90 0x7f74e4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.822+0000 7f74effff640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4003f40 0x7f74e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.822+0000 7f74f6d9d640 1 -- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f815aa90 msgr2=0x7f74e4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.822+0000 7f74f6d9d640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f815aa90 0x7f74e4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.822+0000 7f74f6d9d640 1 -- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4003f40 msgr2=0x7f74e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.822+0000 7f74f6d9d640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4003f40 0x7f74e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.822+0000 7f74f6d9d640 1 -- 172.21.15.16:0/521156753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e4003520 con 0x7f74e4004d00 2023-11-13T11:02:46.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.822+0000 7f74effff640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4003f40 0x7f74e40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:46.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.823+0000 7f74f6d9d640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e4004d00 0x7f74e4005180 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f74f000a790 tx=0x7f74e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18db88ed8687784d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.823+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f0034630 con 0x7f74e4004d00 2023-11-13T11:02:46.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.823+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f002f070 con 0x7f74e4004d00 2023-11-13T11:02:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.823+0000 7f74f5d9b640 1 -- 172.21.15.16:0/521156753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f0034a30 con 0x7f74e4004d00 2023-11-13T11:02:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.823+0000 7f74f759e640 1 -- 172.21.15.16:0/521156753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e4004d00 msgr2=0x7f74e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.823+0000 7f74f759e640 1 --2- 172.21.15.16:0/521156753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74e4004d00 0x7f74e4005180 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f74f000a790 tx=0x7f74e40057f0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.824+0000 7f74f6d9d640 1 -- 172.21.15.16:0/521156753 reap_dead start 2023-11-13T11:02:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.824+0000 7f74f759e640 1 -- 172.21.15.16:0/521156753 shutdown_connections 2023-11-13T11:02:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.824+0000 7f74f759e640 1 -- 172.21.15.16:0/521156753 >> 172.21.15.16:0/521156753 conn(0x7f74f80b04a0 msgr2=0x7f74f80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.824+0000 7f74f759e640 1 -- 172.21.15.16:0/521156753 shutdown_connections 2023-11-13T11:02:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.824+0000 7f74f759e640 1 -- 172.21.15.16:0/521156753 wait complete. 2023-11-13T11:02:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.826+0000 7f74f759e640 1 Processor -- start 2023-11-13T11:02:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.826+0000 7f74f759e640 1 -- start start 2023-11-13T11:02:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74f759e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4004d00 0x7f74f8149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74f759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f813e540 0x7f74f814a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74f759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f8142220 0x7f74f81426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74f759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f812c940 con 0x7f74f813e540 2023-11-13T11:02:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74f759e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f812cc20 con 0x7f74e4004d00 2023-11-13T11:02:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74f759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f812cf00 con 0x7f74f8142220 2023-11-13T11:02:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f813e540 0x7f74f814a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f813e540 0x7f74f814a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38930/0 (socket says 172.21.15.16:38930) 2023-11-13T11:02:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74effff640 1 -- 172.21.15.16:0/3144419552 learned_addr learned my addr 172.21.15.16:0/3144419552 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74f659c640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4004d00 0x7f74f8149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74effff640 1 -- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f8142220 msgr2=0x7f74f81426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74effff640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f8142220 0x7f74f81426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74effff640 1 -- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4004d00 msgr2=0x7f74f8149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74effff640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4004d00 0x7f74f8149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74effff640 1 -- 172.21.15.16:0/3144419552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f812d1e0 con 0x7f74f813e540 2023-11-13T11:02:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.827+0000 7f74f659c640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4004d00 0x7f74f8149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.828+0000 7f74effff640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f813e540 0x7f74f814a010 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f74e00109d0 tx=0x7f74e0010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.828+0000 7f74eeffd640 1 -- 172.21.15.16:0/3144419552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000eba0 con 0x7f74f813e540 2023-11-13T11:02:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.828+0000 7f74eeffd640 1 -- 172.21.15.16:0/3144419552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e000ed00 con 0x7f74f813e540 2023-11-13T11:02:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.828+0000 7f74eeffd640 1 -- 172.21.15.16:0/3144419552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e001c620 con 0x7f74f813e540 2023-11-13T11:02:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.828+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f812d4c0 con 0x7f74f813e540 2023-11-13T11:02:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.828+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f8150ad0 con 0x7f74f813e540 2023-11-13T11:02:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.829+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f804e4d0 con 0x7f74f813e540 2023-11-13T11:02:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.830+0000 7f74eeffd640 1 -- 172.21.15.16:0/3144419552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74e001c780 con 0x7f74f813e540 2023-11-13T11:02:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.831+0000 7f74eeffd640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74d806ae70 0x7f74d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.831+0000 7f74eeffd640 1 -- 172.21.15.16:0/3144419552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f74e0089e30 con 0x7f74f813e540 2023-11-13T11:02:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.832+0000 7f74f659c640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74d806ae70 0x7f74d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.833+0000 7f74f659c640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74d806ae70 0x7f74d806d330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f74f8147a00 tx=0x7f74e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:46.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:46.836+0000 7f74eeffd640 1 -- 172.21.15.16:0/3144419552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74e0054d70 con 0x7f74f813e540 2023-11-13T11:02:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.068+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74f806c220 con 0x7f74f813e540 2023-11-13T11:02:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.069+0000 7f74eeffd640 1 -- 172.21.15.16:0/3144419552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f74e0058390 con 0x7f74f813e540 2023-11-13T11:02:47.074 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:47.075 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:39.012251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-11-13T11:02:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74d806ae70 msgr2=0x7f74d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74d806ae70 0x7f74d806d330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f74f8147a00 tx=0x7f74e8008040 comp rx=0 tx=0).stop 2023-11-13T11:02:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f813e540 msgr2=0x7f74f814a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f813e540 0x7f74f814a010 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f74e00109d0 tx=0x7f74e0010e90 comp rx=0 tx=0).stop 2023-11-13T11:02:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 shutdown_connections 2023-11-13T11:02:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f74d806ae70 0x7f74d806d330 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74f8142220 0x7f74f81426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f813e540 0x7f74f814a010 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 --2- 172.21.15.16:0/3144419552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e4004d00 0x7f74f8149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 >> 172.21.15.16:0/3144419552 conn(0x7f74f80b04a0 msgr2=0x7f74f806fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 shutdown_connections 2023-11-13T11:02:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.072+0000 7f74f759e640 1 -- 172.21.15.16:0/3144419552 wait complete. 2023-11-13T11:02:47.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2023-11-13T11:02:47.336 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:47.336 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:02:47.337 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:47.353 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-13T11:02:47.353 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:02:47.354 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:02:47.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:47.414 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2023-11-13T11:02:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-13T11:02:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-13T11:02:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287535 s, 36.5 MB/s 2023-11-13T11:02:47.507 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2023-11-13T11:02:47.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:47.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.735+0000 7f6dae547640 1 Processor -- start 2023-11-13T11:02:47.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.735+0000 7f6dae547640 1 -- start start 2023-11-13T11:02:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.737+0000 7f6dae547640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6da80cb190 0x7f6da815b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.737+0000 7f6dae547640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6da80716b0 0x7f6da806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.737+0000 7f6dae547640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da8158c20 con 0x7f6da806a190 2023-11-13T11:02:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.737+0000 7f6dae547640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da80b1830 con 0x7f6da80716b0 2023-11-13T11:02:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.737+0000 7f6dae547640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da815b780 con 0x7f6da80cb190 2023-11-13T11:02:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.737+0000 7f6dacd44640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6da80716b0 0x7f6da806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.737+0000 7f6dacd44640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6da80716b0 0x7f6da806f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38934/0 (socket says 172.21.15.16:38934) 2023-11-13T11:02:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.737+0000 7f6dacd44640 1 -- 172.21.15.16:0/1581351825 learned_addr learned my addr 172.21.15.16:0/1581351825 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.738+0000 7f6dad545640 1 --2- 172.21.15.16:0/1581351825 >> v2:172.21.15.184:3300/0 conn(0x7f6da80cb190 0x7f6da815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.738+0000 7f6dacd44640 1 -- 172.21.15.16:0/1581351825 >> v2:172.21.15.184:3300/0 conn(0x7f6da80cb190 msgr2=0x7f6da815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.738+0000 7f6dacd44640 1 --2- 172.21.15.16:0/1581351825 >> v2:172.21.15.184:3300/0 conn(0x7f6da80cb190 0x7f6da815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.738+0000 7f6dacd44640 1 -- 172.21.15.16:0/1581351825 >> v1:172.21.15.184:6789/0 conn(0x7f6da806a190 legacy=0x7f6da80b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.738+0000 7f6dacd44640 1 -- 172.21.15.16:0/1581351825 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da815ba60 con 0x7f6da80716b0 2023-11-13T11:02:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.738+0000 7f6dad545640 1 --2- 172.21.15.16:0/1581351825 >> v2:172.21.15.184:3300/0 conn(0x7f6da80cb190 0x7f6da815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:47.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.739+0000 7f6dacd44640 1 --2- 172.21.15.16:0/1581351825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80716b0 0x7f6da806f970 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6da80b1560 tx=0x7f6d9802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4197cfcfa57543e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:47.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.739+0000 7f6d9ffff640 1 -- 172.21.15.16:0/1581351825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98031890 con 0x7f6da80716b0 2023-11-13T11:02:47.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.739+0000 7f6d9ffff640 1 -- 172.21.15.16:0/1581351825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d980319f0 con 0x7f6da80716b0 2023-11-13T11:02:47.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.739+0000 7f6d9ffff640 1 -- 172.21.15.16:0/1581351825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98031d80 con 0x7f6da80716b0 2023-11-13T11:02:47.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.740+0000 7f6dae547640 1 -- 172.21.15.16:0/1581351825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80716b0 msgr2=0x7f6da806f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.740+0000 7f6dae547640 1 --2- 172.21.15.16:0/1581351825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80716b0 0x7f6da806f970 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6da80b1560 tx=0x7f6d9802b930 comp rx=0 tx=0).stop 2023-11-13T11:02:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.740+0000 7f6dae547640 1 -- 172.21.15.16:0/1581351825 shutdown_connections 2023-11-13T11:02:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.740+0000 7f6dae547640 1 --2- 172.21.15.16:0/1581351825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80716b0 0x7f6da806f970 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.740+0000 7f6dae547640 1 --2- 172.21.15.16:0/1581351825 >> v2:172.21.15.184:3300/0 conn(0x7f6da80cb190 0x7f6da815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.740+0000 7f6dae547640 1 -- 172.21.15.16:0/1581351825 >> 172.21.15.16:0/1581351825 conn(0x7f6da80b04a0 msgr2=0x7f6da80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.740+0000 7f6dae547640 1 -- 172.21.15.16:0/1581351825 shutdown_connections 2023-11-13T11:02:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.740+0000 7f6dae547640 1 -- 172.21.15.16:0/1581351825 wait complete. 2023-11-13T11:02:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.742+0000 7f6dae547640 1 Processor -- start 2023-11-13T11:02:47.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.742+0000 7f6dae547640 1 -- start start 2023-11-13T11:02:47.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.743+0000 7f6dae547640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da806a190 0x7f6da8147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.743+0000 7f6dae547640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da80716b0 0x7f6da80aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.743+0000 7f6dae547640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80cb190 0x7f6da80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.743+0000 7f6dae547640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81479c0 con 0x7f6da80cb190 2023-11-13T11:02:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.743+0000 7f6dae547640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81f6190 con 0x7f6da806a190 2023-11-13T11:02:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.743+0000 7f6dae547640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81f6470 con 0x7f6da80716b0 2023-11-13T11:02:47.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.743+0000 7f6dadd46640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80cb190 0x7f6da80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:47.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.743+0000 7f6dadd46640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80cb190 0x7f6da80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38950/0 (socket says 172.21.15.16:38950) 2023-11-13T11:02:47.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.743+0000 7f6dadd46640 1 -- 172.21.15.16:0/3697146843 learned_addr learned my addr 172.21.15.16:0/3697146843 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:47.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.744+0000 7f6dad545640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da806a190 0x7f6da8147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:47.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.744+0000 7f6dacd44640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da80716b0 0x7f6da80aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.744+0000 7f6dadd46640 1 -- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da80716b0 msgr2=0x7f6da80aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.744+0000 7f6dadd46640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da80716b0 0x7f6da80aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.744+0000 7f6dadd46640 1 -- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da806a190 msgr2=0x7f6da8147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.744+0000 7f6dadd46640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da806a190 0x7f6da8147480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.744+0000 7f6dadd46640 1 -- 172.21.15.16:0/3697146843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d980094d0 con 0x7f6da80cb190 2023-11-13T11:02:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.744+0000 7f6dadd46640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80cb190 0x7f6da80ce740 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6da000b970 tx=0x7f6da000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.745+0000 7f6d9dffb640 1 -- 172.21.15.16:0/3697146843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000cb40 con 0x7f6da80cb190 2023-11-13T11:02:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.745+0000 7f6d9dffb640 1 -- 172.21.15.16:0/3697146843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da000cca0 con 0x7f6da80cb190 2023-11-13T11:02:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.745+0000 7f6d9dffb640 1 -- 172.21.15.16:0/3697146843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0014680 con 0x7f6da80cb190 2023-11-13T11:02:47.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.745+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da81f6750 con 0x7f6da80cb190 2023-11-13T11:02:47.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.745+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da80af250 con 0x7f6da80cb190 2023-11-13T11:02:47.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.746+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da80ce360 con 0x7f6da80cb190 2023-11-13T11:02:47.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.747+0000 7f6d9dffb640 1 -- 172.21.15.16:0/3697146843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6da001d3b0 con 0x7f6da80cb190 2023-11-13T11:02:47.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.748+0000 7f6d9dffb640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8c06ae40 0x7f6d8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:47.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.748+0000 7f6d9dffb640 1 -- 172.21.15.16:0/3697146843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6da00895e0 con 0x7f6da80cb190 2023-11-13T11:02:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.749+0000 7f6dad545640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8c06ae40 0x7f6d8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.750+0000 7f6dad545640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8c06ae40 0x7f6d8c06d300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6da4008850 tx=0x7f6da4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.752+0000 7f6d9dffb640 1 -- 172.21.15.16:0/3697146843 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6da0054500 con 0x7f6da80cb190 2023-11-13T11:02:47.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.983+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6da804e4d0 con 0x7f6da80cb190 2023-11-13T11:02:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.984+0000 7f6d9dffb640 1 -- 172.21.15.16:0/3697146843 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f6da0057b20 con 0x7f6da80cb190 2023-11-13T11:02:47.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:47.990 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:39.012251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2023-11-13T11:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8c06ae40 msgr2=0x7f6d8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8c06ae40 0x7f6d8c06d300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6da4008850 tx=0x7f6da4008040 comp rx=0 tx=0).stop 2023-11-13T11:02:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80cb190 msgr2=0x7f6da80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80cb190 0x7f6da80ce740 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6da000b970 tx=0x7f6da000be30 comp rx=0 tx=0).stop 2023-11-13T11:02:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 shutdown_connections 2023-11-13T11:02:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8c06ae40 0x7f6d8c06d300 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6da80cb190 0x7f6da80ce740 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6da80716b0 0x7f6da80aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 --2- 172.21.15.16:0/3697146843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6da806a190 0x7f6da8147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 >> 172.21.15.16:0/3697146843 conn(0x7f6da80b04a0 msgr2=0x7f6da815a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.987+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 shutdown_connections 2023-11-13T11:02:47.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:47.988+0000 7f6dae547640 1 -- 172.21.15.16:0/3697146843 wait complete. 2023-11-13T11:02:48.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2023-11-13T11:02:48.248 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:48.248 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:02:48.249 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:48.268 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-13T11:02:48.269 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2023-11-13T11:02:48.269 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T11:02:48.269 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2023-11-13T11:02:48.270 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:02:48.270 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2023-11-13T11:02:48.271 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T11:02:48.411 INFO:teuthology.orchestra.run.smithi016.stdout:1099511630412 2023-11-13T11:02:48.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T11:02:48.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2023-11-13T11:02:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.698+0000 7f12a2d3c640 1 Processor -- start 2023-11-13T11:02:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.698+0000 7f12a2d3c640 1 -- start start 2023-11-13T11:02:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.700+0000 7f12a2d3c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f129c15b5c0 0x7f129c164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.700+0000 7f12a2d3c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f129c161640 0x7f129c0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.700+0000 7f12a2d3c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c165b40 con 0x7f129c0b2970 2023-11-13T11:02:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.700+0000 7f12a2d3c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c0b2de0 con 0x7f129c15b5c0 2023-11-13T11:02:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.700+0000 7f12a2d3c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c1652a0 con 0x7f129c161640 2023-11-13T11:02:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.700+0000 7f12a1d3a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f129c15b5c0 0x7f129c164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.700+0000 7f12a253b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f129c0b2970 0x7f129c0ad520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39998/0 (socket says 172.21.15.16:39998) 2023-11-13T11:02:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.700+0000 7f12a253b640 1 -- 172.21.15.16:0/1484990059 learned_addr learned my addr 172.21.15.16:0/1484990059 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.701+0000 7f12a1539640 1 --2- 172.21.15.16:0/1484990059 >> v2:172.21.15.184:3300/0 conn(0x7f129c161640 0x7f129c0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.701+0000 7f12a1d3a640 1 -- 172.21.15.16:0/1484990059 >> v2:172.21.15.184:3300/0 conn(0x7f129c161640 msgr2=0x7f129c0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.701+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1484990059 >> v2:172.21.15.184:3300/0 conn(0x7f129c161640 0x7f129c0b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.701+0000 7f12a1d3a640 1 -- 172.21.15.16:0/1484990059 >> v1:172.21.15.16:6789/0 conn(0x7f129c0b2970 legacy=0x7f129c0ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.701+0000 7f12a1d3a640 1 -- 172.21.15.16:0/1484990059 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c165580 con 0x7f129c15b5c0 2023-11-13T11:02:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.702+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1484990059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 0x7f129c164c10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f129c06dd70 tx=0x7f129802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed733d8c1bd2342b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.702+0000 7f12a0d38640 1 -- 172.21.15.16:0/1484990059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298031830 con 0x7f129c15b5c0 2023-11-13T11:02:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.702+0000 7f12a0d38640 1 -- 172.21.15.16:0/1484990059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298031990 con 0x7f129c15b5c0 2023-11-13T11:02:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.702+0000 7f12a0d38640 1 -- 172.21.15.16:0/1484990059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298031d20 con 0x7f129c15b5c0 2023-11-13T11:02:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.703+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1484990059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 msgr2=0x7f129c164c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.703+0000 7f12a2d3c640 1 --2- 172.21.15.16:0/1484990059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 0x7f129c164c10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f129c06dd70 tx=0x7f129802b8d0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.703+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1484990059 shutdown_connections 2023-11-13T11:02:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.703+0000 7f12a2d3c640 1 --2- 172.21.15.16:0/1484990059 >> v2:172.21.15.184:3300/0 conn(0x7f129c161640 0x7f129c0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.703+0000 7f12a2d3c640 1 --2- 172.21.15.16:0/1484990059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 0x7f129c164c10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.703+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1484990059 >> 172.21.15.16:0/1484990059 conn(0x7f129c0b0970 msgr2=0x7f129c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:48.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.703+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1484990059 shutdown_connections 2023-11-13T11:02:48.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.703+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1484990059 wait complete. 2023-11-13T11:02:48.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.705+0000 7f12a2d3c640 1 Processor -- start 2023-11-13T11:02:48.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.705+0000 7f12a2d3c640 1 -- start start 2023-11-13T11:02:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.706+0000 7f12a2d3c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129c0b2970 0x7f129c147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.706+0000 7f12a2d3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 0x7f129c15c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.706+0000 7f12a2d3c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129c161640 0x7f129c160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.706+0000 7f12a2d3c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c15d660 con 0x7f129c15b5c0 2023-11-13T11:02:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.706+0000 7f12a2d3c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c0ca700 con 0x7f129c161640 2023-11-13T11:02:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.706+0000 7f12a2d3c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c147e90 con 0x7f129c0b2970 2023-11-13T11:02:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1539640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 0x7f129c15c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1539640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 0x7f129c15c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38976/0 (socket says 172.21.15.16:38976) 2023-11-13T11:02:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1539640 1 -- 172.21.15.16:0/1498684065 learned_addr learned my addr 172.21.15.16:0/1498684065 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129c0b2970 0x7f129c147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a253b640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129c161640 0x7f129c160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1539640 1 -- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129c0b2970 msgr2=0x7f129c147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1539640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129c0b2970 0x7f129c147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1539640 1 -- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129c161640 msgr2=0x7f129c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1539640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129c161640 0x7f129c160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1539640 1 -- 172.21.15.16:0/1498684065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298009400 con 0x7f129c15b5c0 2023-11-13T11:02:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a253b640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129c161640 0x7f129c160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.707+0000 7f12a1539640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 0x7f129c15c650 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f128c00d830 tx=0x7f128c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.708+0000 7f12927fc640 1 -- 172.21.15.16:0/1498684065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c00b6c0 con 0x7f129c15b5c0 2023-11-13T11:02:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.708+0000 7f12927fc640 1 -- 172.21.15.16:0/1498684065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c00b820 con 0x7f129c15b5c0 2023-11-13T11:02:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.708+0000 7f12927fc640 1 -- 172.21.15.16:0/1498684065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c00bbb0 con 0x7f129c15b5c0 2023-11-13T11:02:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.708+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1498684065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c15d0a0 con 0x7f129c15b5c0 2023-11-13T11:02:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.708+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1498684065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c147b20 con 0x7f129c15b5c0 2023-11-13T11:02:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.710+0000 7f12927fc640 1 -- 172.21.15.16:0/1498684065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f128c01e460 con 0x7f129c15b5c0 2023-11-13T11:02:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.711+0000 7f12927fc640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f128006ae40 0x7f128006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.711+0000 7f12927fc640 1 -- 172.21.15.16:0/1498684065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f128c089fe0 con 0x7f129c15b5c0 2023-11-13T11:02:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.712+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f128006ae40 0x7f128006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.712+0000 7f12a6c0d740 1 Processor -- start 2023-11-13T11:02:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.712+0000 7f12a6c0d740 1 -- start start 2023-11-13T11:02:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb4848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb4544f0 0x5633bb481740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb458050 0x5633bb481c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bb3c7d20 con 0x5633bb4844d0 2023-11-13T11:02:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bb3c5410 con 0x5633bb4544f0 2023-11-13T11:02:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633bb3c0d00 con 0x5633bb458050 2023-11-13T11:02:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f128006ae40 0x7f128006d300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1298036000 tx=0x7f129803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a1d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb4848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a1d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb4848b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38984/0 (socket says 172.21.15.16:38984) 2023-11-13T11:02:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.713+0000 7f12a1d3a640 1 -- 172.21.15.16:0/1169580778 learned_addr learned my addr 172.21.15.16:0/1169580778 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.714+0000 7f12a1539640 1 --2- 172.21.15.16:0/1169580778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb4544f0 0x5633bb481740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.714+0000 7f12a253b640 1 --2- 172.21.15.16:0/1169580778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb458050 0x5633bb481c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.714+0000 7f12a1d3a640 1 -- 172.21.15.16:0/1169580778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb458050 msgr2=0x5633bb481c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.714+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1169580778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb458050 0x5633bb481c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.714+0000 7f12a1d3a640 1 -- 172.21.15.16:0/1169580778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb4544f0 msgr2=0x5633bb481740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.714+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1169580778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb4544f0 0x5633bb481740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.714+0000 7f12a1d3a640 1 -- 172.21.15.16:0/1169580778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633bb37a4f0 con 0x5633bb4844d0 2023-11-13T11:02:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.714+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1169580778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb4848b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f129802bee0 tx=0x7f1298030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dced610b5806b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f127f7fe640 1 -- 172.21.15.16:0/1169580778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129803ae40 con 0x5633bb4844d0 2023-11-13T11:02:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f127f7fe640 1 -- 172.21.15.16:0/1169580778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298031c90 con 0x5633bb4844d0 2023-11-13T11:02:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f127f7fe640 1 -- 172.21.15.16:0/1169580778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129804f480 con 0x5633bb4844d0 2023-11-13T11:02:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1169580778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 msgr2=0x5633bb4848b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1169580778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb4848b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f129802bee0 tx=0x7f1298030000 comp rx=0 tx=0).stop 2023-11-13T11:02:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1169580778 shutdown_connections 2023-11-13T11:02:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1169580778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb458050 0x5633bb481c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1169580778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb4544f0 0x5633bb481740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1169580778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb4848b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1169580778 >> 172.21.15.16:0/1169580778 conn(0x5633bb456760 msgr2=0x5633bb49f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.715+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1169580778 shutdown_connections 2023-11-13T11:02:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.716+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1169580778 wait complete. 2023-11-13T11:02:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.716+0000 7f12a6c0d740 1 -- start start 2023-11-13T11:02:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.716+0000 7f12a6c0d740 1 Processor -- start 2023-11-13T11:02:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 0x5633bb4878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb4844d0 0x5633bb47e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb46c030 0x5633bb4b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bb37afa0 con 0x5633bb46c030 2023-11-13T11:02:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bb37ac40 con 0x5633bb458050 2023-11-13T11:02:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633bb0b9f90 con 0x5633bb4844d0 2023-11-13T11:02:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a6c0d740 10 client.0 _ll_register_callbacks cb 0x5633bb37f370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a253b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb46c030 0x5633bb4b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a253b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb46c030 0x5633bb4b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38998/0 (socket says 172.21.15.16:38998) 2023-11-13T11:02:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a253b640 1 -- 172.21.15.16:0/1216966659 learned_addr learned my addr 172.21.15.16:0/1216966659 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 0x5633bb4878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a1539640 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb4844d0 0x5633bb47e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a253b640 1 -- 172.21.15.16:0/1216966659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb4844d0 msgr2=0x5633bb47e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a253b640 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb4844d0 0x5633bb47e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a253b640 1 -- 172.21.15.16:0/1216966659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 msgr2=0x5633bb4878f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a253b640 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 0x5633bb4878f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.718+0000 7f12a253b640 1 -- 172.21.15.16:0/1216966659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c165580 con 0x5633bb46c030 2023-11-13T11:02:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.719+0000 7f12a253b640 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb46c030 0x5633bb4b29e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f129400f7e0 tx=0x7f129400fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863e0b968ddfb23f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.719+0000 7f127cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.719+0000 7f127cff9640 1 -- 172.21.15.16:0/1216966659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129400d940 con 0x5633bb46c030 2023-11-13T11:02:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.719+0000 7f127cff9640 1 -- 172.21.15.16:0/1216966659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f129400daa0 con 0x5633bb46c030 2023-11-13T11:02:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.719+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1216966659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633bb3b9ce0 con 0x5633bb46c030 2023-11-13T11:02:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.719+0000 7f127cff9640 1 -- 172.21.15.16:0/1216966659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1294001bb0 con 0x5633bb46c030 2023-11-13T11:02:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.719+0000 7f12a6c0d740 10 client.6558 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.719+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1216966659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5633bb3bf380 con 0x5633bb46c030 2023-11-13T11:02:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.720+0000 7f127cff9640 1 -- 172.21.15.16:0/1216966659 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633bb3b9ce0 con 0x5633bb46c030 2023-11-13T11:02:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.720+0000 7f127cff9640 1 -- 172.21.15.16:0/1216966659 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1294012070 con 0x5633bb46c030 2023-11-13T11:02:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.720+0000 7f12a6c0d740 10 client.6558 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.720+0000 7f12a6c0d740 10 client.6558 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.720+0000 7f12a6c0d740 20 client.6558 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.720+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633bb46c4b0 0x5633bb46e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.720+0000 7f12a6c0d740 4 client.6558 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.720+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1216966659 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633bb3b9a00 con 0x5633bb46c4b0 2023-11-13T11:02:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.720+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633bb46c4b0 0x5633bb46e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.721+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633bb46c4b0 0x5633bb46e060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80bfad0b4d2f1ed server_cookie=f7772ff306010537 in_seq=0 out_seq=0 2023-11-13T11:02:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.721+0000 7f127cff9640 10 client.6558 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.721+0000 7f127cff9640 1 -- 172.21.15.16:0/1216966659 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5633bb3b9a00 con 0x5633bb46c4b0 2023-11-13T11:02:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.721+0000 7f127cff9640 10 client.6558 handle_command_reply: tid=0 2023-11-13T11:02:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.721+0000 7f12a6c0d740 1 client.6558 shutdown 2023-11-13T11:02:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1216966659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb46c030 msgr2=0x5633bb4b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb46c030 0x5633bb4b29e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f129400f7e0 tx=0x7f129400fca0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1216966659 shutdown_connections 2023-11-13T11:02:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633bb46c4b0 0x5633bb46e060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb46c030 0x5633bb4b29e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb4844d0 0x5633bb47e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1216966659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 0x5633bb4878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f127cff9640 0 client.6558 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1216966659 >> 172.21.15.16:0/1216966659 conn(0x5633bb4544f0 msgr2=0x5633bb458a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1216966659 shutdown_connections 2023-11-13T11:02:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1216966659 wait complete. 2023-11-13T11:02:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.722+0000 7f12a6c0d740 20 client.6558 trim_cache size 0 max 16384 2023-11-13T11:02:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.735+0000 7f12a6c0d740 1 Processor -- start 2023-11-13T11:02:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.735+0000 7f12a6c0d740 1 -- start start 2023-11-13T11:02:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.736+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb458050 0x5633bb4585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.736+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb46c030 0x5633bb481190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.736+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb4844d0 0x5633bb4816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.736+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bb4af310 con 0x5633bb458050 2023-11-13T11:02:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.736+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bb455ec0 con 0x5633bb46c030 2023-11-13T11:02:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.736+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633bb374f70 con 0x5633bb4844d0 2023-11-13T11:02:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.736+0000 7f12a1d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb458050 0x5633bb4585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.736+0000 7f12a1d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb458050 0x5633bb4585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39002/0 (socket says 172.21.15.16:39002) 2023-11-13T11:02:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.736+0000 7f12a1d3a640 1 -- 172.21.15.16:0/2593069696 learned_addr learned my addr 172.21.15.16:0/2593069696 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f12a1d3a640 1 -- 172.21.15.16:0/2593069696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb4844d0 msgr2=0x5633bb4816d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/2593069696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb4844d0 0x5633bb4816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f12a1539640 1 --2- 172.21.15.16:0/2593069696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb46c030 0x5633bb481190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f12a1d3a640 1 -- 172.21.15.16:0/2593069696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb46c030 msgr2=0x5633bb481190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/2593069696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb46c030 0x5633bb481190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f12a1d3a640 1 -- 172.21.15.16:0/2593069696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633bb35ccf0 con 0x5633bb458050 2023-11-13T11:02:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/2593069696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb458050 0x5633bb4585f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1298047000 tx=0x7f1298031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa9aa64673f0854 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f127ffff640 1 -- 172.21.15.16:0/2593069696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298057be0 con 0x5633bb458050 2023-11-13T11:02:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f127ffff640 1 -- 172.21.15.16:0/2593069696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298057d40 con 0x5633bb458050 2023-11-13T11:02:48.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.737+0000 7f127ffff640 1 -- 172.21.15.16:0/2593069696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12980575b0 con 0x5633bb458050 2023-11-13T11:02:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.738+0000 7f12a6c0d740 1 -- 172.21.15.16:0/2593069696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb458050 msgr2=0x5633bb4585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.738+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/2593069696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb458050 0x5633bb4585f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1298047000 tx=0x7f1298031000 comp rx=0 tx=0).stop 2023-11-13T11:02:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.738+0000 7f12a6c0d740 1 -- 172.21.15.16:0/2593069696 shutdown_connections 2023-11-13T11:02:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.738+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/2593069696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb4844d0 0x5633bb4816d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.738+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/2593069696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb46c030 0x5633bb481190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.738+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/2593069696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb458050 0x5633bb4585f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.738+0000 7f12a6c0d740 1 -- 172.21.15.16:0/2593069696 >> 172.21.15.16:0/2593069696 conn(0x5633bb4544f0 msgr2=0x5633bb470db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.738+0000 7f12a6c0d740 1 -- 172.21.15.16:0/2593069696 shutdown_connections 2023-11-13T11:02:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.738+0000 7f12a6c0d740 1 -- 172.21.15.16:0/2593069696 wait complete. 2023-11-13T11:02:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.739+0000 7f12a6c0d740 1 -- start start 2023-11-13T11:02:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.739+0000 7f12a6c0d740 1 Processor -- start 2023-11-13T11:02:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 0x5633bb47f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb46c030 0x5633bb47fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a6c0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb47d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bb50b430 con 0x5633bb4844d0 2023-11-13T11:02:48.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633bb50ad70 con 0x5633bb458050 2023-11-13T11:02:48.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a6c0d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633bb50b220 con 0x5633bb46c030 2023-11-13T11:02:48.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a6c0d740 10 client.0 _ll_register_callbacks cb 0x7f12940018e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:48.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a253b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb47d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a253b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb47d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39016/0 (socket says 172.21.15.16:39016) 2023-11-13T11:02:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.740+0000 7f12a253b640 1 -- 172.21.15.16:0/1688302326 learned_addr learned my addr 172.21.15.16:0/1688302326 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f12a253b640 1 -- 172.21.15.16:0/1688302326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb46c030 msgr2=0x5633bb47fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f12a253b640 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb46c030 0x5633bb47fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f12a253b640 1 -- 172.21.15.16:0/1688302326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 msgr2=0x5633bb47f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 0x5633bb47f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f12a253b640 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 0x5633bb47f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f12a253b640 1 -- 172.21.15.16:0/1688302326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633bb37a4f0 con 0x5633bb4844d0 2023-11-13T11:02:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f12a253b640 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb47d0f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f129400fce0 tx=0x7f1294001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15af72e50cc08a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f127cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f127cff9640 1 -- 172.21.15.16:0/1688302326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129400d520 con 0x5633bb4844d0 2023-11-13T11:02:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f127cff9640 1 -- 172.21.15.16:0/1688302326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f1294007760 con 0x5633bb4844d0 2023-11-13T11:02:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1688302326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1298031c90 con 0x5633bb4844d0 2023-11-13T11:02:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f127cff9640 1 -- 172.21.15.16:0/1688302326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1294007920 con 0x5633bb4844d0 2023-11-13T11:02:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.741+0000 7f12a6c0d740 10 client.6564 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:48.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1688302326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5633bb3822e0 con 0x5633bb4844d0 2023-11-13T11:02:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f127cff9640 1 -- 172.21.15.16:0/1688302326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298031c90 con 0x5633bb4844d0 2023-11-13T11:02:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f127cff9640 1 -- 172.21.15.16:0/1688302326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f129402e420 con 0x5633bb4844d0 2023-11-13T11:02:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f12a6c0d740 10 client.6564 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f12a6c0d740 10 client.6564 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f12a6c0d740 20 client.6564 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:48.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633bb4b9680 0x5633bb47d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f12a6c0d740 4 client.6564 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-11-13T11:02:48.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1688302326 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5633bb0c17e0 con 0x5633bb4b9680 2023-11-13T11:02:48.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.742+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633bb4b9680 0x5633bb47d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a1d3a640 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633bb4b9680 0x5633bb47d630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=263b711a780febad server_cookie=5238ef93670f1613 in_seq=0 out_seq=0 2023-11-13T11:02:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f127cff9640 10 client.6564 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f127cff9640 1 -- 172.21.15.16:0/1688302326 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5633bb0c17e0 con 0x5633bb4b9680 2023-11-13T11:02:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f127cff9640 10 client.6564 handle_command_reply: tid=0 2023-11-13T11:02:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 client.6564 shutdown 2023-11-13T11:02:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1688302326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 msgr2=0x5633bb47d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb47d0f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f129400fce0 tx=0x7f1294001250 comp rx=0 tx=0).stop 2023-11-13T11:02:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1688302326 shutdown_connections 2023-11-13T11:02:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5633bb4b9680 0x5633bb47d630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bb4844d0 0x5633bb47d0f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5633bb46c030 0x5633bb47fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 --2- 172.21.15.16:0/1688302326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5633bb458050 0x5633bb47f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f127cff9640 0 client.6564 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1688302326 >> 172.21.15.16:0/1688302326 conn(0x5633bb4544f0 msgr2=0x5633bb481190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1688302326 shutdown_connections 2023-11-13T11:02:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.743+0000 7f12a6c0d740 1 -- 172.21.15.16:0/1688302326 wait complete. 2023-11-13T11:02:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a6c0d740 20 client.6564 trim_cache size 0 max 16384 2023-11-13T11:02:48.765 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2023-11-13T11:02:48.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f128006ae40 msgr2=0x7f128006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f128006ae40 0x7f128006d300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1298036000 tx=0x7f129803c040 comp rx=0 tx=0).stop 2023-11-13T11:02:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1498684065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 msgr2=0x7f129c15c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 0x7f129c15c650 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f128c00d830 tx=0x7f128c00dcf0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1498684065 shutdown_connections 2023-11-13T11:02:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f128006ae40 0x7f128006d300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f129c161640 0x7f129c160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c15b5c0 0x7f129c15c650 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 --2- 172.21.15.16:0/1498684065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f129c0b2970 0x7f129c147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1498684065 >> 172.21.15.16:0/1498684065 conn(0x7f129c0b0970 msgr2=0x7f129c06bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1498684065 shutdown_connections 2023-11-13T11:02:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.744+0000 7f12a2d3c640 1 -- 172.21.15.16:0/1498684065 wait complete. 2023-11-13T11:02:48.769 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4c.00000000 parent 2023-11-13T11:02:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.878+0000 7fd1d3f70f40 1 Processor -- start 2023-11-13T11:02:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.878+0000 7fd1d3f70f40 1 -- start start 2023-11-13T11:02:48.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.879+0000 7fd1d3f70f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0e9b440 0x5568e0f3bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.879+0000 7fd1d3f70f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568e0eb58b0 0x5568e0f3b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.879+0000 7fd1d3f70f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568e0f3faa0 0x5568e0f3b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.879+0000 7fd1d3f70f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e0e17690 con 0x5568e0e9b440 2023-11-13T11:02:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.879+0000 7fd1d3f70f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e0dcebc0 con 0x5568e0eb58b0 2023-11-13T11:02:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.879+0000 7fd1d3f70f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568e0db19c0 con 0x5568e0f3faa0 2023-11-13T11:02:48.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.879+0000 7fd1d1a72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0e9b440 0x5568e0f3bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.879+0000 7fd1d1a72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0e9b440 0x5568e0f3bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39032/0 (socket says 172.21.15.16:39032) 2023-11-13T11:02:48.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.879+0000 7fd1d1a72640 1 -- 172.21.15.16:0/3470208551 learned_addr learned my addr 172.21.15.16:0/3470208551 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:48.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1cbfff640 1 --2- 172.21.15.16:0/3470208551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568e0eb58b0 0x5568e0f3b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1d1a72640 1 -- 172.21.15.16:0/3470208551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568e0f3faa0 msgr2=0x5568e0f3b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1d1a72640 1 --2- 172.21.15.16:0/3470208551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568e0f3faa0 0x5568e0f3b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1d1a72640 1 -- 172.21.15.16:0/3470208551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568e0eb58b0 msgr2=0x5568e0f3b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1d1a72640 1 --2- 172.21.15.16:0/3470208551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568e0eb58b0 0x5568e0f3b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1d1a72640 1 -- 172.21.15.16:0/3470208551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc005de0 con 0x5568e0e9b440 2023-11-13T11:02:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1d1a72640 1 --2- 172.21.15.16:0/3470208551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0e9b440 0x5568e0f3bf50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400a7a0 tx=0x7fd1c400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac28cceca4d0b20 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1d0a70640 1 -- 172.21.15.16:0/3470208551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400f070 con 0x5568e0e9b440 2023-11-13T11:02:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1d0a70640 1 -- 172.21.15.16:0/3470208551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c4007600 con 0x5568e0e9b440 2023-11-13T11:02:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.880+0000 7fd1d0a70640 1 -- 172.21.15.16:0/3470208551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4007990 con 0x5568e0e9b440 2023-11-13T11:02:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.881+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3470208551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0e9b440 msgr2=0x5568e0f3bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.881+0000 7fd1d3f70f40 1 --2- 172.21.15.16:0/3470208551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0e9b440 0x5568e0f3bf50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400a7a0 tx=0x7fd1c400a890 comp rx=0 tx=0).stop 2023-11-13T11:02:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.881+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3470208551 shutdown_connections 2023-11-13T11:02:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.881+0000 7fd1d3f70f40 1 --2- 172.21.15.16:0/3470208551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568e0f3faa0 0x5568e0f3b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.881+0000 7fd1d3f70f40 1 --2- 172.21.15.16:0/3470208551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568e0eb58b0 0x5568e0f3b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.881+0000 7fd1d3f70f40 1 --2- 172.21.15.16:0/3470208551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0e9b440 0x5568e0f3bf50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.881+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3470208551 >> 172.21.15.16:0/3470208551 conn(0x5568e0eb64c0 msgr2=0x5568e0e97910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.881+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3470208551 shutdown_connections 2023-11-13T11:02:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.881+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3470208551 wait complete. 2023-11-13T11:02:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.882+0000 7fd1d3f70f40 1 Processor -- start 2023-11-13T11:02:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.882+0000 7fd1d3f70f40 1 -- start start 2023-11-13T11:02:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.883+0000 7fd1d3f70f40 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568e0e9b440 0x5568e0f24f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.883+0000 7fd1d3f70f40 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0eb58b0 0x5568e0f25810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.883+0000 7fd1d3f70f40 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568e0f3faa0 0x5568e0f3b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.883+0000 7fd1d3f70f40 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e0da5520 con 0x5568e0eb58b0 2023-11-13T11:02:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.883+0000 7fd1d3f70f40 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568e0f282b0 con 0x5568e0e9b440 2023-11-13T11:02:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.883+0000 7fd1d3f70f40 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568e0f28440 con 0x5568e0f3faa0 2023-11-13T11:02:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0eb58b0 0x5568e0f25810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0eb58b0 0x5568e0f25810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39038/0 (socket says 172.21.15.16:39038) 2023-11-13T11:02:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1cbfff640 1 -- 172.21.15.16:0/3712317333 learned_addr learned my addr 172.21.15.16:0/3712317333 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1d2273640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568e0f3faa0 0x5568e0f3b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1d1a72640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568e0e9b440 0x5568e0f24f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1cbfff640 1 -- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568e0f3faa0 msgr2=0x5568e0f3b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1cbfff640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568e0f3faa0 0x5568e0f3b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1cbfff640 1 -- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568e0e9b440 msgr2=0x5568e0f24f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1cbfff640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5568e0e9b440 0x5568e0f24f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1cbfff640 1 -- 172.21.15.16:0/3712317333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c400a420 con 0x5568e0eb58b0 2023-11-13T11:02:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.884+0000 7fd1d2273640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5568e0f3faa0 0x5568e0f3b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.885+0000 7fd1cbfff640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0eb58b0 0x5568e0f25810 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc01cc40 tx=0x7fd1bc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.885+0000 7fd1caffd640 1 -- 172.21.15.16:0/3712317333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc052990 con 0x5568e0eb58b0 2023-11-13T11:02:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.885+0000 7fd1caffd640 1 -- 172.21.15.16:0/3712317333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc052af0 con 0x5568e0eb58b0 2023-11-13T11:02:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.885+0000 7fd1caffd640 1 -- 172.21.15.16:0/3712317333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc052e80 con 0x5568e0eb58b0 2023-11-13T11:02:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.885+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5568e0dbd680 con 0x5568e0eb58b0 2023-11-13T11:02:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.885+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568e0db0830 con 0x5568e0eb58b0 2023-11-13T11:02:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.888+0000 7fd1caffd640 1 -- 172.21.15.16:0/3712317333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1bc059070 con 0x5568e0eb58b0 2023-11-13T11:02:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.889+0000 7fd1caffd640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1b4069820 0x7fd1b406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.889+0000 7fd1caffd640 1 -- 172.21.15.16:0/3712317333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fd1bc0ca110 con 0x5568e0eb58b0 2023-11-13T11:02:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.889+0000 7fd1d3f70f40 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x5568e0f28de0 0x5568e0f29230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.890+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 --> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] -- osd_op(unknown.0.0:1 8.3a 8:5fa3b718:::10000000a4c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57) v8 -- 0x5568e0f25d50 con 0x5568e0f28de0 2023-11-13T11:02:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.890+0000 7fd1d1a72640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1b4069820 0x7fd1b406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.890+0000 7fd1d2273640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x5568e0f28de0 0x5568e0f29230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.891+0000 7fd1d1a72640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1b4069820 0x7fd1b406bce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000c00 tx=0x7fd1c400c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.891+0000 7fd1d2273640 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x5568e0f28de0 0x5568e0f29230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d2273640 1 -- 172.21.15.16:0/3712317333 <== osd.7 v2:172.21.15.184:6824/3047573298 1 ==== osd_op_reply(1 10000000a4c.00000000 [rmxattr] v57'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd1cc006040 con 0x5568e0f28de0 2023-11-13T11:02:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x5568e0f28de0 msgr2=0x5568e0f29230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x5568e0f28de0 0x5568e0f29230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1b4069820 msgr2=0x7fd1b406bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd1b4069820 0x7fd1b406bce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000c00 tx=0x7fd1c400c000 comp rx=0 tx=0).stop 2023-11-13T11:02:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0eb58b0 msgr2=0x5568e0f25810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 --2- 172.21.15.16:0/3712317333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568e0eb58b0 0x5568e0f25810 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc01cc40 tx=0x7fd1bc002fb0 comp rx=0 tx=0).stop 2023-11-13T11:02:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d2273640 1 -- 172.21.15.16:0/3712317333 reap_dead start 2023-11-13T11:02:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 shutdown_connections 2023-11-13T11:02:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 >> 172.21.15.16:0/3712317333 conn(0x5568e0eb64c0 msgr2=0x5568e0e97520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 shutdown_connections 2023-11-13T11:02:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:48.894+0000 7fd1d3f70f40 1 -- 172.21.15.16:0/3712317333 wait complete. 2023-11-13T11:02:48.915 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2023-11-13T11:02:48.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2023-11-13T11:02:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.134+0000 7f9ff2178640 1 Processor -- start 2023-11-13T11:02:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.134+0000 7f9ff2178640 1 -- start start 2023-11-13T11:02:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.136+0000 7f9ff2178640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9fec0ce9b0 0x7f9fec0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.136+0000 7f9ff2178640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec144950 con 0x7f9fec0ce550 2023-11-13T11:02:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.136+0000 7f9ff2178640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec0ce300 con 0x7f9fec144570 2023-11-13T11:02:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.136+0000 7f9ff2178640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fec0cd260 con 0x7f9fec0ce9b0 2023-11-13T11:02:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.136+0000 7f9fe3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9fec0ce550 0x7f9fec0b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40004/0 (socket says 172.21.15.16:40004) 2023-11-13T11:02:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.136+0000 7f9fe3fff640 1 -- 172.21.15.16:0/3317290919 learned_addr learned my addr 172.21.15.16:0/3317290919 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.136+0000 7f9febfff640 1 --2- 172.21.15.16:0/3317290919 >> v2:172.21.15.184:3300/0 conn(0x7f9fec0ce9b0 0x7f9fec0b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311020466 0 0) 0x7f9fec144950 con 0x7f9fec0ce550 2023-11-13T11:02:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd8003520 con 0x7f9fec0ce550 2023-11-13T11:02:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 313244482 0 0) 0x7f9fd8003520 con 0x7f9fec0ce550 2023-11-13T11:02:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 >> v2:172.21.15.184:3300/0 conn(0x7f9fec0ce9b0 msgr2=0x7f9fec0b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 --2- 172.21.15.16:0/3317290919 >> v2:172.21.15.184:3300/0 conn(0x7f9fec0ce9b0 0x7f9fec0b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 >> v1:172.21.15.184:6789/0 conn(0x7f9fec144570 legacy=0x7f9fec15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec0cd540 con 0x7f9fec0ce550 2023-11-13T11:02:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f9fd4001670 con 0x7f9fec0ce550 2023-11-13T11:02:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 >> v1:172.21.15.16:6789/0 conn(0x7f9fec0ce550 legacy=0x7f9fec0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd80039c0 0x7f9fd8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fd8005180 0x7f9fd800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec0cd260 con 0x7f9fd80047c0 2023-11-13T11:02:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd800c080 con 0x7f9fd80039c0 2023-11-13T11:02:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.137+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd800c240 con 0x7f9fd8005180 2023-11-13T11:02:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9fe3fff640 1 -- 172.21.15.16:0/3317290919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fd8005180 msgr2=0x7f9fd800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd80039c0 0x7f9fd8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9febfff640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fd8005180 0x7f9fd800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fd8005180 0x7f9fd800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9fe3fff640 1 -- 172.21.15.16:0/3317290919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd80039c0 msgr2=0x7f9fd8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd80039c0 0x7f9fd8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9fe3fff640 1 -- 172.21.15.16:0/3317290919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8003520 con 0x7f9fd80047c0 2023-11-13T11:02:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9febfff640 1 -- 172.21.15.16:0/3317290919 reap_dead start 2023-11-13T11:02:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.138+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd80039c0 0x7f9fd8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.139+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c0e0 tx=0x7f9fd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503d4b0f0f21b83d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.139+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4005ad0 con 0x7f9fd80047c0 2023-11-13T11:02:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.139+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd4005c50 con 0x7f9fd80047c0 2023-11-13T11:02:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.139+0000 7f9feaffd640 1 -- 172.21.15.16:0/3317290919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4007070 con 0x7f9fd80047c0 2023-11-13T11:02:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.140+0000 7f9ff2178640 1 -- 172.21.15.16:0/3317290919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 msgr2=0x7f9fd8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.140+0000 7f9ff2178640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c0e0 tx=0x7f9fd80057f0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.140+0000 7f9ff2178640 1 -- 172.21.15.16:0/3317290919 shutdown_connections 2023-11-13T11:02:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.140+0000 7f9ff2178640 1 --2- 172.21.15.16:0/3317290919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fd8004c40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.140+0000 7f9ff2178640 1 -- 172.21.15.16:0/3317290919 >> 172.21.15.16:0/3317290919 conn(0x7f9fec06a190 msgr2=0x7f9fec06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.140+0000 7f9ff2178640 1 -- 172.21.15.16:0/3317290919 shutdown_connections 2023-11-13T11:02:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.140+0000 7f9ff2178640 1 -- 172.21.15.16:0/3317290919 wait complete. 2023-11-13T11:02:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.142+0000 7f9ff2178640 1 Processor -- start 2023-11-13T11:02:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.142+0000 7f9ff2178640 1 -- start start 2023-11-13T11:02:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.143+0000 7f9ff2178640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fec142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.143+0000 7f9ff2178640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec140050 0x7f9fec1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.143+0000 7f9ff2178640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec1f5640 0x7f9fec1f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.143+0000 7f9ff2178640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec15a8b0 con 0x7f9fd80047c0 2023-11-13T11:02:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.143+0000 7f9ff2178640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1f6030 con 0x7f9fec140050 2023-11-13T11:02:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.143+0000 7f9ff2178640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec142c70 con 0x7f9fec1f5640 2023-11-13T11:02:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9feb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fec142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9feb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fec142590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39056/0 (socket says 172.21.15.16:39056) 2023-11-13T11:02:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9feb7fe640 1 -- 172.21.15.16:0/1381329307 learned_addr learned my addr 172.21.15.16:0/1381329307 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec140050 0x7f9fec1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9feb7fe640 1 -- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec1f5640 msgr2=0x7f9fec1f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec1f5640 0x7f9fec1f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9feb7fe640 1 -- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec140050 msgr2=0x7f9fec1584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec140050 0x7f9fec1584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9feb7fe640 1 -- 172.21.15.16:0/1381329307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec142f50 con 0x7f9fd80047c0 2023-11-13T11:02:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.144+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fec142590 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00d430 tx=0x7f9fdc00d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.145+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/1381329307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00b070 con 0x7f9fd80047c0 2023-11-13T11:02:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.145+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/1381329307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc003610 con 0x7f9fd80047c0 2023-11-13T11:02:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.145+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/1381329307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0039c0 con 0x7f9fd80047c0 2023-11-13T11:02:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.145+0000 7f9ff2178640 1 -- 172.21.15.16:0/1381329307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec143230 con 0x7f9fd80047c0 2023-11-13T11:02:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.145+0000 7f9ff2178640 1 -- 172.21.15.16:0/1381329307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec1404d0 con 0x7f9fd80047c0 2023-11-13T11:02:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.148+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/1381329307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fdc003b20 con 0x7f9fd80047c0 2023-11-13T11:02:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.149+0000 7f9fe8ff9640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fcc06ae40 0x7f9fcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.149+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/1381329307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f9fdc088f80 con 0x7f9fd80047c0 2023-11-13T11:02:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.149+0000 7f9ff4de9740 1 Processor -- start 2023-11-13T11:02:49.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.150+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fcc06ae40 0x7f9fcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.150+0000 7f9ff4de9740 1 -- start start 2023-11-13T11:02:49.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c6fd70 0x55e446c70150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fcc06ae40 0x7f9fcc06d300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4034000 tx=0x7f9fd4009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c3fd90 0x55e446c6cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c6d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c00d80 con 0x55e446c438f0 2023-11-13T11:02:49.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446bb3180 con 0x55e446c3fd90 2023-11-13T11:02:49.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e446bb0870 con 0x55e446c6fd70 2023-11-13T11:02:49.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9febfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c6d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9febfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c6d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39066/0 (socket says 172.21.15.16:39066) 2023-11-13T11:02:49.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9febfff640 1 -- 172.21.15.16:0/709772651 learned_addr learned my addr 172.21.15.16:0/709772651 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/709772651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c3fd90 0x55e446c6cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.151+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/709772651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c6fd70 0x55e446c70150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.152+0000 7f9febfff640 1 -- 172.21.15.16:0/709772651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c6fd70 msgr2=0x55e446c70150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.152+0000 7f9febfff640 1 --2- 172.21.15.16:0/709772651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c6fd70 0x55e446c70150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.152+0000 7f9febfff640 1 -- 172.21.15.16:0/709772651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c3fd90 msgr2=0x55e446c6cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.152+0000 7f9febfff640 1 --2- 172.21.15.16:0/709772651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c3fd90 0x55e446c6cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.152+0000 7f9febfff640 1 -- 172.21.15.16:0/709772651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e446b6eab0 con 0x55e446c438f0 2023-11-13T11:02:49.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.152+0000 7f9febfff640 1 --2- 172.21.15.16:0/709772651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c6d520 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0b30c0 tx=0x7f9fe4007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142707274bf19e3e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.152+0000 7f9fe17fa640 1 -- 172.21.15.16:0/709772651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4019070 con 0x55e446c438f0 2023-11-13T11:02:49.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.152+0000 7f9fe17fa640 1 -- 172.21.15.16:0/709772651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe400f050 con 0x55e446c438f0 2023-11-13T11:02:49.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.152+0000 7f9fe17fa640 1 -- 172.21.15.16:0/709772651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4003bd0 con 0x55e446c438f0 2023-11-13T11:02:49.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.153+0000 7f9ff4de9740 1 -- 172.21.15.16:0/709772651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 msgr2=0x55e446c6d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.153+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/709772651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c6d520 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0b30c0 tx=0x7f9fe4007870 comp rx=0 tx=0).stop 2023-11-13T11:02:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.153+0000 7f9ff4de9740 1 -- 172.21.15.16:0/709772651 shutdown_connections 2023-11-13T11:02:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.153+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/709772651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c6d520 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.153+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/709772651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c3fd90 0x55e446c6cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.153+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/709772651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c6fd70 0x55e446c70150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.153+0000 7f9ff4de9740 1 -- 172.21.15.16:0/709772651 >> 172.21.15.16:0/709772651 conn(0x55e446c42000 msgr2=0x55e446c8ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.153+0000 7f9ff4de9740 1 -- 172.21.15.16:0/709772651 shutdown_connections 2023-11-13T11:02:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.153+0000 7f9ff4de9740 1 -- 172.21.15.16:0/709772651 wait complete. 2023-11-13T11:02:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.154+0000 7f9ff4de9740 1 -- start start 2023-11-13T11:02:49.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.154+0000 7f9ff4de9740 1 Processor -- start 2023-11-13T11:02:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.155+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c65050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.155+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 0x55e446c53fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.155+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 0x55e446c72fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.155+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446bac160 con 0x55e446c438f0 2023-11-13T11:02:49.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.155+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446b48290 con 0x55e446c6fd70 2023-11-13T11:02:49.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.155+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e446b65c10 con 0x55e446c56cb0 2023-11-13T11:02:49.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.155+0000 7f9ff4de9740 10 client.0 _ll_register_callbacks cb 0x55e446b68270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:49.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.155+0000 7f9feb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c65050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9feb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c65050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39078/0 (socket says 172.21.15.16:39078) 2023-11-13T11:02:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9feb7fe640 1 -- 172.21.15.16:0/140882153 learned_addr learned my addr 172.21.15.16:0/140882153 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9febfff640 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 0x55e446c72fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 0x55e446c53fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9feb7fe640 1 -- 172.21.15.16:0/140882153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 msgr2=0x55e446c72fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 0x55e446c72fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9feb7fe640 1 -- 172.21.15.16:0/140882153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 msgr2=0x55e446c53fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 0x55e446c53fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9feb7fe640 1 -- 172.21.15.16:0/140882153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe4009da0 con 0x55e446c438f0 2023-11-13T11:02:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9febfff640 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 0x55e446c72fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c65050 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc057b10 tx=0x7f9fdc057c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31eaa984029442a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.156+0000 7f9fca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9fca7fc640 1 -- 172.21.15.16:0/140882153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc055f20 con 0x55e446c438f0 2023-11-13T11:02:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9ff4de9740 1 -- 172.21.15.16:0/140882153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e446ba5140 con 0x55e446c438f0 2023-11-13T11:02:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9fca7fc640 1 -- 172.21.15.16:0/140882153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f9fdc06f210 con 0x55e446c438f0 2023-11-13T11:02:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9fca7fc640 1 -- 172.21.15.16:0/140882153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fdc06f3d0 con 0x55e446c438f0 2023-11-13T11:02:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9ff4de9740 10 client.6585 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9ff4de9740 1 -- 172.21.15.16:0/140882153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55e446baa7e0 con 0x55e446c438f0 2023-11-13T11:02:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9fca7fc640 1 -- 172.21.15.16:0/140882153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e446ba5140 con 0x55e446c438f0 2023-11-13T11:02:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9fca7fc640 1 -- 172.21.15.16:0/140882153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9fdc084020 con 0x55e446c438f0 2023-11-13T11:02:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9ff4de9740 10 client.6585 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9ff4de9740 10 client.6585 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9ff4de9740 20 client.6585 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55e446c69f20 0x55e446c6a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9ff4de9740 4 client.6585 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.157+0000 7f9ff4de9740 1 -- 172.21.15.16:0/140882153 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e446ba4e60 con 0x55e446c69f20 2023-11-13T11:02:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.158+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55e446c69f20 0x55e446c6a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.158+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55e446c69f20 0x55e446c6a330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382c29b0561a1b70 server_cookie=7199375bbd9fd15d in_seq=0 out_seq=0 2023-11-13T11:02:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.158+0000 7f9fca7fc640 10 client.6585 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9fca7fc640 1 -- 172.21.15.16:0/140882153 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e446ba4e60 con 0x55e446c69f20 2023-11-13T11:02:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9fca7fc640 10 client.6585 handle_command_reply: tid=0 2023-11-13T11:02:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 client.6585 shutdown 2023-11-13T11:02:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 -- 172.21.15.16:0/140882153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 msgr2=0x55e446c65050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c65050 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc057b10 tx=0x7f9fdc057c40 comp rx=0 tx=0).stop 2023-11-13T11:02:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 -- 172.21.15.16:0/140882153 shutdown_connections 2023-11-13T11:02:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 0x55e446c72fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 0x55e446c53fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55e446c69f20 0x55e446c6a330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/140882153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c65050 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 -- 172.21.15.16:0/140882153 >> 172.21.15.16:0/140882153 conn(0x55e446c3fd90 msgr2=0x55e446c73a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9fca7fc640 0 client.6585 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:49.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.159+0000 7f9ff4de9740 1 -- 172.21.15.16:0/140882153 shutdown_connections 2023-11-13T11:02:49.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.160+0000 7f9ff4de9740 1 -- 172.21.15.16:0/140882153 wait complete. 2023-11-13T11:02:49.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.160+0000 7f9ff4de9740 20 client.6585 trim_cache size 0 max 16384 2023-11-13T11:02:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.177+0000 7f9ff4de9740 1 Processor -- start 2023-11-13T11:02:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.177+0000 7f9ff4de9740 1 -- start start 2023-11-13T11:02:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c438f0 0x55e446ca3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c56cb0 0x55e446cad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c6fd70 0x55e446c45f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4468a5390 con 0x55e446c56cb0 2023-11-13T11:02:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c54380 con 0x55e446c438f0 2023-11-13T11:02:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e446c6c8e0 con 0x55e446c6fd70 2023-11-13T11:02:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9fe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c56cb0 0x55e446cad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9fe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c56cb0 0x55e446cad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39086/0 (socket says 172.21.15.16:39086) 2023-11-13T11:02:49.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9fe3fff640 1 -- 172.21.15.16:0/553804100 learned_addr learned my addr 172.21.15.16:0/553804100 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9fe3fff640 1 -- 172.21.15.16:0/553804100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c6fd70 msgr2=0x55e446c45f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/553804100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c6fd70 0x55e446c45f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/553804100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c438f0 0x55e446ca3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9fe3fff640 1 -- 172.21.15.16:0/553804100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c438f0 msgr2=0x55e446ca3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/553804100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c438f0 0x55e446ca3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.178+0000 7f9fe3fff640 1 -- 172.21.15.16:0/553804100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e446b47b20 con 0x55e446c56cb0 2023-11-13T11:02:49.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.179+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/553804100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c56cb0 0x55e446cad450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c1d0 tx=0x7f9fd4007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7764dc8721b6cb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.179+0000 7f9fe1ffb640 1 -- 172.21.15.16:0/553804100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40074f0 con 0x55e446c56cb0 2023-11-13T11:02:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.179+0000 7f9fe1ffb640 1 -- 172.21.15.16:0/553804100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd404a040 con 0x55e446c56cb0 2023-11-13T11:02:49.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.179+0000 7f9fe1ffb640 1 -- 172.21.15.16:0/553804100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd404f430 con 0x55e446c56cb0 2023-11-13T11:02:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 -- 172.21.15.16:0/553804100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c56cb0 msgr2=0x55e446cad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/553804100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c56cb0 0x55e446cad450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c1d0 tx=0x7f9fd4007000 comp rx=0 tx=0).stop 2023-11-13T11:02:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 -- 172.21.15.16:0/553804100 shutdown_connections 2023-11-13T11:02:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/553804100 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c6fd70 0x55e446c45f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/553804100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c56cb0 0x55e446cad450 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/553804100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c438f0 0x55e446ca3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 -- 172.21.15.16:0/553804100 >> 172.21.15.16:0/553804100 conn(0x55e446c3fd90 msgr2=0x55e446c43df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 -- 172.21.15.16:0/553804100 shutdown_connections 2023-11-13T11:02:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 -- 172.21.15.16:0/553804100 wait complete. 2023-11-13T11:02:49.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.180+0000 7f9ff4de9740 1 -- start start 2023-11-13T11:02:49.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.181+0000 7f9ff4de9740 1 Processor -- start 2023-11-13T11:02:49.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c44b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 0x55e446c6b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9ff4de9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 0x55e446c6b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c63310 con 0x55e446c438f0 2023-11-13T11:02:49.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e446c453c0 con 0x55e446c6fd70 2023-11-13T11:02:49.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9ff4de9740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e446c67d20 con 0x55e446c56cb0 2023-11-13T11:02:49.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9ff4de9740 10 client.0 _ll_register_callbacks cb 0x7f9fdc053f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9feb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c44b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9feb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c44b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39092/0 (socket says 172.21.15.16:39092) 2023-11-13T11:02:49.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3940537774 learned_addr learned my addr 172.21.15.16:0/3940537774 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9febfff640 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 0x55e446c6b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3940537774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 msgr2=0x55e446c6b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 0x55e446c6b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3940537774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 msgr2=0x55e446c6b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 0x55e446c6b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3940537774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd8003520 con 0x55e446c438f0 2023-11-13T11:02:49.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.182+0000 7f9febfff640 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 0x55e446c6b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c44b30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0545a0 tx=0x7f9fdc087160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891730ecfebfc3ac server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9fca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9fca7fc640 1 -- 172.21.15.16:0/3940537774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc05bc20 con 0x55e446c438f0 2023-11-13T11:02:49.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9fca7fc640 1 -- 172.21.15.16:0/3940537774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f9fdc07fc30 con 0x55e446c438f0 2023-11-13T11:02:49.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9ff4de9740 1 -- 172.21.15.16:0/3940537774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fe400f050 con 0x55e446c438f0 2023-11-13T11:02:49.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9fca7fc640 1 -- 172.21.15.16:0/3940537774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fdc07fdf0 con 0x55e446c438f0 2023-11-13T11:02:49.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9ff4de9740 10 client.6591 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:49.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9ff4de9740 1 -- 172.21.15.16:0/3940537774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55e446b6d310 con 0x55e446c438f0 2023-11-13T11:02:49.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9fca7fc640 1 -- 172.21.15.16:0/3940537774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc067020 con 0x55e446c438f0 2023-11-13T11:02:49.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9fca7fc640 1 -- 172.21.15.16:0/3940537774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9fdc07c070 con 0x55e446c438f0 2023-11-13T11:02:49.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9ff4de9740 10 client.6591 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:49.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9ff4de9740 10 client.6591 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:49.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9ff4de9740 20 client.6591 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:49.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55e446c3b730 0x55e446c63730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9ff4de9740 4 client.6591 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2023-11-13T11:02:49.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.183+0000 7f9ff4de9740 1 -- 172.21.15.16:0/3940537774 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x55e4468acbe0 con 0x55e446c3b730 2023-11-13T11:02:49.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.184+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55e446c3b730 0x55e446c63730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.184+0000 7f9fe3fff640 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55e446c3b730 0x55e446c63730 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9db64875ddf2117 server_cookie=d676d90ecf70d7d0 in_seq=0 out_seq=0 2023-11-13T11:02:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.184+0000 7f9fca7fc640 10 client.6591 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9fca7fc640 1 -- 172.21.15.16:0/3940537774 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e4468acbe0 con 0x55e446c3b730 2023-11-13T11:02:49.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9fca7fc640 10 client.6591 handle_command_reply: tid=0 2023-11-13T11:02:49.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 client.6591 shutdown 2023-11-13T11:02:49.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 -- 172.21.15.16:0/3940537774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 msgr2=0x55e446c44b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c44b30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0545a0 tx=0x7f9fdc087160 comp rx=0 tx=0).stop 2023-11-13T11:02:49.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 -- 172.21.15.16:0/3940537774 shutdown_connections 2023-11-13T11:02:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e446c6fd70 0x55e446c6b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55e446c56cb0 0x55e446c6b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55e446c3b730 0x55e446c63730 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 --2- 172.21.15.16:0/3940537774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e446c438f0 0x55e446c44b30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 -- 172.21.15.16:0/3940537774 >> 172.21.15.16:0/3940537774 conn(0x55e446c3fd90 msgr2=0x55e446c53630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9fca7fc640 0 client.6591 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:49.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 -- 172.21.15.16:0/3940537774 shutdown_connections 2023-11-13T11:02:49.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 1 -- 172.21.15.16:0/3940537774 wait complete. 2023-11-13T11:02:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.185+0000 7f9ff4de9740 20 client.6591 trim_cache size 0 max 16384 2023-11-13T11:02:49.225 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2023-11-13T11:02:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "48a0cfc1-ccc4-4fc5-b4b7-94a1026a05b8", 2023-11-13T11:02:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2023-11-13T11:02:49.227 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:49.228 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f0f71a64640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4c [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v808 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6183=0-4194304@1} caps={6183=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=6183 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55fd3fc66c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a4c:[<0x10000000001/new_file_1 v808>,<0x10000000000/ceph-qa-suite v135>,<0x1/scrub_test_path v93>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-11-13T11:02:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 -- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fcc06ae40 msgr2=0x7f9fcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fcc06ae40 0x7f9fcc06d300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4034000 tx=0x7f9fd4009030 comp rx=0 tx=0).stop 2023-11-13T11:02:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 -- 172.21.15.16:0/1381329307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 msgr2=0x7f9fec142590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fec142590 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00d430 tx=0x7f9fdc00d8f0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 -- 172.21.15.16:0/1381329307 shutdown_connections 2023-11-13T11:02:49.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fcc06ae40 0x7f9fcc06d300 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fec1f5640 0x7f9fec1f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fec140050 0x7f9fec1584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 --2- 172.21.15.16:0/1381329307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd80047c0 0x7f9fec142590 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 -- 172.21.15.16:0/1381329307 >> 172.21.15.16:0/1381329307 conn(0x7f9fec06a190 msgr2=0x7f9fec06fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 -- 172.21.15.16:0/1381329307 shutdown_connections 2023-11-13T11:02:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.186+0000 7f9ff2178640 1 -- 172.21.15.16:0/1381329307 wait complete. 2023-11-13T11:02:49.234 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '48a0cfc1-ccc4-4fc5-b4b7-94a1026a05b8', 'mode': 'asynchronous'}' 2023-11-13T11:02:49.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-13T11:02:49.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.466+0000 7f4e4abe2640 1 Processor -- start 2023-11-13T11:02:49.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.466+0000 7f4e4abe2640 1 -- start start 2023-11-13T11:02:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.468+0000 7f4e4abe2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4e4415b8e0 0x7f4e4406c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.468+0000 7f4e4abe2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4406b860 con 0x7f4e4415bcc0 2023-11-13T11:02:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.468+0000 7f4e4abe2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4406bb40 con 0x7f4e440ce260 2023-11-13T11:02:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.468+0000 7f4e4abe2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4406be20 con 0x7f4e4415b8e0 2023-11-13T11:02:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.468+0000 7f4e493df640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4e4415bcc0 0x7f4e440cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40020/0 (socket says 172.21.15.16:40020) 2023-11-13T11:02:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.468+0000 7f4e493df640 1 -- 172.21.15.16:0/4023082490 learned_addr learned my addr 172.21.15.16:0/4023082490 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.469+0000 7f4e49be0640 1 --2- 172.21.15.16:0/4023082490 >> v2:172.21.15.184:3300/0 conn(0x7f4e4415b8e0 0x7f4e4406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.469+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574319597 0 0) 0x7f4e4406b860 con 0x7f4e4415bcc0 2023-11-13T11:02:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.469+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e30003520 con 0x7f4e4415bcc0 2023-11-13T11:02:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.469+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3614493391 0 0) 0x7f4e30003520 con 0x7f4e4415bcc0 2023-11-13T11:02:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.469+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 >> v2:172.21.15.184:3300/0 conn(0x7f4e4415b8e0 msgr2=0x7f4e4406c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.469+0000 7f4e48bde640 1 --2- 172.21.15.16:0/4023082490 >> v2:172.21.15.184:3300/0 conn(0x7f4e4415b8e0 0x7f4e4406c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.469+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 >> v1:172.21.15.184:6789/0 conn(0x7f4e440ce260 legacy=0x7f4e440cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.469+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4406c100 con 0x7f4e4415bcc0 2023-11-13T11:02:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.469+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f4e34001670 con 0x7f4e4415bcc0 2023-11-13T11:02:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 >> v1:172.21.15.16:6789/0 conn(0x7f4e4415bcc0 legacy=0x7f4e440cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e48bde640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e300039c0 0x7f4e30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e48bde640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e300047c0 0x7f4e30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e48bde640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e30005180 0x7f4e3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4406be20 con 0x7f4e30005180 2023-11-13T11:02:49.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3000c080 con 0x7f4e300039c0 2023-11-13T11:02:49.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3000c240 con 0x7f4e300047c0 2023-11-13T11:02:49.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e4a3e1640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e30005180 0x7f4e3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e49be0640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e300039c0 0x7f4e30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e493df640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e300047c0 0x7f4e30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e4a3e1640 1 -- 172.21.15.16:0/4023082490 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e300047c0 msgr2=0x7f4e30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.470+0000 7f4e4a3e1640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e300047c0 0x7f4e30004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.471+0000 7f4e4a3e1640 1 -- 172.21.15.16:0/4023082490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e300039c0 msgr2=0x7f4e30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.471+0000 7f4e4a3e1640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e300039c0 0x7f4e30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.471+0000 7f4e4a3e1640 1 -- 172.21.15.16:0/4023082490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e30003520 con 0x7f4e30005180 2023-11-13T11:02:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.471+0000 7f4e4a3e1640 1 -- 172.21.15.16:0/4023082490 reap_dead start 2023-11-13T11:02:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.472+0000 7f4e4a3e1640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e30005180 0x7f4e3000b980 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c001e20 tx=0x7f4e300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e09224c5b43e6d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.472+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c003b70 con 0x7f4e30005180 2023-11-13T11:02:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.472+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c003cd0 con 0x7f4e30005180 2023-11-13T11:02:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.472+0000 7f4e48bde640 1 -- 172.21.15.16:0/4023082490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c005310 con 0x7f4e30005180 2023-11-13T11:02:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.472+0000 7f4e4abe2640 1 -- 172.21.15.16:0/4023082490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e30005180 msgr2=0x7f4e3000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.472+0000 7f4e4abe2640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e30005180 0x7f4e3000b980 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c001e20 tx=0x7f4e300057f0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.473+0000 7f4e4abe2640 1 -- 172.21.15.16:0/4023082490 shutdown_connections 2023-11-13T11:02:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.473+0000 7f4e4abe2640 1 --2- 172.21.15.16:0/4023082490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e30005180 0x7f4e3000b980 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.473+0000 7f4e4abe2640 1 -- 172.21.15.16:0/4023082490 >> 172.21.15.16:0/4023082490 conn(0x7f4e440b0970 msgr2=0x7f4e440b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.473+0000 7f4e4abe2640 1 -- 172.21.15.16:0/4023082490 shutdown_connections 2023-11-13T11:02:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.473+0000 7f4e4abe2640 1 -- 172.21.15.16:0/4023082490 wait complete. 2023-11-13T11:02:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.475+0000 7f4e4abe2640 1 Processor -- start 2023-11-13T11:02:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.475+0000 7f4e4abe2640 1 -- start start 2023-11-13T11:02:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.476+0000 7f4e4abe2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e30005180 0x7f4e4415e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.476+0000 7f4e4abe2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e440c37f0 0x7f4e440c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.476+0000 7f4e4abe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4414dda0 0x7f4e4414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.476+0000 7f4e4abe2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440c4ef0 con 0x7f4e4414dda0 2023-11-13T11:02:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.476+0000 7f4e4abe2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4414e760 con 0x7f4e30005180 2023-11-13T11:02:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.476+0000 7f4e4abe2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440c4270 con 0x7f4e440c37f0 2023-11-13T11:02:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.476+0000 7f4e4a3e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4414dda0 0x7f4e4414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.476+0000 7f4e4a3e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4414dda0 0x7f4e4414e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39114/0 (socket says 172.21.15.16:39114) 2023-11-13T11:02:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.476+0000 7f4e4a3e1640 1 -- 172.21.15.16:0/2175479172 learned_addr learned my addr 172.21.15.16:0/2175479172 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e49be0640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e30005180 0x7f4e4415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e493df640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e440c37f0 0x7f4e440c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e4a3e1640 1 -- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e440c37f0 msgr2=0x7f4e440c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e4a3e1640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e440c37f0 0x7f4e440c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e4a3e1640 1 -- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e30005180 msgr2=0x7f4e4415e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e4a3e1640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e30005180 0x7f4e4415e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e4a3e1640 1 -- 172.21.15.16:0/2175479172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e440c4550 con 0x7f4e4414dda0 2023-11-13T11:02:49.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e4a3e1640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4414dda0 0x7f4e4414e220 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c001f40 tx=0x7f4e3c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e3a7fc640 1 -- 172.21.15.16:0/2175479172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c006d50 con 0x7f4e4414dda0 2023-11-13T11:02:49.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e3a7fc640 1 -- 172.21.15.16:0/2175479172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c007030 con 0x7f4e4414dda0 2023-11-13T11:02:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e3a7fc640 1 -- 172.21.15.16:0/2175479172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0073c0 con 0x7f4e4414dda0 2023-11-13T11:02:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.477+0000 7f4e4abe2640 1 -- 172.21.15.16:0/2175479172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e440c4830 con 0x7f4e4414dda0 2023-11-13T11:02:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.478+0000 7f4e4abe2640 1 -- 172.21.15.16:0/2175479172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e441564f0 con 0x7f4e4414dda0 2023-11-13T11:02:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.480+0000 7f4e3a7fc640 1 -- 172.21.15.16:0/2175479172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e3c007520 con 0x7f4e4414dda0 2023-11-13T11:02:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.481+0000 7f4e3a7fc640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e2806ae40 0x7f4e2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.481+0000 7f4e3a7fc640 1 -- 172.21.15.16:0/2175479172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4e3c0ab2e0 con 0x7f4e4414dda0 2023-11-13T11:02:49.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.481+0000 7f4e4e9d7740 1 Processor -- start 2023-11-13T11:02:49.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.481+0000 7f4e4e9d7740 1 -- start start 2023-11-13T11:02:49.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.482+0000 7f4e49be0640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e2806ae40 0x7f4e2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.482+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b6d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.482+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b700650 0x55b46b703000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.482+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b6d3750 0x55b46b703540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.482+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46b645e90 con 0x55b46b700270 2023-11-13T11:02:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.482+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46b643580 con 0x55b46b6d3750 2023-11-13T11:02:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.482+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b46b63ee70 con 0x55b46b700650 2023-11-13T11:02:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b6d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b6d3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39118/0 (socket says 172.21.15.16:39118) 2023-11-13T11:02:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 -- 172.21.15.16:0/506432985 learned_addr learned my addr 172.21.15.16:0/506432985 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e2806ae40 0x7f4e2806d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009990 tx=0x7f4e40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e493df640 1 --2- 172.21.15.16:0/506432985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b700650 0x55b46b703000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e4a3e1640 1 --2- 172.21.15.16:0/506432985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b6d3750 0x55b46b703540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 -- 172.21.15.16:0/506432985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b700650 msgr2=0x55b46b703000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 --2- 172.21.15.16:0/506432985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b700650 0x55b46b703000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 -- 172.21.15.16:0/506432985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b6d3750 msgr2=0x55b46b703540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 --2- 172.21.15.16:0/506432985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b6d3750 0x55b46b703540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 -- 172.21.15.16:0/506432985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b46b609020 con 0x55b46b700270 2023-11-13T11:02:49.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.483+0000 7f4e49be0640 1 --2- 172.21.15.16:0/506432985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b6d3210 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4e40013cf0 tx=0x7f4e40016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29caed07ebf44a7b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.484+0000 7f4e277fe640 1 -- 172.21.15.16:0/506432985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40023440 con 0x55b46b700270 2023-11-13T11:02:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.484+0000 7f4e277fe640 1 -- 172.21.15.16:0/506432985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e400235a0 con 0x55b46b700270 2023-11-13T11:02:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.484+0000 7f4e277fe640 1 -- 172.21.15.16:0/506432985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40023910 con 0x55b46b700270 2023-11-13T11:02:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.484+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/506432985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 msgr2=0x55b46b6d3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.484+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/506432985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b6d3210 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4e40013cf0 tx=0x7f4e40016000 comp rx=0 tx=0).stop 2023-11-13T11:02:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.485+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/506432985 shutdown_connections 2023-11-13T11:02:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.485+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/506432985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b6d3750 0x55b46b703540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.485+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/506432985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b700650 0x55b46b703000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.485+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/506432985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b6d3210 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.485+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/506432985 >> 172.21.15.16:0/506432985 conn(0x55b46b6d5480 msgr2=0x55b46b71e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.485+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/506432985 shutdown_connections 2023-11-13T11:02:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.485+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/506432985 wait complete. 2023-11-13T11:02:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.485+0000 7f4e4e9d7740 1 -- start start 2023-11-13T11:02:49.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.486+0000 7f4e4e9d7740 1 Processor -- start 2023-11-13T11:02:49.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b71eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700650 0x55b46b721510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b6fb670 0x55b46b721a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46b5db480 con 0x55b46b700270 2023-11-13T11:02:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46b5f8dc0 con 0x55b46b700650 2023-11-13T11:02:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b46b337f90 con 0x55b46b6fb670 2023-11-13T11:02:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e4e9d7740 10 client.0 _ll_register_callbacks cb 0x55b46b350be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e49be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b71eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e49be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b71eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39120/0 (socket says 172.21.15.16:39120) 2023-11-13T11:02:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e49be0640 1 -- 172.21.15.16:0/2100772111 learned_addr learned my addr 172.21.15.16:0/2100772111 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e493df640 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700650 0x55b46b721510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e49be0640 1 -- 172.21.15.16:0/2100772111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b6fb670 msgr2=0x55b46b721a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e49be0640 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b6fb670 0x55b46b721a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e49be0640 1 -- 172.21.15.16:0/2100772111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700650 msgr2=0x55b46b721510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e49be0640 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700650 0x55b46b721510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e49be0640 1 -- 172.21.15.16:0/2100772111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e40013a20 con 0x55b46b700270 2023-11-13T11:02:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.487+0000 7f4e493df640 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700650 0x55b46b721510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.488+0000 7f4e49be0640 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b71eab0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4e40016cc0 tx=0x7f4e40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bf79cfb8b0bb44 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.488+0000 7f4e24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.488+0000 7f4e24ff9640 1 -- 172.21.15.16:0/2100772111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e400239e0 con 0x55b46b700270 2023-11-13T11:02:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.488+0000 7f4e24ff9640 1 -- 172.21.15.16:0/2100772111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4e40023b40 con 0x55b46b700270 2023-11-13T11:02:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.488+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/2100772111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b46b637e50 con 0x55b46b700270 2023-11-13T11:02:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.488+0000 7f4e24ff9640 1 -- 172.21.15.16:0/2100772111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e40023d00 con 0x55b46b700270 2023-11-13T11:02:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.488+0000 7f4e4e9d7740 10 client.6603 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.488+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/2100772111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55b46b63d4f0 con 0x55b46b700270 2023-11-13T11:02:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.489+0000 7f4e24ff9640 1 -- 172.21.15.16:0/2100772111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b46b637e50 con 0x55b46b700270 2023-11-13T11:02:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.489+0000 7f4e24ff9640 1 -- 172.21.15.16:0/2100772111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e40004020 con 0x55b46b700270 2023-11-13T11:02:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.489+0000 7f4e4e9d7740 10 client.6603 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.489+0000 7f4e4e9d7740 10 client.6603 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.489+0000 7f4e4e9d7740 20 client.6603 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.489+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55b46b705510 0x55b46b705920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.489+0000 7f4e4e9d7740 4 client.6603 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.489+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/2100772111 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b46b637b70 con 0x55b46b705510 2023-11-13T11:02:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.489+0000 7f4e493df640 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55b46b705510 0x55b46b705920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.490+0000 7f4e493df640 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55b46b705510 0x55b46b705920 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dfc10b167686ab0 server_cookie=b3f054447e9a2f10 in_seq=0 out_seq=0 2023-11-13T11:02:49.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.490+0000 7f4e24ff9640 10 client.6603 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:49.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.490+0000 7f4e24ff9640 1 -- 172.21.15.16:0/2100772111 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b46b637b70 con 0x55b46b705510 2023-11-13T11:02:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.490+0000 7f4e24ff9640 10 client.6603 handle_command_reply: tid=0 2023-11-13T11:02:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.490+0000 7f4e4e9d7740 1 client.6603 shutdown 2023-11-13T11:02:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/2100772111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 msgr2=0x55b46b71eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b71eab0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4e40016cc0 tx=0x7f4e40004730 comp rx=0 tx=0).stop 2023-11-13T11:02:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/2100772111 shutdown_connections 2023-11-13T11:02:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b6fb670 0x55b46b721a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700650 0x55b46b721510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55b46b705510 0x55b46b705920 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/2100772111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b700270 0x55b46b71eab0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e24ff9640 0 client.6603 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/2100772111 >> 172.21.15.16:0/2100772111 conn(0x55b46b6d3750 msgr2=0x55b46b6d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/2100772111 shutdown_connections 2023-11-13T11:02:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/2100772111 wait complete. 2023-11-13T11:02:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.491+0000 7f4e4e9d7740 20 client.6603 trim_cache size 0 max 16384 2023-11-13T11:02:49.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.508+0000 7f4e4e9d7740 1 Processor -- start 2023-11-13T11:02:49.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.508+0000 7f4e4e9d7740 1 -- start start 2023-11-13T11:02:49.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b6d3750 0x55b46b6e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b707930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b700270 0x55b46b707e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46b720870 con 0x55b46b6fb670 2023-11-13T11:02:49.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46b7066a0 con 0x55b46b6d3750 2023-11-13T11:02:49.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b46b705ce0 con 0x55b46b700270 2023-11-13T11:02:49.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e493df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b707930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e493df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b707930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39122/0 (socket says 172.21.15.16:39122) 2023-11-13T11:02:49.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e493df640 1 -- 172.21.15.16:0/1028293697 learned_addr learned my addr 172.21.15.16:0/1028293697 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e493df640 1 -- 172.21.15.16:0/1028293697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b700270 msgr2=0x55b46b707e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e493df640 1 --2- 172.21.15.16:0/1028293697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b700270 0x55b46b707e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e493df640 1 -- 172.21.15.16:0/1028293697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b6d3750 msgr2=0x55b46b6e9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e493df640 1 --2- 172.21.15.16:0/1028293697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b6d3750 0x55b46b6e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.509+0000 7f4e493df640 1 -- 172.21.15.16:0/1028293697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b46b5dad10 con 0x55b46b6fb670 2023-11-13T11:02:49.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.510+0000 7f4e493df640 1 --2- 172.21.15.16:0/1028293697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b707930 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4e34000ff0 tx=0x7f4e34007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae681d79b162f68 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.510+0000 7f4e27fff640 1 -- 172.21.15.16:0/1028293697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34004c00 con 0x55b46b6fb670 2023-11-13T11:02:49.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.510+0000 7f4e27fff640 1 -- 172.21.15.16:0/1028293697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e34004d60 con 0x55b46b6fb670 2023-11-13T11:02:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.510+0000 7f4e27fff640 1 -- 172.21.15.16:0/1028293697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34007240 con 0x55b46b6fb670 2023-11-13T11:02:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.510+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1028293697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 msgr2=0x55b46b707930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.510+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1028293697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b707930 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4e34000ff0 tx=0x7f4e34007a90 comp rx=0 tx=0).stop 2023-11-13T11:02:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.511+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1028293697 shutdown_connections 2023-11-13T11:02:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.511+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1028293697 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b700270 0x55b46b707e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.511+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1028293697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b707930 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.511+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1028293697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b6d3750 0x55b46b6e9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.511+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1028293697 >> 172.21.15.16:0/1028293697 conn(0x55b46b6d96e0 msgr2=0x55b46b706c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.511+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1028293697 shutdown_connections 2023-11-13T11:02:49.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.511+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1028293697 wait complete. 2023-11-13T11:02:49.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.511+0000 7f4e4e9d7740 1 -- start start 2023-11-13T11:02:49.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.512+0000 7f4e4e9d7740 1 Processor -- start 2023-11-13T11:02:49.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.512+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b6d3750 0x55b46b6d7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b6d7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e4e9d7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700270 0x55b46b72c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46b72f800 con 0x55b46b6fb670 2023-11-13T11:02:49.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46b72f9a0 con 0x55b46b700270 2023-11-13T11:02:49.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e4e9d7740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b46b6ec8b0 con 0x55b46b6d3750 2023-11-13T11:02:49.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e4e9d7740 10 client.0 _ll_register_callbacks cb 0x7f4e400095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:49.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e493df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b6d7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e493df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b6d7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39126/0 (socket says 172.21.15.16:39126) 2023-11-13T11:02:49.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e493df640 1 -- 172.21.15.16:0/1453317403 learned_addr learned my addr 172.21.15.16:0/1453317403 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:49.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e49be0640 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b6d3750 0x55b46b6d7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e493df640 1 -- 172.21.15.16:0/1453317403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b6d3750 msgr2=0x55b46b6d7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e493df640 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b6d3750 0x55b46b6d7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e493df640 1 -- 172.21.15.16:0/1453317403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700270 msgr2=0x55b46b72c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:49.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e493df640 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700270 0x55b46b72c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.513+0000 7f4e493df640 1 -- 172.21.15.16:0/1453317403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4406c100 con 0x55b46b6fb670 2023-11-13T11:02:49.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e493df640 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b6d7ff0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4e34001eb0 tx=0x7f4e34001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e443834107b1f011 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:49.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:49.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e24ff9640 1 -- 172.21.15.16:0/1453317403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34007450 con 0x55b46b6fb670 2023-11-13T11:02:49.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e24ff9640 1 -- 172.21.15.16:0/1453317403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4e3401c020 con 0x55b46b6fb670 2023-11-13T11:02:49.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1453317403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e400235a0 con 0x55b46b6fb670 2023-11-13T11:02:49.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e24ff9640 1 -- 172.21.15.16:0/1453317403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e34004b80 con 0x55b46b6fb670 2023-11-13T11:02:49.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e4e9d7740 10 client.6609 fetch_fsmap learned FSMap version 29 2023-11-13T11:02:49.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1453317403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55b46b600300 con 0x55b46b6fb670 2023-11-13T11:02:49.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e24ff9640 1 -- 172.21.15.16:0/1453317403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e400235a0 con 0x55b46b6fb670 2023-11-13T11:02:49.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e24ff9640 1 -- 172.21.15.16:0/1453317403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e340075b0 con 0x55b46b6fb670 2023-11-13T11:02:49.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.514+0000 7f4e4e9d7740 10 client.6609 fetch_fsmap finished waiting for FSMap version 29 2023-11-13T11:02:49.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.515+0000 7f4e4e9d7740 10 client.6609 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:49.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.515+0000 7f4e4e9d7740 20 client.6609 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:49.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.515+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55b46b700650 0x55b46b72cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:49.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.515+0000 7f4e4e9d7740 4 client.6609 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-13T11:02:49.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.515+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1453317403 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b46b33f7e0 con 0x55b46b700650 2023-11-13T11:02:49.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.515+0000 7f4e49be0640 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55b46b700650 0x55b46b72cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:49.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.515+0000 7f4e49be0640 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55b46b700650 0x55b46b72cb40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=138f6e490ce591d1 server_cookie=f7221b38e3cdd91c in_seq=0 out_seq=0 2023-11-13T11:02:49.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.515+0000 7f4e24ff9640 10 client.6609 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:49.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e24ff9640 1 -- 172.21.15.16:0/1453317403 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x55b46b33f7e0 con 0x55b46b700650 2023-11-13T11:02:49.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e24ff9640 10 client.6609 handle_command_reply: tid=0 2023-11-13T11:02:49.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 client.6609 shutdown 2023-11-13T11:02:49.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1453317403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 msgr2=0x55b46b6d7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b6d7ff0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4e34001eb0 tx=0x7f4e34001930 comp rx=0 tx=0).stop 2023-11-13T11:02:49.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1453317403 shutdown_connections 2023-11-13T11:02:49.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46b700270 0x55b46b72c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55b46b700650 0x55b46b72cb40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b46b6fb670 0x55b46b6d7ff0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 --2- 172.21.15.16:0/1453317403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55b46b6d3750 0x55b46b6d7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1453317403 >> 172.21.15.16:0/1453317403 conn(0x55b46b6d96e0 msgr2=0x55b46b707930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e24ff9640 0 client.6609 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1453317403 shutdown_connections 2023-11-13T11:02:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 1 -- 172.21.15.16:0/1453317403 wait complete. 2023-11-13T11:02:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.516+0000 7f4e4e9d7740 20 client.6609 trim_cache size 0 max 16384 2023-11-13T11:02:49.540 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-13T11:02:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2023-11-13T11:02:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "48a0cfc1-ccc4-4fc5-b4b7-94a1026a05b8": { 2023-11-13T11:02:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2023-11-13T11:02:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "48a0cfc1-ccc4-4fc5-b4b7-94a1026a05b8", 2023-11-13T11:02:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "" 2023-11-13T11:02:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:02:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:02:49.542 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 -- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e2806ae40 msgr2=0x7f4e2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e2806ae40 0x7f4e2806d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009990 tx=0x7f4e40008040 comp rx=0 tx=0).stop 2023-11-13T11:02:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 -- 172.21.15.16:0/2175479172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4414dda0 msgr2=0x7f4e4414e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4414dda0 0x7f4e4414e220 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c001f40 tx=0x7f4e3c0014a0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 -- 172.21.15.16:0/2175479172 shutdown_connections 2023-11-13T11:02:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4e2806ae40 0x7f4e2806d300 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4414dda0 0x7f4e4414e220 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e440c37f0 0x7f4e440c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 --2- 172.21.15.16:0/2175479172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e30005180 0x7f4e4415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 -- 172.21.15.16:0/2175479172 >> 172.21.15.16:0/2175479172 conn(0x7f4e440b0970 msgr2=0x7f4e440caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 -- 172.21.15.16:0/2175479172 shutdown_connections 2023-11-13T11:02:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:49.517+0000 7f4e4abe2640 1 -- 172.21.15.16:0/2175479172 wait complete. 2023-11-13T11:02:49.595 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:02:49.604 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:02:54.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-13T11:02:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.781+0000 7f6e6bfff640 1 Processor -- start 2023-11-13T11:02:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.781+0000 7f6e6bfff640 1 -- start start 2023-11-13T11:02:54.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.783+0000 7f6e6bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e6c0ce260 0x7f6e6c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.783+0000 7f6e6bfff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c06d010 con 0x7f6e6c15bcc0 2023-11-13T11:02:54.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.783+0000 7f6e6bfff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c06b860 con 0x7f6e6c15b8e0 2023-11-13T11:02:54.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.783+0000 7f6e6bfff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c06bb40 con 0x7f6e6c0ce260 2023-11-13T11:02:54.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.783+0000 7f6e6a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6e6c15bcc0 0x7f6e6c0cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41340/0 (socket says 172.21.15.16:41340) 2023-11-13T11:02:54.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.783+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/2386485108 learned_addr learned my addr 172.21.15.16:0/2386485108 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/2386485108 >> v2:172.21.15.184:3300/0 conn(0x7f6e6c0ce260 0x7f6e6c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247680990 0 0) 0x7f6e6c06d010 con 0x7f6e6c15bcc0 2023-11-13T11:02:54.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e54003520 con 0x7f6e6c15bcc0 2023-11-13T11:02:54.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2374940019 0 0) 0x7f6e54003520 con 0x7f6e6c15bcc0 2023-11-13T11:02:54.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 >> v2:172.21.15.184:3300/0 conn(0x7f6e6c0ce260 msgr2=0x7f6e6c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 --2- 172.21.15.16:0/2386485108 >> v2:172.21.15.184:3300/0 conn(0x7f6e6c0ce260 0x7f6e6c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 >> v1:172.21.15.184:6789/0 conn(0x7f6e6c15b8e0 legacy=0x7f6e6c06c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c06be20 con 0x7f6e6c15bcc0 2023-11-13T11:02:54.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f6e60001670 con 0x7f6e6c15bcc0 2023-11-13T11:02:54.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 >> v1:172.21.15.16:6789/0 conn(0x7f6e6c15bcc0 legacy=0x7f6e6c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540039c0 0x7f6e54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e540047c0 0x7f6e54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e54005180 0x7f6e5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c06bb40 con 0x7f6e540047c0 2023-11-13T11:02:54.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5400c080 con 0x7f6e54005180 2023-11-13T11:02:54.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.784+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5400c240 con 0x7f6e540039c0 2023-11-13T11:02:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6a7fc640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e540047c0 0x7f6e54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6affd640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540039c0 0x7f6e54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e54005180 0x7f6e5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/2386485108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540039c0 msgr2=0x7f6e54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6a7fc640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540039c0 0x7f6e54003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/2386485108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e54005180 msgr2=0x7f6e5400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6a7fc640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e54005180 0x7f6e5400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/2386485108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54003520 con 0x7f6e540047c0 2023-11-13T11:02:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/2386485108 reap_dead start 2023-11-13T11:02:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.785+0000 7f6e6affd640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e540039c0 0x7f6e54003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.786+0000 7f6e6a7fc640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e540047c0 0x7f6e54004c40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000c0e0 tx=0x7f6e540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4f216a6afa803c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60005ad0 con 0x7f6e540047c0 2023-11-13T11:02:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60005c30 con 0x7f6e540047c0 2023-11-13T11:02:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e69ffb640 1 -- 172.21.15.16:0/2386485108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6002ca50 con 0x7f6e540047c0 2023-11-13T11:02:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e6bfff640 1 -- 172.21.15.16:0/2386485108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e540047c0 msgr2=0x7f6e54004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e6bfff640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e540047c0 0x7f6e54004c40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000c0e0 tx=0x7f6e540057f0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e6bfff640 1 -- 172.21.15.16:0/2386485108 shutdown_connections 2023-11-13T11:02:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e6bfff640 1 --2- 172.21.15.16:0/2386485108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e540047c0 0x7f6e54004c40 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e6bfff640 1 -- 172.21.15.16:0/2386485108 >> 172.21.15.16:0/2386485108 conn(0x7f6e6c0b0970 msgr2=0x7f6e6c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e6bfff640 1 -- 172.21.15.16:0/2386485108 shutdown_connections 2023-11-13T11:02:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.787+0000 7f6e6bfff640 1 -- 172.21.15.16:0/2386485108 wait complete. 2023-11-13T11:02:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.789+0000 7f6e6bfff640 1 Processor -- start 2023-11-13T11:02:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.789+0000 7f6e6bfff640 1 -- start start 2023-11-13T11:02:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.790+0000 7f6e6bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e540047c0 0x7f6e6c15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.790+0000 7f6e6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6c151810 0x7f6e6c151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.790+0000 7f6e6bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c15a000 0x7f6e6c06f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.790+0000 7f6e6bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c159be0 con 0x7f6e6c151810 2023-11-13T11:02:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.790+0000 7f6e6bfff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15a4b0 con 0x7f6e540047c0 2023-11-13T11:02:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.790+0000 7f6e6bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0b3120 con 0x7f6e6c15a000 2023-11-13T11:02:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6c151810 0x7f6e6c151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6c151810 0x7f6e6c151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52778/0 (socket says 172.21.15.16:52778) 2023-11-13T11:02:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/929383734 learned_addr learned my addr 172.21.15.16:0/929383734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c15a000 msgr2=0x7f6e6c06f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6a7fc640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c15a000 0x7f6e6c06f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e540047c0 msgr2=0x7f6e6c15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6affd640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e540047c0 0x7f6e6c15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6a7fc640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e540047c0 0x7f6e6c15a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6a7fc640 1 -- 172.21.15.16:0/929383734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c0b3400 con 0x7f6e6c151810 2023-11-13T11:02:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.791+0000 7f6e6affd640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e540047c0 0x7f6e6c15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.792+0000 7f6e6a7fc640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6c151810 0x7f6e6c151c90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6e60000c00 tx=0x7f6e6002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.792+0000 7f6e537fe640 1 -- 172.21.15.16:0/929383734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6003c040 con 0x7f6e6c151810 2023-11-13T11:02:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.792+0000 7f6e537fe640 1 -- 172.21.15.16:0/929383734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6002cc40 con 0x7f6e6c151810 2023-11-13T11:02:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.792+0000 7f6e6bfff640 1 -- 172.21.15.16:0/929383734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c159520 con 0x7f6e6c151810 2023-11-13T11:02:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.792+0000 7f6e537fe640 1 -- 172.21.15.16:0/929383734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60007070 con 0x7f6e6c151810 2023-11-13T11:02:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.792+0000 7f6e6bfff640 1 -- 172.21.15.16:0/929383734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c152470 con 0x7f6e6c151810 2023-11-13T11:02:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.794+0000 7f6e537fe640 1 -- 172.21.15.16:0/929383734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e60032070 con 0x7f6e6c151810 2023-11-13T11:02:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.795+0000 7f6e537fe640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6e5806ae70 0x7f6e5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.796+0000 7f6e537fe640 1 -- 172.21.15.16:0/929383734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6e600aac00 con 0x7f6e6c151810 2023-11-13T11:02:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.796+0000 7f6e741b4740 1 Processor -- start 2023-11-13T11:02:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.796+0000 7f6e741b4740 1 -- start start 2023-11-13T11:02:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.796+0000 7f6e6affd640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6e5806ae70 0x7f6e5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca5ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7650 0x5570ca5fa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5ca750 0x5570ca5fa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e741b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ca53ce90 con 0x5570ca5ca750 2023-11-13T11:02:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e741b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ca53a580 con 0x5570ca5f7650 2023-11-13T11:02:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e741b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570ca535e70 con 0x5570ca5f7270 2023-11-13T11:02:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5ca750 0x5570ca5fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e6affd640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6e5806ae70 0x7f6e5806d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c15d950 tx=0x7f6e5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5ca750 0x5570ca5fa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52782/0 (socket says 172.21.15.16:52782) 2023-11-13T11:02:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.797+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/2340079803 learned_addr learned my addr 172.21.15.16:0/2340079803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:54.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.798+0000 7f6e6a7fc640 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7650 0x5570ca5fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.798+0000 7f6e6affd640 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca5ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.798+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/2340079803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 msgr2=0x5570ca5ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.798+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca5ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.798+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/2340079803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7650 msgr2=0x5570ca5fa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.798+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7650 0x5570ca5fa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.798+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/2340079803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570ca500020 con 0x5570ca5ca750 2023-11-13T11:02:54.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.798+0000 7f6e6affd640 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca5ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:54.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.798+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5ca750 0x5570ca5fa540 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0cc8a0 tx=0x7f6e640077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591e392fba0179e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:54.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e50ff9640 1 -- 172.21.15.16:0/2340079803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640147e0 con 0x5570ca5ca750 2023-11-13T11:02:54.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e50ff9640 1 -- 172.21.15.16:0/2340079803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6400f050 con 0x5570ca5ca750 2023-11-13T11:02:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e50ff9640 1 -- 172.21.15.16:0/2340079803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64014b40 con 0x5570ca5ca750 2023-11-13T11:02:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e741b4740 1 -- 172.21.15.16:0/2340079803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5ca750 msgr2=0x5570ca5fa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5ca750 0x5570ca5fa540 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0cc8a0 tx=0x7f6e640077c0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e741b4740 1 -- 172.21.15.16:0/2340079803 shutdown_connections 2023-11-13T11:02:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5ca750 0x5570ca5fa540 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7650 0x5570ca5fa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2340079803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca5ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e741b4740 1 -- 172.21.15.16:0/2340079803 >> 172.21.15.16:0/2340079803 conn(0x5570ca5cc480 msgr2=0x5570ca615000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.799+0000 7f6e741b4740 1 -- 172.21.15.16:0/2340079803 shutdown_connections 2023-11-13T11:02:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.800+0000 7f6e741b4740 1 -- 172.21.15.16:0/2340079803 wait complete. 2023-11-13T11:02:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.800+0000 7f6e741b4740 1 -- start start 2023-11-13T11:02:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.800+0000 7f6e741b4740 1 Processor -- start 2023-11-13T11:02:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca5e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7650 0x5570ca618510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f2700 0x5570ca618a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e741b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ca4d2480 con 0x5570ca5f2700 2023-11-13T11:02:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e741b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ca4efdc0 con 0x5570ca5f7650 2023-11-13T11:02:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e741b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570ca22ef90 con 0x5570ca5f7270 2023-11-13T11:02:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e741b4740 10 client.0 _ll_register_callbacks cb 0x5570ca247be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f2700 0x5570ca618a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f2700 0x5570ca618a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52790/0 (socket says 172.21.15.16:52790) 2023-11-13T11:02:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/2568919450 learned_addr learned my addr 172.21.15.16:0/2568919450 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/2568919450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 msgr2=0x5570ca5e3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca5e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/2568919450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7650 msgr2=0x5570ca618510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7650 0x5570ca618510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.802+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/2568919450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e64009d20 con 0x5570ca5f2700 2023-11-13T11:02:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.803+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f2700 0x5570ca618a50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6e64016000 tx=0x7f6e6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c9007f535e28f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.803+0000 7f6e49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.803+0000 7f6e49ffb640 1 -- 172.21.15.16:0/2568919450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64003b90 con 0x5570ca5f2700 2023-11-13T11:02:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.803+0000 7f6e49ffb640 1 -- 172.21.15.16:0/2568919450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6e64009420 con 0x5570ca5f2700 2023-11-13T11:02:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.803+0000 7f6e741b4740 1 -- 172.21.15.16:0/2568919450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570ca52ee50 con 0x5570ca5f2700 2023-11-13T11:02:54.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.803+0000 7f6e49ffb640 1 -- 172.21.15.16:0/2568919450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e64007eb0 con 0x5570ca5f2700 2023-11-13T11:02:54.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.803+0000 7f6e741b4740 10 client.6621 fetch_fsmap learned FSMap version 30 2023-11-13T11:02:54.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.803+0000 7f6e741b4740 1 -- 172.21.15.16:0/2568919450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x5570ca5344f0 con 0x5570ca5f2700 2023-11-13T11:02:54.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.804+0000 7f6e49ffb640 1 -- 172.21.15.16:0/2568919450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5570ca52ee50 con 0x5570ca5f2700 2023-11-13T11:02:54.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.804+0000 7f6e49ffb640 1 -- 172.21.15.16:0/2568919450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e64003cf0 con 0x5570ca5f2700 2023-11-13T11:02:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.804+0000 7f6e741b4740 10 client.6621 fetch_fsmap finished waiting for FSMap version 30 2023-11-13T11:02:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.804+0000 7f6e741b4740 10 client.6621 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.804+0000 7f6e741b4740 20 client.6621 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.804+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5570ca5e2ca0 0x5570ca5e30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.804+0000 7f6e741b4740 4 client.6621 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.804+0000 7f6e741b4740 1 -- 172.21.15.16:0/2568919450 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570ca52eb70 con 0x5570ca5e2ca0 2023-11-13T11:02:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.804+0000 7f6e6affd640 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5570ca5e2ca0 0x5570ca5e30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.805+0000 7f6e6affd640 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5570ca5e2ca0 0x5570ca5e30b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3565b7d07e03c1 server_cookie=caa0cadfda4ea516 in_seq=0 out_seq=0 2023-11-13T11:02:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.805+0000 7f6e49ffb640 10 client.6621 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.805+0000 7f6e49ffb640 1 -- 172.21.15.16:0/2568919450 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5570ca52eb70 con 0x5570ca5e2ca0 2023-11-13T11:02:54.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.805+0000 7f6e49ffb640 10 client.6621 handle_command_reply: tid=0 2023-11-13T11:02:54.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.805+0000 7f6e741b4740 1 client.6621 shutdown 2023-11-13T11:02:54.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 -- 172.21.15.16:0/2568919450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f2700 msgr2=0x5570ca618a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f2700 0x5570ca618a50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6e64016000 tx=0x7f6e6400b040 comp rx=0 tx=0).stop 2023-11-13T11:02:54.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 -- 172.21.15.16:0/2568919450 shutdown_connections 2023-11-13T11:02:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5570ca5e2ca0 0x5570ca5e30b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f2700 0x5570ca618a50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7650 0x5570ca618510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 --2- 172.21.15.16:0/2568919450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca5e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 -- 172.21.15.16:0/2568919450 >> 172.21.15.16:0/2568919450 conn(0x5570ca5ca750 msgr2=0x5570ca617660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e49ffb640 0 client.6621 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 -- 172.21.15.16:0/2568919450 shutdown_connections 2023-11-13T11:02:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 1 -- 172.21.15.16:0/2568919450 wait complete. 2023-11-13T11:02:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.806+0000 7f6e741b4740 20 client.6621 trim_cache size 0 max 16384 2023-11-13T11:02:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.822+0000 7f6e741b4740 1 Processor -- start 2023-11-13T11:02:54.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.822+0000 7f6e741b4740 1 -- start start 2023-11-13T11:02:54.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.823+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f2700 0x5570ca5fa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.823+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca616f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.823+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.823+0000 7f6e741b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ca618eb0 con 0x5570ca5f7650 2023-11-13T11:02:54.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.823+0000 7f6e741b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ca6174c0 con 0x5570ca5f2700 2023-11-13T11:02:54.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.823+0000 7f6e741b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570ca5f6030 con 0x5570ca5f7270 2023-11-13T11:02:54.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52792/0 (socket says 172.21.15.16:52792) 2023-11-13T11:02:54.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/427621781 learned_addr learned my addr 172.21.15.16:0/427621781 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6affd640 1 --2- 172.21.15.16:0/427621781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f2700 0x5570ca5fa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/427621781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 msgr2=0x5570ca616f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/427621781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca616f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/427621781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f2700 msgr2=0x5570ca5fa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/427621781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f2700 0x5570ca5fa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/427621781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570ca4d1d10 con 0x5570ca5f7650 2023-11-13T11:02:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.824+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/427621781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5cdd70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6e64014dc0 tx=0x7f6e64020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abeafa8af0244ab7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e517fa640 1 -- 172.21.15.16:0/427621781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64014270 con 0x5570ca5f7650 2023-11-13T11:02:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e517fa640 1 -- 172.21.15.16:0/427621781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e640148f0 con 0x5570ca5f7650 2023-11-13T11:02:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e517fa640 1 -- 172.21.15.16:0/427621781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64015bf0 con 0x5570ca5f7650 2023-11-13T11:02:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e741b4740 1 -- 172.21.15.16:0/427621781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 msgr2=0x5570ca5cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e741b4740 1 --2- 172.21.15.16:0/427621781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5cdd70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6e64014dc0 tx=0x7f6e64020000 comp rx=0 tx=0).stop 2023-11-13T11:02:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e741b4740 1 -- 172.21.15.16:0/427621781 shutdown_connections 2023-11-13T11:02:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e741b4740 1 --2- 172.21.15.16:0/427621781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5cdd70 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e741b4740 1 --2- 172.21.15.16:0/427621781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f7270 0x5570ca616f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e741b4740 1 --2- 172.21.15.16:0/427621781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f2700 0x5570ca5fa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e741b4740 1 -- 172.21.15.16:0/427621781 >> 172.21.15.16:0/427621781 conn(0x5570ca5ca750 msgr2=0x5570ca617660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e741b4740 1 -- 172.21.15.16:0/427621781 shutdown_connections 2023-11-13T11:02:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.825+0000 7f6e741b4740 1 -- 172.21.15.16:0/427621781 wait complete. 2023-11-13T11:02:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.826+0000 7f6e741b4740 1 -- start start 2023-11-13T11:02:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.826+0000 7f6e741b4740 1 Processor -- start 2023-11-13T11:02:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.827+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f2700 0x5570ca67fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.827+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7270 0x5570ca6162a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.827+0000 7f6e741b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.827+0000 7f6e741b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ca5cf010 con 0x5570ca5f7650 2023-11-13T11:02:54.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.827+0000 7f6e741b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ca5f9460 con 0x5570ca5f7270 2023-11-13T11:02:54.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.827+0000 7f6e741b4740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570ca5f96b0 con 0x5570ca5f2700 2023-11-13T11:02:54.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.827+0000 7f6e741b4740 10 client.0 _ll_register_callbacks cb 0x7f6e64009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:54.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52808/0 (socket says 172.21.15.16:52808) 2023-11-13T11:02:54.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/164254606 learned_addr learned my addr 172.21.15.16:0/164254606 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:54.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/164254606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f2700 msgr2=0x5570ca67fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f2700 0x5570ca67fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/164254606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7270 msgr2=0x5570ca6162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:54.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7270 0x5570ca6162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e6b7fe640 1 -- 172.21.15.16:0/164254606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570ca5344f0 con 0x5570ca5f7650 2023-11-13T11:02:54.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e6b7fe640 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5f1040 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6e64007cc0 tx=0x7f6e64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f02161907fe554 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:54.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.828+0000 7f6e49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:54.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e49ffb640 1 -- 172.21.15.16:0/164254606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64003b90 con 0x5570ca5f7650 2023-11-13T11:02:54.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e49ffb640 1 -- 172.21.15.16:0/164254606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6e64003cf0 con 0x5570ca5f7650 2023-11-13T11:02:54.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e741b4740 1 -- 172.21.15.16:0/164254606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e6400f050 con 0x5570ca5f7650 2023-11-13T11:02:54.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e49ffb640 1 -- 172.21.15.16:0/164254606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e64004040 con 0x5570ca5f7650 2023-11-13T11:02:54.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e741b4740 10 client.6627 fetch_fsmap learned FSMap version 30 2023-11-13T11:02:54.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e741b4740 1 -- 172.21.15.16:0/164254606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x5570ca4f7300 con 0x5570ca5f7650 2023-11-13T11:02:54.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e49ffb640 1 -- 172.21.15.16:0/164254606 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6400f050 con 0x5570ca5f7650 2023-11-13T11:02:54.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e49ffb640 1 -- 172.21.15.16:0/164254606 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e640041a0 con 0x5570ca5f7650 2023-11-13T11:02:54.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e741b4740 10 client.6627 fetch_fsmap finished waiting for FSMap version 30 2023-11-13T11:02:54.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e741b4740 10 client.6627 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:54.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e741b4740 20 client.6627 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:54.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e741b4740 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5570ca5fa5b0 0x5570ca5f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:54.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e741b4740 4 client.6627 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-13T11:02:54.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.829+0000 7f6e741b4740 1 -- 172.21.15.16:0/164254606 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5570ca2367e0 con 0x5570ca5fa5b0 2023-11-13T11:02:54.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.830+0000 7f6e6affd640 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5570ca5fa5b0 0x5570ca5f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:54.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.830+0000 7f6e6affd640 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5570ca5fa5b0 0x5570ca5f88a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768da09fbe24101a server_cookie=dc59254d4fdee36a in_seq=0 out_seq=0 2023-11-13T11:02:54.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.830+0000 7f6e49ffb640 10 client.6627 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:54.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.830+0000 7f6e49ffb640 1 -- 172.21.15.16:0/164254606 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5570ca2367e0 con 0x5570ca5fa5b0 2023-11-13T11:02:54.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.830+0000 7f6e49ffb640 10 client.6627 handle_command_reply: tid=0 2023-11-13T11:02:54.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.830+0000 7f6e741b4740 1 client.6627 shutdown 2023-11-13T11:02:54.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 -- 172.21.15.16:0/164254606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 msgr2=0x5570ca5f1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5f1040 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6e64007cc0 tx=0x7f6e64009750 comp rx=0 tx=0).stop 2023-11-13T11:02:54.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 -- 172.21.15.16:0/164254606 shutdown_connections 2023-11-13T11:02:54.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x5570ca5fa5b0 0x5570ca5f88a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570ca5f7650 0x5570ca5f1040 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5570ca5f7270 0x5570ca6162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 --2- 172.21.15.16:0/164254606 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5570ca5f2700 0x5570ca67fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 -- 172.21.15.16:0/164254606 >> 172.21.15.16:0/164254606 conn(0x5570ca5ca750 msgr2=0x5570ca5cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:54.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e49ffb640 0 client.6627 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:54.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 -- 172.21.15.16:0/164254606 shutdown_connections 2023-11-13T11:02:54.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 1 -- 172.21.15.16:0/164254606 wait complete. 2023-11-13T11:02:54.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.831+0000 7f6e741b4740 20 client.6627 trim_cache size 0 max 16384 2023-11-13T11:02:54.855 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2023-11-13T11:02:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2023-11-13T11:02:54.856 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:54.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 -- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6e5806ae70 msgr2=0x7f6e5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6e5806ae70 0x7f6e5806d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c15d950 tx=0x7f6e5c009040 comp rx=0 tx=0).stop 2023-11-13T11:02:54.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 -- 172.21.15.16:0/929383734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6c151810 msgr2=0x7f6e6c151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:54.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6c151810 0x7f6e6c151c90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6e60000c00 tx=0x7f6e6002e070 comp rx=0 tx=0).stop 2023-11-13T11:02:54.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 -- 172.21.15.16:0/929383734 shutdown_connections 2023-11-13T11:02:54.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6e5806ae70 0x7f6e5806d330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e6c15a000 0x7f6e6c06f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6c151810 0x7f6e6c151c90 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 --2- 172.21.15.16:0/929383734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e540047c0 0x7f6e6c15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:54.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 -- 172.21.15.16:0/929383734 >> 172.21.15.16:0/929383734 conn(0x7f6e6c0b0970 msgr2=0x7f6e6c1562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:54.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 -- 172.21.15.16:0/929383734 shutdown_connections 2023-11-13T11:02:54.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:54.832+0000 7f6e6bfff640 1 -- 172.21.15.16:0/929383734 wait complete. 2023-11-13T11:02:54.860 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-13T11:02:54.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2023-11-13T11:02:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.081+0000 7ff28c976640 1 Processor -- start 2023-11-13T11:02:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.082+0000 7ff28c976640 1 -- start start 2023-11-13T11:02:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.083+0000 7ff28c976640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880cf350 0x7ff2880ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.083+0000 7ff28c976640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28815b360 con 0x7ff28815af80 2023-11-13T11:02:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.083+0000 7ff28c976640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2880b3250 con 0x7ff2880cc770 2023-11-13T11:02:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.083+0000 7ff28c976640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880b3530 con 0x7ff2880cf350 2023-11-13T11:02:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.084+0000 7ff28659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff28815af80 0x7ff2880cec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41350/0 (socket says 172.21.15.16:41350) 2023-11-13T11:02:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.084+0000 7ff28659c640 1 -- 172.21.15.16:0/3549795049 learned_addr learned my addr 172.21.15.16:0/3549795049 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.084+0000 7ff27ffff640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880cf350 0x7ff2880ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.084+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582831574 0 0) 0x7ff28815b360 con 0x7ff28815af80 2023-11-13T11:02:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.084+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff274003520 con 0x7ff28815af80 2023-11-13T11:02:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2256403988 0 0) 0x7ff274003520 con 0x7ff28815af80 2023-11-13T11:02:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880cf350 msgr2=0x7ff2880ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880cf350 0x7ff2880ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 >> v1:172.21.15.184:6789/0 conn(0x7ff2880cc770 legacy=0x7ff28806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880b0110 con 0x7ff28815af80 2023-11-13T11:02:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7ff278001670 con 0x7ff28815af80 2023-11-13T11:02:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 >> v1:172.21.15.16:6789/0 conn(0x7ff28815af80 legacy=0x7ff2880cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2740039a0 0x7ff274003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740047c0 0x7ff274004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880cf350 0x7ff2740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880b3530 con 0x7ff2740039a0 2023-11-13T11:02:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27400bab0 con 0x7ff2740047c0 2023-11-13T11:02:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27400bc70 con 0x7ff2880cf350 2023-11-13T11:02:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.085+0000 7ff28659c640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2740039a0 0x7ff274003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.086+0000 7ff27ffff640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740047c0 0x7ff274004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.086+0000 7ff28659c640 1 -- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880cf350 msgr2=0x7ff2740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.086+0000 7ff286d9d640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880cf350 0x7ff2740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.086+0000 7ff28659c640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880cf350 0x7ff2740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.086+0000 7ff28659c640 1 -- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740047c0 msgr2=0x7ff274004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.086+0000 7ff28659c640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740047c0 0x7ff274004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.086+0000 7ff28659c640 1 -- 172.21.15.16:0/3549795049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274003520 con 0x7ff2740039a0 2023-11-13T11:02:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.086+0000 7ff286d9d640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880cf350 0x7ff2740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.087+0000 7ff28659c640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2740039a0 0x7ff274003dc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c240 tx=0x7ff2740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d00fc52a3693176 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.087+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278005ad0 con 0x7ff2740039a0 2023-11-13T11:02:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.087+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff278005c30 con 0x7ff2740039a0 2023-11-13T11:02:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.087+0000 7ff285d9b640 1 -- 172.21.15.16:0/3549795049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27802c8b0 con 0x7ff2740039a0 2023-11-13T11:02:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.088+0000 7ff28c976640 1 -- 172.21.15.16:0/3549795049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2740039a0 msgr2=0x7ff274003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.088+0000 7ff28c976640 1 --2- 172.21.15.16:0/3549795049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2740039a0 0x7ff274003dc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c240 tx=0x7ff2740057f0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.088+0000 7ff286d9d640 1 -- 172.21.15.16:0/3549795049 reap_dead start 2023-11-13T11:02:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.088+0000 7ff28c976640 1 -- 172.21.15.16:0/3549795049 shutdown_connections 2023-11-13T11:02:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.088+0000 7ff28c976640 1 -- 172.21.15.16:0/3549795049 >> 172.21.15.16:0/3549795049 conn(0x7ff2880b2600 msgr2=0x7ff2880b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.088+0000 7ff28c976640 1 -- 172.21.15.16:0/3549795049 shutdown_connections 2023-11-13T11:02:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.088+0000 7ff28c976640 1 -- 172.21.15.16:0/3549795049 wait complete. 2023-11-13T11:02:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.090+0000 7ff28c976640 1 Processor -- start 2023-11-13T11:02:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.090+0000 7ff28c976640 1 -- start start 2023-11-13T11:02:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.091+0000 7ff28c976640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740039a0 0x7ff2880b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.091+0000 7ff28c976640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880c70c0 0x7ff2880c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.091+0000 7ff28c976640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880c52b0 0x7ff2880c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.091+0000 7ff28c976640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880b5f30 con 0x7ff2880c52b0 2023-11-13T11:02:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.091+0000 7ff28c976640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880c3f50 con 0x7ff2740039a0 2023-11-13T11:02:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.091+0000 7ff28c976640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880c41a0 con 0x7ff2880c70c0 2023-11-13T11:02:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.091+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880c52b0 0x7ff2880c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.091+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880c52b0 0x7ff2880c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52826/0 (socket says 172.21.15.16:52826) 2023-11-13T11:02:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.091+0000 7ff286d9d640 1 -- 172.21.15.16:0/3037229546 learned_addr learned my addr 172.21.15.16:0/3037229546 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.092+0000 7ff28659c640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740039a0 0x7ff2880b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.092+0000 7ff27ffff640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880c70c0 0x7ff2880c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.092+0000 7ff286d9d640 1 -- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880c70c0 msgr2=0x7ff2880c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.092+0000 7ff286d9d640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880c70c0 0x7ff2880c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.092+0000 7ff286d9d640 1 -- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740039a0 msgr2=0x7ff2880b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.092+0000 7ff286d9d640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740039a0 0x7ff2880b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.092+0000 7ff286d9d640 1 -- 172.21.15.16:0/3037229546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880b4440 con 0x7ff2880c52b0 2023-11-13T11:02:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.092+0000 7ff286d9d640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880c52b0 0x7ff2880c5730 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff28815e980 tx=0x7ff28000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.092+0000 7ff27effd640 1 -- 172.21.15.16:0/3037229546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280006d50 con 0x7ff2880c52b0 2023-11-13T11:02:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.093+0000 7ff27effd640 1 -- 172.21.15.16:0/3037229546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff280003970 con 0x7ff2880c52b0 2023-11-13T11:02:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.093+0000 7ff28c976640 1 -- 172.21.15.16:0/3037229546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2880b4720 con 0x7ff2880c52b0 2023-11-13T11:02:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.093+0000 7ff27effd640 1 -- 172.21.15.16:0/3037229546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280003d20 con 0x7ff2880c52b0 2023-11-13T11:02:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.093+0000 7ff28c976640 1 -- 172.21.15.16:0/3037229546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2880c6c40 con 0x7ff2880c52b0 2023-11-13T11:02:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.095+0000 7ff27effd640 1 -- 172.21.15.16:0/3037229546 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff280005070 con 0x7ff2880c52b0 2023-11-13T11:02:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.096+0000 7ff27effd640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.096+0000 7ff27effd640 1 -- 172.21.15.16:0/3037229546 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7ff280089590 con 0x7ff2880c52b0 2023-11-13T11:02:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.097+0000 7ff28659c640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.097+0000 7ff28f5e1740 1 Processor -- start 2023-11-13T11:02:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.097+0000 7ff28f5e1740 1 -- start start 2023-11-13T11:02:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61940ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61910720 0x560e6193d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6187fd20 con 0x560e61914280 2023-11-13T11:02:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6187d410 con 0x560e61910720 2023-11-13T11:02:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e61878d00 con 0x560e61940700 2023-11-13T11:02:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff28659c640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff278034000 tx=0x7ff278009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52834/0 (socket says 172.21.15.16:52834) 2023-11-13T11:02:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff286d9d640 1 -- 172.21.15.16:0/152340012 learned_addr learned my addr 172.21.15.16:0/152340012 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.098+0000 7ff27ffff640 1 --2- 172.21.15.16:0/152340012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61910720 0x560e6193d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff28659c640 1 --2- 172.21.15.16:0/152340012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61940ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff286d9d640 1 -- 172.21.15.16:0/152340012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 msgr2=0x560e61940ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff286d9d640 1 --2- 172.21.15.16:0/152340012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61940ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff286d9d640 1 -- 172.21.15.16:0/152340012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61910720 msgr2=0x560e6193d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff286d9d640 1 --2- 172.21.15.16:0/152340012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61910720 0x560e6193d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff286d9d640 1 -- 172.21.15.16:0/152340012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e618324f0 con 0x560e61914280 2023-11-13T11:02:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff286d9d640 1 --2- 172.21.15.16:0/152340012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193deb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff280001550 tx=0x7ff280057e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d7a1bb521acd81 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff267fff640 1 -- 172.21.15.16:0/152340012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28005c160 con 0x560e61914280 2023-11-13T11:02:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff267fff640 1 -- 172.21.15.16:0/152340012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff280058210 con 0x560e61914280 2023-11-13T11:02:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.099+0000 7ff267fff640 1 -- 172.21.15.16:0/152340012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28005c360 con 0x560e61914280 2023-11-13T11:02:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.100+0000 7ff28f5e1740 1 -- 172.21.15.16:0/152340012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 msgr2=0x560e6193deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.100+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/152340012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193deb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff280001550 tx=0x7ff280057e80 comp rx=0 tx=0).stop 2023-11-13T11:02:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.100+0000 7ff28f5e1740 1 -- 172.21.15.16:0/152340012 shutdown_connections 2023-11-13T11:02:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.100+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/152340012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193deb0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.100+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/152340012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61910720 0x560e6193d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.100+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/152340012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61940ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.100+0000 7ff28f5e1740 1 -- 172.21.15.16:0/152340012 >> 172.21.15.16:0/152340012 conn(0x560e61912990 msgr2=0x560e6195b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.100+0000 7ff28f5e1740 1 -- 172.21.15.16:0/152340012 shutdown_connections 2023-11-13T11:02:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.100+0000 7ff28f5e1740 1 -- 172.21.15.16:0/152340012 wait complete. 2023-11-13T11:02:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.101+0000 7ff28f5e1740 1 -- start start 2023-11-13T11:02:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.101+0000 7ff28f5e1740 1 Processor -- start 2023-11-13T11:02:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.102+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61914280 0x560e6196ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.102+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61941270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.102+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61943bb0 0x560e61915db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.102+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e61832fa0 con 0x560e61943bb0 2023-11-13T11:02:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.102+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e61832c40 con 0x560e61914280 2023-11-13T11:02:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.102+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e61571f90 con 0x560e61940700 2023-11-13T11:02:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.102+0000 7ff28f5e1740 10 client.0 _ll_register_callbacks cb 0x560e61837370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61943bb0 0x560e61915db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61943bb0 0x560e61915db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52850/0 (socket says 172.21.15.16:52850) 2023-11-13T11:02:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff286d9d640 1 -- 172.21.15.16:0/107389930 learned_addr learned my addr 172.21.15.16:0/107389930 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff28659c640 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61914280 0x560e6196ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff286d9d640 1 -- 172.21.15.16:0/107389930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 msgr2=0x560e61941270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff286d9d640 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61941270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff286d9d640 1 -- 172.21.15.16:0/107389930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61914280 msgr2=0x560e6196ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff286d9d640 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61914280 0x560e6196ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff286d9d640 1 -- 172.21.15.16:0/107389930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2880c6c40 con 0x560e61943bb0 2023-11-13T11:02:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff286d9d640 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61943bb0 0x560e61915db0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff280000f80 tx=0x7ff2800014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565eea78896ccf86 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff2657fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.103+0000 7ff2657fa640 1 -- 172.21.15.16:0/107389930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280057900 con 0x560e61943bb0 2023-11-13T11:02:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff2657fa640 1 -- 172.21.15.16:0/107389930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7ff280060510 con 0x560e61943bb0 2023-11-13T11:02:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff28f5e1740 1 -- 172.21.15.16:0/107389930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e61871ce0 con 0x560e61943bb0 2023-11-13T11:02:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff2657fa640 1 -- 172.21.15.16:0/107389930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff28005c360 con 0x560e61943bb0 2023-11-13T11:02:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff28f5e1740 10 client.6639 fetch_fsmap learned FSMap version 30 2023-11-13T11:02:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff28f5e1740 1 -- 172.21.15.16:0/107389930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x560e61877380 con 0x560e61943bb0 2023-11-13T11:02:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff2657fa640 1 -- 172.21.15.16:0/107389930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560e61871ce0 con 0x560e61943bb0 2023-11-13T11:02:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff2657fa640 1 -- 172.21.15.16:0/107389930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff280060080 con 0x560e61943bb0 2023-11-13T11:02:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff28f5e1740 10 client.6639 fetch_fsmap finished waiting for FSMap version 30 2023-11-13T11:02:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff28f5e1740 10 client.6639 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff28f5e1740 20 client.6639 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x560e6195be60 0x560e61958900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff28f5e1740 4 client.6639 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:55.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.104+0000 7ff28f5e1740 1 -- 172.21.15.16:0/107389930 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e61871a00 con 0x560e6195be60 2023-11-13T11:02:55.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.105+0000 7ff28659c640 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x560e6195be60 0x560e61958900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.105+0000 7ff28659c640 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x560e6195be60 0x560e61958900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d7d58c6eac5f2f server_cookie=b11c9de8d716a337 in_seq=0 out_seq=0 2023-11-13T11:02:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.105+0000 7ff2657fa640 10 client.6639 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff2657fa640 1 -- 172.21.15.16:0/107389930 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e61871a00 con 0x560e6195be60 2023-11-13T11:02:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff2657fa640 10 client.6639 handle_command_reply: tid=0 2023-11-13T11:02:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff28f5e1740 1 client.6639 shutdown 2023-11-13T11:02:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff28f5e1740 1 -- 172.21.15.16:0/107389930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61943bb0 msgr2=0x560e61915db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61943bb0 0x560e61915db0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff280000f80 tx=0x7ff2800014a0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff28f5e1740 1 -- 172.21.15.16:0/107389930 shutdown_connections 2023-11-13T11:02:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x560e6195be60 0x560e61958900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61943bb0 0x560e61915db0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61941270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/107389930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61914280 0x560e6196ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff2657fa640 0 client.6639 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.106+0000 7ff28f5e1740 1 -- 172.21.15.16:0/107389930 >> 172.21.15.16:0/107389930 conn(0x560e61910720 msgr2=0x560e619443a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.107+0000 7ff28f5e1740 1 -- 172.21.15.16:0/107389930 shutdown_connections 2023-11-13T11:02:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.107+0000 7ff28f5e1740 1 -- 172.21.15.16:0/107389930 wait complete. 2023-11-13T11:02:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.107+0000 7ff28f5e1740 20 client.6639 trim_cache size 0 max 16384 2023-11-13T11:02:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.120+0000 7ff28f5e1740 1 Processor -- start 2023-11-13T11:02:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.120+0000 7ff28f5e1740 1 -- start start 2023-11-13T11:02:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.121+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e619594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.121+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61944320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.121+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61943bb0 0x560e6195ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.121+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e619176d0 con 0x560e61914280 2023-11-13T11:02:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.121+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e61917370 con 0x560e61943bb0 2023-11-13T11:02:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.121+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e61916f20 con 0x560e61940700 2023-11-13T11:02:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.121+0000 7ff28659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e619594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.121+0000 7ff28659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e619594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52862/0 (socket says 172.21.15.16:52862) 2023-11-13T11:02:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.121+0000 7ff28659c640 1 -- 172.21.15.16:0/219255600 learned_addr learned my addr 172.21.15.16:0/219255600 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.122+0000 7ff28659c640 1 -- 172.21.15.16:0/219255600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 msgr2=0x560e61944320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.122+0000 7ff28659c640 1 --2- 172.21.15.16:0/219255600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61944320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.122+0000 7ff28659c640 1 -- 172.21.15.16:0/219255600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61943bb0 msgr2=0x560e6195ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.122+0000 7ff28659c640 1 --2- 172.21.15.16:0/219255600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61943bb0 0x560e6195ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.122+0000 7ff28659c640 1 -- 172.21.15.16:0/219255600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e61814cf0 con 0x560e61914280 2023-11-13T11:02:55.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.122+0000 7ff28659c640 1 --2- 172.21.15.16:0/219255600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e619594b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff278007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2490fb78c34bc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.122+0000 7ff27cff9640 1 -- 172.21.15.16:0/219255600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27803c040 con 0x560e61914280 2023-11-13T11:02:55.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.122+0000 7ff27cff9640 1 -- 172.21.15.16:0/219255600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff278049070 con 0x560e61914280 2023-11-13T11:02:55.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.122+0000 7ff27cff9640 1 -- 172.21.15.16:0/219255600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278007530 con 0x560e61914280 2023-11-13T11:02:55.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.123+0000 7ff28f5e1740 1 -- 172.21.15.16:0/219255600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 msgr2=0x560e619594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.123+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/219255600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e619594b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff278007000 comp rx=0 tx=0).stop 2023-11-13T11:02:55.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.123+0000 7ff28f5e1740 1 -- 172.21.15.16:0/219255600 shutdown_connections 2023-11-13T11:02:55.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.123+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/219255600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61943bb0 0x560e6195ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.123+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/219255600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61940700 0x560e61944320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.123+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/219255600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e619594b0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.123+0000 7ff28f5e1740 1 -- 172.21.15.16:0/219255600 >> 172.21.15.16:0/219255600 conn(0x560e61910720 msgr2=0x560e6196eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.123+0000 7ff28f5e1740 1 -- 172.21.15.16:0/219255600 shutdown_connections 2023-11-13T11:02:55.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.123+0000 7ff28f5e1740 1 -- 172.21.15.16:0/219255600 wait complete. 2023-11-13T11:02:55.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.124+0000 7ff28f5e1740 1 -- start start 2023-11-13T11:02:55.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.124+0000 7ff28f5e1740 1 Processor -- start 2023-11-13T11:02:55.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61940700 0x560e619c8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28f5e1740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61943bb0 0x560e61940d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e61942db0 con 0x560e61914280 2023-11-13T11:02:55.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6195eb50 con 0x560e61940700 2023-11-13T11:02:55.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28f5e1740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e61942ad0 con 0x560e61943bb0 2023-11-13T11:02:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28f5e1740 10 client.0 _ll_register_callbacks cb 0x7ff28006e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52878/0 (socket says 172.21.15.16:52878) 2023-11-13T11:02:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.125+0000 7ff28659c640 1 -- 172.21.15.16:0/3227751074 learned_addr learned my addr 172.21.15.16:0/3227751074 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff27ffff640 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61940700 0x560e619c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff286d9d640 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61943bb0 0x560e61940d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff28659c640 1 -- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61943bb0 msgr2=0x560e61940d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff28659c640 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61943bb0 0x560e61940d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff28659c640 1 -- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61940700 msgr2=0x560e619c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff28659c640 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61940700 0x560e619c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff28659c640 1 -- 172.21.15.16:0/3227751074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff274003520 con 0x560e61914280 2023-11-13T11:02:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff27ffff640 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61940700 0x560e619c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff286d9d640 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61943bb0 0x560e61940d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff28659c640 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193da10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c210 tx=0x7ff27802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3836c94cc8d95438 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff2657fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.126+0000 7ff2657fa640 1 -- 172.21.15.16:0/3227751074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278007680 con 0x560e61914280 2023-11-13T11:02:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff2657fa640 1 -- 172.21.15.16:0/3227751074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7ff278050030 con 0x560e61914280 2023-11-13T11:02:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff28f5e1740 1 -- 172.21.15.16:0/3227751074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff280058210 con 0x560e61914280 2023-11-13T11:02:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff2657fa640 1 -- 172.21.15.16:0/3227751074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2780501f0 con 0x560e61914280 2023-11-13T11:02:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff28f5e1740 10 client.6645 fetch_fsmap learned FSMap version 30 2023-11-13T11:02:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff28f5e1740 1 -- 172.21.15.16:0/3227751074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x560e6183a2e0 con 0x560e61914280 2023-11-13T11:02:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff2657fa640 1 -- 172.21.15.16:0/3227751074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff280058210 con 0x560e61914280 2023-11-13T11:02:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff2657fa640 1 -- 172.21.15.16:0/3227751074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff278050350 con 0x560e61914280 2023-11-13T11:02:55.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff28f5e1740 10 client.6645 fetch_fsmap finished waiting for FSMap version 30 2023-11-13T11:02:55.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff28f5e1740 10 client.6645 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-13T11:02:55.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff28f5e1740 20 client.6645 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:55.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x560e61941250 0x560e6193b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff28f5e1740 4 client.6645 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-11-13T11:02:55.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.127+0000 7ff28f5e1740 1 -- 172.21.15.16:0/3227751074 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e615797e0 con 0x560e61941250 2023-11-13T11:02:55.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.128+0000 7ff27ffff640 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x560e61941250 0x560e6193b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.128+0000 7ff27ffff640 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x560e61941250 0x560e6193b900 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4a3659a7473678 server_cookie=a59bb6670d5c25d6 in_seq=0 out_seq=0 2023-11-13T11:02:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.128+0000 7ff2657fa640 10 client.6645 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.128+0000 7ff2657fa640 1 -- 172.21.15.16:0/3227751074 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x560e615797e0 con 0x560e61941250 2023-11-13T11:02:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.128+0000 7ff2657fa640 10 client.6645 handle_command_reply: tid=0 2023-11-13T11:02:55.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.128+0000 7ff28f5e1740 1 client.6645 shutdown 2023-11-13T11:02:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 -- 172.21.15.16:0/3227751074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 msgr2=0x560e6193da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193da10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c210 tx=0x7ff27802e070 comp rx=0 tx=0).stop 2023-11-13T11:02:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 -- 172.21.15.16:0/3227751074 shutdown_connections 2023-11-13T11:02:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560e61943bb0 0x560e61940d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e61940700 0x560e619c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x560e61941250 0x560e6193b900 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 --2- 172.21.15.16:0/3227751074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e61914280 0x560e6193da10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 -- 172.21.15.16:0/3227751074 >> 172.21.15.16:0/3227751074 conn(0x560e61910720 msgr2=0x560e6197ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff2657fa640 0 client.6645 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 -- 172.21.15.16:0/3227751074 shutdown_connections 2023-11-13T11:02:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 1 -- 172.21.15.16:0/3227751074 wait complete. 2023-11-13T11:02:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.129+0000 7ff28f5e1740 20 client.6645 trim_cache size 0 max 16384 2023-11-13T11:02:55.162 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-13T11:02:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:02:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2023-11-13T11:02:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1488910146, 2023-11-13T11:02:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511630412, 2023-11-13T11:02:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2023-11-13T11:02:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:02:55.167 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-13T11:02:55.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 -- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 msgr2=0x7ff26806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff278034000 tx=0x7ff278009030 comp rx=0 tx=0).stop 2023-11-13T11:02:55.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 -- 172.21.15.16:0/3037229546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880c52b0 msgr2=0x7ff2880c5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880c52b0 0x7ff2880c5730 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff28815e980 tx=0x7ff28000dc00 comp rx=0 tx=0).stop 2023-11-13T11:02:55.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 -- 172.21.15.16:0/3037229546 shutdown_connections 2023-11-13T11:02:55.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880c52b0 0x7ff2880c5730 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2880c70c0 0x7ff2880c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 --2- 172.21.15.16:0/3037229546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2740039a0 0x7ff2880b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 -- 172.21.15.16:0/3037229546 >> 172.21.15.16:0/3037229546 conn(0x7ff2880b2600 msgr2=0x7ff2880ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 -- 172.21.15.16:0/3037229546 shutdown_connections 2023-11-13T11:02:55.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.130+0000 7ff28c976640 1 -- 172.21.15.16:0/3037229546 wait complete. 2023-11-13T11:02:55.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:55.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.397+0000 7f9fc8f97640 1 Processor -- start 2023-11-13T11:02:55.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.397+0000 7f9fc8f97640 1 -- start start 2023-11-13T11:02:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.399+0000 7f9fc8f97640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc40d18b0 0x7f9fc40d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.399+0000 7f9fc8f97640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fc40b1d00 0x7f9fc40b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.399+0000 7f9fc8f97640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40d0aa0 con 0x7f9fc415d170 2023-11-13T11:02:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.399+0000 7f9fc8f97640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc40b0080 con 0x7f9fc40b1d00 2023-11-13T11:02:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.399+0000 7f9fc8f97640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40d0fd0 con 0x7f9fc40d18b0 2023-11-13T11:02:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.399+0000 7f9fc2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fc40b1d00 0x7f9fc40b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.399+0000 7f9fc2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fc40b1d00 0x7f9fc40b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52888/0 (socket says 172.21.15.16:52888) 2023-11-13T11:02:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.399+0000 7f9fc2d9d640 1 -- 172.21.15.16:0/2273037489 learned_addr learned my addr 172.21.15.16:0/2273037489 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.399+0000 7f9fc1d9b640 1 --2- 172.21.15.16:0/2273037489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc40d18b0 0x7f9fc40d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.400+0000 7f9fc2d9d640 1 -- 172.21.15.16:0/2273037489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc40d18b0 msgr2=0x7f9fc40d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.400+0000 7f9fc2d9d640 1 --2- 172.21.15.16:0/2273037489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc40d18b0 0x7f9fc40d37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.400+0000 7f9fc2d9d640 1 -- 172.21.15.16:0/2273037489 >> v1:172.21.15.184:6789/0 conn(0x7f9fc415d170 legacy=0x7f9fc40d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.400+0000 7f9fc2d9d640 1 -- 172.21.15.16:0/2273037489 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40d0cf0 con 0x7f9fc40b1d00 2023-11-13T11:02:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.401+0000 7f9fc2d9d640 1 --2- 172.21.15.16:0/2273037489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40b1d00 0x7f9fc40b04a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009e50 tx=0x7f9fbc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefc9a42d7f7a10d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.401+0000 7f9fc159a640 1 -- 172.21.15.16:0/2273037489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc02ca80 con 0x7f9fc40b1d00 2023-11-13T11:02:55.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.401+0000 7f9fc159a640 1 -- 172.21.15.16:0/2273037489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fbc02cbe0 con 0x7f9fc40b1d00 2023-11-13T11:02:55.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.401+0000 7f9fc159a640 1 -- 172.21.15.16:0/2273037489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0343f0 con 0x7f9fc40b1d00 2023-11-13T11:02:55.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.402+0000 7f9fc8f97640 1 -- 172.21.15.16:0/2273037489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40b1d00 msgr2=0x7f9fc40b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.402+0000 7f9fc8f97640 1 --2- 172.21.15.16:0/2273037489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40b1d00 0x7f9fc40b04a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009e50 tx=0x7f9fbc02bee0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.402+0000 7f9fc8f97640 1 -- 172.21.15.16:0/2273037489 shutdown_connections 2023-11-13T11:02:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.402+0000 7f9fc8f97640 1 --2- 172.21.15.16:0/2273037489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40b1d00 0x7f9fc40b04a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.402+0000 7f9fc8f97640 1 --2- 172.21.15.16:0/2273037489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc40d18b0 0x7f9fc40d37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.402+0000 7f9fc8f97640 1 -- 172.21.15.16:0/2273037489 >> 172.21.15.16:0/2273037489 conn(0x7f9fc40d2cc0 msgr2=0x7f9fc40d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.402+0000 7f9fc8f97640 1 -- 172.21.15.16:0/2273037489 shutdown_connections 2023-11-13T11:02:55.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.402+0000 7f9fc8f97640 1 -- 172.21.15.16:0/2273037489 wait complete. 2023-11-13T11:02:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.404+0000 7f9fc8f97640 1 Processor -- start 2023-11-13T11:02:55.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.404+0000 7f9fc8f97640 1 -- start start 2023-11-13T11:02:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.405+0000 7f9fc8f97640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40b1d00 0x7f9fc40b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.405+0000 7f9fc8f97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40d18b0 0x7f9fc4159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.405+0000 7f9fc8f97640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d170 0x7f9fc40b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.405+0000 7f9fc8f97640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40b80d0 con 0x7f9fc40d18b0 2023-11-13T11:02:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.405+0000 7f9fc8f97640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40b3a40 con 0x7f9fc40b1d00 2023-11-13T11:02:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.405+0000 7f9fc8f97640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40b2280 con 0x7f9fc415d170 2023-11-13T11:02:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.405+0000 7f9fc1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40d18b0 0x7f9fc4159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40d18b0 0x7f9fc4159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52904/0 (socket says 172.21.15.16:52904) 2023-11-13T11:02:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc1d9b640 1 -- 172.21.15.16:0/1559123922 learned_addr learned my addr 172.21.15.16:0/1559123922 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc259c640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40b1d00 0x7f9fc40b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc2d9d640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d170 0x7f9fc40b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc1d9b640 1 -- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d170 msgr2=0x7f9fc40b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc1d9b640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d170 0x7f9fc40b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc1d9b640 1 -- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40b1d00 msgr2=0x7f9fc40b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc1d9b640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40b1d00 0x7f9fc40b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc1d9b640 1 -- 172.21.15.16:0/1559123922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc009b30 con 0x7f9fc40d18b0 2023-11-13T11:02:55.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.406+0000 7f9fc1d9b640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40d18b0 0x7f9fc4159530 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00d800 tx=0x7f9fac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.407+0000 7f9fbaffd640 1 -- 172.21.15.16:0/1559123922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac00b6c0 con 0x7f9fc40d18b0 2023-11-13T11:02:55.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.407+0000 7f9fbaffd640 1 -- 172.21.15.16:0/1559123922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac00b820 con 0x7f9fc40d18b0 2023-11-13T11:02:55.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.407+0000 7f9fbaffd640 1 -- 172.21.15.16:0/1559123922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac00bbb0 con 0x7f9fc40d18b0 2023-11-13T11:02:55.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.407+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc40b2560 con 0x7f9fc40d18b0 2023-11-13T11:02:55.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.407+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc40b2b20 con 0x7f9fc40d18b0 2023-11-13T11:02:55.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.408+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc40677a0 con 0x7f9fc40d18b0 2023-11-13T11:02:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.409+0000 7f9fbaffd640 1 -- 172.21.15.16:0/1559123922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fac00bd10 con 0x7f9fc40d18b0 2023-11-13T11:02:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.410+0000 7f9fbaffd640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fa406ae40 0x7f9fa406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.410+0000 7f9fbaffd640 1 -- 172.21.15.16:0/1559123922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f9fac089d10 con 0x7f9fc40d18b0 2023-11-13T11:02:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.411+0000 7f9fc259c640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fa406ae40 0x7f9fa406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.412+0000 7f9fc259c640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fa406ae40 0x7f9fa406d300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.415+0000 7f9fbaffd640 1 -- 172.21.15.16:0/1559123922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fac054c40 con 0x7f9fc40d18b0 2023-11-13T11:02:55.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.649+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fc40b13f0 con 0x7f9fc40d18b0 2023-11-13T11:02:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.650+0000 7f9fbaffd640 1 -- 172.21.15.16:0/1559123922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f9fac058260 con 0x7f9fc40d18b0 2023-11-13T11:02:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:55.656 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:51.012959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":21,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:55.656 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 30 2023-11-13T11:02:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fa406ae40 msgr2=0x7f9fa406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fa406ae40 0x7f9fa406d300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4008040 comp rx=0 tx=0).stop 2023-11-13T11:02:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40d18b0 msgr2=0x7f9fc4159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40d18b0 0x7f9fc4159530 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00d800 tx=0x7f9fac00dcc0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 shutdown_connections 2023-11-13T11:02:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9fa406ae40 0x7f9fa406d300 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fc415d170 0x7f9fc40b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc40d18b0 0x7f9fc4159530 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 --2- 172.21.15.16:0/1559123922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fc40b1d00 0x7f9fc40b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 >> 172.21.15.16:0/1559123922 conn(0x7f9fc40d2cc0 msgr2=0x7f9fc415b160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 shutdown_connections 2023-11-13T11:02:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.653+0000 7f9fc8f97640 1 -- 172.21.15.16:0/1559123922 wait complete. 2023-11-13T11:02:55.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 1488910146 2023-11-13T11:02:55.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.904+0000 7fc2be5e9640 1 Processor -- start 2023-11-13T11:02:55.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.905+0000 7fc2be5e9640 1 -- start start 2023-11-13T11:02:55.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.906+0000 7fc2be5e9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2b80cf090 0x7fc2b81622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.906+0000 7fc2be5e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b8164dd0 0x7fc2b8165210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.906+0000 7fc2be5e9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc2b80b19d0 0x7fc2b80b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.906+0000 7fc2be5e9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b80b23a0 con 0x7fc2b80cf090 2023-11-13T11:02:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.906+0000 7fc2be5e9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b80cf470 con 0x7fc2b80b19d0 2023-11-13T11:02:55.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.906+0000 7fc2be5e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8144660 con 0x7fc2b8164dd0 2023-11-13T11:02:55.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.906+0000 7fc2b7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2b80cf090 0x7fc2b81622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2b7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2b80cf090 0x7fc2b81622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52914/0 (socket says 172.21.15.16:52914) 2023-11-13T11:02:55.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2b7fff640 1 -- 172.21.15.16:0/3513947444 learned_addr learned my addr 172.21.15.16:0/3513947444 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/3513947444 >> v2:172.21.15.184:3300/0 conn(0x7fc2b80b19d0 0x7fc2b80b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2b7fff640 1 -- 172.21.15.16:0/3513947444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b8164dd0 msgr2=0x7fc2b8165210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/3513947444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b8164dd0 0x7fc2b8165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/3513947444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b8164dd0 0x7fc2b8165210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2b7fff640 1 -- 172.21.15.16:0/3513947444 >> v2:172.21.15.184:3300/0 conn(0x7fc2b80b19d0 msgr2=0x7fc2b80b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/3513947444 >> v2:172.21.15.184:3300/0 conn(0x7fc2b80b19d0 0x7fc2b80b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2b7fff640 1 -- 172.21.15.16:0/3513947444 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b8161e60 con 0x7fc2b80cf090 2023-11-13T11:02:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/3513947444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b8164dd0 0x7fc2b8165210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.907+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/3513947444 >> v2:172.21.15.184:3300/0 conn(0x7fc2b80b19d0 0x7fc2b80b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.908+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/3513947444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b80cf090 0x7fc2b81622f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80cf890 tx=0x7fc2a802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba00c98c2b5ff419 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2b6ffd640 1 -- 172.21.15.16:0/3513947444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a802c9f0 con 0x7fc2b80cf090 2023-11-13T11:02:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2b6ffd640 1 -- 172.21.15.16:0/3513947444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a802cb50 con 0x7fc2b80cf090 2023-11-13T11:02:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2b6ffd640 1 -- 172.21.15.16:0/3513947444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80343f0 con 0x7fc2b80cf090 2023-11-13T11:02:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2be5e9640 1 -- 172.21.15.16:0/3513947444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b80cf090 msgr2=0x7fc2b81622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/3513947444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b80cf090 0x7fc2b81622f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80cf890 tx=0x7fc2a802be50 comp rx=0 tx=0).stop 2023-11-13T11:02:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2be5e9640 1 -- 172.21.15.16:0/3513947444 shutdown_connections 2023-11-13T11:02:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/3513947444 >> v2:172.21.15.184:3300/0 conn(0x7fc2b80b19d0 0x7fc2b80b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/3513947444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b8164dd0 0x7fc2b8165210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/3513947444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b80cf090 0x7fc2b81622f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2be5e9640 1 -- 172.21.15.16:0/3513947444 >> 172.21.15.16:0/3513947444 conn(0x7fc2b806a190 msgr2=0x7fc2b806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2be5e9640 1 -- 172.21.15.16:0/3513947444 shutdown_connections 2023-11-13T11:02:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.909+0000 7fc2be5e9640 1 -- 172.21.15.16:0/3513947444 wait complete. 2023-11-13T11:02:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.911+0000 7fc2be5e9640 1 Processor -- start 2023-11-13T11:02:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.911+0000 7fc2be5e9640 1 -- start start 2023-11-13T11:02:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.912+0000 7fc2be5e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b80b19d0 0x7fc2b8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.912+0000 7fc2be5e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b80cf090 0x7fc2b8143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.912+0000 7fc2be5e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b8164dd0 0x7fc2b81588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.912+0000 7fc2be5e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b81fb010 con 0x7fc2b8164dd0 2023-11-13T11:02:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.912+0000 7fc2be5e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8158720 con 0x7fc2b80b19d0 2023-11-13T11:02:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.912+0000 7fc2be5e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b81fa140 con 0x7fc2b80cf090 2023-11-13T11:02:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.913+0000 7fc2bcb86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b8164dd0 0x7fc2b81588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.913+0000 7fc2bcb86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b8164dd0 0x7fc2b81588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52920/0 (socket says 172.21.15.16:52920) 2023-11-13T11:02:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.913+0000 7fc2bcb86640 1 -- 172.21.15.16:0/619280461 learned_addr learned my addr 172.21.15.16:0/619280461 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.913+0000 7fc2bcb86640 1 -- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b80cf090 msgr2=0x7fc2b8143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.913+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b80cf090 0x7fc2b8143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.913+0000 7fc2bcb86640 1 -- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b80b19d0 msgr2=0x7fc2b8155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.913+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b80b19d0 0x7fc2b8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.913+0000 7fc2bcb86640 1 -- 172.21.15.16:0/619280461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8009b30 con 0x7fc2b8164dd0 2023-11-13T11:02:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.913+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b8164dd0 0x7fc2b81588c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c7c0 tx=0x7fc2b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.914+0000 7fc2b4ff9640 1 -- 172.21.15.16:0/619280461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0004230 con 0x7fc2b8164dd0 2023-11-13T11:02:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.914+0000 7fc2b4ff9640 1 -- 172.21.15.16:0/619280461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b0004390 con 0x7fc2b8164dd0 2023-11-13T11:02:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.914+0000 7fc2b4ff9640 1 -- 172.21.15.16:0/619280461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0007860 con 0x7fc2b8164dd0 2023-11-13T11:02:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.914+0000 7fc2be5e9640 1 -- 172.21.15.16:0/619280461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b81fa390 con 0x7fc2b8164dd0 2023-11-13T11:02:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.914+0000 7fc2be5e9640 1 -- 172.21.15.16:0/619280461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b81fa950 con 0x7fc2b8164dd0 2023-11-13T11:02:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.917+0000 7fc2b4ff9640 1 -- 172.21.15.16:0/619280461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc2b00079c0 con 0x7fc2b8164dd0 2023-11-13T11:02:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.918+0000 7fc2b4ff9640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc29806ae40 0x7fc29806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.918+0000 7fc2b4ff9640 1 -- 172.21.15.16:0/619280461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc2b0089a70 con 0x7fc2b8164dd0 2023-11-13T11:02:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.918+0000 7fc2c117c740 1 Processor -- start 2023-11-13T11:02:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.919+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc29806ae40 0x7fc29806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.919+0000 7fc2c117c740 1 -- start start 2023-11-13T11:02:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.919+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c133603c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.919+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d38a0 0x55c133600af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.919+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c133601030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.919+0000 7fc2c117c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c133545c70 con 0x55c1335d7400 2023-11-13T11:02:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.919+0000 7fc2c117c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c133543360 con 0x55c133603880 2023-11-13T11:02:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.919+0000 7fc2c117c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c13353ec50 con 0x55c1335d38a0 2023-11-13T11:02:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc29806ae40 0x7fc29806d300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8035000 tx=0x7fc2a802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2bcb86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c133601030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2bcb86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c133601030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52926/0 (socket says 172.21.15.16:52926) 2023-11-13T11:02:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2bcb86640 1 -- 172.21.15.16:0/4127999098 learned_addr learned my addr 172.21.15.16:0/4127999098 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/4127999098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c133603c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/4127999098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d38a0 0x55c133600af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2bcb86640 1 -- 172.21.15.16:0/4127999098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d38a0 msgr2=0x55c133600af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/4127999098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d38a0 0x55c133600af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2bcb86640 1 -- 172.21.15.16:0/4127999098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 msgr2=0x55c133603c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/4127999098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c133603c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.920+0000 7fc2bcb86640 1 -- 172.21.15.16:0/4127999098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c133501660 con 0x55c1335d7400 2023-11-13T11:02:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.921+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/4127999098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c133601030 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0004770 tx=0x7fc2b00582f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc61944b0350fe5 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.921+0000 7fc2adffb640 1 -- 172.21.15.16:0/4127999098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0060980 con 0x55c1335d7400 2023-11-13T11:02:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.921+0000 7fc2adffb640 1 -- 172.21.15.16:0/4127999098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b00567c0 con 0x55c1335d7400 2023-11-13T11:02:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.921+0000 7fc2adffb640 1 -- 172.21.15.16:0/4127999098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0054d90 con 0x55c1335d7400 2023-11-13T11:02:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.921+0000 7fc2c117c740 1 -- 172.21.15.16:0/4127999098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 msgr2=0x55c133601030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.921+0000 7fc2c117c740 1 --2- 172.21.15.16:0/4127999098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c133601030 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0004770 tx=0x7fc2b00582f0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.922+0000 7fc2c117c740 1 -- 172.21.15.16:0/4127999098 shutdown_connections 2023-11-13T11:02:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.922+0000 7fc2c117c740 1 --2- 172.21.15.16:0/4127999098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c133601030 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.922+0000 7fc2c117c740 1 --2- 172.21.15.16:0/4127999098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d38a0 0x55c133600af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.922+0000 7fc2c117c740 1 --2- 172.21.15.16:0/4127999098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c133603c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.922+0000 7fc2c117c740 1 -- 172.21.15.16:0/4127999098 >> 172.21.15.16:0/4127999098 conn(0x55c1335d5b10 msgr2=0x55c13361e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.922+0000 7fc2c117c740 1 -- 172.21.15.16:0/4127999098 shutdown_connections 2023-11-13T11:02:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.922+0000 7fc2c117c740 1 -- 172.21.15.16:0/4127999098 wait complete. 2023-11-13T11:02:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.922+0000 7fc2c117c740 1 -- start start 2023-11-13T11:02:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.923+0000 7fc2c117c740 1 Processor -- start 2023-11-13T11:02:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c1335f8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c133621ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335f8c90 0x55c1336220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2c117c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1334db0b0 con 0x55c1335d7400 2023-11-13T11:02:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2c117c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1334f8940 con 0x55c133603880 2023-11-13T11:02:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2c117c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1332a1f50 con 0x55c1335f8c90 2023-11-13T11:02:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2c117c740 10 client.0 _ll_register_callbacks cb 0x55c1334fd070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c1335f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c1335f8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52934/0 (socket says 172.21.15.16:52934) 2023-11-13T11:02:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2b7fff640 1 -- 172.21.15.16:0/3049972175 learned_addr learned my addr 172.21.15.16:0/3049972175 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335f8c90 0x55c1336220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c133621ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2b7fff640 1 -- 172.21.15.16:0/3049972175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335f8c90 msgr2=0x55c1336220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335f8c90 0x55c1336220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2b7fff640 1 -- 172.21.15.16:0/3049972175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 msgr2=0x55c133621ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.924+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c133621ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.925+0000 7fc2b7fff640 1 -- 172.21.15.16:0/3049972175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b81fa950 con 0x55c1335d7400 2023-11-13T11:02:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.925+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c133621ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.925+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c1335f8620 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc2a802c460 tx=0x7fc2a802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5fa7ec46970c38 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.925+0000 7fc296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.925+0000 7fc296ffd640 1 -- 172.21.15.16:0/3049972175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8050070 con 0x55c1335d7400 2023-11-13T11:02:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.925+0000 7fc296ffd640 1 -- 172.21.15.16:0/3049972175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc2a8040e00 con 0x55c1335d7400 2023-11-13T11:02:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.925+0000 7fc2c117c740 1 -- 172.21.15.16:0/3049972175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1334e3880 con 0x55c1335d7400 2023-11-13T11:02:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.925+0000 7fc296ffd640 1 -- 172.21.15.16:0/3049972175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2a803eb30 con 0x55c1335d7400 2023-11-13T11:02:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc2c117c740 10 client.6663 fetch_fsmap learned FSMap version 30 2023-11-13T11:02:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc2c117c740 1 -- 172.21.15.16:0/3049972175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55c13353d2d0 con 0x55c1335d7400 2023-11-13T11:02:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc296ffd640 1 -- 172.21.15.16:0/3049972175 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1334e3880 con 0x55c1335d7400 2023-11-13T11:02:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc296ffd640 1 -- 172.21.15.16:0/3049972175 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc2a8053020 con 0x55c1335d7400 2023-11-13T11:02:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc2c117c740 10 client.6663 fetch_fsmap finished waiting for FSMap version 30 2023-11-13T11:02:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc2c117c740 10 client.6663 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T11:02:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc2c117c740 20 client.6663 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc2c117c740 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55c1335fe9f0 0x55c133605210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc2c117c740 4 client.6663 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:02:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc2c117c740 1 -- 172.21.15.16:0/3049972175 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c133537c30 con 0x55c1335fe9f0 2023-11-13T11:02:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.926+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55c1335fe9f0 0x55c133605210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.927+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55c1335fe9f0 0x55c133605210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1abfca63e27dc1 server_cookie=c32cf1beeab5b541 in_seq=0 out_seq=0 2023-11-13T11:02:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.927+0000 7fc296ffd640 10 client.6663 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc296ffd640 1 -- 172.21.15.16:0/3049972175 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c133537c30 con 0x55c1335fe9f0 2023-11-13T11:02:55.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc296ffd640 10 client.6663 handle_command_reply: tid=0 2023-11-13T11:02:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 client.6663 shutdown 2023-11-13T11:02:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 -- 172.21.15.16:0/3049972175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 msgr2=0x55c1335f8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c1335f8620 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc2a802c460 tx=0x7fc2a802e070 comp rx=0 tx=0).stop 2023-11-13T11:02:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 -- 172.21.15.16:0/3049972175 shutdown_connections 2023-11-13T11:02:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335f8c90 0x55c1336220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c133621ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55c1335fe9f0 0x55c133605210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 --2- 172.21.15.16:0/3049972175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335d7400 0x55c1335f8620 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 -- 172.21.15.16:0/3049972175 >> 172.21.15.16:0/3049972175 conn(0x55c1335d38a0 msgr2=0x55c1335d7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc296ffd640 0 client.6663 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 -- 172.21.15.16:0/3049972175 shutdown_connections 2023-11-13T11:02:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 1 -- 172.21.15.16:0/3049972175 wait complete. 2023-11-13T11:02:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.928+0000 7fc2c117c740 20 client.6663 trim_cache size 0 max 16384 2023-11-13T11:02:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.942+0000 7fc2c117c740 1 Processor -- start 2023-11-13T11:02:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.942+0000 7fc2c117c740 1 -- start start 2023-11-13T11:02:55.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.943+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d7400 0x55c1335ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.943+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1335f8c90 0x55c133605c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.943+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c133603880 0x55c1336061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.943+0000 7fc2c117c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1335ffbe0 con 0x55c133603880 2023-11-13T11:02:55.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.943+0000 7fc2c117c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1335cf840 con 0x55c1335f8c90 2023-11-13T11:02:55.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.943+0000 7fc2c117c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1335d7140 con 0x55c1335d7400 2023-11-13T11:02:55.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2bcb86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c133603880 0x55c1336061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2bcb86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c133603880 0x55c1336061b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52950/0 (socket says 172.21.15.16:52950) 2023-11-13T11:02:55.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2bcb86640 1 -- 172.21.15.16:0/1953130800 learned_addr learned my addr 172.21.15.16:0/1953130800 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/1953130800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d7400 0x55c1335ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/1953130800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1335f8c90 0x55c133605c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2bcb86640 1 -- 172.21.15.16:0/1953130800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d7400 msgr2=0x55c1335ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/1953130800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d7400 0x55c1335ff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2bcb86640 1 -- 172.21.15.16:0/1953130800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1335f8c90 msgr2=0x55c133605c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/1953130800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1335f8c90 0x55c133605c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.944+0000 7fc2bcb86640 1 -- 172.21.15.16:0/1953130800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1334da940 con 0x55c133603880 2023-11-13T11:02:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.945+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/1953130800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c133603880 0x55c1336061b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0004770 tx=0x7fc2b005a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5fedf5aa82cb97 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.945+0000 7fc2ae7fc640 1 -- 172.21.15.16:0/1953130800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0004500 con 0x55c133603880 2023-11-13T11:02:55.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.945+0000 7fc2ae7fc640 1 -- 172.21.15.16:0/1953130800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b0060980 con 0x55c133603880 2023-11-13T11:02:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.945+0000 7fc2ae7fc640 1 -- 172.21.15.16:0/1953130800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0054d90 con 0x55c133603880 2023-11-13T11:02:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.945+0000 7fc2c117c740 1 -- 172.21.15.16:0/1953130800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c133603880 msgr2=0x55c1336061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.945+0000 7fc2c117c740 1 --2- 172.21.15.16:0/1953130800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c133603880 0x55c1336061b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0004770 tx=0x7fc2b005a000 comp rx=0 tx=0).stop 2023-11-13T11:02:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.946+0000 7fc2c117c740 1 -- 172.21.15.16:0/1953130800 shutdown_connections 2023-11-13T11:02:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.946+0000 7fc2c117c740 1 --2- 172.21.15.16:0/1953130800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c133603880 0x55c1336061b0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.946+0000 7fc2c117c740 1 --2- 172.21.15.16:0/1953130800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c1335f8c90 0x55c133605c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.946+0000 7fc2c117c740 1 --2- 172.21.15.16:0/1953130800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d7400 0x55c1335ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.946+0000 7fc2c117c740 1 -- 172.21.15.16:0/1953130800 >> 172.21.15.16:0/1953130800 conn(0x55c1335d38a0 msgr2=0x55c1335d7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.946+0000 7fc2c117c740 1 -- 172.21.15.16:0/1953130800 shutdown_connections 2023-11-13T11:02:55.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.946+0000 7fc2c117c740 1 -- 172.21.15.16:0/1953130800 wait complete. 2023-11-13T11:02:55.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.946+0000 7fc2c117c740 1 -- start start 2023-11-13T11:02:55.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.947+0000 7fc2c117c740 1 Processor -- start 2023-11-13T11:02:55.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d7400 0x55c133605c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335f8c90 0x55c1336061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2c117c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c1335f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2c117c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c133630500 con 0x55c1335f8c90 2023-11-13T11:02:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2c117c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c133671f70 con 0x55c133603880 2023-11-13T11:02:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2c117c740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1336721c0 con 0x55c1335d7400 2023-11-13T11:02:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2c117c740 10 client.0 _ll_register_callbacks cb 0x55c1335d36a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:02:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335f8c90 0x55c1336061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335f8c90 0x55c1336061b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52966/0 (socket says 172.21.15.16:52966) 2023-11-13T11:02:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2b77fe640 1 -- 172.21.15.16:0/2434829305 learned_addr learned my addr 172.21.15.16:0/2434829305 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2b77fe640 1 -- 172.21.15.16:0/2434829305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d7400 msgr2=0x55c133605c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d7400 0x55c133605c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2b77fe640 1 -- 172.21.15.16:0/2434829305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 msgr2=0x55c1335f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2bcb86640 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c1335f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c1335f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.948+0000 7fc2b77fe640 1 -- 172.21.15.16:0/2434829305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c133501660 con 0x55c1335f8c90 2023-11-13T11:02:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.949+0000 7fc2b77fe640 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335f8c90 0x55c1336061b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0004700 tx=0x7fc2a0018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ad965b58221717 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.949+0000 7fc296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:02:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.949+0000 7fc296ffd640 1 -- 172.21.15.16:0/2434829305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0014810 con 0x55c1335f8c90 2023-11-13T11:02:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.949+0000 7fc296ffd640 1 -- 172.21.15.16:0/2434829305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc2a000b040 con 0x55c1335f8c90 2023-11-13T11:02:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.949+0000 7fc2c117c740 1 -- 172.21.15.16:0/2434829305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2b00567c0 con 0x55c1335f8c90 2023-11-13T11:02:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.949+0000 7fc296ffd640 1 -- 172.21.15.16:0/2434829305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2a0014970 con 0x55c1335f8c90 2023-11-13T11:02:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.949+0000 7fc2c117c740 10 client.6669 fetch_fsmap learned FSMap version 30 2023-11-13T11:02:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.949+0000 7fc2c117c740 1 -- 172.21.15.16:0/2434829305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55c1334ffec0 con 0x55c1335f8c90 2023-11-13T11:02:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.950+0000 7fc296ffd640 1 -- 172.21.15.16:0/2434829305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b00567c0 con 0x55c1335f8c90 2023-11-13T11:02:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.950+0000 7fc296ffd640 1 -- 172.21.15.16:0/2434829305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc2a0014ad0 con 0x55c1335f8c90 2023-11-13T11:02:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.950+0000 7fc2c117c740 10 client.6669 fetch_fsmap finished waiting for FSMap version 30 2023-11-13T11:02:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.950+0000 7fc2c117c740 10 client.6669 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-13T11:02:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.950+0000 7fc2c117c740 20 client.6669 populate_metadata read hostname 'smithi016' 2023-11-13T11:02:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.950+0000 7fc2c117c740 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55c1335f5930 0x55c1335f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.950+0000 7fc2c117c740 4 client.6669 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1488910146}] 2023-11-13T11:02:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.950+0000 7fc2c117c740 1 -- 172.21.15.16:0/2434829305 --> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1488910146}) v1 -- 0x55c133537950 con 0x55c1335f5930 2023-11-13T11:02:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.950+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55c1335f5930 0x55c1335f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2b7fff640 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55c1335f5930 0x55c1335f5d10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b9b1e49bb9229c server_cookie=59c28fd981db440a in_seq=0 out_seq=0 2023-11-13T11:02:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc296ffd640 10 client.6669 ms_handle_connect on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc296ffd640 1 -- 172.21.15.16:0/2434829305 <== mds.0 v2:172.21.15.16:6834/3328237273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55c133537950 con 0x55c1335f5930 2023-11-13T11:02:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc296ffd640 10 client.6669 handle_command_reply: tid=0 2023-11-13T11:02:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 client.6669 shutdown 2023-11-13T11:02:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 -- 172.21.15.16:0/2434829305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335f8c90 msgr2=0x55c1336061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335f8c90 0x55c1336061b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0004700 tx=0x7fc2a0018420 comp rx=0 tx=0).stop 2023-11-13T11:02:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 -- 172.21.15.16:0/2434829305 shutdown_connections 2023-11-13T11:02:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c133603880 0x55c1335f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.16:6834/3328237273,v1:172.21.15.16:6835/3328237273] conn(0x55c1335f5930 0x55c1335f5d10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1335f8c90 0x55c1336061b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 --2- 172.21.15.16:0/2434829305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c1335d7400 0x55c133605c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 -- 172.21.15.16:0/2434829305 >> 172.21.15.16:0/2434829305 conn(0x55c1335d38a0 msgr2=0x55c1335f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc296ffd640 0 client.6669 ms_handle_reset on v2:172.21.15.16:6834/3328237273 2023-11-13T11:02:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 -- 172.21.15.16:0/2434829305 shutdown_connections 2023-11-13T11:02:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.951+0000 7fc2c117c740 1 -- 172.21.15.16:0/2434829305 wait complete. 2023-11-13T11:02:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2c117c740 20 client.6669 trim_cache size 0 max 16384 2023-11-13T11:02:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 -- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc29806ae40 msgr2=0x7fc29806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc29806ae40 0x7fc29806d300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8035000 tx=0x7fc2a802e000 comp rx=0 tx=0).stop 2023-11-13T11:02:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 -- 172.21.15.16:0/619280461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b8164dd0 msgr2=0x7fc2b81588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b8164dd0 0x7fc2b81588c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c7c0 tx=0x7fc2b000cc80 comp rx=0 tx=0).stop 2023-11-13T11:02:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 -- 172.21.15.16:0/619280461 shutdown_connections 2023-11-13T11:02:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc29806ae40 0x7fc29806d300 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2b8164dd0 0x7fc2b81588c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2b80cf090 0x7fc2b8143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 --2- 172.21.15.16:0/619280461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b80b19d0 0x7fc2b8155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 -- 172.21.15.16:0/619280461 >> 172.21.15.16:0/619280461 conn(0x7fc2b806a190 msgr2=0x7fc2b815d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 -- 172.21.15.16:0/619280461 shutdown_connections 2023-11-13T11:02:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:55.952+0000 7fc2be5e9640 1 -- 172.21.15.16:0/619280461 wait complete. 2023-11-13T11:02:55.974 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2023-11-13T11:02:55.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:02:55.980 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-13T11:02:55.972+0000 7fe1578d7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-13T11:02:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.191+0000 7f2938856640 1 Processor -- start 2023-11-13T11:02:56.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.191+0000 7f2938856640 1 -- start start 2023-11-13T11:02:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.193+0000 7f2938856640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29340ccc60 0x7f2934071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.193+0000 7f2938856640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2934071980 0x7f293406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.193+0000 7f2938856640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29340cd0e0 con 0x7f2934155410 2023-11-13T11:02:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.193+0000 7f2938856640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2934071e10 con 0x7f2934071980 2023-11-13T11:02:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.193+0000 7f2938856640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293415b500 con 0x7f29340ccc60 2023-11-13T11:02:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.194+0000 7f2932d9d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2934071980 0x7f293406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.194+0000 7f292bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29340ccc60 0x7f2934071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.194+0000 7f292bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29340ccc60 0x7f2934071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:35212/0 (socket says 172.21.15.16:35212) 2023-11-13T11:02:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.194+0000 7f292bfff640 1 -- 172.21.15.16:0/1651736162 learned_addr learned my addr 172.21.15.16:0/1651736162 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.195+0000 7f292bfff640 1 -- 172.21.15.16:0/1651736162 >> v2:172.21.15.184:3300/0 conn(0x7f2934071980 msgr2=0x7f293406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.195+0000 7f292bfff640 1 --2- 172.21.15.16:0/1651736162 >> v2:172.21.15.184:3300/0 conn(0x7f2934071980 0x7f293406fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.195+0000 7f292bfff640 1 -- 172.21.15.16:0/1651736162 >> v1:172.21.15.184:6789/0 conn(0x7f2934155410 legacy=0x7f29340ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.195+0000 7f292bfff640 1 -- 172.21.15.16:0/1651736162 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293415b7e0 con 0x7f29340ccc60 2023-11-13T11:02:56.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.197+0000 7f292bfff640 1 --2- 172.21.15.16:0/1651736162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29340ccc60 0x7f2934071440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f291c009870 tx=0x7f291c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c65c21bffbea5d75 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:56.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.197+0000 7f2931d9b640 1 -- 172.21.15.16:0/1651736162 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c030840 con 0x7f29340ccc60 2023-11-13T11:02:56.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.197+0000 7f2931d9b640 1 -- 172.21.15.16:0/1651736162 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c0309a0 con 0x7f29340ccc60 2023-11-13T11:02:56.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.197+0000 7f2931d9b640 1 -- 172.21.15.16:0/1651736162 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c030d30 con 0x7f29340ccc60 2023-11-13T11:02:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.198+0000 7f2938856640 1 -- 172.21.15.16:0/1651736162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29340ccc60 msgr2=0x7f2934071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.198+0000 7f2938856640 1 --2- 172.21.15.16:0/1651736162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29340ccc60 0x7f2934071440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f291c009870 tx=0x7f291c02b980 comp rx=0 tx=0).stop 2023-11-13T11:02:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.198+0000 7f2938856640 1 -- 172.21.15.16:0/1651736162 shutdown_connections 2023-11-13T11:02:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.198+0000 7f2938856640 1 --2- 172.21.15.16:0/1651736162 >> v2:172.21.15.184:3300/0 conn(0x7f2934071980 0x7f293406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.198+0000 7f2938856640 1 --2- 172.21.15.16:0/1651736162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29340ccc60 0x7f2934071440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.198+0000 7f2938856640 1 -- 172.21.15.16:0/1651736162 >> 172.21.15.16:0/1651736162 conn(0x7f293406a190 msgr2=0x7f293406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.198+0000 7f2938856640 1 -- 172.21.15.16:0/1651736162 shutdown_connections 2023-11-13T11:02:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.199+0000 7f2938856640 1 -- 172.21.15.16:0/1651736162 wait complete. 2023-11-13T11:02:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.201+0000 7f2938856640 1 Processor -- start 2023-11-13T11:02:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.201+0000 7f2938856640 1 -- start start 2023-11-13T11:02:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.202+0000 7f2938856640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2934071980 0x7f2934147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.202+0000 7f2938856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340ccc60 0x7f29340b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.202+0000 7f2938856640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2934155410 0x7f29341e3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.202+0000 7f2938856640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340b1480 con 0x7f29340ccc60 2023-11-13T11:02:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.202+0000 7f2938856640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2934148270 con 0x7f2934071980 2023-11-13T11:02:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.202+0000 7f2938856640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2934153b00 con 0x7f2934155410 2023-11-13T11:02:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f292bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340ccc60 0x7f29340b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f292bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340ccc60 0x7f29340b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52970/0 (socket says 172.21.15.16:52970) 2023-11-13T11:02:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f292bfff640 1 -- 172.21.15.16:0/3296640312 learned_addr learned my addr 172.21.15.16:0/3296640312 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f293259c640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2934071980 0x7f2934147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f2932d9d640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2934155410 0x7f29341e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f292bfff640 1 -- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2934155410 msgr2=0x7f29341e3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f292bfff640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2934155410 0x7f29341e3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f292bfff640 1 -- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2934071980 msgr2=0x7f2934147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f292bfff640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2934071980 0x7f2934147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.203+0000 7f292bfff640 1 -- 172.21.15.16:0/3296640312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0094d0 con 0x7f29340ccc60 2023-11-13T11:02:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.204+0000 7f292bfff640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340ccc60 0x7f29340b18a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f291c000c00 tx=0x7f291c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.204+0000 7f292affd640 1 -- 172.21.15.16:0/3296640312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c039c90 con 0x7f29340ccc60 2023-11-13T11:02:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.204+0000 7f292affd640 1 -- 172.21.15.16:0/3296640312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c039df0 con 0x7f29340ccc60 2023-11-13T11:02:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.204+0000 7f292affd640 1 -- 172.21.15.16:0/3296640312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c043640 con 0x7f29340ccc60 2023-11-13T11:02:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.204+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29341f2e80 con 0x7f29340ccc60 2023-11-13T11:02:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.204+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2934153820 con 0x7f29340ccc60 2023-11-13T11:02:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.206+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2934067090 con 0x7f29340ccc60 2023-11-13T11:02:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.207+0000 7f292affd640 1 -- 172.21.15.16:0/3296640312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f291c0437a0 con 0x7f29340ccc60 2023-11-13T11:02:56.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.208+0000 7f292affd640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f291406ae40 0x7f291406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:56.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.208+0000 7f292affd640 1 -- 172.21.15.16:0/3296640312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f291c03e070 con 0x7f29340ccc60 2023-11-13T11:02:56.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.209+0000 7f293259c640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f291406ae40 0x7f291406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:56.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.210+0000 7f293259c640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f291406ae40 0x7f291406d300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f29341e5a40 tx=0x7f2924009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.213+0000 7f292affd640 1 -- 172.21.15.16:0/3296640312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f291c0776c0 con 0x7f29340ccc60 2023-11-13T11:02:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.448+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29340ae1d0 con 0x7f29340ccc60 2023-11-13T11:02:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.450+0000 7f292affd640 1 -- 172.21.15.16:0/3296640312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f291c07ace0 con 0x7f29340ccc60 2023-11-13T11:02:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:56.455 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:55.012799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:02:56.456 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2023-11-13T11:02:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f291406ae40 msgr2=0x7f291406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f291406ae40 0x7f291406d300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f29341e5a40 tx=0x7f2924009040 comp rx=0 tx=0).stop 2023-11-13T11:02:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340ccc60 msgr2=0x7f29340b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340ccc60 0x7f29340b18a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f291c000c00 tx=0x7f291c004730 comp rx=0 tx=0).stop 2023-11-13T11:02:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 shutdown_connections 2023-11-13T11:02:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f291406ae40 0x7f291406d300 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2934155410 0x7f29341e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29340ccc60 0x7f29340b18a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 --2- 172.21.15.16:0/3296640312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2934071980 0x7f2934147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.452+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 >> 172.21.15.16:0/3296640312 conn(0x7f293406a190 msgr2=0x7f29340b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.453+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 shutdown_connections 2023-11-13T11:02:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:56.453+0000 7f2938856640 1 -- 172.21.15.16:0/3296640312 wait complete. 2023-11-13T11:02:56.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2023-11-13T11:02:56.709 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:02:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:02:56.710 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:02:56.726 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-13T11:02:56.726 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-13T11:02:56.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:02:56.867 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:02:56.993 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-13T11:02:56.994 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:02:56.994 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:02:57.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:02:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.275+0000 7f4b2bfff640 1 Processor -- start 2023-11-13T11:02:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.275+0000 7f4b2bfff640 1 -- start start 2023-11-13T11:02:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.277+0000 7f4b2bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c06a8a0 0x7f4b2c070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.277+0000 7f4b2bfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4b2c0708a0 0x7f4b2c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.277+0000 7f4b2bfff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c070d50 con 0x7f4b2c15aa90 2023-11-13T11:02:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.277+0000 7f4b2bfff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c15d630 con 0x7f4b2c0708a0 2023-11-13T11:02:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.277+0000 7f4b2bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c15d910 con 0x7f4b2c06a8a0 2023-11-13T11:02:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.278+0000 7f4b2affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4b2c15aa90 0x7f4b2c06a190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41362/0 (socket says 172.21.15.16:41362) 2023-11-13T11:02:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.278+0000 7f4b2affd640 1 -- 172.21.15.16:0/4183545091 learned_addr learned my addr 172.21.15.16:0/4183545091 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.278+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345333627 0 0) 0x7f4b2c070d50 con 0x7f4b2c15aa90 2023-11-13T11:02:57.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.278+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b18005730 con 0x7f4b2c15aa90 2023-11-13T11:02:57.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.278+0000 7f4b227fc640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c06a8a0 0x7f4b2c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.278+0000 7f4b2b7fe640 1 --2- 172.21.15.16:0/4183545091 >> v2:172.21.15.184:3300/0 conn(0x7f4b2c0708a0 0x7f4b2c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89788793 0 0) 0x7f4b18005730 con 0x7f4b2c15aa90 2023-11-13T11:02:57.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c06a8a0 msgr2=0x7f4b2c070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:57.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c06a8a0 0x7f4b2c070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 >> v2:172.21.15.184:3300/0 conn(0x7f4b2c0708a0 msgr2=0x7f4b2c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:57.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 --2- 172.21.15.16:0/4183545091 >> v2:172.21.15.184:3300/0 conn(0x7f4b2c0708a0 0x7f4b2c06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c15dbf0 con 0x7f4b2c15aa90 2023-11-13T11:02:57.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f4b1c001670 con 0x7f4b2c15aa90 2023-11-13T11:02:57.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 >> v1:172.21.15.16:6789/0 conn(0x7f4b2c15aa90 legacy=0x7f4b2c06a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b18005b80 0x7f4b18005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b18006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c06a8a0 0x7f4b180073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c15d630 con 0x7f4b180069d0 2023-11-13T11:02:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c15d910 con 0x7f4b18005b80 2023-11-13T11:02:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1800dd20 con 0x7f4b2c06a8a0 2023-11-13T11:02:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b227fc640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b18006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.279+0000 7f4b2affd640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b18005b80 0x7f4b18005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.280+0000 7f4b2b7fe640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c06a8a0 0x7f4b180073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.280+0000 7f4b227fc640 1 -- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c06a8a0 msgr2=0x7f4b180073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.280+0000 7f4b227fc640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c06a8a0 0x7f4b180073b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.280+0000 7f4b227fc640 1 -- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b18005b80 msgr2=0x7f4b18005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.280+0000 7f4b227fc640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b18005b80 0x7f4b18005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.280+0000 7f4b227fc640 1 -- 172.21.15.16:0/4183545091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18005730 con 0x7f4b180069d0 2023-11-13T11:02:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.280+0000 7f4b2b7fe640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c06a8a0 0x7f4b180073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.281+0000 7f4b227fc640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b18006e50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4b14007360 tx=0x7f4b18007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc9015077c7f991 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.281+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1402ccb0 con 0x7f4b180069d0 2023-11-13T11:02:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.281+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1402ce10 con 0x7f4b180069d0 2023-11-13T11:02:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.281+0000 7f4b2a7fc640 1 -- 172.21.15.16:0/4183545091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14034660 con 0x7f4b180069d0 2023-11-13T11:02:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.282+0000 7f4b2bfff640 1 -- 172.21.15.16:0/4183545091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 msgr2=0x7f4b18006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.282+0000 7f4b2bfff640 1 --2- 172.21.15.16:0/4183545091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b18006e50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4b14007360 tx=0x7f4b18007a00 comp rx=0 tx=0).stop 2023-11-13T11:02:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.282+0000 7f4b2b7fe640 1 -- 172.21.15.16:0/4183545091 reap_dead start 2023-11-13T11:02:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.282+0000 7f4b2bfff640 1 -- 172.21.15.16:0/4183545091 shutdown_connections 2023-11-13T11:02:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.282+0000 7f4b2bfff640 1 -- 172.21.15.16:0/4183545091 >> 172.21.15.16:0/4183545091 conn(0x7f4b2c0b04a0 msgr2=0x7f4b2c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.282+0000 7f4b2bfff640 1 -- 172.21.15.16:0/4183545091 shutdown_connections 2023-11-13T11:02:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.282+0000 7f4b2bfff640 1 -- 172.21.15.16:0/4183545091 wait complete. 2023-11-13T11:02:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.284+0000 7f4b2bfff640 1 Processor -- start 2023-11-13T11:02:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.284+0000 7f4b2bfff640 1 -- start start 2023-11-13T11:02:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.285+0000 7f4b2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b2c148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.285+0000 7f4b2bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c06c440 0x7f4b2c06c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.285+0000 7f4b2bfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c1f1fc0 0x7f4b2c1f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.285+0000 7f4b2bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c06ce00 con 0x7f4b180069d0 2023-11-13T11:02:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.285+0000 7f4b2bfff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c1f2d30 con 0x7f4b2c06c440 2023-11-13T11:02:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.285+0000 7f4b2bfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c0a89f0 con 0x7f4b2c1f1fc0 2023-11-13T11:02:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.285+0000 7f4b2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b2c148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b2c148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52986/0 (socket says 172.21.15.16:52986) 2023-11-13T11:02:57.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b2affd640 1 -- 172.21.15.16:0/719924012 learned_addr learned my addr 172.21.15.16:0/719924012 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b227fc640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c06c440 0x7f4b2c06c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b2b7fe640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c1f1fc0 0x7f4b2c1f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b2affd640 1 -- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c1f1fc0 msgr2=0x7f4b2c1f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b2affd640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c1f1fc0 0x7f4b2c1f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b2affd640 1 -- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c06c440 msgr2=0x7f4b2c06c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b2affd640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c06c440 0x7f4b2c06c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b2affd640 1 -- 172.21.15.16:0/719924012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0a8cd0 con 0x7f4b180069d0 2023-11-13T11:02:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b227fc640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c06c440 0x7f4b2c06c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.286+0000 7f4b2affd640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b2c148200 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00e7f0 tx=0x7f4b1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:57.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.287+0000 7f4b23fff640 1 -- 172.21.15.16:0/719924012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00c870 con 0x7f4b180069d0 2023-11-13T11:02:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.287+0000 7f4b23fff640 1 -- 172.21.15.16:0/719924012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c00c9d0 con 0x7f4b180069d0 2023-11-13T11:02:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.287+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c0a8fb0 con 0x7f4b180069d0 2023-11-13T11:02:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.287+0000 7f4b23fff640 1 -- 172.21.15.16:0/719924012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00cd60 con 0x7f4b180069d0 2023-11-13T11:02:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.287+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c148740 con 0x7f4b180069d0 2023-11-13T11:02:57.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.288+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2c0675b0 con 0x7f4b180069d0 2023-11-13T11:02:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.289+0000 7f4b23fff640 1 -- 172.21.15.16:0/719924012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b1c01d460 con 0x7f4b180069d0 2023-11-13T11:02:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.290+0000 7f4b23fff640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b0c06ae60 0x7f4b0c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.290+0000 7f4b23fff640 1 -- 172.21.15.16:0/719924012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4b1c089880 con 0x7f4b180069d0 2023-11-13T11:02:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.291+0000 7f4b227fc640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b0c06ae60 0x7f4b0c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.292+0000 7f4b227fc640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b0c06ae60 0x7f4b0c06d320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4b14036000 tx=0x7f4b14030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:57.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.294+0000 7f4b23fff640 1 -- 172.21.15.16:0/719924012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b1c054710 con 0x7f4b180069d0 2023-11-13T11:02:57.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.488+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b2c04e4d0 con 0x7f4b180069d0 2023-11-13T11:02:57.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.491+0000 7f4b23fff640 1 -- 172.21.15.16:0/719924012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f4b1c057d30 con 0x7f4b180069d0 2023-11-13T11:02:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:02:57.497 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:02:08.953574+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:01:36.305098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-13T11:01:37.461852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.4","mappings":[{"from":2,"to":5}]},{"pgid":"8.20","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:02:57.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b0c06ae60 msgr2=0x7f4b0c06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b0c06ae60 0x7f4b0c06d320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4b14036000 tx=0x7f4b14030000 comp rx=0 tx=0).stop 2023-11-13T11:02:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 msgr2=0x7f4b2c148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b2c148200 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00e7f0 tx=0x7f4b1c00ecb0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 shutdown_connections 2023-11-13T11:02:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b0c06ae60 0x7f4b0c06d320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b2c1f1fc0 0x7f4b2c1f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c06c440 0x7f4b2c06c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 --2- 172.21.15.16:0/719924012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b180069d0 0x7f4b2c148200 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 >> 172.21.15.16:0/719924012 conn(0x7f4b2c0b04a0 msgr2=0x7f4b2c157b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:57.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 shutdown_connections 2023-11-13T11:02:57.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.494+0000 7f4b2bfff640 1 -- 172.21.15.16:0/719924012 wait complete. 2023-11-13T11:02:57.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T11:01:36.305098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T11:01:37.461852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:02:57.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:02:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.725+0000 7f3c2de52640 1 Processor -- start 2023-11-13T11:02:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.725+0000 7f3c2de52640 1 -- start start 2023-11-13T11:02:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.727+0000 7f3c2de52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c2815b5c0 0x7f3c28164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.727+0000 7f3c2de52640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c28161640 0x7f3c280b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.727+0000 7f3c2de52640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c28165b40 con 0x7f3c280b2970 2023-11-13T11:02:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.727+0000 7f3c2de52640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c280b2de0 con 0x7f3c28161640 2023-11-13T11:02:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.727+0000 7f3c2de52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c281652a0 con 0x7f3c2815b5c0 2023-11-13T11:02:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.727+0000 7f3c27fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c28161640 0x7f3c280b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.728+0000 7f3c2d651640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3c280b2970 0x7f3c280ad520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41366/0 (socket says 172.21.15.16:41366) 2023-11-13T11:02:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.728+0000 7f3c2d651640 1 -- 172.21.15.16:0/2556842658 learned_addr learned my addr 172.21.15.16:0/2556842658 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:02:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.728+0000 7f3c2ce50640 1 --2- 172.21.15.16:0/2556842658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c2815b5c0 0x7f3c28164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.728+0000 7f3c27fff640 1 -- 172.21.15.16:0/2556842658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c2815b5c0 msgr2=0x7f3c28164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.728+0000 7f3c27fff640 1 --2- 172.21.15.16:0/2556842658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c2815b5c0 0x7f3c28164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.728+0000 7f3c27fff640 1 -- 172.21.15.16:0/2556842658 >> v1:172.21.15.16:6789/0 conn(0x7f3c280b2970 legacy=0x7f3c280ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.728+0000 7f3c27fff640 1 -- 172.21.15.16:0/2556842658 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28165580 con 0x7f3c28161640 2023-11-13T11:02:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.728+0000 7f3c277fe640 1 -- 172.21.15.16:0/2556842658 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553960043 0 0) 0x7f3c28165b40 con 0x7f3c280b2970 2023-11-13T11:02:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.729+0000 7f3c27fff640 1 --2- 172.21.15.16:0/2556842658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 0x7f3c280b2430 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097a0 tx=0x7f3c1402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b21069afdedf9c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.730+0000 7f3c277fe640 1 -- 172.21.15.16:0/2556842658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14031830 con 0x7f3c28161640 2023-11-13T11:02:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.730+0000 7f3c277fe640 1 -- 172.21.15.16:0/2556842658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c14031990 con 0x7f3c28161640 2023-11-13T11:02:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.730+0000 7f3c277fe640 1 -- 172.21.15.16:0/2556842658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14031d20 con 0x7f3c28161640 2023-11-13T11:02:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.730+0000 7f3c2de52640 1 -- 172.21.15.16:0/2556842658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 msgr2=0x7f3c280b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.730+0000 7f3c2de52640 1 --2- 172.21.15.16:0/2556842658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 0x7f3c280b2430 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097a0 tx=0x7f3c1402b8d0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.730+0000 7f3c2de52640 1 -- 172.21.15.16:0/2556842658 shutdown_connections 2023-11-13T11:02:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.730+0000 7f3c2de52640 1 --2- 172.21.15.16:0/2556842658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 0x7f3c280b2430 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.730+0000 7f3c2de52640 1 --2- 172.21.15.16:0/2556842658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c2815b5c0 0x7f3c28164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.730+0000 7f3c2de52640 1 -- 172.21.15.16:0/2556842658 >> 172.21.15.16:0/2556842658 conn(0x7f3c280b0970 msgr2=0x7f3c280b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.731+0000 7f3c2de52640 1 -- 172.21.15.16:0/2556842658 shutdown_connections 2023-11-13T11:02:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.731+0000 7f3c2de52640 1 -- 172.21.15.16:0/2556842658 wait complete. 2023-11-13T11:02:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.732+0000 7f3c2de52640 1 Processor -- start 2023-11-13T11:02:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.732+0000 7f3c2de52640 1 -- start start 2023-11-13T11:02:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.733+0000 7f3c2de52640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c280b2970 0x7f3c280aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.733+0000 7f3c2de52640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c2815b5c0 0x7f3c280a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.733+0000 7f3c2de52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 0x7f3c2815fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.733+0000 7f3c2de52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c280a8920 con 0x7f3c28161640 2023-11-13T11:02:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.733+0000 7f3c2de52640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2815eb40 con 0x7f3c2815b5c0 2023-11-13T11:02:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.733+0000 7f3c2de52640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2815ee20 con 0x7f3c280b2970 2023-11-13T11:02:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c2d651640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 0x7f3c2815fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c2d651640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 0x7f3c2815fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53012/0 (socket says 172.21.15.16:53012) 2023-11-13T11:02:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c2d651640 1 -- 172.21.15.16:0/4008199254 learned_addr learned my addr 172.21.15.16:0/4008199254 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c27fff640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c2815b5c0 0x7f3c280a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c2ce50640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c280b2970 0x7f3c280aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c2d651640 1 -- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c280b2970 msgr2=0x7f3c280aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c2d651640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c280b2970 0x7f3c280aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c2d651640 1 -- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c2815b5c0 msgr2=0x7f3c280a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c2d651640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c2815b5c0 0x7f3c280a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c2d651640 1 -- 172.21.15.16:0/4008199254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c14009400 con 0x7f3c28161640 2023-11-13T11:02:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.734+0000 7f3c27fff640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c2815b5c0 0x7f3c280a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.735+0000 7f3c2d651640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 0x7f3c2815fbe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c7e0 tx=0x7f3c2000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:57.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.735+0000 7f3c257fa640 1 -- 172.21.15.16:0/4008199254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000a640 con 0x7f3c28161640 2023-11-13T11:02:57.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.735+0000 7f3c257fa640 1 -- 172.21.15.16:0/4008199254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2000a7c0 con 0x7f3c28161640 2023-11-13T11:02:57.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.735+0000 7f3c257fa640 1 -- 172.21.15.16:0/4008199254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000ab70 con 0x7f3c28161640 2023-11-13T11:02:57.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.735+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2815f100 con 0x7f3c28161640 2023-11-13T11:02:57.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.735+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2815f6c0 con 0x7f3c28161640 2023-11-13T11:02:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.736+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2815f8b0 con 0x7f3c28161640 2023-11-13T11:02:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.737+0000 7f3c257fa640 1 -- 172.21.15.16:0/4008199254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c2000acd0 con 0x7f3c28161640 2023-11-13T11:02:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.738+0000 7f3c257fa640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c0c06ae40 0x7f3c0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.738+0000 7f3c257fa640 1 -- 172.21.15.16:0/4008199254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f3c20088dc0 con 0x7f3c28161640 2023-11-13T11:02:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.739+0000 7f3c2ce50640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c0c06ae40 0x7f3c0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.740+0000 7f3c2ce50640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c0c06ae40 0x7f3c0c06d300 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3c280b3b60 tx=0x7f3c1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.743+0000 7f3c257fa640 1 -- 172.21.15.16:0/4008199254 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c20053ce0 con 0x7f3c28161640 2023-11-13T11:02:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.942+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c2804e4d0 con 0x7f3c28161640 2023-11-13T11:02:57.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.943+0000 7f3c257fa640 1 -- 172.21.15.16:0/4008199254 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c20018070 con 0x7f3c28161640 2023-11-13T11:02:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:02:57.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.945+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c0c06ae40 msgr2=0x7f3c0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.945+0000 7f3c2de52640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c0c06ae40 0x7f3c0c06d300 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3c280b3b60 tx=0x7f3c1c008040 comp rx=0 tx=0).stop 2023-11-13T11:02:57.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.945+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 msgr2=0x7f3c2815fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:57.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.945+0000 7f3c2de52640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 0x7f3c2815fbe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c7e0 tx=0x7f3c2000cca0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.946+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 shutdown_connections 2023-11-13T11:02:57.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.946+0000 7f3c2de52640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3c0c06ae40 0x7f3c0c06d300 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.946+0000 7f3c2de52640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c28161640 0x7f3c2815fbe0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.946+0000 7f3c2de52640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c2815b5c0 0x7f3c280a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.946+0000 7f3c2de52640 1 --2- 172.21.15.16:0/4008199254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c280b2970 0x7f3c280aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:57.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.946+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 >> 172.21.15.16:0/4008199254 conn(0x7f3c280b0970 msgr2=0x7f3c2806bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:57.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.946+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 shutdown_connections 2023-11-13T11:02:57.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:57.946+0000 7f3c2de52640 1 -- 172.21.15.16:0/4008199254 wait complete. 2023-11-13T11:02:57.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T11:02:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.194+0000 7f3e96ce5640 1 Processor -- start 2023-11-13T11:02:58.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.195+0000 7f3e96ce5640 1 -- start start 2023-11-13T11:02:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.196+0000 7f3e96ce5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e900d1850 0x7f3e900d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.196+0000 7f3e96ce5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e900d2d00 0x7f3e900c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.196+0000 7f3e96ce5640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e900ce0e0 con 0x7f3e9015ce70 2023-11-13T11:02:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.196+0000 7f3e96ce5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e900ce3c0 con 0x7f3e900d2d00 2023-11-13T11:02:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.196+0000 7f3e96ce5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900ce6a0 con 0x7f3e900d1850 2023-11-13T11:02:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.197+0000 7f3e964e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e900d2d00 0x7f3e900c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.197+0000 7f3e964e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e900d2d00 0x7f3e900c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53026/0 (socket says 172.21.15.16:53026) 2023-11-13T11:02:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.197+0000 7f3e964e4640 1 -- 172.21.15.16:0/2818120658 learned_addr learned my addr 172.21.15.16:0/2818120658 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.197+0000 7f3e954e2640 1 --2- 172.21.15.16:0/2818120658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e900d1850 0x7f3e900d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.197+0000 7f3e964e4640 1 -- 172.21.15.16:0/2818120658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e900d1850 msgr2=0x7f3e900d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.197+0000 7f3e964e4640 1 --2- 172.21.15.16:0/2818120658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e900d1850 0x7f3e900d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.197+0000 7f3e964e4640 1 -- 172.21.15.16:0/2818120658 >> v1:172.21.15.184:6789/0 conn(0x7f3e9015ce70 legacy=0x7f3e900cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.197+0000 7f3e964e4640 1 -- 172.21.15.16:0/2818120658 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900ce980 con 0x7f3e900d2d00 2023-11-13T11:02:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.199+0000 7f3e964e4640 1 --2- 172.21.15.16:0/2818120658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 0x7f3e900c9c30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009e50 tx=0x7f3e8802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfca0e5875f5598 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.199+0000 7f3e94ce1640 1 -- 172.21.15.16:0/2818120658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8802caf0 con 0x7f3e900d2d00 2023-11-13T11:02:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.199+0000 7f3e94ce1640 1 -- 172.21.15.16:0/2818120658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8802cc50 con 0x7f3e900d2d00 2023-11-13T11:02:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.199+0000 7f3e94ce1640 1 -- 172.21.15.16:0/2818120658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e880343f0 con 0x7f3e900d2d00 2023-11-13T11:02:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.200+0000 7f3e96ce5640 1 -- 172.21.15.16:0/2818120658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 msgr2=0x7f3e900c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.200+0000 7f3e96ce5640 1 --2- 172.21.15.16:0/2818120658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 0x7f3e900c9c30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009e50 tx=0x7f3e8802bf50 comp rx=0 tx=0).stop 2023-11-13T11:02:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.200+0000 7f3e96ce5640 1 -- 172.21.15.16:0/2818120658 shutdown_connections 2023-11-13T11:02:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.200+0000 7f3e96ce5640 1 --2- 172.21.15.16:0/2818120658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 0x7f3e900c9c30 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.200+0000 7f3e96ce5640 1 --2- 172.21.15.16:0/2818120658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e900d1850 0x7f3e900d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.200+0000 7f3e96ce5640 1 -- 172.21.15.16:0/2818120658 >> 172.21.15.16:0/2818120658 conn(0x7f3e9008c580 msgr2=0x7f3e9009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.200+0000 7f3e96ce5640 1 -- 172.21.15.16:0/2818120658 shutdown_connections 2023-11-13T11:02:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.200+0000 7f3e96ce5640 1 -- 172.21.15.16:0/2818120658 wait complete. 2023-11-13T11:02:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.202+0000 7f3e96ce5640 1 Processor -- start 2023-11-13T11:02:58.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.202+0000 7f3e96ce5640 1 -- start start 2023-11-13T11:02:58.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.203+0000 7f3e96ce5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e900d1850 0x7f3e9015ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.203+0000 7f3e96ce5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 0x7f3e901e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.203+0000 7f3e96ce5640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e9015ce70 0x7f3e901e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.203+0000 7f3e96ce5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90149380 con 0x7f3e900d2d00 2023-11-13T11:02:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.203+0000 7f3e96ce5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90148d50 con 0x7f3e900d1850 2023-11-13T11:02:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.203+0000 7f3e96ce5640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901e6460 con 0x7f3e9015ce70 2023-11-13T11:02:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.203+0000 7f3e954e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 0x7f3e901e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.203+0000 7f3e954e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 0x7f3e901e58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53040/0 (socket says 172.21.15.16:53040) 2023-11-13T11:02:58.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.203+0000 7f3e954e2640 1 -- 172.21.15.16:0/513113545 learned_addr learned my addr 172.21.15.16:0/513113545 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:58.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e95ce3640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e900d1850 0x7f3e9015ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e954e2640 1 -- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e9015ce70 msgr2=0x7f3e901e5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e954e2640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e9015ce70 0x7f3e901e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e954e2640 1 -- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e900d1850 msgr2=0x7f3e9015ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e954e2640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e900d1850 0x7f3e9015ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e954e2640 1 -- 172.21.15.16:0/513113545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88009b30 con 0x7f3e900d2d00 2023-11-13T11:02:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e95ce3640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e900d1850 0x7f3e9015ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e954e2640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 0x7f3e901e58a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000d5f0 tx=0x7f3e8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e867fc640 1 -- 172.21.15.16:0/513113545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e800125f0 con 0x7f3e900d2d00 2023-11-13T11:02:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.204+0000 7f3e867fc640 1 -- 172.21.15.16:0/513113545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e80012750 con 0x7f3e900d2d00 2023-11-13T11:02:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.205+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e901e6740 con 0x7f3e900d2d00 2023-11-13T11:02:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.205+0000 7f3e867fc640 1 -- 172.21.15.16:0/513113545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80012b00 con 0x7f3e900d2d00 2023-11-13T11:02:58.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.205+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e901604c0 con 0x7f3e900d2d00 2023-11-13T11:02:58.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.206+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9015a6a0 con 0x7f3e900d2d00 2023-11-13T11:02:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.207+0000 7f3e867fc640 1 -- 172.21.15.16:0/513113545 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e80012c60 con 0x7f3e900d2d00 2023-11-13T11:02:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.208+0000 7f3e867fc640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e7406ae40 0x7f3e7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.208+0000 7f3e867fc640 1 -- 172.21.15.16:0/513113545 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f3e80089c20 con 0x7f3e900d2d00 2023-11-13T11:02:58.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.209+0000 7f3e95ce3640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e7406ae40 0x7f3e7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.210+0000 7f3e95ce3640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e7406ae40 0x7f3e7406d300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000f60 tx=0x7f3e8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:58.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.213+0000 7f3e867fc640 1 -- 172.21.15.16:0/513113545 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e80054b50 con 0x7f3e900d2d00 2023-11-13T11:02:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.407+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e9004e4d0 con 0x7f3e900d2d00 2023-11-13T11:02:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.408+0000 7f3e867fc640 1 -- 172.21.15.16:0/513113545 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e80058170 con 0x7f3e900d2d00 2023-11-13T11:02:58.413 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T11:02:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e7406ae40 msgr2=0x7f3e7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e7406ae40 0x7f3e7406d300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000f60 tx=0x7f3e8c009040 comp rx=0 tx=0).stop 2023-11-13T11:02:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 msgr2=0x7f3e901e58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 0x7f3e901e58a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000d5f0 tx=0x7f3e8000dab0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 shutdown_connections 2023-11-13T11:02:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e7406ae40 0x7f3e7406d300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e9015ce70 0x7f3e901e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e900d2d00 0x7f3e901e58a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 --2- 172.21.15.16:0/513113545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e900d1850 0x7f3e9015ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 >> 172.21.15.16:0/513113545 conn(0x7f3e9008c580 msgr2=0x7f3e9006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 shutdown_connections 2023-11-13T11:02:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.411+0000 7f3e96ce5640 1 -- 172.21.15.16:0/513113545 wait complete. 2023-11-13T11:02:58.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T11:02:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.652+0000 7fbcfa0d1640 1 Processor -- start 2023-11-13T11:02:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.652+0000 7fbcfa0d1640 1 -- start start 2023-11-13T11:02:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.653+0000 7fbcfa0d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbcf415ce70 0x7fbcf40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.653+0000 7fbcfa0d1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcf40d1850 0x7fbcf40c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.653+0000 7fbcfa0d1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf40ce0e0 con 0x7fbcf40cdbb0 2023-11-13T11:02:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.653+0000 7fbcfa0d1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf40ce3c0 con 0x7fbcf40d1850 2023-11-13T11:02:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.653+0000 7fbcfa0d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf40ce6a0 con 0x7fbcf415ce70 2023-11-13T11:02:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.654+0000 7fbcf98d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcf40d1850 0x7fbcf40c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.654+0000 7fbcf98d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcf40d1850 0x7fbcf40c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53052/0 (socket says 172.21.15.16:53052) 2023-11-13T11:02:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.654+0000 7fbcf98d0640 1 -- 172.21.15.16:0/1201393318 learned_addr learned my addr 172.21.15.16:0/1201393318 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.654+0000 7fbcf90cf640 1 --2- 172.21.15.16:0/1201393318 >> v2:172.21.15.184:3300/0 conn(0x7fbcf415ce70 0x7fbcf40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.654+0000 7fbcf98d0640 1 -- 172.21.15.16:0/1201393318 >> v2:172.21.15.184:3300/0 conn(0x7fbcf415ce70 msgr2=0x7fbcf40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.654+0000 7fbcf98d0640 1 --2- 172.21.15.16:0/1201393318 >> v2:172.21.15.184:3300/0 conn(0x7fbcf415ce70 0x7fbcf40cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.654+0000 7fbcf98d0640 1 -- 172.21.15.16:0/1201393318 >> v1:172.21.15.16:6789/0 conn(0x7fbcf40cdbb0 legacy=0x7fbcf40d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.654+0000 7fbcf98d0640 1 -- 172.21.15.16:0/1201393318 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf40ce980 con 0x7fbcf40d1850 2023-11-13T11:02:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.654+0000 7fbcf90cf640 1 --2- 172.21.15.16:0/1201393318 >> v2:172.21.15.184:3300/0 conn(0x7fbcf415ce70 0x7fbcf40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.655+0000 7fbcf98d0640 1 --2- 172.21.15.16:0/1201393318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf40d1850 0x7fbcf40c9c30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbcf408c400 tx=0x7fbcec02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea45214c9809880d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.655+0000 7fbcebfff640 1 -- 172.21.15.16:0/1201393318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec02cb10 con 0x7fbcf40d1850 2023-11-13T11:02:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcebfff640 1 -- 172.21.15.16:0/1201393318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec02cc70 con 0x7fbcf40d1850 2023-11-13T11:02:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcebfff640 1 -- 172.21.15.16:0/1201393318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec0343f0 con 0x7fbcf40d1850 2023-11-13T11:02:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1201393318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf40d1850 msgr2=0x7fbcf40c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcfa0d1640 1 --2- 172.21.15.16:0/1201393318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf40d1850 0x7fbcf40c9c30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbcf408c400 tx=0x7fbcec02bf70 comp rx=0 tx=0).stop 2023-11-13T11:02:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1201393318 shutdown_connections 2023-11-13T11:02:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcfa0d1640 1 --2- 172.21.15.16:0/1201393318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf40d1850 0x7fbcf40c9c30 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcfa0d1640 1 --2- 172.21.15.16:0/1201393318 >> v2:172.21.15.184:3300/0 conn(0x7fbcf415ce70 0x7fbcf40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1201393318 >> 172.21.15.16:0/1201393318 conn(0x7fbcf408c580 msgr2=0x7fbcf409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1201393318 shutdown_connections 2023-11-13T11:02:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.656+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1201393318 wait complete. 2023-11-13T11:02:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.658+0000 7fbcfa0d1640 1 Processor -- start 2023-11-13T11:02:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.658+0000 7fbcfa0d1640 1 -- start start 2023-11-13T11:02:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.659+0000 7fbcfa0d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf40cdbb0 0x7fbcf41f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.659+0000 7fbcfa0d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf40d1850 0x7fbcf40b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.659+0000 7fbcfa0d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf415ce70 0x7fbcf415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.659+0000 7fbcfa0d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf41f6a30 con 0x7fbcf415ce70 2023-11-13T11:02:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.659+0000 7fbcfa0d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf41f5b40 con 0x7fbcf40cdbb0 2023-11-13T11:02:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.659+0000 7fbcfa0d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf41f5e20 con 0x7fbcf40d1850 2023-11-13T11:02:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.659+0000 7fbcf98d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf415ce70 0x7fbcf415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf98d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf415ce70 0x7fbcf415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53066/0 (socket says 172.21.15.16:53066) 2023-11-13T11:02:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf98d0640 1 -- 172.21.15.16:0/1806275892 learned_addr learned my addr 172.21.15.16:0/1806275892 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf98d0640 1 -- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf40d1850 msgr2=0x7fbcf40b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf90cf640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf40cdbb0 0x7fbcf41f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf88ce640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf40d1850 0x7fbcf40b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf98d0640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf40d1850 0x7fbcf40b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf98d0640 1 -- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf40cdbb0 msgr2=0x7fbcf41f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf98d0640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf40cdbb0 0x7fbcf41f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf98d0640 1 -- 172.21.15.16:0/1806275892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec009b30 con 0x7fbcf415ce70 2023-11-13T11:02:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbcf98d0640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf415ce70 0x7fbcf415a160 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbcec036040 tx=0x7fbcec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.660+0000 7fbce9ffb640 1 -- 172.21.15.16:0/1806275892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec009510 con 0x7fbcf415ce70 2023-11-13T11:02:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.661+0000 7fbce9ffb640 1 -- 172.21.15.16:0/1806275892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec03e450 con 0x7fbcf415ce70 2023-11-13T11:02:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.661+0000 7fbce9ffb640 1 -- 172.21.15.16:0/1806275892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec03e800 con 0x7fbcf415ce70 2023-11-13T11:02:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.661+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf41f6100 con 0x7fbcf415ce70 2023-11-13T11:02:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.661+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf41f66c0 con 0x7fbcf415ce70 2023-11-13T11:02:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.662+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf409c810 con 0x7fbcf415ce70 2023-11-13T11:02:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.663+0000 7fbce9ffb640 1 -- 172.21.15.16:0/1806275892 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcec03e960 con 0x7fbcf415ce70 2023-11-13T11:02:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.664+0000 7fbce9ffb640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcd806ae70 0x7fbcd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.664+0000 7fbce9ffb640 1 -- 172.21.15.16:0/1806275892 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fbcec0ac5c0 con 0x7fbcf415ce70 2023-11-13T11:02:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.664+0000 7fbcf90cf640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcd806ae70 0x7fbcd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.665+0000 7fbcf90cf640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcd806ae70 0x7fbcd806d330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b840 tx=0x7fbcf0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.668+0000 7fbce9ffb640 1 -- 172.21.15.16:0/1806275892 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbcf409c810 con 0x7fbcf415ce70 2023-11-13T11:02:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.862+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbcf409c810 con 0x7fbcf415ce70 2023-11-13T11:02:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.863+0000 7fbce9ffb640 1 -- 172.21.15.16:0/1806275892 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbcf409c810 con 0x7fbcf415ce70 2023-11-13T11:02:58.869 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T11:02:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.865+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcd806ae70 msgr2=0x7fbcd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.865+0000 7fbcfa0d1640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcd806ae70 0x7fbcd806d330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b840 tx=0x7fbcf0008040 comp rx=0 tx=0).stop 2023-11-13T11:02:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.865+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf415ce70 msgr2=0x7fbcf415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.865+0000 7fbcfa0d1640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf415ce70 0x7fbcf415a160 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbcec036040 tx=0x7fbcec004a40 comp rx=0 tx=0).stop 2023-11-13T11:02:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.866+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 shutdown_connections 2023-11-13T11:02:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.866+0000 7fbcfa0d1640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcd806ae70 0x7fbcd806d330 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.866+0000 7fbcfa0d1640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf415ce70 0x7fbcf415a160 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.866+0000 7fbcfa0d1640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf40d1850 0x7fbcf40b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.866+0000 7fbcfa0d1640 1 --2- 172.21.15.16:0/1806275892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf40cdbb0 0x7fbcf41f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.866+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 >> 172.21.15.16:0/1806275892 conn(0x7fbcf408c580 msgr2=0x7fbcf415f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.866+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 shutdown_connections 2023-11-13T11:02:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:58.866+0000 7fbcfa0d1640 1 -- 172.21.15.16:0/1806275892 wait complete. 2023-11-13T11:02:58.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T11:02:59.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.093+0000 7f674857c640 1 Processor -- start 2023-11-13T11:02:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.094+0000 7f674857c640 1 -- start start 2023-11-13T11:02:59.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.095+0000 7f674857c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f67400b1c70 0x7f67400cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:59.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.095+0000 7f674857c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67400cd9c0 0x7f67400ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:59.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.095+0000 7f674857c640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674006c710 con 0x7f674015de20 2023-11-13T11:02:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.095+0000 7f674857c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674006b860 con 0x7f67400cd9c0 2023-11-13T11:02:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.095+0000 7f674857c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674006bb00 con 0x7f67400b1c70 2023-11-13T11:02:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.096+0000 7f6746b19640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67400cd9c0 0x7f67400ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.096+0000 7f6746b19640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67400cd9c0 0x7f67400ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53070/0 (socket says 172.21.15.16:53070) 2023-11-13T11:02:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.096+0000 7f6746b19640 1 -- 172.21.15.16:0/1389066934 learned_addr learned my addr 172.21.15.16:0/1389066934 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.096+0000 7f6745b17640 1 --2- 172.21.15.16:0/1389066934 >> v2:172.21.15.184:3300/0 conn(0x7f67400b1c70 0x7f67400cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.096+0000 7f6746b19640 1 -- 172.21.15.16:0/1389066934 >> v2:172.21.15.184:3300/0 conn(0x7f67400b1c70 msgr2=0x7f67400cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.096+0000 7f6746b19640 1 --2- 172.21.15.16:0/1389066934 >> v2:172.21.15.184:3300/0 conn(0x7f67400b1c70 0x7f67400cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.096+0000 7f6746b19640 1 -- 172.21.15.16:0/1389066934 >> v1:172.21.15.184:6789/0 conn(0x7f674015de20 legacy=0x7f674015e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:02:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.096+0000 7f6746b19640 1 -- 172.21.15.16:0/1389066934 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674006bde0 con 0x7f67400cd9c0 2023-11-13T11:02:59.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.096+0000 7f6745b17640 1 --2- 172.21.15.16:0/1389066934 >> v2:172.21.15.184:3300/0 conn(0x7f67400b1c70 0x7f67400cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:02:59.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.097+0000 7f6746b19640 1 --2- 172.21.15.16:0/1389066934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67400cd9c0 0x7f67400ce6d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6740071000 tx=0x7f673802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69657d8566934d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:59.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f6745316640 1 -- 172.21.15.16:0/1389066934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380317e0 con 0x7f67400cd9c0 2023-11-13T11:02:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f6745316640 1 -- 172.21.15.16:0/1389066934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6738031940 con 0x7f67400cd9c0 2023-11-13T11:02:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f6745316640 1 -- 172.21.15.16:0/1389066934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738031cd0 con 0x7f67400cd9c0 2023-11-13T11:02:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f674857c640 1 -- 172.21.15.16:0/1389066934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67400cd9c0 msgr2=0x7f67400ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f674857c640 1 --2- 172.21.15.16:0/1389066934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67400cd9c0 0x7f67400ce6d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6740071000 tx=0x7f673802b880 comp rx=0 tx=0).stop 2023-11-13T11:02:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f674857c640 1 -- 172.21.15.16:0/1389066934 shutdown_connections 2023-11-13T11:02:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f674857c640 1 --2- 172.21.15.16:0/1389066934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67400cd9c0 0x7f67400ce6d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f674857c640 1 --2- 172.21.15.16:0/1389066934 >> v2:172.21.15.184:3300/0 conn(0x7f67400b1c70 0x7f67400cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f674857c640 1 -- 172.21.15.16:0/1389066934 >> 172.21.15.16:0/1389066934 conn(0x7f67400b2630 msgr2=0x7f67400b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f674857c640 1 -- 172.21.15.16:0/1389066934 shutdown_connections 2023-11-13T11:02:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.098+0000 7f674857c640 1 -- 172.21.15.16:0/1389066934 wait complete. 2023-11-13T11:02:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.100+0000 7f674857c640 1 Processor -- start 2023-11-13T11:02:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.100+0000 7f674857c640 1 -- start start 2023-11-13T11:02:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.101+0000 7f674857c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67400b1c70 0x7f6740152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:59.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.101+0000 7f674857c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67400cd9c0 0x7f6740150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.101+0000 7f674857c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674015de20 0x7f6740150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.101+0000 7f674857c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6740151ee0 con 0x7f674015de20 2023-11-13T11:02:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.101+0000 7f674857c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67401528b0 con 0x7f67400cd9c0 2023-11-13T11:02:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.101+0000 7f674857c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67401505d0 con 0x7f67400b1c70 2023-11-13T11:02:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6746b19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674015de20 0x7f6740150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6746b19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674015de20 0x7f6740150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53086/0 (socket says 172.21.15.16:53086) 2023-11-13T11:02:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6746b19640 1 -- 172.21.15.16:0/1460502820 learned_addr learned my addr 172.21.15.16:0/1460502820 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:02:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6745b17640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67400cd9c0 0x7f6740150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6746b19640 1 -- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67400b1c70 msgr2=0x7f6740152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6746318640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67400b1c70 0x7f6740152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6746b19640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67400b1c70 0x7f6740152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6746b19640 1 -- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67400cd9c0 msgr2=0x7f6740150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6746b19640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67400cd9c0 0x7f6740150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.102+0000 7f6746b19640 1 -- 172.21.15.16:0/1460502820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67380094d0 con 0x7f674015de20 2023-11-13T11:02:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.103+0000 7f6746b19640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674015de20 0x7f6740150960 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f673802b8c0 tx=0x7f67380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.103+0000 7f673effd640 1 -- 172.21.15.16:0/1460502820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738032940 con 0x7f674015de20 2023-11-13T11:02:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.103+0000 7f673effd640 1 -- 172.21.15.16:0/1460502820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6738032ac0 con 0x7f674015de20 2023-11-13T11:02:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.103+0000 7f673effd640 1 -- 172.21.15.16:0/1460502820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738032e70 con 0x7f674015de20 2023-11-13T11:02:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.103+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67401512d0 con 0x7f674015de20 2023-11-13T11:02:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.103+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6740151890 con 0x7f674015de20 2023-11-13T11:02:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.104+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6740152090 con 0x7f674015de20 2023-11-13T11:02:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.105+0000 7f673effd640 1 -- 172.21.15.16:0/1460502820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f67380453f0 con 0x7f674015de20 2023-11-13T11:02:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.106+0000 7f673effd640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f672c06ae40 0x7f672c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:02:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.106+0000 7f673effd640 1 -- 172.21.15.16:0/1460502820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f67380abca0 con 0x7f674015de20 2023-11-13T11:02:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.107+0000 7f6746318640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f672c06ae40 0x7f672c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:02:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.108+0000 7f6746318640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f672c06ae40 0x7f672c06d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6730007dc0 tx=0x7f67300072f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:02:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.111+0000 7f673effd640 1 -- 172.21.15.16:0/1460502820 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6740152090 con 0x7f674015de20 2023-11-13T11:02:59.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.311+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6740152090 con 0x7f674015de20 2023-11-13T11:02:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.312+0000 7f673effd640 1 -- 172.21.15.16:0/1460502820 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6740152090 con 0x7f674015de20 2023-11-13T11:02:59.317 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T11:02:59.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f672c06ae40 msgr2=0x7f672c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f672c06ae40 0x7f672c06d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6730007dc0 tx=0x7f67300072f0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674015de20 msgr2=0x7f6740150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:02:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674015de20 0x7f6740150960 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f673802b8c0 tx=0x7f67380046c0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 shutdown_connections 2023-11-13T11:02:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f672c06ae40 0x7f672c06d300 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f674015de20 0x7f6740150960 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67400cd9c0 0x7f6740150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 --2- 172.21.15.16:0/1460502820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67400b1c70 0x7f6740152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:02:59.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.314+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 >> 172.21.15.16:0/1460502820 conn(0x7f67400b2630 msgr2=0x7f67400ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:02:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.315+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 shutdown_connections 2023-11-13T11:02:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:02:59.315+0000 7f674857c640 1 -- 172.21.15.16:0/1460502820 wait complete. 2023-11-13T11:02:59.337 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-13T11:02:59.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:59.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-13T11:02:59.427 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[111941]: fuse finished with error 0 and tester_r 0 2023-11-13T11:02:59.458 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:02:59.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:59.459 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-13T11:02:59.650 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:02:59.651 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:02:59.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:02:59.692 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:02:59.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:59.693 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-13T11:02:59.775 INFO:teuthology.orchestra.run.smithi184.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-13T11:02:59.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:02:59.777 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:02:59.778 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:02:59.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:02:59.829 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:02:59.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:59.830 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-13T11:02:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-13T11:02:59.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:02:59.932 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:02:59.932 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:02:59.992 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:02:59.993 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:02:59.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:02:59.993 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-13T11:03:00.051 INFO:teuthology.orchestra.run.smithi184.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-13T11:03:00.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:03:00.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.276+0000 7fcd8967e640 1 Processor -- start 2023-11-13T11:03:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.276+0000 7fcd8967e640 1 -- start start 2023-11-13T11:03:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.278+0000 7fcd8967e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd840cb190 0x7fcd8415b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.278+0000 7fcd8967e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd840716b0 0x7fcd8406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.278+0000 7fcd8967e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd84158c20 con 0x7fcd8406a190 2023-11-13T11:03:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.278+0000 7fcd8967e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd840b1830 con 0x7fcd840716b0 2023-11-13T11:03:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.278+0000 7fcd8967e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8415b780 con 0x7fcd840cb190 2023-11-13T11:03:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.278+0000 7fcd88e7d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd8406a190 0x7fcd840b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41390/0 (socket says 172.21.15.16:41390) 2023-11-13T11:03:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.279+0000 7fcd88e7d640 1 -- 172.21.15.16:0/4122350412 learned_addr learned my addr 172.21.15.16:0/4122350412 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.279+0000 7fcd7bfff640 1 --2- 172.21.15.16:0/4122350412 >> v2:172.21.15.16:3300/0 conn(0x7fcd840716b0 0x7fcd8406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.279+0000 7fcd83fff640 1 --2- 172.21.15.16:0/4122350412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd840cb190 0x7fcd8415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.279+0000 7fcd837fe640 1 -- 172.21.15.16:0/4122350412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313325493 0 0) 0x7fcd84158c20 con 0x7fcd8406a190 2023-11-13T11:03:00.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.279+0000 7fcd837fe640 1 -- 172.21.15.16:0/4122350412 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd70003520 con 0x7fcd8406a190 2023-11-13T11:03:00.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.279+0000 7fcd7bfff640 1 -- 172.21.15.16:0/4122350412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd840cb190 msgr2=0x7fcd8415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:00.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.279+0000 7fcd7bfff640 1 --2- 172.21.15.16:0/4122350412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd840cb190 0x7fcd8415b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.279+0000 7fcd7bfff640 1 -- 172.21.15.16:0/4122350412 >> v1:172.21.15.16:6789/0 conn(0x7fcd8406a190 legacy=0x7fcd840b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.279+0000 7fcd7bfff640 1 -- 172.21.15.16:0/4122350412 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8415ba60 con 0x7fcd840716b0 2023-11-13T11:03:00.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.281+0000 7fcd7bfff640 1 --2- 172.21.15.16:0/4122350412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 0x7fcd8406f970 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0097a0 tx=0x7fcd6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575082d6f03b374f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:00.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.281+0000 7fcd837fe640 1 -- 172.21.15.16:0/4122350412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c031830 con 0x7fcd840716b0 2023-11-13T11:03:00.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.281+0000 7fcd837fe640 1 -- 172.21.15.16:0/4122350412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c031990 con 0x7fcd840716b0 2023-11-13T11:03:00.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.281+0000 7fcd837fe640 1 -- 172.21.15.16:0/4122350412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c031d20 con 0x7fcd840716b0 2023-11-13T11:03:00.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.282+0000 7fcd8967e640 1 -- 172.21.15.16:0/4122350412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 msgr2=0x7fcd8406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.282+0000 7fcd8967e640 1 --2- 172.21.15.16:0/4122350412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 0x7fcd8406f970 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0097a0 tx=0x7fcd6c02b8d0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.282+0000 7fcd8967e640 1 -- 172.21.15.16:0/4122350412 shutdown_connections 2023-11-13T11:03:00.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.282+0000 7fcd8967e640 1 --2- 172.21.15.16:0/4122350412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 0x7fcd8406f970 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.282+0000 7fcd8967e640 1 --2- 172.21.15.16:0/4122350412 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd840cb190 0x7fcd8415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.282+0000 7fcd8967e640 1 -- 172.21.15.16:0/4122350412 >> 172.21.15.16:0/4122350412 conn(0x7fcd840b04a0 msgr2=0x7fcd840b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:00.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.282+0000 7fcd8967e640 1 -- 172.21.15.16:0/4122350412 shutdown_connections 2023-11-13T11:03:00.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.282+0000 7fcd8967e640 1 -- 172.21.15.16:0/4122350412 wait complete. 2023-11-13T11:03:00.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.284+0000 7fcd8967e640 1 Processor -- start 2023-11-13T11:03:00.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.284+0000 7fcd8967e640 1 -- start start 2023-11-13T11:03:00.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.285+0000 7fcd8967e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8406a190 0x7fcd840a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.285+0000 7fcd8967e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 0x7fcd840c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.285+0000 7fcd8967e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd840cb190 0x7fcd841f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.285+0000 7fcd8967e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840c9600 con 0x7fcd840716b0 2023-11-13T11:03:00.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.285+0000 7fcd8967e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840c8d60 con 0x7fcd840cb190 2023-11-13T11:03:00.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.285+0000 7fcd8967e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840c9040 con 0x7fcd8406a190 2023-11-13T11:03:00.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.285+0000 7fcd7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 0x7fcd840c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.285+0000 7fcd7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 0x7fcd840c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53098/0 (socket says 172.21.15.16:53098) 2023-11-13T11:03:00.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.285+0000 7fcd7bfff640 1 -- 172.21.15.16:0/2036736694 learned_addr learned my addr 172.21.15.16:0/2036736694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:00.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd83fff640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8406a190 0x7fcd840a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd88e7d640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd840cb190 0x7fcd841f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd7bfff640 1 -- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8406a190 msgr2=0x7fcd840a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd7bfff640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8406a190 0x7fcd840a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd7bfff640 1 -- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd840cb190 msgr2=0x7fcd841f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd7bfff640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd840cb190 0x7fcd841f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd7bfff640 1 -- 172.21.15.16:0/2036736694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c009400 con 0x7fcd840716b0 2023-11-13T11:03:00.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd88e7d640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd840cb190 0x7fcd841f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:00.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd7bfff640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 0x7fcd840c97c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c02bee0 tx=0x7fcd6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:00.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.286+0000 7fcd817fa640 1 -- 172.21.15.16:0/2036736694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c03a630 con 0x7fcd840716b0 2023-11-13T11:03:00.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.287+0000 7fcd817fa640 1 -- 172.21.15.16:0/2036736694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c03a7b0 con 0x7fcd840716b0 2023-11-13T11:03:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.287+0000 7fcd817fa640 1 -- 172.21.15.16:0/2036736694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c03ab60 con 0x7fcd840716b0 2023-11-13T11:03:00.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.287+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd840c9320 con 0x7fcd840716b0 2023-11-13T11:03:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.287+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd840a91c0 con 0x7fcd840716b0 2023-11-13T11:03:00.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.288+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd840a9350 con 0x7fcd840716b0 2023-11-13T11:03:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.289+0000 7fcd817fa640 1 -- 172.21.15.16:0/2036736694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd6c03acc0 con 0x7fcd840716b0 2023-11-13T11:03:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.290+0000 7fcd817fa640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd6406ae40 0x7fcd6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.290+0000 7fcd817fa640 1 -- 172.21.15.16:0/2036736694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fcd6c0ab050 con 0x7fcd840716b0 2023-11-13T11:03:00.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.291+0000 7fcd83fff640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd6406ae40 0x7fcd6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.292+0000 7fcd83fff640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd6406ae40 0x7fcd6406d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcd840ca5e0 tx=0x7fcd74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:00.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.295+0000 7fcd817fa640 1 -- 172.21.15.16:0/2036736694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd6c076010 con 0x7fcd840716b0 2023-11-13T11:03:00.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.533+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd8406bdb0 con 0x7fcd840716b0 2023-11-13T11:03:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.534+0000 7fcd817fa640 1 -- 172.21.15.16:0/2036736694 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fcd8406bdb0 con 0x7fcd840716b0 2023-11-13T11:03:00.540 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:00.540 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:59.012901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:03:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 32 2023-11-13T11:03:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd6406ae40 msgr2=0x7fcd6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd6406ae40 0x7fcd6406d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcd840ca5e0 tx=0x7fcd74008040 comp rx=0 tx=0).stop 2023-11-13T11:03:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 msgr2=0x7fcd840c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 0x7fcd840c97c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c02bee0 tx=0x7fcd6c0046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 shutdown_connections 2023-11-13T11:03:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd6406ae40 0x7fcd6406d300 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd840cb190 0x7fcd841f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840716b0 0x7fcd840c97c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 --2- 172.21.15.16:0/2036736694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd8406a190 0x7fcd840a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 >> 172.21.15.16:0/2036736694 conn(0x7fcd840b04a0 msgr2=0x7fcd841580d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:00.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 shutdown_connections 2023-11-13T11:03:00.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.537+0000 7fcd8967e640 1 -- 172.21.15.16:0/2036736694 wait complete. 2023-11-13T11:03:00.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.782+0000 7f0baa63f640 1 Processor -- start 2023-11-13T11:03:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.782+0000 7f0baa63f640 1 -- start start 2023-11-13T11:03:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.783+0000 7f0baa63f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40cfeb0 0x7f0ba40d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.783+0000 7f0baa63f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba40cebd0 con 0x7f0ba40cd950 2023-11-13T11:03:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.783+0000 7f0baa63f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba40cdfc0 con 0x7f0ba415d150 2023-11-13T11:03:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.783+0000 7f0baa63f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40d3ac0 con 0x7f0ba40cfeb0 2023-11-13T11:03:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.784+0000 7f0ba37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0ba40cd950 0x7f0ba40cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41400/0 (socket says 172.21.15.16:41400) 2023-11-13T11:03:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.784+0000 7f0ba37fe640 1 -- 172.21.15.16:0/2882629307 learned_addr learned my addr 172.21.15.16:0/2882629307 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.784+0000 7f0ba8bdc640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40cfeb0 0x7f0ba40d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.784+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094176221 0 0) 0x7f0ba40cebd0 con 0x7f0ba40cd950 2023-11-13T11:03:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.784+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b8c003520 con 0x7f0ba40cd950 2023-11-13T11:03:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189551044 0 0) 0x7f0b8c003520 con 0x7f0ba40cd950 2023-11-13T11:03:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40cfeb0 msgr2=0x7f0ba40d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40cfeb0 0x7f0ba40d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 >> v1:172.21.15.184:6789/0 conn(0x7f0ba415d150 legacy=0x7f0ba4161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40ce2a0 con 0x7f0ba40cd950 2023-11-13T11:03:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f0b98001670 con 0x7f0ba40cd950 2023-11-13T11:03:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 >> v1:172.21.15.16:6789/0 conn(0x7f0ba40cd950 legacy=0x7f0ba40cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b8c0039a0 0x7f0b8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40cfeb0 0x7f0b8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0b8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c00ba30 con 0x7f0b8c004d00 2023-11-13T11:03:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c00bb70 con 0x7f0b8c0039a0 2023-11-13T11:03:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c00bd30 con 0x7f0ba40cfeb0 2023-11-13T11:03:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.785+0000 7f0ba8bdc640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0b8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.786+0000 7f0ba37fe640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40cfeb0 0x7f0b8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.786+0000 7f0ba3fff640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b8c0039a0 0x7f0b8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.786+0000 7f0ba8bdc640 1 -- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40cfeb0 msgr2=0x7f0b8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.786+0000 7f0ba8bdc640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40cfeb0 0x7f0b8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.786+0000 7f0ba8bdc640 1 -- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b8c0039a0 msgr2=0x7f0b8c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.786+0000 7f0ba8bdc640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b8c0039a0 0x7f0b8c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.786+0000 7f0ba8bdc640 1 -- 172.21.15.16:0/2882629307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c003520 con 0x7f0b8c004d00 2023-11-13T11:03:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.786+0000 7f0ba3fff640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b8c0039a0 0x7f0b8c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.787+0000 7f0ba8bdc640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0b8c005180 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40d02c0 tx=0x7f0b8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc87f0960f9630e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.787+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c02c660 con 0x7f0b8c004d00 2023-11-13T11:03:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.787+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c02c7c0 con 0x7f0b8c004d00 2023-11-13T11:03:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.787+0000 7f0ba2ffd640 1 -- 172.21.15.16:0/2882629307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c02cb50 con 0x7f0b8c004d00 2023-11-13T11:03:00.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.788+0000 7f0baa63f640 1 -- 172.21.15.16:0/2882629307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 msgr2=0x7f0b8c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.788+0000 7f0baa63f640 1 --2- 172.21.15.16:0/2882629307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0b8c005180 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40d02c0 tx=0x7f0b8c0057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.788+0000 7f0ba8bdc640 1 -- 172.21.15.16:0/2882629307 reap_dead start 2023-11-13T11:03:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.788+0000 7f0baa63f640 1 -- 172.21.15.16:0/2882629307 shutdown_connections 2023-11-13T11:03:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.788+0000 7f0baa63f640 1 -- 172.21.15.16:0/2882629307 >> 172.21.15.16:0/2882629307 conn(0x7f0ba409c030 msgr2=0x7f0ba40d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.788+0000 7f0baa63f640 1 -- 172.21.15.16:0/2882629307 shutdown_connections 2023-11-13T11:03:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.788+0000 7f0baa63f640 1 -- 172.21.15.16:0/2882629307 wait complete. 2023-11-13T11:03:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.790+0000 7f0baa63f640 1 Processor -- start 2023-11-13T11:03:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.790+0000 7f0baa63f640 1 -- start start 2023-11-13T11:03:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.791+0000 7f0baa63f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0ba41f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.791+0000 7f0baa63f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40b4650 0x7f0ba40b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.791+0000 7f0baa63f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba40c8740 0x7f0ba40c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.791+0000 7f0baa63f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40b5010 con 0x7f0b8c004d00 2023-11-13T11:03:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.791+0000 7f0baa63f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40b7a70 con 0x7f0ba40c8740 2023-11-13T11:03:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.791+0000 7f0baa63f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba41f2470 con 0x7f0ba40b4650 2023-11-13T11:03:00.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.791+0000 7f0ba3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0ba41f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.791+0000 7f0ba3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0ba41f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53122/0 (socket says 172.21.15.16:53122) 2023-11-13T11:03:00.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.791+0000 7f0ba3fff640 1 -- 172.21.15.16:0/38148726 learned_addr learned my addr 172.21.15.16:0/38148726 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:00.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba8bdc640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba40c8740 0x7f0ba40c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba37fe640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40b4650 0x7f0ba40b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba3fff640 1 -- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40b4650 msgr2=0x7f0ba40b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba3fff640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40b4650 0x7f0ba40b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba3fff640 1 -- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba40c8740 msgr2=0x7f0ba40c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba3fff640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba40c8740 0x7f0ba40c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba3fff640 1 -- 172.21.15.16:0/38148726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40b69d0 con 0x7f0b8c004d00 2023-11-13T11:03:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba8bdc640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba40c8740 0x7f0ba40c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba3fff640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0ba41f1f30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400db10 tx=0x7f0b940020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba0ff9640 1 -- 172.21.15.16:0/38148726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94003da0 con 0x7f0b8c004d00 2023-11-13T11:03:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba0ff9640 1 -- 172.21.15.16:0/38148726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b94006b50 con 0x7f0b8c004d00 2023-11-13T11:03:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0ba0ff9640 1 -- 172.21.15.16:0/38148726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400b740 con 0x7f0b8c004d00 2023-11-13T11:03:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.792+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba40b6cb0 con 0x7f0b8c004d00 2023-11-13T11:03:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.793+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba40b7270 con 0x7f0b8c004d00 2023-11-13T11:03:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.794+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba4067090 con 0x7f0b8c004d00 2023-11-13T11:03:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.795+0000 7f0ba0ff9640 1 -- 172.21.15.16:0/38148726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b9400b8a0 con 0x7f0b8c004d00 2023-11-13T11:03:00.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.796+0000 7f0ba0ff9640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b9006ae40 0x7f0b9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:00.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.796+0000 7f0ba0ff9640 1 -- 172.21.15.16:0/38148726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f0b94088d10 con 0x7f0b8c004d00 2023-11-13T11:03:00.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.797+0000 7f0ba37fe640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b9006ae40 0x7f0b9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.798+0000 7f0ba37fe640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b9006ae40 0x7f0b9006d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001200 tx=0x7f0b98001150 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.801+0000 7f0ba0ff9640 1 -- 172.21.15.16:0/38148726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b94053cf0 con 0x7f0b8c004d00 2023-11-13T11:03:01.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.996+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ba40cf940 con 0x7f0b8c004d00 2023-11-13T11:03:01.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:00.999+0000 7f0ba0ff9640 1 -- 172.21.15.16:0/38148726 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16821 (secure 0 0 0) 0x7f0b94018090 con 0x7f0b8c004d00 2023-11-13T11:03:01.005 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:02:08.953574+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:01:36.305098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-13T11:01:37.461852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.4","mappings":[{"from":2,"to":5}]},{"pgid":"8.20","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:01.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b9006ae40 msgr2=0x7f0b9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b9006ae40 0x7f0b9006d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001200 tx=0x7f0b98001150 comp rx=0 tx=0).stop 2023-11-13T11:03:01.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 msgr2=0x7f0ba41f1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0ba41f1f30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400db10 tx=0x7f0b940020c0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 shutdown_connections 2023-11-13T11:03:01.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b9006ae40 0x7f0b9006d300 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ba40c8740 0x7f0ba40c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ba40b4650 0x7f0ba40b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 --2- 172.21.15.16:0/38148726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c004d00 0x7f0ba41f1f30 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 >> 172.21.15.16:0/38148726 conn(0x7f0ba409c030 msgr2=0x7f0ba40d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:01.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 shutdown_connections 2023-11-13T11:03:01.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.002+0000 7f0baa63f640 1 -- 172.21.15.16:0/38148726 wait complete. 2023-11-13T11:03:01.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T11:01:36.305098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T11:01:37.461852+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:03:01.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:03:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.240+0000 7fa60a8d7640 1 Processor -- start 2023-11-13T11:03:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.241+0000 7fa60a8d7640 1 -- start start 2023-11-13T11:03:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.242+0000 7fa60a8d7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa60415ce70 0x7fa6040cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.242+0000 7fa60a8d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6040cdbb0 0x7fa6040d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.242+0000 7fa60a8d7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040cac40 con 0x7fa6040d1850 2023-11-13T11:03:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.242+0000 7fa60a8d7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6040ce0e0 con 0x7fa6040cdbb0 2023-11-13T11:03:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.242+0000 7fa60a8d7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040ce3c0 con 0x7fa60415ce70 2023-11-13T11:03:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa6090d4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6040cdbb0 0x7fa6040d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa6090d4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6040cdbb0 0x7fa6040d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53132/0 (socket says 172.21.15.16:53132) 2023-11-13T11:03:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa6090d4640 1 -- 172.21.15.16:0/3622816251 learned_addr learned my addr 172.21.15.16:0/3622816251 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa60a0d6640 1 --1- 172.21.15.16:0/3622816251 >> v1:172.21.15.16:6789/0 conn(0x7fa6040d1850 0x7fa6040cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41416/0 (socket says 172.21.15.16:41416) 2023-11-13T11:03:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa6098d5640 1 --2- 172.21.15.16:0/3622816251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa60415ce70 0x7fa6040cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa6090d4640 1 -- 172.21.15.16:0/3622816251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa60415ce70 msgr2=0x7fa6040cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa6090d4640 1 --2- 172.21.15.16:0/3622816251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa60415ce70 0x7fa6040cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa6090d4640 1 -- 172.21.15.16:0/3622816251 >> v1:172.21.15.16:6789/0 conn(0x7fa6040d1850 legacy=0x7fa6040cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa6090d4640 1 -- 172.21.15.16:0/3622816251 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040ce6a0 con 0x7fa6040cdbb0 2023-11-13T11:03:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.243+0000 7fa6088d3640 1 -- 172.21.15.16:0/3622816251 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608889946 0 0) 0x7fa6040cac40 con 0x7fa6040d1850 2023-11-13T11:03:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.245+0000 7fa6090d4640 1 --2- 172.21.15.16:0/3622816251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 0x7fa6040d27c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40098a0 tx=0x7fa5f402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce962b1d0f7bb1c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.245+0000 7fa6088d3640 1 -- 172.21.15.16:0/3622816251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4030840 con 0x7fa6040cdbb0 2023-11-13T11:03:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.245+0000 7fa6088d3640 1 -- 172.21.15.16:0/3622816251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f40309a0 con 0x7fa6040cdbb0 2023-11-13T11:03:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.245+0000 7fa6088d3640 1 -- 172.21.15.16:0/3622816251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4030d30 con 0x7fa6040cdbb0 2023-11-13T11:03:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.245+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3622816251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 msgr2=0x7fa6040d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.245+0000 7fa60a8d7640 1 --2- 172.21.15.16:0/3622816251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 0x7fa6040d27c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40098a0 tx=0x7fa5f402b980 comp rx=0 tx=0).stop 2023-11-13T11:03:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.246+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3622816251 shutdown_connections 2023-11-13T11:03:01.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.246+0000 7fa60a8d7640 1 --2- 172.21.15.16:0/3622816251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 0x7fa6040d27c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.246+0000 7fa60a8d7640 1 --2- 172.21.15.16:0/3622816251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa60415ce70 0x7fa6040cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.246+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3622816251 >> 172.21.15.16:0/3622816251 conn(0x7fa60408c580 msgr2=0x7fa60409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:01.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.246+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3622816251 shutdown_connections 2023-11-13T11:03:01.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.246+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3622816251 wait complete. 2023-11-13T11:03:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.247+0000 7fa60a8d7640 1 Processor -- start 2023-11-13T11:03:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.248+0000 7fa60a8d7640 1 -- start start 2023-11-13T11:03:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.248+0000 7fa60a8d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 0x7fa604151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.248+0000 7fa60a8d7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6040d1850 0x7fa6040b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa60a8d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa60415ce70 0x7fa6040c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa60a8d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040c8660 con 0x7fa6040cdbb0 2023-11-13T11:03:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa60a8d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041523e0 con 0x7fa60415ce70 2023-11-13T11:03:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa60a8d7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040c79e0 con 0x7fa6040d1850 2023-11-13T11:03:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa6098d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 0x7fa604151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa6098d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 0x7fa604151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53138/0 (socket says 172.21.15.16:53138) 2023-11-13T11:03:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa6098d5640 1 -- 172.21.15.16:0/3366902346 learned_addr learned my addr 172.21.15.16:0/3366902346 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa60a0d6640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa60415ce70 0x7fa6040c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa6090d4640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6040d1850 0x7fa6040b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa6098d5640 1 -- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6040d1850 msgr2=0x7fa6040b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa6098d5640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6040d1850 0x7fa6040b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa6098d5640 1 -- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa60415ce70 msgr2=0x7fa6040c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa6098d5640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa60415ce70 0x7fa6040c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa6098d5640 1 -- 172.21.15.16:0/3366902346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40094d0 con 0x7fa6040cdbb0 2023-11-13T11:03:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.249+0000 7fa60a0d6640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa60415ce70 0x7fa6040c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.250+0000 7fa6098d5640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 0x7fa604151ea0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d5f0 tx=0x7fa60000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.250+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/3366902346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6000125f0 con 0x7fa6040cdbb0 2023-11-13T11:03:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.250+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/3366902346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa600012750 con 0x7fa6040cdbb0 2023-11-13T11:03:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.250+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/3366902346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600012b00 con 0x7fa6040cdbb0 2023-11-13T11:03:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.250+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6040c7cc0 con 0x7fa6040cdbb0 2023-11-13T11:03:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.250+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6040c8280 con 0x7fa6040cdbb0 2023-11-13T11:03:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.251+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6040c88d0 con 0x7fa6040cdbb0 2023-11-13T11:03:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.252+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/3366902346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa600012c60 con 0x7fa6040cdbb0 2023-11-13T11:03:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.253+0000 7fa5fa7fc640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa5e806ae40 0x7fa5e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.253+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/3366902346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fa600089c70 con 0x7fa6040cdbb0 2023-11-13T11:03:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.254+0000 7fa6090d4640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa5e806ae40 0x7fa5e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.255+0000 7fa6090d4640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa5e806ae40 0x7fa5e806d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4036000 tx=0x7fa5f403b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.258+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/3366902346 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa600054b90 con 0x7fa6040cdbb0 2023-11-13T11:03:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.449+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6040675b0 con 0x7fa6040cdbb0 2023-11-13T11:03:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.450+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/3366902346 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6000581b0 con 0x7fa6040cdbb0 2023-11-13T11:03:01.455 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:03:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa5e806ae40 msgr2=0x7fa5e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa5e806ae40 0x7fa5e806d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4036000 tx=0x7fa5f403b040 comp rx=0 tx=0).stop 2023-11-13T11:03:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 msgr2=0x7fa604151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 0x7fa604151ea0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d5f0 tx=0x7fa60000dab0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 shutdown_connections 2023-11-13T11:03:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa5e806ae40 0x7fa5e806d300 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa60415ce70 0x7fa6040c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6040d1850 0x7fa6040b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 --2- 172.21.15.16:0/3366902346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6040cdbb0 0x7fa604151ea0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 >> 172.21.15.16:0/3366902346 conn(0x7fa60408c580 msgr2=0x7fa6040cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 shutdown_connections 2023-11-13T11:03:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.453+0000 7fa60a8d7640 1 -- 172.21.15.16:0/3366902346 wait complete. 2023-11-13T11:03:01.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T11:03:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.699+0000 7fa2b4f0a640 1 Processor -- start 2023-11-13T11:03:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.699+0000 7fa2b4f0a640 1 -- start start 2023-11-13T11:03:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.700+0000 7fa2b4f0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2b00d2ed0 0x7fa2b0167a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.700+0000 7fa2b4f0a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b00cac40 con 0x7fa2b00d1850 2023-11-13T11:03:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.700+0000 7fa2b4f0a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b00ce0e0 con 0x7fa2b015ce70 2023-11-13T11:03:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.700+0000 7fa2b4f0a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b00ce3c0 con 0x7fa2b00d2ed0 2023-11-13T11:03:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.701+0000 7fa2aeffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa2b00d1850 0x7fa2b00d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41418/0 (socket says 172.21.15.16:41418) 2023-11-13T11:03:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.701+0000 7fa2aeffd640 1 -- 172.21.15.16:0/369128869 learned_addr learned my addr 172.21.15.16:0/369128869 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.701+0000 7fa2affff640 1 --2- 172.21.15.16:0/369128869 >> v2:172.21.15.16:3300/0 conn(0x7fa2b00d2ed0 0x7fa2b0167a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.701+0000 7fa2ae7fc640 1 -- 172.21.15.16:0/369128869 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116020905 0 0) 0x7fa2b00cac40 con 0x7fa2b00d1850 2023-11-13T11:03:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.701+0000 7fa2ae7fc640 1 -- 172.21.15.16:0/369128869 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa29c003520 con 0x7fa2b00d1850 2023-11-13T11:03:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.701+0000 7fa2affff640 1 -- 172.21.15.16:0/369128869 >> v1:172.21.15.184:6789/0 conn(0x7fa2b015ce70 legacy=0x7fa2b00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.701+0000 7fa2affff640 1 -- 172.21.15.16:0/369128869 >> v1:172.21.15.16:6789/0 conn(0x7fa2b00d1850 legacy=0x7fa2b00d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.701+0000 7fa2affff640 1 -- 172.21.15.16:0/369128869 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00ce6a0 con 0x7fa2b00d2ed0 2023-11-13T11:03:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.703+0000 7fa2affff640 1 --2- 172.21.15.16:0/369128869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b00d2ed0 0x7fa2b0167a10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097a0 tx=0x7fa2a802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233a09fe61e484da server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.703+0000 7fa2ae7fc640 1 -- 172.21.15.16:0/369128869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8030610 con 0x7fa2b00d2ed0 2023-11-13T11:03:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.703+0000 7fa2ae7fc640 1 -- 172.21.15.16:0/369128869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8030770 con 0x7fa2b00d2ed0 2023-11-13T11:03:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.703+0000 7fa2ae7fc640 1 -- 172.21.15.16:0/369128869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8030ae0 con 0x7fa2b00d2ed0 2023-11-13T11:03:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.703+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/369128869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b00d2ed0 msgr2=0x7fa2b0167a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.703+0000 7fa2b4f0a640 1 --2- 172.21.15.16:0/369128869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b00d2ed0 0x7fa2b0167a10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097a0 tx=0x7fa2a802b960 comp rx=0 tx=0).stop 2023-11-13T11:03:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.704+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/369128869 shutdown_connections 2023-11-13T11:03:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.704+0000 7fa2b4f0a640 1 --2- 172.21.15.16:0/369128869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b00d2ed0 0x7fa2b0167a10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.704+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/369128869 >> 172.21.15.16:0/369128869 conn(0x7fa2b008c580 msgr2=0x7fa2b009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:01.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.704+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/369128869 shutdown_connections 2023-11-13T11:03:01.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.704+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/369128869 wait complete. 2023-11-13T11:03:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.706+0000 7fa2b4f0a640 1 Processor -- start 2023-11-13T11:03:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.706+0000 7fa2b4f0a640 1 -- start start 2023-11-13T11:03:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2b4f0a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2b00d1850 0x7fa2b00a0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2b4f0a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2b00d2ed0 0x7fa2b014ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2b4f0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b015ce70 0x7fa2b009d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2b4f0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00a0890 con 0x7fa2b015ce70 2023-11-13T11:03:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2b4f0a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b009fec0 con 0x7fa2b00d2ed0 2023-11-13T11:03:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2b4f0a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b009f2b0 con 0x7fa2b00d1850 2023-11-13T11:03:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b015ce70 0x7fa2b009d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b015ce70 0x7fa2b009d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53148/0 (socket says 172.21.15.16:53148) 2023-11-13T11:03:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2affff640 1 -- 172.21.15.16:0/936689408 learned_addr learned my addr 172.21.15.16:0/936689408 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2aeffd640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2b00d2ed0 0x7fa2b014ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.707+0000 7fa2af7fe640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2b00d1850 0x7fa2b00a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2affff640 1 -- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2b00d1850 msgr2=0x7fa2b00a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2affff640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2b00d1850 0x7fa2b00a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2affff640 1 -- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2b00d2ed0 msgr2=0x7fa2b014ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2affff640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2b00d2ed0 0x7fa2b014ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2affff640 1 -- 172.21.15.16:0/936689408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a8009400 con 0x7fa2b015ce70 2023-11-13T11:03:01.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2aeffd640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2b00d2ed0 0x7fa2b014ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:01.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2affff640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b015ce70 0x7fa2b009d4d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097a0 tx=0x7fa2a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:01.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2a7fff640 1 -- 172.21.15.16:0/936689408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8039c10 con 0x7fa2b015ce70 2023-11-13T11:03:01.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2a7fff640 1 -- 172.21.15.16:0/936689408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a803f040 con 0x7fa2b015ce70 2023-11-13T11:03:01.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2a7fff640 1 -- 172.21.15.16:0/936689408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8006e10 con 0x7fa2b015ce70 2023-11-13T11:03:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b009f590 con 0x7fa2b015ce70 2023-11-13T11:03:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.708+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b009fb50 con 0x7fa2b015ce70 2023-11-13T11:03:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.710+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b008c300 con 0x7fa2b015ce70 2023-11-13T11:03:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.711+0000 7fa2a7fff640 1 -- 172.21.15.16:0/936689408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2a8004030 con 0x7fa2b015ce70 2023-11-13T11:03:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.712+0000 7fa2a7fff640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa29006ae70 0x7fa29006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.712+0000 7fa2a7fff640 1 -- 172.21.15.16:0/936689408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fa2a80ab5e0 con 0x7fa2b015ce70 2023-11-13T11:03:01.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.713+0000 7fa2af7fe640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa29006ae70 0x7fa29006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:01.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.714+0000 7fa2af7fe640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa29006ae70 0x7fa29006d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0007fd0 tx=0x7fa2a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:01.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.717+0000 7fa2a7fff640 1 -- 172.21.15.16:0/936689408 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2a80765c0 con 0x7fa2b015ce70 2023-11-13T11:03:01.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.908+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2b004e4d0 con 0x7fa2b015ce70 2023-11-13T11:03:01.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.909+0000 7fa2a7fff640 1 -- 172.21.15.16:0/936689408 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2a8079be0 con 0x7fa2b015ce70 2023-11-13T11:03:01.915 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T11:03:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa29006ae70 msgr2=0x7fa29006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa29006ae70 0x7fa29006d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0007fd0 tx=0x7fa2a0009040 comp rx=0 tx=0).stop 2023-11-13T11:03:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b015ce70 msgr2=0x7fa2b009d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b015ce70 0x7fa2b009d4d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097a0 tx=0x7fa2a80046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 shutdown_connections 2023-11-13T11:03:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa29006ae70 0x7fa29006d330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2b015ce70 0x7fa2b009d4d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2b00d2ed0 0x7fa2b014ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 --2- 172.21.15.16:0/936689408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2b00d1850 0x7fa2b00a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 >> 172.21.15.16:0/936689408 conn(0x7fa2b008c580 msgr2=0x7fa2b00d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 shutdown_connections 2023-11-13T11:03:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:01.912+0000 7fa2b4f0a640 1 -- 172.21.15.16:0/936689408 wait complete. 2023-11-13T11:03:01.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T11:03:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.166+0000 7f7480e61640 1 Processor -- start 2023-11-13T11:03:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.166+0000 7f7480e61640 1 -- start start 2023-11-13T11:03:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.168+0000 7f7480e61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f747c144570 0x7f747c15f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.168+0000 7f7480e61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c0ce550 0x7f747c0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.168+0000 7f7480e61640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747c0ceef0 con 0x7f747c06bb30 2023-11-13T11:03:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.168+0000 7f7480e61640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747c144950 con 0x7f747c144570 2023-11-13T11:03:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.168+0000 7f7480e61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c15f740 con 0x7f747c0ce550 2023-11-13T11:03:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.169+0000 7f747a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f747c144570 0x7f747c15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.169+0000 7f747ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f747c06bb30 0x7f747c06bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41430/0 (socket says 172.21.15.16:41430) 2023-11-13T11:03:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.169+0000 7f747ad9d640 1 -- 172.21.15.16:0/2866175333 learned_addr learned my addr 172.21.15.16:0/2866175333 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.169+0000 7f7479d9b640 1 --2- 172.21.15.16:0/2866175333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c0ce550 0x7f747c0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.169+0000 7f747a59c640 1 -- 172.21.15.16:0/2866175333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c0ce550 msgr2=0x7f747c0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.169+0000 7f747a59c640 1 --2- 172.21.15.16:0/2866175333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c0ce550 0x7f747c0ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.169+0000 7f747a59c640 1 -- 172.21.15.16:0/2866175333 >> v1:172.21.15.16:6789/0 conn(0x7f747c06bb30 legacy=0x7f747c06bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.169+0000 7f747959a640 1 -- 172.21.15.16:0/2866175333 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655894419 0 0) 0x7f747c0ceef0 con 0x7f747c06bb30 2023-11-13T11:03:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.169+0000 7f747a59c640 1 -- 172.21.15.16:0/2866175333 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0ce300 con 0x7f747c144570 2023-11-13T11:03:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.170+0000 7f747a59c640 1 --2- 172.21.15.16:0/2866175333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 0x7f747c15f200 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f746c009d90 tx=0x7f746c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef4f5918c7c89dd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:02.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f747959a640 1 -- 172.21.15.16:0/2866175333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c02cc50 con 0x7f747c144570 2023-11-13T11:03:02.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f747959a640 1 -- 172.21.15.16:0/2866175333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c02cdb0 con 0x7f747c144570 2023-11-13T11:03:02.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f747959a640 1 -- 172.21.15.16:0/2866175333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c034660 con 0x7f747c144570 2023-11-13T11:03:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f7480e61640 1 -- 172.21.15.16:0/2866175333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 msgr2=0x7f747c15f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f7480e61640 1 --2- 172.21.15.16:0/2866175333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 0x7f747c15f200 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f746c009d90 tx=0x7f746c02bea0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f7480e61640 1 -- 172.21.15.16:0/2866175333 shutdown_connections 2023-11-13T11:03:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f7480e61640 1 --2- 172.21.15.16:0/2866175333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c0ce550 0x7f747c0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f7480e61640 1 --2- 172.21.15.16:0/2866175333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 0x7f747c15f200 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f7480e61640 1 -- 172.21.15.16:0/2866175333 >> 172.21.15.16:0/2866175333 conn(0x7f747c06a190 msgr2=0x7f747c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:02.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f7480e61640 1 -- 172.21.15.16:0/2866175333 shutdown_connections 2023-11-13T11:03:02.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.171+0000 7f7480e61640 1 -- 172.21.15.16:0/2866175333 wait complete. 2023-11-13T11:03:02.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.173+0000 7f7480e61640 1 Processor -- start 2023-11-13T11:03:02.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.173+0000 7f7480e61640 1 -- start start 2023-11-13T11:03:02.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.174+0000 7f7480e61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c06bb30 0x7f747c1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.174+0000 7f7480e61640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f747c0ce550 0x7f747c139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.174+0000 7f7480e61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 0x7f747c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.174+0000 7f7480e61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c071110 con 0x7f747c144570 2023-11-13T11:03:02.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.174+0000 7f7480e61640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c0713f0 con 0x7f747c0ce550 2023-11-13T11:03:02.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.174+0000 7f7480e61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c1548e0 con 0x7f747c06bb30 2023-11-13T11:03:02.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 0x7f747c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 0x7f747c157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53170/0 (socket says 172.21.15.16:53170) 2023-11-13T11:03:02.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747ad9d640 1 -- 172.21.15.16:0/1341837589 learned_addr learned my addr 172.21.15.16:0/1341837589 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:02.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f7479d9b640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f747c0ce550 0x7f747c139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747a59c640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c06bb30 0x7f747c1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747ad9d640 1 -- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c06bb30 msgr2=0x7f747c1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747ad9d640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c06bb30 0x7f747c1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747ad9d640 1 -- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f747c0ce550 msgr2=0x7f747c139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747ad9d640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f747c0ce550 0x7f747c139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747ad9d640 1 -- 172.21.15.16:0/1341837589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c009a70 con 0x7f747c144570 2023-11-13T11:03:02.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f7479d9b640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f747c0ce550 0x7f747c139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:02.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747a59c640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c06bb30 0x7f747c1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:02.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.175+0000 7f747ad9d640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 0x7f747c157840 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f747400c450 tx=0x7f747400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:02.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.176+0000 7f7472ffd640 1 -- 172.21.15.16:0/1341837589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740074e0 con 0x7f747c144570 2023-11-13T11:03:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.176+0000 7f7472ffd640 1 -- 172.21.15.16:0/1341837589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7474007660 con 0x7f747c144570 2023-11-13T11:03:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.176+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747c154bc0 con 0x7f747c144570 2023-11-13T11:03:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.176+0000 7f7472ffd640 1 -- 172.21.15.16:0/1341837589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474004a20 con 0x7f747c144570 2023-11-13T11:03:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.176+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747c13a390 con 0x7f747c144570 2023-11-13T11:03:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.177+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747c071610 con 0x7f747c144570 2023-11-13T11:03:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.178+0000 7f7472ffd640 1 -- 172.21.15.16:0/1341837589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7474004b80 con 0x7f747c144570 2023-11-13T11:03:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.179+0000 7f7472ffd640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f745c06ae40 0x7f745c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.179+0000 7f7472ffd640 1 -- 172.21.15.16:0/1341837589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7474088980 con 0x7f747c144570 2023-11-13T11:03:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.179+0000 7f747a59c640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f745c06ae40 0x7f745c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.181+0000 7f747a59c640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f745c06ae40 0x7f745c06d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f746c036000 tx=0x7f746c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.184+0000 7f7472ffd640 1 -- 172.21.15.16:0/1341837589 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f747c071610 con 0x7f747c144570 2023-11-13T11:03:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.379+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f747c154db0 con 0x7f747c144570 2023-11-13T11:03:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.380+0000 7f7472ffd640 1 -- 172.21.15.16:0/1341837589 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7474018020 con 0x7f747c144570 2023-11-13T11:03:02.385 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T11:03:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f745c06ae40 msgr2=0x7f745c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f745c06ae40 0x7f745c06d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f746c036000 tx=0x7f746c030000 comp rx=0 tx=0).stop 2023-11-13T11:03:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 msgr2=0x7f747c157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 0x7f747c157840 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f747400c450 tx=0x7f747400c910 comp rx=0 tx=0).stop 2023-11-13T11:03:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 shutdown_connections 2023-11-13T11:03:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f745c06ae40 0x7f745c06d300 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747c144570 0x7f747c157840 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f747c0ce550 0x7f747c139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 --2- 172.21.15.16:0/1341837589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f747c06bb30 0x7f747c1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 >> 172.21.15.16:0/1341837589 conn(0x7f747c06a190 msgr2=0x7f747c0b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 shutdown_connections 2023-11-13T11:03:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.383+0000 7f7480e61640 1 -- 172.21.15.16:0/1341837589 wait complete. 2023-11-13T11:03:02.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.627+0000 7f8678f5a640 1 Processor -- start 2023-11-13T11:03:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.627+0000 7f8678f5a640 1 -- start start 2023-11-13T11:03:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.628+0000 7f8678f5a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f867415d9f0 0x7f867406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.629+0000 7f8678f5a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f867415e240 0x7f86740b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.629+0000 7f8678f5a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867415a7e0 con 0x7f867415ddd0 2023-11-13T11:03:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.629+0000 7f8678f5a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867415b070 con 0x7f867415e240 2023-11-13T11:03:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.629+0000 7f8678f5a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867415aab0 con 0x7f867415d9f0 2023-11-13T11:03:02.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.629+0000 7f8672d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f867415e240 0x7f86740b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.629+0000 7f8672d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f867415e240 0x7f86740b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53186/0 (socket says 172.21.15.16:53186) 2023-11-13T11:03:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.629+0000 7f8672d9d640 1 -- 172.21.15.16:0/2418461479 learned_addr learned my addr 172.21.15.16:0/2418461479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.629+0000 7f867259c640 1 --2- 172.21.15.16:0/2418461479 >> v2:172.21.15.184:3300/0 conn(0x7f867415d9f0 0x7f867406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.630+0000 7f8672d9d640 1 -- 172.21.15.16:0/2418461479 >> v2:172.21.15.184:3300/0 conn(0x7f867415d9f0 msgr2=0x7f867406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:02.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.630+0000 7f8672d9d640 1 --2- 172.21.15.16:0/2418461479 >> v2:172.21.15.184:3300/0 conn(0x7f867415d9f0 0x7f867406d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.630+0000 7f8672d9d640 1 -- 172.21.15.16:0/2418461479 >> v1:172.21.15.184:6789/0 conn(0x7f867415ddd0 legacy=0x7f867415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.630+0000 7f8672d9d640 1 -- 172.21.15.16:0/2418461479 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867415ad90 con 0x7f867415e240 2023-11-13T11:03:02.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.631+0000 7f8672d9d640 1 --2- 172.21.15.16:0/2418461479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415e240 0x7f86740b2630 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f86740b2600 tx=0x7f866c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25652e0995f8a1a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.631+0000 7f867159a640 1 -- 172.21.15.16:0/2418461479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0317e0 con 0x7f867415e240 2023-11-13T11:03:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.631+0000 7f867159a640 1 -- 172.21.15.16:0/2418461479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c031940 con 0x7f867415e240 2023-11-13T11:03:02.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.631+0000 7f867159a640 1 -- 172.21.15.16:0/2418461479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c031cd0 con 0x7f867415e240 2023-11-13T11:03:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.632+0000 7f8678f5a640 1 -- 172.21.15.16:0/2418461479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415e240 msgr2=0x7f86740b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.632+0000 7f8678f5a640 1 --2- 172.21.15.16:0/2418461479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415e240 0x7f86740b2630 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f86740b2600 tx=0x7f866c02b880 comp rx=0 tx=0).stop 2023-11-13T11:03:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.632+0000 7f8678f5a640 1 -- 172.21.15.16:0/2418461479 shutdown_connections 2023-11-13T11:03:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.632+0000 7f8678f5a640 1 --2- 172.21.15.16:0/2418461479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415e240 0x7f86740b2630 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.632+0000 7f8678f5a640 1 --2- 172.21.15.16:0/2418461479 >> v2:172.21.15.184:3300/0 conn(0x7f867415d9f0 0x7f867406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.632+0000 7f8678f5a640 1 -- 172.21.15.16:0/2418461479 >> 172.21.15.16:0/2418461479 conn(0x7f867406a190 msgr2=0x7f867406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.632+0000 7f8678f5a640 1 -- 172.21.15.16:0/2418461479 shutdown_connections 2023-11-13T11:03:02.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.632+0000 7f8678f5a640 1 -- 172.21.15.16:0/2418461479 wait complete. 2023-11-13T11:03:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.634+0000 7f8678f5a640 1 Processor -- start 2023-11-13T11:03:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.634+0000 7f8678f5a640 1 -- start start 2023-11-13T11:03:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.635+0000 7f8678f5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415d9f0 0x7f86741423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.635+0000 7f8678f5a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f867415ddd0 0x7f86740cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.635+0000 7f8678f5a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f867415e240 0x7f86740cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.635+0000 7f8678f5a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86740ce9f0 con 0x7f867415d9f0 2023-11-13T11:03:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.635+0000 7f8678f5a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86741557b0 con 0x7f867415e240 2023-11-13T11:03:02.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.635+0000 7f8678f5a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8674155a90 con 0x7f867415ddd0 2023-11-13T11:03:02.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.635+0000 7f867259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415d9f0 0x7f86741423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.635+0000 7f867259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415d9f0 0x7f86741423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53200/0 (socket says 172.21.15.16:53200) 2023-11-13T11:03:02.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.635+0000 7f867259c640 1 -- 172.21.15.16:0/1550465487 learned_addr learned my addr 172.21.15.16:0/1550465487 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:02.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f8672d9d640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f867415e240 0x7f86740cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f867259c640 1 -- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f867415ddd0 msgr2=0x7f86740cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f867259c640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f867415ddd0 0x7f86740cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f867259c640 1 -- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f867415e240 msgr2=0x7f86740cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f867259c640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f867415e240 0x7f86740cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f867259c640 1 -- 172.21.15.16:0/1550465487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0094d0 con 0x7f867415d9f0 2023-11-13T11:03:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f867259c640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415d9f0 0x7f86741423e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8664007e40 tx=0x7f866400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f866affd640 1 -- 172.21.15.16:0/1550465487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664014620 con 0x7f867415d9f0 2023-11-13T11:03:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f866affd640 1 -- 172.21.15.16:0/1550465487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866400ee90 con 0x7f867415d9f0 2023-11-13T11:03:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f866affd640 1 -- 172.21.15.16:0/1550465487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86640149c0 con 0x7f867415d9f0 2023-11-13T11:03:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8674155d70 con 0x7f867415d9f0 2023-11-13T11:03:02.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.636+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8674142920 con 0x7f867415d9f0 2023-11-13T11:03:02.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.637+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8674067090 con 0x7f867415d9f0 2023-11-13T11:03:02.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.639+0000 7f866affd640 1 -- 172.21.15.16:0/1550465487 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8664014b20 con 0x7f867415d9f0 2023-11-13T11:03:02.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.640+0000 7f866affd640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f865406ae40 0x7f865406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:02.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.640+0000 7f866affd640 1 -- 172.21.15.16:0/1550465487 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f866408a1b0 con 0x7f867415d9f0 2023-11-13T11:03:02.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.641+0000 7f8671d9b640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f865406ae40 0x7f865406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.642+0000 7f8671d9b640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f865406ae40 0x7f865406d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8674158e10 tx=0x7f865c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.644+0000 7f866affd640 1 -- 172.21.15.16:0/1550465487 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8664055040 con 0x7f867415d9f0 2023-11-13T11:03:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.875+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8674150610 con 0x7f867415d9f0 2023-11-13T11:03:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.876+0000 7f866affd640 1 -- 172.21.15.16:0/1550465487 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f8664058660 con 0x7f867415d9f0 2023-11-13T11:03:02.881 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:02.881 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:59.012901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:03:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 32 2023-11-13T11:03:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.878+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f865406ae40 msgr2=0x7f865406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.878+0000 7f8678f5a640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f865406ae40 0x7f865406d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8674158e10 tx=0x7f865c0079b0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.878+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415d9f0 msgr2=0x7f86741423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.878+0000 7f8678f5a640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415d9f0 0x7f86741423e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8664007e40 tx=0x7f866400e4b0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.879+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 shutdown_connections 2023-11-13T11:03:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.879+0000 7f8678f5a640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f865406ae40 0x7f865406d300 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.879+0000 7f8678f5a640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f867415e240 0x7f86740cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.879+0000 7f8678f5a640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f867415ddd0 0x7f86740cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.879+0000 7f8678f5a640 1 --2- 172.21.15.16:0/1550465487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867415d9f0 0x7f86741423e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.879+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 >> 172.21.15.16:0/1550465487 conn(0x7f867406a190 msgr2=0x7f867415cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.879+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 shutdown_connections 2023-11-13T11:03:02.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:02.879+0000 7f8678f5a640 1 -- 172.21.15.16:0/1550465487 wait complete. 2023-11-13T11:03:02.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.122+0000 7ff0a1c6f640 1 Processor -- start 2023-11-13T11:03:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.122+0000 7ff0a1c6f640 1 -- start start 2023-11-13T11:03:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.124+0000 7ff0a1c6f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15d9f0 0x7ff09c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.124+0000 7ff0a1c6f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff09c15ddd0 0x7ff09c15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.124+0000 7ff0a1c6f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c15aab0 con 0x7ff09c15a610 2023-11-13T11:03:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.124+0000 7ff0a1c6f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c15b070 con 0x7ff09c15ddd0 2023-11-13T11:03:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.124+0000 7ff0a1c6f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c15ad90 con 0x7ff09c15d9f0 2023-11-13T11:03:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.124+0000 7ff09bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff09c15a610 0x7ff09c0b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33302/0 (socket says 172.21.15.16:33302) 2023-11-13T11:03:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.124+0000 7ff09bfff640 1 -- 172.21.15.16:0/3116728779 learned_addr learned my addr 172.21.15.16:0/3116728779 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.124+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/3116728779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15d9f0 0x7ff09c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.125+0000 7ff093fff640 1 --2- 172.21.15.16:0/3116728779 >> v2:172.21.15.184:3300/0 conn(0x7ff09c15ddd0 0x7ff09c15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.125+0000 7ff09affd640 1 -- 172.21.15.16:0/3116728779 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919782776 0 0) 0x7ff09c15aab0 con 0x7ff09c15a610 2023-11-13T11:03:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.125+0000 7ff09affd640 1 -- 172.21.15.16:0/3116728779 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff088003520 con 0x7ff09c15a610 2023-11-13T11:03:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.125+0000 7ff093fff640 1 -- 172.21.15.16:0/3116728779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15d9f0 msgr2=0x7ff09c06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.125+0000 7ff093fff640 1 --2- 172.21.15.16:0/3116728779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15d9f0 0x7ff09c06d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.125+0000 7ff093fff640 1 -- 172.21.15.16:0/3116728779 >> v1:172.21.15.16:6789/0 conn(0x7ff09c15a610 legacy=0x7ff09c0b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.125+0000 7ff09affd640 1 -- 172.21.15.16:0/3116728779 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922925270 0 0) 0x7ff088003520 con 0x7ff09c15a610 2023-11-13T11:03:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.126+0000 7ff093fff640 1 -- 172.21.15.16:0/3116728779 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c06cca0 con 0x7ff09c15ddd0 2023-11-13T11:03:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.126+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/3116728779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15d9f0 0x7ff09c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T11:03:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.127+0000 7ff093fff640 1 --2- 172.21.15.16:0/3116728779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c15ddd0 0x7ff09c15a0d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff0840097a0 tx=0x7ff08402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b473a397562f0d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff09affd640 1 -- 172.21.15.16:0/3116728779 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0940019f0 con 0x7ff09c15ddd0 2023-11-13T11:03:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff09affd640 1 -- 172.21.15.16:0/3116728779 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08402e050 con 0x7ff09c15ddd0 2023-11-13T11:03:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff09affd640 1 -- 172.21.15.16:0/3116728779 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084030730 con 0x7ff09c15ddd0 2023-11-13T11:03:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/3116728779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c15ddd0 msgr2=0x7ff09c15a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff0a1c6f640 1 --2- 172.21.15.16:0/3116728779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c15ddd0 0x7ff09c15a0d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff0840097a0 tx=0x7ff08402b710 comp rx=0 tx=0).stop 2023-11-13T11:03:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/3116728779 shutdown_connections 2023-11-13T11:03:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff0a1c6f640 1 --2- 172.21.15.16:0/3116728779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c15ddd0 0x7ff09c15a0d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff0a1c6f640 1 --2- 172.21.15.16:0/3116728779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15d9f0 0x7ff09c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/3116728779 >> 172.21.15.16:0/3116728779 conn(0x7ff09c06a190 msgr2=0x7ff09c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/3116728779 shutdown_connections 2023-11-13T11:03:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.128+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/3116728779 wait complete. 2023-11-13T11:03:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.130+0000 7ff0a1c6f640 1 Processor -- start 2023-11-13T11:03:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.130+0000 7ff0a1c6f640 1 -- start start 2023-11-13T11:03:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.131+0000 7ff0a1c6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09c15a610 0x7ff09c15b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.131+0000 7ff0a1c6f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c15d9f0 0x7ff09c14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.131+0000 7ff0a1c6f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15ddd0 0x7ff09c14cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.131+0000 7ff0a1c6f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0cd8f0 con 0x7ff09c15a610 2023-11-13T11:03:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.131+0000 7ff0a1c6f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0ccce0 con 0x7ff09c15d9f0 2023-11-13T11:03:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.131+0000 7ff0a1c6f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0ccfc0 con 0x7ff09c15ddd0 2023-11-13T11:03:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09c15a610 0x7ff09c15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09c15a610 0x7ff09c15b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58210/0 (socket says 172.21.15.16:58210) 2023-11-13T11:03:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09b7fe640 1 -- 172.21.15.16:0/2113148701 learned_addr learned my addr 172.21.15.16:0/2113148701 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff093fff640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c15d9f0 0x7ff09c14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09bfff640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15ddd0 0x7ff09c14cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09b7fe640 1 -- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15ddd0 msgr2=0x7ff09c14cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15ddd0 0x7ff09c14cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09b7fe640 1 -- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c15d9f0 msgr2=0x7ff09c14c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c15d9f0 0x7ff09c14c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09b7fe640 1 -- 172.21.15.16:0/2113148701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c009400 con 0x7ff09c15a610 2023-11-13T11:03:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.132+0000 7ff09b7fe640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09c15a610 0x7ff09c15b420 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff08c010df0 tx=0x7ff08c00e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.133+0000 7ff098ff9640 1 -- 172.21.15.16:0/2113148701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c009e20 con 0x7ff09c15a610 2023-11-13T11:03:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.133+0000 7ff098ff9640 1 -- 172.21.15.16:0/2113148701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c011660 con 0x7ff09c15a610 2023-11-13T11:03:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.133+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff084009400 con 0x7ff09c15a610 2023-11-13T11:03:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.133+0000 7ff098ff9640 1 -- 172.21.15.16:0/2113148701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0119f0 con 0x7ff09c15a610 2023-11-13T11:03:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.133+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c0cd580 con 0x7ff09c15a610 2023-11-13T11:03:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.134+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c067090 con 0x7ff09c15a610 2023-11-13T11:03:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.136+0000 7ff098ff9640 1 -- 172.21.15.16:0/2113148701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff08c011b50 con 0x7ff09c15a610 2023-11-13T11:03:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.137+0000 7ff098ff9640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff07c06ae40 0x7ff07c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.137+0000 7ff098ff9640 1 -- 172.21.15.16:0/2113148701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7ff08c08be20 con 0x7ff09c15a610 2023-11-13T11:03:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.138+0000 7ff093fff640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff07c06ae40 0x7ff07c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.139+0000 7ff093fff640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff07c06ae40 0x7ff07c06d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff084034000 tx=0x7ff084008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.141+0000 7ff098ff9640 1 -- 172.21.15.16:0/2113148701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff08c056cb0 con 0x7ff09c15a610 2023-11-13T11:03:03.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.337+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff09c06c6a0 con 0x7ff09c15a610 2023-11-13T11:03:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.340+0000 7ff098ff9640 1 -- 172.21.15.16:0/2113148701 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16821 (secure 0 0 0) 0x7ff08c01e020 con 0x7ff09c15a610 2023-11-13T11:03:03.345 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:03.346 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:02:08.953574+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:01:36.305098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-13T11:01:37.461852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.4","mappings":[{"from":2,"to":5}]},{"pgid":"8.20","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.342+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff07c06ae40 msgr2=0x7ff07c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.342+0000 7ff0a1c6f640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff07c06ae40 0x7ff07c06d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff084034000 tx=0x7ff084008030 comp rx=0 tx=0).stop 2023-11-13T11:03:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.342+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09c15a610 msgr2=0x7ff09c15b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.342+0000 7ff0a1c6f640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09c15a610 0x7ff09c15b420 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff08c010df0 tx=0x7ff08c00e730 comp rx=0 tx=0).stop 2023-11-13T11:03:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.343+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 shutdown_connections 2023-11-13T11:03:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.343+0000 7ff0a1c6f640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff07c06ae40 0x7ff07c06d300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.343+0000 7ff0a1c6f640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff09c15ddd0 0x7ff09c14cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.343+0000 7ff0a1c6f640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff09c15d9f0 0x7ff09c14c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.343+0000 7ff0a1c6f640 1 --2- 172.21.15.16:0/2113148701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09c15a610 0x7ff09c15b420 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.343+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 >> 172.21.15.16:0/2113148701 conn(0x7ff09c06a190 msgr2=0x7ff09c1534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.343+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 shutdown_connections 2023-11-13T11:03:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.343+0000 7ff0a1c6f640 1 -- 172.21.15.16:0/2113148701 wait complete. 2023-11-13T11:03:03.368 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-13T11:03:03.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.586+0000 7ffb51852640 1 Processor -- start 2023-11-13T11:03:03.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.586+0000 7ffb51852640 1 -- start start 2023-11-13T11:03:03.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.588+0000 7ffb51852640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffb4c15cc90 0x7ffb4c06a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.588+0000 7ffb51852640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb4c071320 0x7ffb4c06a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.588+0000 7ffb51852640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4c06fc50 con 0x7ffb4c070cf0 2023-11-13T11:03:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.588+0000 7ffb51852640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c06ff30 con 0x7ffb4c071320 2023-11-13T11:03:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.588+0000 7ffb51852640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c070210 con 0x7ffb4c15cc90 2023-11-13T11:03:03.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb4bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffb4c070cf0 0x7ffb4c06b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33312/0 (socket says 172.21.15.16:33312) 2023-11-13T11:03:03.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb4bfff640 1 -- 172.21.15.16:0/2131038898 learned_addr learned my addr 172.21.15.16:0/2131038898 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb51051640 1 --2- 172.21.15.16:0/2131038898 >> v2:172.21.15.16:3300/0 conn(0x7ffb4c071320 0x7ffb4c06a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb50850640 1 --2- 172.21.15.16:0/2131038898 >> v2:172.21.15.184:3300/0 conn(0x7ffb4c15cc90 0x7ffb4c06a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb4b7fe640 1 -- 172.21.15.16:0/2131038898 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323903315 0 0) 0x7ffb4c06fc50 con 0x7ffb4c070cf0 2023-11-13T11:03:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb4b7fe640 1 -- 172.21.15.16:0/2131038898 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb34003520 con 0x7ffb4c070cf0 2023-11-13T11:03:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb51051640 1 -- 172.21.15.16:0/2131038898 >> v2:172.21.15.184:3300/0 conn(0x7ffb4c15cc90 msgr2=0x7ffb4c06a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:03.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb51051640 1 --2- 172.21.15.16:0/2131038898 >> v2:172.21.15.184:3300/0 conn(0x7ffb4c15cc90 0x7ffb4c06a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb51051640 1 -- 172.21.15.16:0/2131038898 >> v1:172.21.15.16:6789/0 conn(0x7ffb4c070cf0 legacy=0x7ffb4c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.589+0000 7ffb51051640 1 -- 172.21.15.16:0/2131038898 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c0704f0 con 0x7ffb4c071320 2023-11-13T11:03:03.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.590+0000 7ffb51051640 1 --2- 172.21.15.16:0/2131038898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 0x7ffb4c06a6d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0ae150 tx=0x7ffb4402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3a0f4e529ff5a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:03.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.591+0000 7ffb4b7fe640 1 -- 172.21.15.16:0/2131038898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4402cc80 con 0x7ffb4c071320 2023-11-13T11:03:03.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.591+0000 7ffb4b7fe640 1 -- 172.21.15.16:0/2131038898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4402cde0 con 0x7ffb4c071320 2023-11-13T11:03:03.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.591+0000 7ffb4b7fe640 1 -- 172.21.15.16:0/2131038898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44034660 con 0x7ffb4c071320 2023-11-13T11:03:03.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.591+0000 7ffb51852640 1 -- 172.21.15.16:0/2131038898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 msgr2=0x7ffb4c06a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.591+0000 7ffb51852640 1 --2- 172.21.15.16:0/2131038898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 0x7ffb4c06a6d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0ae150 tx=0x7ffb4402bed0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.591+0000 7ffb51852640 1 -- 172.21.15.16:0/2131038898 shutdown_connections 2023-11-13T11:03:03.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.591+0000 7ffb51852640 1 --2- 172.21.15.16:0/2131038898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 0x7ffb4c06a6d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.591+0000 7ffb51852640 1 --2- 172.21.15.16:0/2131038898 >> v2:172.21.15.184:3300/0 conn(0x7ffb4c15cc90 0x7ffb4c06a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.591+0000 7ffb51852640 1 -- 172.21.15.16:0/2131038898 >> 172.21.15.16:0/2131038898 conn(0x7ffb4c0b04a0 msgr2=0x7ffb4c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.592+0000 7ffb51852640 1 -- 172.21.15.16:0/2131038898 shutdown_connections 2023-11-13T11:03:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.592+0000 7ffb51852640 1 -- 172.21.15.16:0/2131038898 wait complete. 2023-11-13T11:03:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.593+0000 7ffb51852640 1 Processor -- start 2023-11-13T11:03:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.594+0000 7ffb51852640 1 -- start start 2023-11-13T11:03:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.594+0000 7ffb51852640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb4c070cf0 0x7ffb4c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb51852640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 0x7ffb4c1f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb51852640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c15cc90 0x7ffb4c13eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb51852640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c141970 con 0x7ffb4c071320 2023-11-13T11:03:03.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb51852640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c141660 con 0x7ffb4c15cc90 2023-11-13T11:03:03.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb51852640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c1f6260 con 0x7ffb4c070cf0 2023-11-13T11:03:03.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 0x7ffb4c1f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 0x7ffb4c1f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58226/0 (socket says 172.21.15.16:58226) 2023-11-13T11:03:03.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb4bfff640 1 -- 172.21.15.16:0/3790822181 learned_addr learned my addr 172.21.15.16:0/3790822181 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:03.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb50850640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb4c070cf0 0x7ffb4c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb4bfff640 1 -- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb4c070cf0 msgr2=0x7ffb4c158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb4bfff640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb4c070cf0 0x7ffb4c158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb4bfff640 1 -- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c15cc90 msgr2=0x7ffb4c13eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb4bfff640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c15cc90 0x7ffb4c13eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.595+0000 7ffb4bfff640 1 -- 172.21.15.16:0/3790822181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb44009a70 con 0x7ffb4c071320 2023-11-13T11:03:03.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.596+0000 7ffb4bfff640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 0x7ffb4c1f5d20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00c500 tx=0x7ffb3c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:03.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.596+0000 7ffb497fa640 1 -- 172.21.15.16:0/3790822181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c005ce0 con 0x7ffb4c071320 2023-11-13T11:03:03.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.596+0000 7ffb497fa640 1 -- 172.21.15.16:0/3790822181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c005e40 con 0x7ffb4c071320 2023-11-13T11:03:03.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.596+0000 7ffb497fa640 1 -- 172.21.15.16:0/3790822181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c007330 con 0x7ffb4c071320 2023-11-13T11:03:03.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.596+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c1f64e0 con 0x7ffb4c071320 2023-11-13T11:03:03.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.596+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c1f6aa0 con 0x7ffb4c071320 2023-11-13T11:03:03.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.597+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c04e4d0 con 0x7ffb4c071320 2023-11-13T11:03:03.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.598+0000 7ffb497fa640 1 -- 172.21.15.16:0/3790822181 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffb3c007550 con 0x7ffb4c071320 2023-11-13T11:03:03.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.599+0000 7ffb497fa640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb3806ae40 0x7ffb3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:03.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.599+0000 7ffb497fa640 1 -- 172.21.15.16:0/3790822181 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7ffb3c088e10 con 0x7ffb4c071320 2023-11-13T11:03:03.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.600+0000 7ffb50850640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb3806ae40 0x7ffb3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.601+0000 7ffb50850640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb3806ae40 0x7ffb3806d300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb40006fd0 tx=0x7ffb40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.603+0000 7ffb497fa640 1 -- 172.21.15.16:0/3790822181 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb3c08c030 con 0x7ffb4c071320 2023-11-13T11:03:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.838+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb4c06d2e0 con 0x7ffb4c071320 2023-11-13T11:03:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.840+0000 7ffb497fa640 1 -- 172.21.15.16:0/3790822181 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4920 (secure 0 0 0) 0x7ffb3c017020 con 0x7ffb4c071320 2023-11-13T11:03:03.845 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:03.845 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:59.012901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:03:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 32 2023-11-13T11:03:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.842+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb3806ae40 msgr2=0x7ffb3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.842+0000 7ffb51852640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb3806ae40 0x7ffb3806d300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb40006fd0 tx=0x7ffb40008040 comp rx=0 tx=0).stop 2023-11-13T11:03:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.842+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 msgr2=0x7ffb4c1f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.842+0000 7ffb51852640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 0x7ffb4c1f5d20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00c500 tx=0x7ffb3c00c9c0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.843+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 shutdown_connections 2023-11-13T11:03:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.843+0000 7ffb51852640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffb3806ae40 0x7ffb3806d300 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.843+0000 7ffb51852640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c15cc90 0x7ffb4c13eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.843+0000 7ffb51852640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb4c071320 0x7ffb4c1f5d20 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.843+0000 7ffb51852640 1 --2- 172.21.15.16:0/3790822181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb4c070cf0 0x7ffb4c158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.843+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 >> 172.21.15.16:0/3790822181 conn(0x7ffb4c0b04a0 msgr2=0x7ffb4c15b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.843+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 shutdown_connections 2023-11-13T11:03:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:03.843+0000 7ffb51852640 1 -- 172.21.15.16:0/3790822181 wait complete. 2023-11-13T11:03:03.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:04.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.090+0000 7f0f4ac94640 1 Processor -- start 2023-11-13T11:03:04.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.090+0000 7f0f4ac94640 1 -- start start 2023-11-13T11:03:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.091+0000 7f0f4ac94640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0f44157670 0x7f0f4406a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.091+0000 7f0f4ac94640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4415d610 con 0x7f0f4406a740 2023-11-13T11:03:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.091+0000 7f0f4ac94640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4415d8d0 con 0x7f0f440af3d0 2023-11-13T11:03:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.091+0000 7f0f4ac94640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4415e550 con 0x7f0f44157670 2023-11-13T11:03:04.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.092+0000 7f0f43fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0f4406a740 0x7f0f440aecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33324/0 (socket says 172.21.15.16:33324) 2023-11-13T11:03:04.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.092+0000 7f0f43fff640 1 -- 172.21.15.16:0/659313003 learned_addr learned my addr 172.21.15.16:0/659313003 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:04.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.092+0000 7f0f48a30640 1 --2- 172.21.15.16:0/659313003 >> v2:172.21.15.184:3300/0 conn(0x7f0f44157670 0x7f0f4406a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.092+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281902158 0 0) 0x7f0f4415d610 con 0x7f0f4406a740 2023-11-13T11:03:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.092+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f34003520 con 0x7f0f4406a740 2023-11-13T11:03:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3383332471 0 0) 0x7f0f34003520 con 0x7f0f4406a740 2023-11-13T11:03:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 >> v2:172.21.15.184:3300/0 conn(0x7f0f44157670 msgr2=0x7f0f4406a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 --2- 172.21.15.16:0/659313003 >> v2:172.21.15.184:3300/0 conn(0x7f0f44157670 0x7f0f4406a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 >> v1:172.21.15.184:6789/0 conn(0x7f0f440af3d0 legacy=0x7f0f440cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4415dbb0 con 0x7f0f4406a740 2023-11-13T11:03:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f0f30001670 con 0x7f0f4406a740 2023-11-13T11:03:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 >> v1:172.21.15.16:6789/0 conn(0x7f0f4406a740 legacy=0x7f0f440aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f34003970 0x7f0f34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f340047c0 0x7f0f34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34005180 0x7f0f3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4415e550 con 0x7f0f34003970 2023-11-13T11:03:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3400c0d0 con 0x7f0f340047c0 2023-11-13T11:03:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3400c290 con 0x7f0f34005180 2023-11-13T11:03:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f48a30640 1 --2- 172.21.15.16:0/659313003 >> v2:172.21.15.184:3300/0 conn(0x7f0f44157670 0x7f0f4406a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:03:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f48a30640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f34003970 0x7f0f34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f43fff640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f340047c0 0x7f0f34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.093+0000 7f0f49231640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34005180 0x7f0f3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.094+0000 7f0f48a30640 1 -- 172.21.15.16:0/659313003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34005180 msgr2=0x7f0f3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.094+0000 7f0f48a30640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34005180 0x7f0f3400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.094+0000 7f0f48a30640 1 -- 172.21.15.16:0/659313003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f340047c0 msgr2=0x7f0f34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.094+0000 7f0f48a30640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f340047c0 0x7f0f34004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.094+0000 7f0f48a30640 1 -- 172.21.15.16:0/659313003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f34003520 con 0x7f0f34003970 2023-11-13T11:03:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.094+0000 7f0f49231640 1 -- 172.21.15.16:0/659313003 reap_dead start 2023-11-13T11:03:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.095+0000 7f0f48a30640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f34003970 0x7f0f34003d90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0f380098a0 tx=0x7f0f340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c91fa2cd55a038 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.095+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38030630 con 0x7f0f34003970 2023-11-13T11:03:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.095+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f38030790 con 0x7f0f34003970 2023-11-13T11:03:04.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.095+0000 7f0f437fe640 1 -- 172.21.15.16:0/659313003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38030b00 con 0x7f0f34003970 2023-11-13T11:03:04.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.096+0000 7f0f4ac94640 1 -- 172.21.15.16:0/659313003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f34003970 msgr2=0x7f0f34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.096+0000 7f0f4ac94640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f34003970 0x7f0f34003d90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0f380098a0 tx=0x7f0f340057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.096+0000 7f0f4ac94640 1 -- 172.21.15.16:0/659313003 shutdown_connections 2023-11-13T11:03:04.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.096+0000 7f0f4ac94640 1 --2- 172.21.15.16:0/659313003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f34003970 0x7f0f34003d90 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.096+0000 7f0f4ac94640 1 -- 172.21.15.16:0/659313003 >> 172.21.15.16:0/659313003 conn(0x7f0f440b18c0 msgr2=0x7f0f440b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.096+0000 7f0f4ac94640 1 -- 172.21.15.16:0/659313003 shutdown_connections 2023-11-13T11:03:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.096+0000 7f0f4ac94640 1 -- 172.21.15.16:0/659313003 wait complete. 2023-11-13T11:03:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.098+0000 7f0f4ac94640 1 Processor -- start 2023-11-13T11:03:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.098+0000 7f0f4ac94640 1 -- start start 2023-11-13T11:03:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.099+0000 7f0f4ac94640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f34003970 0x7f0f4406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.099+0000 7f0f4ac94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f44145da0 0x7f0f44146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.099+0000 7f0f4ac94640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f4406dd40 0x7f0f4406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.099+0000 7f0f4ac94640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f441f70b0 con 0x7f0f44145da0 2023-11-13T11:03:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.099+0000 7f0f4ac94640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4406e700 con 0x7f0f34003970 2023-11-13T11:03:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.099+0000 7f0f4ac94640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f441f6190 con 0x7f0f4406dd40 2023-11-13T11:03:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.099+0000 7f0f43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f44145da0 0x7f0f44146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.099+0000 7f0f43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f44145da0 0x7f0f44146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58254/0 (socket says 172.21.15.16:58254) 2023-11-13T11:03:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.099+0000 7f0f43fff640 1 -- 172.21.15.16:0/2732151430 learned_addr learned my addr 172.21.15.16:0/2732151430 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:04.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f48a30640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f34003970 0x7f0f4406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f49231640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f4406dd40 0x7f0f4406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f43fff640 1 -- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f4406dd40 msgr2=0x7f0f4406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f43fff640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f4406dd40 0x7f0f4406e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f43fff640 1 -- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f34003970 msgr2=0x7f0f4406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f43fff640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f34003970 0x7f0f4406ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f43fff640 1 -- 172.21.15.16:0/2732151430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f441f6430 con 0x7f0f44145da0 2023-11-13T11:03:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f49231640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f4406dd40 0x7f0f4406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f43fff640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f44145da0 0x7f0f44146220 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0f300072c0 tx=0x7f0f30007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f417fa640 1 -- 172.21.15.16:0/2732151430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000f960 con 0x7f0f44145da0 2023-11-13T11:03:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f417fa640 1 -- 172.21.15.16:0/2732151430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3000fac0 con 0x7f0f44145da0 2023-11-13T11:03:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f441f6710 con 0x7f0f44145da0 2023-11-13T11:03:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.100+0000 7f0f417fa640 1 -- 172.21.15.16:0/2732151430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000fe70 con 0x7f0f44145da0 2023-11-13T11:03:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.101+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f441f6cd0 con 0x7f0f44145da0 2023-11-13T11:03:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.102+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f440b0980 con 0x7f0f44145da0 2023-11-13T11:03:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.103+0000 7f0f417fa640 1 -- 172.21.15.16:0/2732151430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f3001d460 con 0x7f0f44145da0 2023-11-13T11:03:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.104+0000 7f0f417fa640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0f2806ae70 0x7f0f2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.104+0000 7f0f417fa640 1 -- 172.21.15.16:0/2732151430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f0f30089910 con 0x7f0f44145da0 2023-11-13T11:03:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.105+0000 7f0f48a30640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0f2806ae70 0x7f0f2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.106+0000 7f0f48a30640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0f2806ae70 0x7f0f2806d330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0f38036000 tx=0x7f0f38031000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.108+0000 7f0f417fa640 1 -- 172.21.15.16:0/2732151430 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f30054830 con 0x7f0f44145da0 2023-11-13T11:03:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.348+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f440677a0 con 0x7f0f44145da0 2023-11-13T11:03:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.349+0000 7f0f417fa640 1 -- 172.21.15.16:0/2732151430 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f0f30057e50 con 0x7f0f44145da0 2023-11-13T11:03:04.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:04.369 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:01:38.428966+0000","modified":"2023-11-13T11:02:59.012901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":23,"addr":"172.21.15.16:6835/3328237273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3328237273},{"type":"v1","addr":"172.21.15.16:6835","nonce":3328237273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-13T11:03:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 32 2023-11-13T11:03:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0f2806ae70 msgr2=0x7f0f2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0f2806ae70 0x7f0f2806d330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0f38036000 tx=0x7f0f38031000 comp rx=0 tx=0).stop 2023-11-13T11:03:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f44145da0 msgr2=0x7f0f44146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f44145da0 0x7f0f44146220 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0f300072c0 tx=0x7f0f30007780 comp rx=0 tx=0).stop 2023-11-13T11:03:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 shutdown_connections 2023-11-13T11:03:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0f2806ae70 0x7f0f2806d330 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f4406dd40 0x7f0f4406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f44145da0 0x7f0f44146220 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 --2- 172.21.15.16:0/2732151430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f34003970 0x7f0f4406ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 >> 172.21.15.16:0/2732151430 conn(0x7f0f440b18c0 msgr2=0x7f0f44156830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 shutdown_connections 2023-11-13T11:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.352+0000 7f0f4ac94640 1 -- 172.21.15.16:0/2732151430 wait complete. 2023-11-13T11:03:04.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.600+0000 7fa11883f640 1 Processor -- start 2023-11-13T11:03:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.600+0000 7fa11883f640 1 -- start start 2023-11-13T11:03:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.602+0000 7fa11883f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1140ccc60 0x7fa114071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.602+0000 7fa11883f640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa114071980 0x7fa11406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.602+0000 7fa11883f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1140cd110 con 0x7fa114155410 2023-11-13T11:03:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.602+0000 7fa11883f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa114071e10 con 0x7fa1140ccc60 2023-11-13T11:03:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.602+0000 7fa11883f640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11415b500 con 0x7fa114071980 2023-11-13T11:03:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.602+0000 7fa10bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1140ccc60 0x7fa114071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.603+0000 7fa10bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1140ccc60 0x7fa114071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58256/0 (socket says 172.21.15.16:58256) 2023-11-13T11:03:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.603+0000 7fa10bfff640 1 -- 172.21.15.16:0/1255493161 learned_addr learned my addr 172.21.15.16:0/1255493161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:04.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.603+0000 7fa112d9d640 1 --2- 172.21.15.16:0/1255493161 >> v2:172.21.15.184:3300/0 conn(0x7fa114071980 0x7fa11406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.603+0000 7fa10bfff640 1 -- 172.21.15.16:0/1255493161 >> v2:172.21.15.184:3300/0 conn(0x7fa114071980 msgr2=0x7fa11406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.603+0000 7fa10bfff640 1 --2- 172.21.15.16:0/1255493161 >> v2:172.21.15.184:3300/0 conn(0x7fa114071980 0x7fa11406fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.603+0000 7fa10bfff640 1 -- 172.21.15.16:0/1255493161 >> v1:172.21.15.16:6789/0 conn(0x7fa114155410 legacy=0x7fa1140ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.603+0000 7fa10bfff640 1 -- 172.21.15.16:0/1255493161 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11415b7e0 con 0x7fa1140ccc60 2023-11-13T11:03:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.604+0000 7fa10bfff640 1 --2- 172.21.15.16:0/1255493161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 0x7fa114071440 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa11415e700 tx=0x7fa0fc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f2de28d3e4a413 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.605+0000 7fa111d9b640 1 -- 172.21.15.16:0/1255493161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc030860 con 0x7fa1140ccc60 2023-11-13T11:03:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.605+0000 7fa111d9b640 1 -- 172.21.15.16:0/1255493161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc0309c0 con 0x7fa1140ccc60 2023-11-13T11:03:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.605+0000 7fa111d9b640 1 -- 172.21.15.16:0/1255493161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc030d50 con 0x7fa1140ccc60 2023-11-13T11:03:04.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.605+0000 7fa11883f640 1 -- 172.21.15.16:0/1255493161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 msgr2=0x7fa114071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.605+0000 7fa11883f640 1 --2- 172.21.15.16:0/1255493161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 0x7fa114071440 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa11415e700 tx=0x7fa0fc02b9a0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.605+0000 7fa11883f640 1 -- 172.21.15.16:0/1255493161 shutdown_connections 2023-11-13T11:03:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.605+0000 7fa11883f640 1 --2- 172.21.15.16:0/1255493161 >> v2:172.21.15.184:3300/0 conn(0x7fa114071980 0x7fa11406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.605+0000 7fa11883f640 1 --2- 172.21.15.16:0/1255493161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 0x7fa114071440 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.605+0000 7fa11883f640 1 -- 172.21.15.16:0/1255493161 >> 172.21.15.16:0/1255493161 conn(0x7fa11406a190 msgr2=0x7fa11406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:04.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.606+0000 7fa11883f640 1 -- 172.21.15.16:0/1255493161 shutdown_connections 2023-11-13T11:03:04.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.606+0000 7fa11883f640 1 -- 172.21.15.16:0/1255493161 wait complete. 2023-11-13T11:03:04.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.607+0000 7fa11883f640 1 Processor -- start 2023-11-13T11:03:04.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.607+0000 7fa11883f640 1 -- start start 2023-11-13T11:03:04.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.608+0000 7fa11883f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa114071980 0x7fa1141578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.608+0000 7fa11883f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 0x7fa1140a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.608+0000 7fa11883f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa114155410 0x7fa1140ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.608+0000 7fa11883f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1140ad130 con 0x7fa1140ccc60 2023-11-13T11:03:04.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.608+0000 7fa11883f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1140a8b60 con 0x7fa114155410 2023-11-13T11:03:04.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.608+0000 7fa11883f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1140a8e40 con 0x7fa114071980 2023-11-13T11:03:04.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 0x7fa1140a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 0x7fa1140a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58266/0 (socket says 172.21.15.16:58266) 2023-11-13T11:03:04.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa10bfff640 1 -- 172.21.15.16:0/2403908630 learned_addr learned my addr 172.21.15.16:0/2403908630 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:04.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa10bfff640 1 -- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa114071980 msgr2=0x7fa1141578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa112d9d640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa114155410 0x7fa1140ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa11259c640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa114071980 0x7fa1141578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa10bfff640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa114071980 0x7fa1141578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa10bfff640 1 -- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa114155410 msgr2=0x7fa1140ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa10bfff640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa114155410 0x7fa1140ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa10bfff640 1 -- 172.21.15.16:0/2403908630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0094d0 con 0x7fa1140ccc60 2023-11-13T11:03:04.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.609+0000 7fa112d9d640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa114155410 0x7fa1140ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:04.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.610+0000 7fa10bfff640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 0x7fa1140a84d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc02b9e0 tx=0x7fa0fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:04.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.610+0000 7fa10affd640 1 -- 172.21.15.16:0/2403908630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc039d60 con 0x7fa1140ccc60 2023-11-13T11:03:04.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.610+0000 7fa10affd640 1 -- 172.21.15.16:0/2403908630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc043400 con 0x7fa1140ccc60 2023-11-13T11:03:04.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.610+0000 7fa10affd640 1 -- 172.21.15.16:0/2403908630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0437b0 con 0x7fa1140ccc60 2023-11-13T11:03:04.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.610+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1140a9120 con 0x7fa1140ccc60 2023-11-13T11:03:04.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.610+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1140a96e0 con 0x7fa1140ccc60 2023-11-13T11:03:04.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.611+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa114067090 con 0x7fa1140ccc60 2023-11-13T11:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.612+0000 7fa10affd640 1 -- 172.21.15.16:0/2403908630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0fc043910 con 0x7fa1140ccc60 2023-11-13T11:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.613+0000 7fa10affd640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0f406ae40 0x7fa0f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:04.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.613+0000 7fa10affd640 1 -- 172.21.15.16:0/2403908630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fa0fc03e070 con 0x7fa1140ccc60 2023-11-13T11:03:04.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.614+0000 7fa11259c640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0f406ae40 0x7fa0f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:04.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.615+0000 7fa11259c640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0f406ae40 0x7fa0f406d300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa1140b2040 tx=0x7fa104009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:04.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.617+0000 7fa10affd640 1 -- 172.21.15.16:0/2403908630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0fc077a50 con 0x7fa1140ccc60 2023-11-13T11:03:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.818+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1140ae1d0 con 0x7fa1140ccc60 2023-11-13T11:03:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.821+0000 7fa10affd640 1 -- 172.21.15.16:0/2403908630 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16821 (secure 0 0 0) 0x7fa0fc07b070 con 0x7fa1140ccc60 2023-11-13T11:03:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:04.827 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:02:08.953574+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:01:36.305098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-13T11:01:37.461852+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.4","mappings":[{"from":2,"to":5}]},{"pgid":"8.20","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.824+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0f406ae40 msgr2=0x7fa0f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.824+0000 7fa11883f640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0f406ae40 0x7fa0f406d300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa1140b2040 tx=0x7fa104009040 comp rx=0 tx=0).stop 2023-11-13T11:03:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.824+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 msgr2=0x7fa1140a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.824+0000 7fa11883f640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 0x7fa1140a84d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc02b9e0 tx=0x7fa0fc004730 comp rx=0 tx=0).stop 2023-11-13T11:03:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.824+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 shutdown_connections 2023-11-13T11:03:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.824+0000 7fa11883f640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa0f406ae40 0x7fa0f406d300 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.824+0000 7fa11883f640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa114155410 0x7fa1140ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.824+0000 7fa11883f640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1140ccc60 0x7fa1140a84d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.824+0000 7fa11883f640 1 --2- 172.21.15.16:0/2403908630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa114071980 0x7fa1141578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.825+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 >> 172.21.15.16:0/2403908630 conn(0x7fa11406a190 msgr2=0x7fa1140b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.825+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 shutdown_connections 2023-11-13T11:03:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:04.825+0000 7fa11883f640 1 -- 172.21.15.16:0/2403908630 wait complete. 2023-11-13T11:03:04.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-13T11:03:05.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.077+0000 7fc7614d1640 1 Processor -- start 2023-11-13T11:03:05.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.078+0000 7fc7614d1640 1 -- start start 2023-11-13T11:03:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.079+0000 7fc7614d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c06a8a0 0x7fc75c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.079+0000 7fc7614d1640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc75c070cf0 0x7fc75c0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.079+0000 7fc7614d1640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75c06fc50 con 0x7fc75c15cc90 2023-11-13T11:03:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.079+0000 7fc7614d1640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75c06ff30 con 0x7fc75c070cf0 2023-11-13T11:03:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.079+0000 7fc7614d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c070210 con 0x7fc75c06a8a0 2023-11-13T11:03:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.080+0000 7fc760cd0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc75c070cf0 0x7fc75c0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.080+0000 7fc753fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c06a8a0 0x7fc75c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.080+0000 7fc760cd0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc75c070cf0 0x7fc75c0b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49724/0 (socket says 172.21.15.16:49724) 2023-11-13T11:03:05.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.080+0000 7fc753fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c06a8a0 0x7fc75c0b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:36732/0 (socket says 172.21.15.16:36732) 2023-11-13T11:03:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.080+0000 7fc760cd0640 1 -- 172.21.15.16:0/3277553689 learned_addr learned my addr 172.21.15.16:0/3277553689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:05.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.081+0000 7fc75b7fe640 1 -- 172.21.15.16:0/3277553689 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779571203 0 0) 0x7fc75c06fc50 con 0x7fc75c15cc90 2023-11-13T11:03:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.081+0000 7fc75b7fe640 1 -- 172.21.15.16:0/3277553689 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc748003520 con 0x7fc75c15cc90 2023-11-13T11:03:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.081+0000 7fc753fff640 1 -- 172.21.15.16:0/3277553689 >> v2:172.21.15.184:3300/0 conn(0x7fc75c070cf0 msgr2=0x7fc75c0b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.081+0000 7fc753fff640 1 --2- 172.21.15.16:0/3277553689 >> v2:172.21.15.184:3300/0 conn(0x7fc75c070cf0 0x7fc75c0b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.081+0000 7fc753fff640 1 -- 172.21.15.16:0/3277553689 >> v1:172.21.15.184:6789/0 conn(0x7fc75c15cc90 legacy=0x7fc75c06a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.081+0000 7fc753fff640 1 -- 172.21.15.16:0/3277553689 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c0704f0 con 0x7fc75c06a8a0 2023-11-13T11:03:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.082+0000 7fc753fff640 1 --2- 172.21.15.16:0/3277553689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c06a8a0 0x7fc75c0b2110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc7440097a0 tx=0x7fc74402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=723e186f4c75dc29 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.083+0000 7fc75b7fe640 1 -- 172.21.15.16:0/3277553689 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74402d040 con 0x7fc75c06a8a0 2023-11-13T11:03:05.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.083+0000 7fc75b7fe640 1 -- 172.21.15.16:0/3277553689 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744031810 con 0x7fc75c06a8a0 2023-11-13T11:03:05.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.083+0000 7fc75b7fe640 1 -- 172.21.15.16:0/3277553689 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744031ba0 con 0x7fc75c06a8a0 2023-11-13T11:03:05.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.084+0000 7fc7614d1640 1 -- 172.21.15.16:0/3277553689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c06a8a0 msgr2=0x7fc75c0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.084+0000 7fc7614d1640 1 --2- 172.21.15.16:0/3277553689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c06a8a0 0x7fc75c0b2110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc7440097a0 tx=0x7fc74402b8b0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.084+0000 7fc7614d1640 1 -- 172.21.15.16:0/3277553689 shutdown_connections 2023-11-13T11:03:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.084+0000 7fc7614d1640 1 --2- 172.21.15.16:0/3277553689 >> v2:172.21.15.184:3300/0 conn(0x7fc75c070cf0 0x7fc75c0b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.084+0000 7fc7614d1640 1 --2- 172.21.15.16:0/3277553689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c06a8a0 0x7fc75c0b2110 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.084+0000 7fc7614d1640 1 -- 172.21.15.16:0/3277553689 >> 172.21.15.16:0/3277553689 conn(0x7fc75c0b04a0 msgr2=0x7fc75c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.084+0000 7fc7614d1640 1 -- 172.21.15.16:0/3277553689 shutdown_connections 2023-11-13T11:03:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.084+0000 7fc7614d1640 1 -- 172.21.15.16:0/3277553689 wait complete. 2023-11-13T11:03:05.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.086+0000 7fc7614d1640 1 Processor -- start 2023-11-13T11:03:05.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.086+0000 7fc7614d1640 1 -- start start 2023-11-13T11:03:05.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.087+0000 7fc7614d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c06a8a0 0x7fc75c15ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.087+0000 7fc7614d1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc75c070cf0 0x7fc75c147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.087+0000 7fc7614d1640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c15cc90 0x7fc75c159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.087+0000 7fc7614d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c160480 con 0x7fc75c06a8a0 2023-11-13T11:03:05.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.087+0000 7fc7614d1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c15a500 con 0x7fc75c070cf0 2023-11-13T11:03:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.087+0000 7fc7614d1640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c1477a0 con 0x7fc75c15cc90 2023-11-13T11:03:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.087+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c06a8a0 0x7fc75c15ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.087+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c06a8a0 0x7fc75c15ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58272/0 (socket says 172.21.15.16:58272) 2023-11-13T11:03:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.087+0000 7fc75bfff640 1 -- 172.21.15.16:0/1834562860 learned_addr learned my addr 172.21.15.16:0/1834562860 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.088+0000 7fc753fff640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc75c070cf0 0x7fc75c147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.088+0000 7fc760cd0640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c15cc90 0x7fc75c159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.088+0000 7fc75bfff640 1 -- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c15cc90 msgr2=0x7fc75c159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.088+0000 7fc75bfff640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c15cc90 0x7fc75c159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.088+0000 7fc75bfff640 1 -- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc75c070cf0 msgr2=0x7fc75c147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.088+0000 7fc75bfff640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc75c070cf0 0x7fc75c147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.088+0000 7fc75bfff640 1 -- 172.21.15.16:0/1834562860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc744009400 con 0x7fc75c06a8a0 2023-11-13T11:03:05.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.088+0000 7fc75bfff640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c06a8a0 0x7fc75c15ff40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00c430 tx=0x7fc74c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:05.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.089+0000 7fc7597fa640 1 -- 172.21.15.16:0/1834562860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0052d0 con 0x7fc75c06a8a0 2023-11-13T11:03:05.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.089+0000 7fc7597fa640 1 -- 172.21.15.16:0/1834562860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c005450 con 0x7fc75c06a8a0 2023-11-13T11:03:05.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.089+0000 7fc7597fa640 1 -- 172.21.15.16:0/1834562860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c003830 con 0x7fc75c06a8a0 2023-11-13T11:03:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.089+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75c153b30 con 0x7fc75c06a8a0 2023-11-13T11:03:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.089+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75c15aa50 con 0x7fc75c06a8a0 2023-11-13T11:03:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.090+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75c04e4d0 con 0x7fc75c06a8a0 2023-11-13T11:03:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.091+0000 7fc7597fa640 1 -- 172.21.15.16:0/1834562860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc74c003990 con 0x7fc75c06a8a0 2023-11-13T11:03:05.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.092+0000 7fc7597fa640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc73c06ae40 0x7fc73c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.092+0000 7fc7597fa640 1 -- 172.21.15.16:0/1834562860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fc74c088e80 con 0x7fc75c06a8a0 2023-11-13T11:03:05.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.093+0000 7fc753fff640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc73c06ae40 0x7fc73c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.094+0000 7fc753fff640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc73c06ae40 0x7fc73c06d300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc744036000 tx=0x7fc74403c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:05.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.096+0000 7fc7597fa640 1 -- 172.21.15.16:0/1834562860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc74c053c90 con 0x7fc75c06a8a0 2023-11-13T11:03:05.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.327+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc75c0713a0 con 0x7fc75c06a8a0 2023-11-13T11:03:05.333 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-13T11:03:05.328+0000 7fe1550d2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-13T11:03:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.331+0000 7fc7597fa640 1 -- 172.21.15.16:0/1834562860 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc74c0572b0 con 0x7fc75c06a8a0 2023-11-13T11:03:05.337 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-13T11:03:05.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc73c06ae40 msgr2=0x7fc73c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc73c06ae40 0x7fc73c06d300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc744036000 tx=0x7fc74403c040 comp rx=0 tx=0).stop 2023-11-13T11:03:05.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c06a8a0 msgr2=0x7fc75c15ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c06a8a0 0x7fc75c15ff40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00c430 tx=0x7fc74c00c8f0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 shutdown_connections 2023-11-13T11:03:05.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc73c06ae40 0x7fc73c06d300 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc75c15cc90 0x7fc75c159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc75c070cf0 0x7fc75c147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 --2- 172.21.15.16:0/1834562860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c06a8a0 0x7fc75c15ff40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.334+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 >> 172.21.15.16:0/1834562860 conn(0x7fc75c0b04a0 msgr2=0x7fc75c0cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.335+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 shutdown_connections 2023-11-13T11:03:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.336+0000 7fc7614d1640 1 -- 172.21.15.16:0/1834562860 wait complete. 2023-11-13T11:03:05.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-13T11:03:05.464 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-13T11:03:05.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.563+0000 7f111bb33640 1 Processor -- start 2023-11-13T11:03:05.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.563+0000 7f111bb33640 1 -- start start 2023-11-13T11:03:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.564+0000 7f111bb33640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f11140ce9b0 0x7f11140b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.565+0000 7f111bb33640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1114144950 con 0x7f11140ce550 2023-11-13T11:03:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.565+0000 7f111bb33640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11140ce300 con 0x7f1114144570 2023-11-13T11:03:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.565+0000 7f111bb33640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11140cd260 con 0x7f11140ce9b0 2023-11-13T11:03:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.565+0000 7f11190ce640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f11140ce550 0x7f11140b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33346/0 (socket says 172.21.15.16:33346) 2023-11-13T11:03:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.565+0000 7f11190ce640 1 -- 172.21.15.16:0/1574928732 learned_addr learned my addr 172.21.15.16:0/1574928732 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:05.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.565+0000 7f111a0d0640 1 --2- 172.21.15.16:0/1574928732 >> v2:172.21.15.184:3300/0 conn(0x7f11140ce9b0 0x7f11140b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.565+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680288562 0 0) 0x7f1114144950 con 0x7f11140ce550 2023-11-13T11:03:05.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.565+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1100003520 con 0x7f11140ce550 2023-11-13T11:03:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4191707711 0 0) 0x7f1100003520 con 0x7f11140ce550 2023-11-13T11:03:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 >> v2:172.21.15.184:3300/0 conn(0x7f11140ce9b0 msgr2=0x7f11140b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 --2- 172.21.15.16:0/1574928732 >> v2:172.21.15.184:3300/0 conn(0x7f11140ce9b0 0x7f11140b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 >> v1:172.21.15.184:6789/0 conn(0x7f1114144570 legacy=0x7f111415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11140cd540 con 0x7f11140ce550 2023-11-13T11:03:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f1104001670 con 0x7f11140ce550 2023-11-13T11:03:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 >> v1:172.21.15.16:6789/0 conn(0x7f11140ce550 legacy=0x7f11140b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1100003970 0x7f1100003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11000047c0 0x7f1100004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1100005180 0x7f110000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11140cd260 con 0x7f1100005180 2023-11-13T11:03:05.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110000c000 con 0x7f1100003970 2023-11-13T11:03:05.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110000c1c0 con 0x7f11000047c0 2023-11-13T11:03:05.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.566+0000 7f111a0d0640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1100005180 0x7f110000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.567+0000 7f11190ce640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11000047c0 0x7f1100004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.567+0000 7f11198cf640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1100003970 0x7f1100003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.567+0000 7f111a0d0640 1 -- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11000047c0 msgr2=0x7f1100004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:05.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.567+0000 7f111a0d0640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11000047c0 0x7f1100004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.567+0000 7f111a0d0640 1 -- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1100003970 msgr2=0x7f1100003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:05.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.567+0000 7f111a0d0640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1100003970 0x7f1100003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.567+0000 7f111a0d0640 1 -- 172.21.15.16:0/1574928732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1100003520 con 0x7f1100005180 2023-11-13T11:03:05.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.567+0000 7f11190ce640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f11000047c0 0x7f1100004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:05.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.567+0000 7f11198cf640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1100003970 0x7f1100003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:05.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.568+0000 7f111a0d0640 1 -- 172.21.15.16:0/1574928732 reap_dead start 2023-11-13T11:03:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.568+0000 7f111a0d0640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1100005180 0x7f110000b980 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f110c009fc0 tx=0x7f11000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6276f12b5405110 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.568+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c02cc30 con 0x7f1100005180 2023-11-13T11:03:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.568+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c00f040 con 0x7f1100005180 2023-11-13T11:03:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.568+0000 7f11188cd640 1 -- 172.21.15.16:0/1574928732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c003990 con 0x7f1100005180 2023-11-13T11:03:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.569+0000 7f111bb33640 1 -- 172.21.15.16:0/1574928732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1100005180 msgr2=0x7f110000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.569+0000 7f111bb33640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1100005180 0x7f110000b980 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f110c009fc0 tx=0x7f11000057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.569+0000 7f111bb33640 1 -- 172.21.15.16:0/1574928732 shutdown_connections 2023-11-13T11:03:05.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.569+0000 7f111bb33640 1 --2- 172.21.15.16:0/1574928732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1100005180 0x7f110000b980 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.569+0000 7f111bb33640 1 -- 172.21.15.16:0/1574928732 >> 172.21.15.16:0/1574928732 conn(0x7f111406a190 msgr2=0x7f111406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.569+0000 7f111bb33640 1 -- 172.21.15.16:0/1574928732 shutdown_connections 2023-11-13T11:03:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.569+0000 7f111bb33640 1 -- 172.21.15.16:0/1574928732 wait complete. 2023-11-13T11:03:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.571+0000 7f111bb33640 1 Processor -- start 2023-11-13T11:03:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.571+0000 7f111bb33640 1 -- start start 2023-11-13T11:03:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111bb33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1100005180 0x7f11141eef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111bb33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1114138a30 0x7f111415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111bb33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1114138eb0 0x7f1114157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111bb33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11141efc80 con 0x7f1114138eb0 2023-11-13T11:03:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111bb33640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111413a260 con 0x7f1114138a30 2023-11-13T11:03:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111bb33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1114139330 con 0x7f1100005180 2023-11-13T11:03:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111a0d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1114138eb0 0x7f1114157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111a0d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1114138eb0 0x7f1114157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58314/0 (socket says 172.21.15.16:58314) 2023-11-13T11:03:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111a0d0640 1 -- 172.21.15.16:0/3672753605 learned_addr learned my addr 172.21.15.16:0/3672753605 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f11198cf640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1100005180 0x7f11141eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111a0d0640 1 -- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1100005180 msgr2=0x7f11141eef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111a0d0640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1100005180 0x7f11141eef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111a0d0640 1 -- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1114138a30 msgr2=0x7f111415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f11190ce640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1114138a30 0x7f111415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.572+0000 7f111a0d0640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1114138a30 0x7f111415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.573+0000 7f111a0d0640 1 -- 172.21.15.16:0/3672753605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11141ef5c0 con 0x7f1114138eb0 2023-11-13T11:03:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.573+0000 7f11198cf640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1100005180 0x7f11141eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.573+0000 7f11190ce640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1114138a30 0x7f111415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.573+0000 7f111a0d0640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1114138eb0 0x7f1114157230 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f110c02c6a0 tx=0x7f110c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.573+0000 7f110a7fc640 1 -- 172.21.15.16:0/3672753605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c02cc30 con 0x7f1114138eb0 2023-11-13T11:03:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.573+0000 7f110a7fc640 1 -- 172.21.15.16:0/3672753605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c003990 con 0x7f1114138eb0 2023-11-13T11:03:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.573+0000 7f110a7fc640 1 -- 172.21.15.16:0/3672753605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c003dc0 con 0x7f1114138eb0 2023-11-13T11:03:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.573+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11141ef8a0 con 0x7f1114138eb0 2023-11-13T11:03:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.573+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111415d6d0 con 0x7f1114138eb0 2023-11-13T11:03:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.574+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11141efdc0 con 0x7f1114138eb0 2023-11-13T11:03:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.575+0000 7f110a7fc640 1 -- 172.21.15.16:0/3672753605 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f110c004030 con 0x7f1114138eb0 2023-11-13T11:03:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.576+0000 7f110a7fc640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f10f806ae40 0x7f10f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.576+0000 7f110a7fc640 1 -- 172.21.15.16:0/3672753605 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f110c0ac210 con 0x7f1114138eb0 2023-11-13T11:03:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.577+0000 7f11198cf640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f10f806ae40 0x7f10f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.578+0000 7f11198cf640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f10f806ae40 0x7f10f806d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1110003dc0 tx=0x7f1110009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.580+0000 7f110a7fc640 1 -- 172.21.15.16:0/3672753605 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11141efdc0 con 0x7f1114138eb0 2023-11-13T11:03:05.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:05.811+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f11141efdc0 con 0x7f1114138eb0 2023-11-13T11:03:06.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.335+0000 7f110a7fc640 1 -- 172.21.15.16:0/3672753605 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f11141efdc0 con 0x7f1114138eb0 2023-11-13T11:03:06.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.338+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f10f806ae40 msgr2=0x7f10f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:06.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.338+0000 7f111bb33640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f10f806ae40 0x7f10f806d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1110003dc0 tx=0x7f1110009040 comp rx=0 tx=0).stop 2023-11-13T11:03:06.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.338+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1114138eb0 msgr2=0x7f1114157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:06.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.338+0000 7f111bb33640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1114138eb0 0x7f1114157230 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f110c02c6a0 tx=0x7f110c009750 comp rx=0 tx=0).stop 2023-11-13T11:03:06.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.339+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 shutdown_connections 2023-11-13T11:03:06.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.339+0000 7f111bb33640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f10f806ae40 0x7f10f806d300 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:06.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.339+0000 7f111bb33640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1114138eb0 0x7f1114157230 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:06.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.339+0000 7f111bb33640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1114138a30 0x7f111415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:06.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.339+0000 7f111bb33640 1 --2- 172.21.15.16:0/3672753605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1100005180 0x7f11141eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:06.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.339+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 >> 172.21.15.16:0/3672753605 conn(0x7f111406a190 msgr2=0x7f111406fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:06.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.339+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 shutdown_connections 2023-11-13T11:03:06.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.339+0000 7f111bb33640 1 -- 172.21.15.16:0/3672753605 wait complete. 2023-11-13T11:03:06.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-13T11:03:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.585+0000 7f3e71a40640 1 Processor -- start 2023-11-13T11:03:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.585+0000 7f3e71a40640 1 -- start start 2023-11-13T11:03:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.587+0000 7f3e71a40640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e6c146320 0x7f3e6c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.587+0000 7f3e71a40640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e6c06c2e0 0x7f3e6c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:06.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.587+0000 7f3e71a40640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6c06c940 con 0x7f3e6c0ce4a0 2023-11-13T11:03:06.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.587+0000 7f3e71a40640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6c0cee60 con 0x7f3e6c146320 2023-11-13T11:03:06.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.587+0000 7f3e71a40640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6c06c720 con 0x7f3e6c06c2e0 2023-11-13T11:03:06.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.587+0000 7f3e70a3e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e6c146320 0x7f3e6c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:06.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.587+0000 7f3e70a3e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e6c146320 0x7f3e6c0b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58318/0 (socket says 172.21.15.16:58318) 2023-11-13T11:03:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.587+0000 7f3e70a3e640 1 -- 172.21.15.16:0/1761342671 learned_addr learned my addr 172.21.15.16:0/1761342671 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.588+0000 7f3e7123f640 1 --2- 172.21.15.16:0/1761342671 >> v2:172.21.15.184:3300/0 conn(0x7f3e6c06c2e0 0x7f3e6c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.588+0000 7f3e70a3e640 1 -- 172.21.15.16:0/1761342671 >> v2:172.21.15.184:3300/0 conn(0x7f3e6c06c2e0 msgr2=0x7f3e6c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.588+0000 7f3e70a3e640 1 --2- 172.21.15.16:0/1761342671 >> v2:172.21.15.184:3300/0 conn(0x7f3e6c06c2e0 0x7f3e6c0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.588+0000 7f3e70a3e640 1 -- 172.21.15.16:0/1761342671 >> v1:172.21.15.184:6789/0 conn(0x7f3e6c0ce4a0 legacy=0x7f3e6c06bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.588+0000 7f3e70a3e640 1 -- 172.21.15.16:0/1761342671 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c070480 con 0x7f3e6c146320 2023-11-13T11:03:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.589+0000 7f3e70a3e640 1 --2- 172.21.15.16:0/1761342671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 0x7f3e6c0b1ad0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0cf760 tx=0x7f3e6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a256bafd4ec9dc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.589+0000 7f3e6b7fe640 1 -- 172.21.15.16:0/1761342671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60031870 con 0x7f3e6c146320 2023-11-13T11:03:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.589+0000 7f3e6b7fe640 1 -- 172.21.15.16:0/1761342671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e600319d0 con 0x7f3e6c146320 2023-11-13T11:03:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.589+0000 7f3e6b7fe640 1 -- 172.21.15.16:0/1761342671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60031d60 con 0x7f3e6c146320 2023-11-13T11:03:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.590+0000 7f3e71a40640 1 -- 172.21.15.16:0/1761342671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 msgr2=0x7f3e6c0b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.590+0000 7f3e71a40640 1 --2- 172.21.15.16:0/1761342671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 0x7f3e6c0b1ad0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0cf760 tx=0x7f3e6002b910 comp rx=0 tx=0).stop 2023-11-13T11:03:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.590+0000 7f3e71a40640 1 -- 172.21.15.16:0/1761342671 shutdown_connections 2023-11-13T11:03:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.590+0000 7f3e71a40640 1 --2- 172.21.15.16:0/1761342671 >> v2:172.21.15.184:3300/0 conn(0x7f3e6c06c2e0 0x7f3e6c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.590+0000 7f3e71a40640 1 --2- 172.21.15.16:0/1761342671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 0x7f3e6c0b1ad0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.590+0000 7f3e71a40640 1 -- 172.21.15.16:0/1761342671 >> 172.21.15.16:0/1761342671 conn(0x7f3e6c06a190 msgr2=0x7f3e6c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.590+0000 7f3e71a40640 1 -- 172.21.15.16:0/1761342671 shutdown_connections 2023-11-13T11:03:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.590+0000 7f3e71a40640 1 -- 172.21.15.16:0/1761342671 wait complete. 2023-11-13T11:03:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.592+0000 7f3e71a40640 1 Processor -- start 2023-11-13T11:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.592+0000 7f3e71a40640 1 -- start start 2023-11-13T11:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e71a40640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e6c06c2e0 0x7f3e6c15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e71a40640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e6c0ce4a0 0x7f3e6c0af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e71a40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 0x7f3e6c141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e71a40640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c1f5d90 con 0x7f3e6c146320 2023-11-13T11:03:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e71a40640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c1f6070 con 0x7f3e6c0ce4a0 2023-11-13T11:03:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e71a40640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c1f6350 con 0x7f3e6c06c2e0 2023-11-13T11:03:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e7123f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 0x7f3e6c141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e7123f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 0x7f3e6c141d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58320/0 (socket says 172.21.15.16:58320) 2023-11-13T11:03:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e7123f640 1 -- 172.21.15.16:0/991927267 learned_addr learned my addr 172.21.15.16:0/991927267 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.593+0000 7f3e6bfff640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e6c0ce4a0 0x7f3e6c0af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e70a3e640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e6c06c2e0 0x7f3e6c15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e7123f640 1 -- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e6c06c2e0 msgr2=0x7f3e6c15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e7123f640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e6c06c2e0 0x7f3e6c15b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e7123f640 1 -- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e6c0ce4a0 msgr2=0x7f3e6c0af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e7123f640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e6c0ce4a0 0x7f3e6c0af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e7123f640 1 -- 172.21.15.16:0/991927267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e600094d0 con 0x7f3e6c146320 2023-11-13T11:03:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e70a3e640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e6c06c2e0 0x7f3e6c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e7123f640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 0x7f3e6c141d20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3e64007e40 tx=0x7f3e6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e697fa640 1 -- 172.21.15.16:0/991927267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64014620 con 0x7f3e6c146320 2023-11-13T11:03:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e697fa640 1 -- 172.21.15.16:0/991927267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6400ee90 con 0x7f3e6c146320 2023-11-13T11:03:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e697fa640 1 -- 172.21.15.16:0/991927267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e640149e0 con 0x7f3e6c146320 2023-11-13T11:03:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c1f6630 con 0x7f3e6c146320 2023-11-13T11:03:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.594+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c1f6bf0 con 0x7f3e6c146320 2023-11-13T11:03:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.595+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c04e4d0 con 0x7f3e6c146320 2023-11-13T11:03:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.596+0000 7f3e697fa640 1 -- 172.21.15.16:0/991927267 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e64014b40 con 0x7f3e6c146320 2023-11-13T11:03:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.597+0000 7f3e697fa640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e5006ae70 0x7f3e5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.597+0000 7f3e697fa640 1 -- 172.21.15.16:0/991927267 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f3e6408a1f0 con 0x7f3e6c146320 2023-11-13T11:03:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.598+0000 7f3e70a3e640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e5006ae70 0x7f3e5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:06.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.599+0000 7f3e70a3e640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e5006ae70 0x7f3e5006d330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3e60036000 tx=0x7f3e6003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:06.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.602+0000 7f3e697fa640 1 -- 172.21.15.16:0/991927267 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e64055030 con 0x7f3e6c146320 2023-11-13T11:03:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:06.794+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3e6c1584f0 con 0x7f3e6c146320 2023-11-13T11:03:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.338+0000 7f3e697fa640 1 -- 172.21.15.16:0/991927267 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3e64058650 con 0x7f3e6c146320 2023-11-13T11:03:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-13T11:03:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.343+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e5006ae70 msgr2=0x7f3e5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.343+0000 7f3e71a40640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e5006ae70 0x7f3e5006d330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3e60036000 tx=0x7f3e6003c040 comp rx=0 tx=0).stop 2023-11-13T11:03:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.343+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 msgr2=0x7f3e6c141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.344+0000 7f3e71a40640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 0x7f3e6c141d20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3e64007e40 tx=0x7f3e6400e4b0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.344+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 shutdown_connections 2023-11-13T11:03:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.344+0000 7f3e71a40640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3e5006ae70 0x7f3e5006d330 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.344+0000 7f3e71a40640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e6c146320 0x7f3e6c141d20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.344+0000 7f3e71a40640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e6c0ce4a0 0x7f3e6c0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.344+0000 7f3e71a40640 1 --2- 172.21.15.16:0/991927267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e6c06c2e0 0x7f3e6c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.344+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 >> 172.21.15.16:0/991927267 conn(0x7f3e6c06a190 msgr2=0x7f3e6c15e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.344+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 shutdown_connections 2023-11-13T11:03:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.345+0000 7f3e71a40640 1 -- 172.21.15.16:0/991927267 wait complete. 2023-11-13T11:03:07.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-13T11:03:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.606+0000 7fce87fff640 1 Processor -- start 2023-11-13T11:03:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.606+0000 7fce87fff640 1 -- start start 2023-11-13T11:03:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce87fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881622e0 0x7fce880b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce87fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fce880b29e0 0x7fce880ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce87fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8806bb10 con 0x7fce8815b650 2023-11-13T11:03:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce87fff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce880b2e70 con 0x7fce880b29e0 2023-11-13T11:03:07.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce87fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce88071830 con 0x7fce881622e0 2023-11-13T11:03:07.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce86ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce8815b650 0x7fce880707a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33354/0 (socket says 172.21.15.16:33354) 2023-11-13T11:03:07.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce86ffd640 1 -- 172.21.15.16:0/2393690797 learned_addr learned my addr 172.21.15.16:0/2393690797 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce877fe640 1 --2- 172.21.15.16:0/2393690797 >> v2:172.21.15.184:3300/0 conn(0x7fce880b29e0 0x7fce880ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296508609 0 0) 0x7fce8806bb10 con 0x7fce8815b650 2023-11-13T11:03:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce867fc640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881622e0 0x7fce880b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.608+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce70003520 con 0x7fce8815b650 2023-11-13T11:03:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 488027262 0 0) 0x7fce70003520 con 0x7fce8815b650 2023-11-13T11:03:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881622e0 msgr2=0x7fce880b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881622e0 0x7fce880b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 >> v2:172.21.15.184:3300/0 conn(0x7fce880b29e0 msgr2=0x7fce880ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 --2- 172.21.15.16:0/2393690797 >> v2:172.21.15.184:3300/0 conn(0x7fce880b29e0 0x7fce880ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce88070f90 con 0x7fce8815b650 2023-11-13T11:03:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fce78001670 con 0x7fce8815b650 2023-11-13T11:03:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 >> v1:172.21.15.16:6789/0 conn(0x7fce8815b650 legacy=0x7fce880707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:07.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce70003970 0x7fce70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:07.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881622e0 0x7fce700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:07.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:07.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce880b2e70 con 0x7fce70004d00 2023-11-13T11:03:07.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce88071830 con 0x7fce70003970 2023-11-13T11:03:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7000b9b0 con 0x7fce881622e0 2023-11-13T11:03:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.609+0000 7fce877fe640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.610+0000 7fce86ffd640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce70003970 0x7fce70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.610+0000 7fce877fe640 1 -- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881622e0 msgr2=0x7fce700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.610+0000 7fce877fe640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881622e0 0x7fce700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.610+0000 7fce877fe640 1 -- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce70003970 msgr2=0x7fce70003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.610+0000 7fce877fe640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce70003970 0x7fce70003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.610+0000 7fce877fe640 1 -- 172.21.15.16:0/2393690797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce70003520 con 0x7fce70004d00 2023-11-13T11:03:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.611+0000 7fce877fe640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce70005180 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fce80009fc0 tx=0x7fce700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e456c74e6c94fb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.611+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8002cc60 con 0x7fce70004d00 2023-11-13T11:03:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.611+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8000f040 con 0x7fce70004d00 2023-11-13T11:03:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.611+0000 7fce85ffb640 1 -- 172.21.15.16:0/2393690797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80034440 con 0x7fce70004d00 2023-11-13T11:03:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.612+0000 7fce87fff640 1 -- 172.21.15.16:0/2393690797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 msgr2=0x7fce70005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.612+0000 7fce87fff640 1 --2- 172.21.15.16:0/2393690797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce70005180 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fce80009fc0 tx=0x7fce700057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.612+0000 7fce877fe640 1 -- 172.21.15.16:0/2393690797 reap_dead start 2023-11-13T11:03:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.612+0000 7fce87fff640 1 -- 172.21.15.16:0/2393690797 shutdown_connections 2023-11-13T11:03:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.612+0000 7fce87fff640 1 -- 172.21.15.16:0/2393690797 >> 172.21.15.16:0/2393690797 conn(0x7fce880b0970 msgr2=0x7fce880b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.612+0000 7fce87fff640 1 -- 172.21.15.16:0/2393690797 shutdown_connections 2023-11-13T11:03:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.612+0000 7fce87fff640 1 -- 172.21.15.16:0/2393690797 wait complete. 2023-11-13T11:03:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.614+0000 7fce87fff640 1 Processor -- start 2023-11-13T11:03:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.614+0000 7fce87fff640 1 -- start start 2023-11-13T11:03:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.615+0000 7fce87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce88165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.615+0000 7fce87fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881f6de0 0x7fce881f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.615+0000 7fce87fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce88163710 0x7fce88163b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.615+0000 7fce87fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce881f77a0 con 0x7fce70004d00 2023-11-13T11:03:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.615+0000 7fce87fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce880a8670 con 0x7fce88163710 2023-11-13T11:03:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.615+0000 7fce87fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce880a8950 con 0x7fce881f6de0 2023-11-13T11:03:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.615+0000 7fce86ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce88165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.615+0000 7fce86ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce88165300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58328/0 (socket says 172.21.15.16:58328) 2023-11-13T11:03:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.615+0000 7fce86ffd640 1 -- 172.21.15.16:0/2987708600 learned_addr learned my addr 172.21.15.16:0/2987708600 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce86ffd640 1 -- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881f6de0 msgr2=0x7fce881f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce86ffd640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881f6de0 0x7fce881f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce86ffd640 1 -- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce88163710 msgr2=0x7fce88163b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce86ffd640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce88163710 0x7fce88163b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce86ffd640 1 -- 172.21.15.16:0/2987708600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce880a8c30 con 0x7fce70004d00 2023-11-13T11:03:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce86ffd640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce88165300 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fce7800d810 tx=0x7fce7800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce6f7fe640 1 -- 172.21.15.16:0/2987708600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce780074e0 con 0x7fce70004d00 2023-11-13T11:03:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce6f7fe640 1 -- 172.21.15.16:0/2987708600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce78007640 con 0x7fce70004d00 2023-11-13T11:03:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce880a8f10 con 0x7fce70004d00 2023-11-13T11:03:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce6f7fe640 1 -- 172.21.15.16:0/2987708600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78004a20 con 0x7fce70004d00 2023-11-13T11:03:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.616+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce881640d0 con 0x7fce70004d00 2023-11-13T11:03:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.617+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8804e4d0 con 0x7fce70004d00 2023-11-13T11:03:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.619+0000 7fce6f7fe640 1 -- 172.21.15.16:0/2987708600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce78004b80 con 0x7fce70004d00 2023-11-13T11:03:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.620+0000 7fce6f7fe640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce7406ae70 0x7fce7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.620+0000 7fce6f7fe640 1 -- 172.21.15.16:0/2987708600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5735+0+0 (secure 0 0 0) 0x7fce780892b0 con 0x7fce70004d00 2023-11-13T11:03:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.621+0000 7fce867fc640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce7406ae70 0x7fce7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.622+0000 7fce867fc640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce7406ae70 0x7fce7406d330 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fce8815fdc0 tx=0x7fce7c009500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.625+0000 7fce6f7fe640 1 -- 172.21.15.16:0/2987708600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce78054340 con 0x7fce70004d00 2023-11-13T11:03:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:07.818+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce88160600 con 0x7fce70004d00 2023-11-13T11:03:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.341+0000 7fce6f7fe640 1 -- 172.21.15.16:0/2987708600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) v1 ==== 169+0+0 (secure 0 0 0) 0x7fce78057960 con 0x7fce70004d00 2023-11-13T11:03:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-13T11:03:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.346+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce7406ae70 msgr2=0x7fce7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.346+0000 7fce87fff640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce7406ae70 0x7fce7406d330 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fce8815fdc0 tx=0x7fce7c009500 comp rx=0 tx=0).stop 2023-11-13T11:03:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.346+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 msgr2=0x7fce88165300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.346+0000 7fce87fff640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce88165300 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fce7800d810 tx=0x7fce7800dcd0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.347+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 shutdown_connections 2023-11-13T11:03:08.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.347+0000 7fce87fff640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fce7406ae70 0x7fce7406d330 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.347+0000 7fce87fff640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce88163710 0x7fce88163b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.347+0000 7fce87fff640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce881f6de0 0x7fce881f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.347+0000 7fce87fff640 1 --2- 172.21.15.16:0/2987708600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce70004d00 0x7fce88165300 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.347+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 >> 172.21.15.16:0/2987708600 conn(0x7fce880b0970 msgr2=0x7fce88161700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.348+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 shutdown_connections 2023-11-13T11:03:08.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.348+0000 7fce87fff640 1 -- 172.21.15.16:0/2987708600 wait complete. 2023-11-13T11:03:08.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-13T11:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.595+0000 7f7135987640 1 Processor -- start 2023-11-13T11:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.595+0000 7f7135987640 1 -- start start 2023-11-13T11:03:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.597+0000 7f7135987640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f713015b8e0 0x7f713006c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.597+0000 7f7135987640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71300ce260 0x7f71300cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.597+0000 7f7135987640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713006b860 con 0x7f713015bcc0 2023-11-13T11:03:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.597+0000 7f7135987640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713006bb40 con 0x7f71300ce260 2023-11-13T11:03:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.597+0000 7f7135987640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713006be20 con 0x7f713015b8e0 2023-11-13T11:03:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.597+0000 7f712ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f713015bcc0 0x7f71300cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33358/0 (socket says 172.21.15.16:33358) 2023-11-13T11:03:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.597+0000 7f712ffff640 1 -- 172.21.15.16:0/2276597989 learned_addr learned my addr 172.21.15.16:0/2276597989 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.597+0000 7f7135186640 1 --2- 172.21.15.16:0/2276597989 >> v2:172.21.15.16:3300/0 conn(0x7f71300ce260 0x7f71300cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.598+0000 7f7134985640 1 --2- 172.21.15.16:0/2276597989 >> v2:172.21.15.184:3300/0 conn(0x7f713015b8e0 0x7f713006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.598+0000 7f712f7fe640 1 -- 172.21.15.16:0/2276597989 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381366270 0 0) 0x7f713006b860 con 0x7f713015bcc0 2023-11-13T11:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.598+0000 7f712f7fe640 1 -- 172.21.15.16:0/2276597989 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7120003520 con 0x7f713015bcc0 2023-11-13T11:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.598+0000 7f7135186640 1 -- 172.21.15.16:0/2276597989 >> v2:172.21.15.184:3300/0 conn(0x7f713015b8e0 msgr2=0x7f713006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.598+0000 7f7135186640 1 --2- 172.21.15.16:0/2276597989 >> v2:172.21.15.184:3300/0 conn(0x7f713015b8e0 0x7f713006c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.598+0000 7f7135186640 1 -- 172.21.15.16:0/2276597989 >> v1:172.21.15.16:6789/0 conn(0x7f713015bcc0 legacy=0x7f71300cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.598+0000 7f7135186640 1 -- 172.21.15.16:0/2276597989 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713006c100 con 0x7f71300ce260 2023-11-13T11:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.599+0000 7f7135186640 1 --2- 172.21.15.16:0/2276597989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71300ce260 0x7f71300cbae0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7130071d70 tx=0x7f712802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6292020ab541a3e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.600+0000 7f712f7fe640 1 -- 172.21.15.16:0/2276597989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712802cc80 con 0x7f71300ce260 2023-11-13T11:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.600+0000 7f712f7fe640 1 -- 172.21.15.16:0/2276597989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712802cde0 con 0x7f71300ce260 2023-11-13T11:03:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.600+0000 7f712f7fe640 1 -- 172.21.15.16:0/2276597989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128034660 con 0x7f71300ce260 2023-11-13T11:03:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.600+0000 7f7135987640 1 -- 172.21.15.16:0/2276597989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71300ce260 msgr2=0x7f71300cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.600+0000 7f7135987640 1 --2- 172.21.15.16:0/2276597989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71300ce260 0x7f71300cbae0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7130071d70 tx=0x7f712802bed0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.600+0000 7f7135987640 1 -- 172.21.15.16:0/2276597989 shutdown_connections 2023-11-13T11:03:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.600+0000 7f7135987640 1 --2- 172.21.15.16:0/2276597989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71300ce260 0x7f71300cbae0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.600+0000 7f7135987640 1 --2- 172.21.15.16:0/2276597989 >> v2:172.21.15.184:3300/0 conn(0x7f713015b8e0 0x7f713006c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.600+0000 7f7135987640 1 -- 172.21.15.16:0/2276597989 >> 172.21.15.16:0/2276597989 conn(0x7f71300b0970 msgr2=0x7f71300b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.601+0000 7f7135987640 1 -- 172.21.15.16:0/2276597989 shutdown_connections 2023-11-13T11:03:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.601+0000 7f7135987640 1 -- 172.21.15.16:0/2276597989 wait complete. 2023-11-13T11:03:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.602+0000 7f7135987640 1 Processor -- start 2023-11-13T11:03:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.603+0000 7f7135987640 1 -- start start 2023-11-13T11:03:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f7135987640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71300ce260 0x7f713014f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f7135987640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f713015b8e0 0x7f7130156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f7135987640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7130071680 0x7f7130157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f7135987640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713014f7e0 con 0x7f713015b8e0 2023-11-13T11:03:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f7135987640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713015a990 con 0x7f7130071680 2023-11-13T11:03:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f7135987640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71300711f0 con 0x7f71300ce260 2023-11-13T11:03:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f712ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f713015b8e0 0x7f7130156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f712ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f713015b8e0 0x7f7130156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58338/0 (socket says 172.21.15.16:58338) 2023-11-13T11:03:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f712ffff640 1 -- 172.21.15.16:0/3645723843 learned_addr learned my addr 172.21.15.16:0/3645723843 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f7134985640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71300ce260 0x7f713014f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f7135186640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7130071680 0x7f7130157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f712ffff640 1 -- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71300ce260 msgr2=0x7f713014f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f712ffff640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71300ce260 0x7f713014f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f712ffff640 1 -- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7130071680 msgr2=0x7f7130157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f712ffff640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7130071680 0x7f7130157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f712ffff640 1 -- 172.21.15.16:0/3645723843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128009a70 con 0x7f713015b8e0 2023-11-13T11:03:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.604+0000 7f7134985640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71300ce260 0x7f713014f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.605+0000 7f712ffff640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f713015b8e0 0x7f7130156b50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f711c00c440 tx=0x7f711c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.605+0000 7f712d7fa640 1 -- 172.21.15.16:0/3645723843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c005ce0 con 0x7f713015b8e0 2023-11-13T11:03:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.605+0000 7f712d7fa640 1 -- 172.21.15.16:0/3645723843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c005e40 con 0x7f713015b8e0 2023-11-13T11:03:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.605+0000 7f712d7fa640 1 -- 172.21.15.16:0/3645723843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c0072b0 con 0x7f713015b8e0 2023-11-13T11:03:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.605+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71300708c0 con 0x7f713015b8e0 2023-11-13T11:03:08.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.605+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7130070e80 con 0x7f713015b8e0 2023-11-13T11:03:08.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.606+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713004e4d0 con 0x7f713015b8e0 2023-11-13T11:03:08.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.607+0000 7f712d7fa640 1 -- 172.21.15.16:0/3645723843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f711c007410 con 0x7f713015b8e0 2023-11-13T11:03:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.608+0000 7f712d7fa640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f711406ae70 0x7f711406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.608+0000 7f712d7fa640 1 -- 172.21.15.16:0/3645723843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f711c088950 con 0x7f713015b8e0 2023-11-13T11:03:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.609+0000 7f7134985640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f711406ae70 0x7f711406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.610+0000 7f7134985640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f711406ae70 0x7f711406d330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.612+0000 7f712d7fa640 1 -- 172.21.15.16:0/3645723843 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f711c053c70 con 0x7f713015b8e0 2023-11-13T11:03:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.828+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7130071330 con 0x7f713015b8e0 2023-11-13T11:03:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.833+0000 7f712d7fa640 1 -- 172.21.15.16:0/3645723843 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f711c0170a0 con 0x7f713015b8e0 2023-11-13T11:03:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f711406ae70 msgr2=0x7f711406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f711406ae70 0x7f711406d330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124008040 comp rx=0 tx=0).stop 2023-11-13T11:03:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f713015b8e0 msgr2=0x7f7130156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f713015b8e0 0x7f7130156b50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f711c00c440 tx=0x7f711c00c900 comp rx=0 tx=0).stop 2023-11-13T11:03:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 shutdown_connections 2023-11-13T11:03:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f711406ae70 0x7f711406d330 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7130071680 0x7f7130157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f713015b8e0 0x7f7130156b50 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 --2- 172.21.15.16:0/3645723843 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71300ce260 0x7f713014f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.836+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 >> 172.21.15.16:0/3645723843 conn(0x7f71300b0970 msgr2=0x7f71300b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.837+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 shutdown_connections 2023-11-13T11:03:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:08.837+0000 7f7135987640 1 -- 172.21.15.16:0/3645723843 wait complete. 2023-11-13T11:03:08.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2023-11-13T11:03:09.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.084+0000 7f1d35f33640 1 Processor -- start 2023-11-13T11:03:09.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.084+0000 7f1d35f33640 1 -- start start 2023-11-13T11:03:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.086+0000 7f1d35f33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d30155410 0x7f1d300ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.086+0000 7f1d35f33640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d300ce990 0x7f1d300cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.086+0000 7f1d35f33640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3015b500 con 0x7f1d30071440 2023-11-13T11:03:09.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.086+0000 7f1d35f33640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3015b7e0 con 0x7f1d300ce990 2023-11-13T11:03:09.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.086+0000 7f1d35f33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3015bac0 con 0x7f1d30155410 2023-11-13T11:03:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.087+0000 7f1d2effd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d300ce990 0x7f1d300cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.087+0000 7f1d2f7fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d30155410 0x7f1d300ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.087+0000 7f1d2effd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1d300ce990 0x7f1d300cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:49850/0 (socket says 172.21.15.16:49850) 2023-11-13T11:03:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.087+0000 7f1d2ffff640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f1d30071440 0x7f1d3006fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:46854/0 (socket says 172.21.15.16:46854) 2023-11-13T11:03:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.087+0000 7f1d2effd640 1 -- 172.21.15.16:0/3728853791 learned_addr learned my addr 172.21.15.16:0/3728853791 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.088+0000 7f1d2effd640 1 -- 172.21.15.16:0/3728853791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d30155410 msgr2=0x7f1d300ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.088+0000 7f1d2effd640 1 --2- 172.21.15.16:0/3728853791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d30155410 0x7f1d300ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.088+0000 7f1d2effd640 1 -- 172.21.15.16:0/3728853791 >> v1:172.21.15.184:6789/0 conn(0x7f1d30071440 legacy=0x7f1d3006fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.088+0000 7f1d2effd640 1 -- 172.21.15.16:0/3728853791 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3015bda0 con 0x7f1d300ce990 2023-11-13T11:03:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.089+0000 7f1d2effd640 1 --2- 172.21.15.16:0/3728853791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d300ce990 0x7f1d300cee10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009870 tx=0x7f1d2402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f00210506ef5b118 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d2e7fc640 1 -- 172.21.15.16:0/3728853791 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24030840 con 0x7f1d300ce990 2023-11-13T11:03:09.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d2e7fc640 1 -- 172.21.15.16:0/3728853791 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d240309a0 con 0x7f1d300ce990 2023-11-13T11:03:09.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d2e7fc640 1 -- 172.21.15.16:0/3728853791 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24030d30 con 0x7f1d300ce990 2023-11-13T11:03:09.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d35f33640 1 -- 172.21.15.16:0/3728853791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d300ce990 msgr2=0x7f1d300cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:09.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d35f33640 1 --2- 172.21.15.16:0/3728853791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d300ce990 0x7f1d300cee10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009870 tx=0x7f1d2402b980 comp rx=0 tx=0).stop 2023-11-13T11:03:09.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d35f33640 1 -- 172.21.15.16:0/3728853791 shutdown_connections 2023-11-13T11:03:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d35f33640 1 --2- 172.21.15.16:0/3728853791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d300ce990 0x7f1d300cee10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d35f33640 1 --2- 172.21.15.16:0/3728853791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d30155410 0x7f1d300ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d35f33640 1 -- 172.21.15.16:0/3728853791 >> 172.21.15.16:0/3728853791 conn(0x7f1d3006a190 msgr2=0x7f1d3006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:09.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d35f33640 1 -- 172.21.15.16:0/3728853791 shutdown_connections 2023-11-13T11:03:09.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.090+0000 7f1d35f33640 1 -- 172.21.15.16:0/3728853791 wait complete. 2023-11-13T11:03:09.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.092+0000 7f1d35f33640 1 Processor -- start 2023-11-13T11:03:09.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.092+0000 7f1d35f33640 1 -- start start 2023-11-13T11:03:09.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.093+0000 7f1d35f33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d30071440 0x7f1d3014fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:09.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.093+0000 7f1d35f33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d300ce990 0x7f1d30159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:09.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.093+0000 7f1d35f33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d30155410 0x7f1d3014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:09.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.093+0000 7f1d35f33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d30156ca0 con 0x7f1d300ce990 2023-11-13T11:03:09.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.093+0000 7f1d35f33640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d30156f80 con 0x7f1d30071440 2023-11-13T11:03:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.093+0000 7f1d35f33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d30157260 con 0x7f1d30155410 2023-11-13T11:03:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.094+0000 7f1d2effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d300ce990 0x7f1d30159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.094+0000 7f1d2effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d300ce990 0x7f1d30159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58344/0 (socket says 172.21.15.16:58344) 2023-11-13T11:03:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.094+0000 7f1d2effd640 1 -- 172.21.15.16:0/1788094231 learned_addr learned my addr 172.21.15.16:0/1788094231 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.094+0000 7f1d2f7fe640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d30071440 0x7f1d3014fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.094+0000 7f1d2effd640 1 -- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d30155410 msgr2=0x7f1d3014d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.094+0000 7f1d2effd640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d30155410 0x7f1d3014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.094+0000 7f1d2effd640 1 -- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d30071440 msgr2=0x7f1d3014fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.094+0000 7f1d2effd640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d30071440 0x7f1d3014fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.094+0000 7f1d2effd640 1 -- 172.21.15.16:0/1788094231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240094d0 con 0x7f1d300ce990 2023-11-13T11:03:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.095+0000 7f1d2effd640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d300ce990 0x7f1d30159890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1d24000c00 tx=0x7f1d24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.095+0000 7f1d17fff640 1 -- 172.21.15.16:0/1788094231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24039c90 con 0x7f1d300ce990 2023-11-13T11:03:09.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.095+0000 7f1d17fff640 1 -- 172.21.15.16:0/1788094231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d24039df0 con 0x7f1d300ce990 2023-11-13T11:03:09.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.095+0000 7f1d17fff640 1 -- 172.21.15.16:0/1788094231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24043640 con 0x7f1d300ce990 2023-11-13T11:03:09.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.095+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d30157540 con 0x7f1d300ce990 2023-11-13T11:03:09.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.095+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d30150260 con 0x7f1d300ce990 2023-11-13T11:03:09.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.096+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d30157a60 con 0x7f1d300ce990 2023-11-13T11:03:09.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.097+0000 7f1d17fff640 1 -- 172.21.15.16:0/1788094231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d240437a0 con 0x7f1d300ce990 2023-11-13T11:03:09.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.098+0000 7f1d17fff640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d1c06ae40 0x7f1d1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:09.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.098+0000 7f1d17fff640 1 -- 172.21.15.16:0/1788094231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1d2403e070 con 0x7f1d300ce990 2023-11-13T11:03:09.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.099+0000 7f1d2f7fe640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d1c06ae40 0x7f1d1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:09.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.100+0000 7f1d2f7fe640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d1c06ae40 0x7f1d1c06d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009a30 tx=0x7f1d20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:09.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.103+0000 7f1d17fff640 1 -- 172.21.15.16:0/1788094231 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d30157a60 con 0x7f1d300ce990 2023-11-13T11:03:09.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:09.367+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f1d30157a60 con 0x7f1d300ce990 2023-11-13T11:03:10.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.347+0000 7f1d17fff640 1 -- 172.21.15.16:0/1788094231 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v347) v1 ==== 143+0+0 (secure 0 0 0) 0x7f1d30157a60 con 0x7f1d300ce990 2023-11-13T11:03:10.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d1c06ae40 msgr2=0x7f1d1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:10.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d1c06ae40 0x7f1d1c06d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009a30 tx=0x7f1d20008040 comp rx=0 tx=0).stop 2023-11-13T11:03:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d300ce990 msgr2=0x7f1d30159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d300ce990 0x7f1d30159890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1d24000c00 tx=0x7f1d24004730 comp rx=0 tx=0).stop 2023-11-13T11:03:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 shutdown_connections 2023-11-13T11:03:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d1c06ae40 0x7f1d1c06d300 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d30155410 0x7f1d3014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d300ce990 0x7f1d30159890 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 --2- 172.21.15.16:0/1788094231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d30071440 0x7f1d3014fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 >> 172.21.15.16:0/1788094231 conn(0x7f1d3006a190 msgr2=0x7f1d300ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:10.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 shutdown_connections 2023-11-13T11:03:10.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.351+0000 7f1d35f33640 1 -- 172.21.15.16:0/1788094231 wait complete. 2023-11-13T11:03:10.378 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2023-11-13T11:03:10.379 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_merge_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) That a directory is merged during scrub. 2023-11-13T11:03:10.379 INFO:tasks.cephfs_test_runner:test_scrub_merge_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-13T11:03:10.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.599+0000 7f8d81013640 1 Processor -- start 2023-11-13T11:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.600+0000 7f8d81013640 1 -- start start 2023-11-13T11:03:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.601+0000 7f8d81013640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c0cb190 0x7f8d7c15b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.601+0000 7f8d81013640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d7c06a190 0x7f8d7c06a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.601+0000 7f8d81013640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7c158c20 con 0x7f8d7c0716b0 2023-11-13T11:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.601+0000 7f8d81013640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7c0b1830 con 0x7f8d7c06a190 2023-11-13T11:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.601+0000 7f8d81013640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c06ab10 con 0x7f8d7c0cb190 2023-11-13T11:03:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d80812640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d7c06a190 0x7f8d7c06a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d80812640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d7c06a190 0x7f8d7c06a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58354/0 (socket says 172.21.15.16:58354) 2023-11-13T11:03:10.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d80812640 1 -- 172.21.15.16:0/87674660 learned_addr learned my addr 172.21.15.16:0/87674660 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:10.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d73fff640 1 --1- 172.21.15.16:0/87674660 >> v1:172.21.15.16:6789/0 conn(0x7f8d7c0716b0 0x7f8d7c06f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33370/0 (socket says 172.21.15.16:33370) 2023-11-13T11:03:10.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d7bfff640 1 --2- 172.21.15.16:0/87674660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c0cb190 0x7f8d7c15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:10.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d80812640 1 -- 172.21.15.16:0/87674660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c0cb190 msgr2=0x7f8d7c15b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:10.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d80812640 1 --2- 172.21.15.16:0/87674660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c0cb190 0x7f8d7c15b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d80812640 1 -- 172.21.15.16:0/87674660 >> v1:172.21.15.16:6789/0 conn(0x7f8d7c0716b0 legacy=0x7f8d7c06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:10.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d80812640 1 -- 172.21.15.16:0/87674660 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c15b780 con 0x7f8d7c06a190 2023-11-13T11:03:10.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.602+0000 7f8d7bfff640 1 --2- 172.21.15.16:0/87674660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c0cb190 0x7f8d7c15b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:10.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.603+0000 7f8d80812640 1 --2- 172.21.15.16:0/87674660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c06a190 0x7f8d7c06a5d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009e50 tx=0x7f8d7402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2ac0ab58f74aec server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:10.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.604+0000 7f8d7b7fe640 1 -- 172.21.15.16:0/87674660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7402cb00 con 0x7f8d7c06a190 2023-11-13T11:03:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.604+0000 7f8d7b7fe640 1 -- 172.21.15.16:0/87674660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7402cc60 con 0x7f8d7c06a190 2023-11-13T11:03:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.604+0000 7f8d7b7fe640 1 -- 172.21.15.16:0/87674660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d740343f0 con 0x7f8d7c06a190 2023-11-13T11:03:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.605+0000 7f8d81013640 1 -- 172.21.15.16:0/87674660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c06a190 msgr2=0x7f8d7c06a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.605+0000 7f8d81013640 1 --2- 172.21.15.16:0/87674660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c06a190 0x7f8d7c06a5d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009e50 tx=0x7f8d7402bf60 comp rx=0 tx=0).stop 2023-11-13T11:03:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.605+0000 7f8d81013640 1 -- 172.21.15.16:0/87674660 shutdown_connections 2023-11-13T11:03:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.605+0000 7f8d81013640 1 --2- 172.21.15.16:0/87674660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c06a190 0x7f8d7c06a5d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.605+0000 7f8d81013640 1 --2- 172.21.15.16:0/87674660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c0cb190 0x7f8d7c15b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.605+0000 7f8d81013640 1 -- 172.21.15.16:0/87674660 >> 172.21.15.16:0/87674660 conn(0x7f8d7c0b04a0 msgr2=0x7f8d7c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.605+0000 7f8d81013640 1 -- 172.21.15.16:0/87674660 shutdown_connections 2023-11-13T11:03:10.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.605+0000 7f8d81013640 1 -- 172.21.15.16:0/87674660 wait complete. 2023-11-13T11:03:10.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.607+0000 7f8d81013640 1 Processor -- start 2023-11-13T11:03:10.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.608+0000 7f8d81013640 1 -- start start 2023-11-13T11:03:10.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.609+0000 7f8d81013640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c06a190 0x7f8d7c1ef640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:10.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.609+0000 7f8d81013640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c0716b0 0x7f8d7c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:10.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.609+0000 7f8d81013640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d7c0cb190 0x7f8d7c06ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:10.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.609+0000 7f8d81013640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0a8670 con 0x7f8d7c0716b0 2023-11-13T11:03:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.609+0000 7f8d81013640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0a8950 con 0x7f8d7c0cb190 2023-11-13T11:03:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.609+0000 7f8d81013640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0a8c30 con 0x7f8d7c06a190 2023-11-13T11:03:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.609+0000 7f8d73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c0716b0 0x7f8d7c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.609+0000 7f8d73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c0716b0 0x7f8d7c1554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58370/0 (socket says 172.21.15.16:58370) 2023-11-13T11:03:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.609+0000 7f8d73fff640 1 -- 172.21.15.16:0/2017804380 learned_addr learned my addr 172.21.15.16:0/2017804380 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:10.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d80812640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d7c0cb190 0x7f8d7c06ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:10.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d73fff640 1 -- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c06a190 msgr2=0x7f8d7c1ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:10.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d73fff640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c06a190 0x7f8d7c1ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d73fff640 1 -- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d7c0cb190 msgr2=0x7f8d7c06ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:10.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d73fff640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d7c0cb190 0x7f8d7c06ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d73fff640 1 -- 172.21.15.16:0/2017804380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74009b30 con 0x7f8d7c0716b0 2023-11-13T11:03:10.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d80812640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d7c0cb190 0x7f8d7c06ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:10.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d73fff640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c0716b0 0x7f8d7c1554d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400c4d0 tx=0x7f8d6400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:10.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d797fa640 1 -- 172.21.15.16:0/2017804380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64007480 con 0x7f8d7c0716b0 2023-11-13T11:03:10.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d797fa640 1 -- 172.21.15.16:0/2017804380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d640075e0 con 0x7f8d7c0716b0 2023-11-13T11:03:10.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d797fa640 1 -- 172.21.15.16:0/2017804380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640049e0 con 0x7f8d7c0716b0 2023-11-13T11:03:10.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.610+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7c0a8f10 con 0x7f8d7c0716b0 2023-11-13T11:03:10.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.611+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7c155a10 con 0x7f8d7c0716b0 2023-11-13T11:03:10.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.612+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7c04e4d0 con 0x7f8d7c0716b0 2023-11-13T11:03:10.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.613+0000 7f8d797fa640 1 -- 172.21.15.16:0/2017804380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d64004b40 con 0x7f8d7c0716b0 2023-11-13T11:03:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.614+0000 7f8d797fa640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8d5c06ae40 0x7f8d5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.614+0000 7f8d797fa640 1 -- 172.21.15.16:0/2017804380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f8d640884e0 con 0x7f8d7c0716b0 2023-11-13T11:03:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.614+0000 7f8d7bfff640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8d5c06ae40 0x7f8d5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.617+0000 7f8d7bfff640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8d5c06ae40 0x7f8d5c06d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c006fd0 tx=0x7f8d6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.619+0000 7f8d797fa640 1 -- 172.21.15.16:0/2017804380 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d640537f0 con 0x7f8d7c0716b0 2023-11-13T11:03:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.823+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d7c157db0 con 0x7f8d7c0716b0 2023-11-13T11:03:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.825+0000 7f8d797fa640 1 -- 172.21.15.16:0/2017804380 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+13203 (secure 0 0 0) 0x7f8d64056e10 con 0x7f8d7c0716b0 2023-11-13T11:03:10.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:10.831 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":60,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:08.339413+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/3328237273":"2023-11-14T11:03:05.329429+0000","172.21.15.16:6834/3328237273":"2023-11-14T11:03:05.329429+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:10.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8d5c06ae40 msgr2=0x7f8d5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:10.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8d5c06ae40 0x7f8d5c06d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c006fd0 tx=0x7f8d6c008040 comp rx=0 tx=0).stop 2023-11-13T11:03:10.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c0716b0 msgr2=0x7f8d7c1554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:10.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c0716b0 0x7f8d7c1554d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400c4d0 tx=0x7f8d6400c990 comp rx=0 tx=0).stop 2023-11-13T11:03:10.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 shutdown_connections 2023-11-13T11:03:10.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8d5c06ae40 0x7f8d5c06d300 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d7c0cb190 0x7f8d7c06ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d7c0716b0 0x7f8d7c1554d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 --2- 172.21.15.16:0/2017804380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d7c06a190 0x7f8d7c1ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:10.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 >> 172.21.15.16:0/2017804380 conn(0x7f8d7c0b04a0 msgr2=0x7f8d7c0cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:10.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 shutdown_connections 2023-11-13T11:03:10.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:10.829+0000 7f8d81013640 1 -- 172.21.15.16:0/2017804380 wait complete. 2023-11-13T11:03:10.852 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:03:10.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:03:11.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.083+0000 7faad6d6c640 1 Processor -- start 2023-11-13T11:03:11.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.083+0000 7faad6d6c640 1 -- start start 2023-11-13T11:03:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.084+0000 7faad6d6c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faad00d1850 0x7faad00d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.084+0000 7faad6d6c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad00d2d00 0x7faad00c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.084+0000 7faad6d6c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad00ce0e0 con 0x7faad015ce70 2023-11-13T11:03:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.084+0000 7faad6d6c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad00ce3c0 con 0x7faad00d1850 2023-11-13T11:03:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.084+0000 7faad6d6c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad00ce6a0 con 0x7faad00d2d00 2023-11-13T11:03:11.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.085+0000 7faad5d6a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faad015ce70 0x7faad00cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33372/0 (socket says 172.21.15.16:33372) 2023-11-13T11:03:11.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.085+0000 7faad5d6a640 1 -- 172.21.15.16:0/1169270206 learned_addr learned my addr 172.21.15.16:0/1169270206 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.085+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529951300 0 0) 0x7faad00ce0e0 con 0x7faad015ce70 2023-11-13T11:03:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.085+0000 7faad5569640 1 --2- 172.21.15.16:0/1169270206 >> v2:172.21.15.184:3300/0 conn(0x7faad00d1850 0x7faad00d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.085+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faabc003520 con 0x7faad015ce70 2023-11-13T11:03:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.085+0000 7faad656b640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad00d2d00 0x7faad00c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3305544343 0 0) 0x7faabc003520 con 0x7faad015ce70 2023-11-13T11:03:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad00d2d00 msgr2=0x7faad00c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad00d2d00 0x7faad00c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 >> v2:172.21.15.184:3300/0 conn(0x7faad00d1850 msgr2=0x7faad00d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 --2- 172.21.15.16:0/1169270206 >> v2:172.21.15.184:3300/0 conn(0x7faad00d1850 0x7faad00d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad00ce980 con 0x7faad015ce70 2023-11-13T11:03:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7faacc001670 con 0x7faad015ce70 2023-11-13T11:03:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 >> v1:172.21.15.16:6789/0 conn(0x7faad015ce70 legacy=0x7faad00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad00d2d00 0x7faabc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc003ef0 0x7faabc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc004d00 0x7faabc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad00ce3c0 con 0x7faabc003ef0 2023-11-13T11:03:11.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad00ce6a0 con 0x7faabc004d00 2023-11-13T11:03:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc00b9d0 con 0x7faad00d2d00 2023-11-13T11:03:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.086+0000 7faad5569640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc003ef0 0x7faabc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.087+0000 7faad656b640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc004d00 0x7faabc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.087+0000 7faad5569640 1 -- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad00d2d00 msgr2=0x7faabc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.087+0000 7faad5d6a640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad00d2d00 0x7faabc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.087+0000 7faad5569640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad00d2d00 0x7faabc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.087+0000 7faad5569640 1 -- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc004d00 msgr2=0x7faabc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.087+0000 7faad5569640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc004d00 0x7faabc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.087+0000 7faad5569640 1 -- 172.21.15.16:0/1169270206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc003520 con 0x7faabc003ef0 2023-11-13T11:03:11.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.087+0000 7faad656b640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc004d00 0x7faabc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:11.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.087+0000 7faad5d6a640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad00d2d00 0x7faabc003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:11.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.088+0000 7faad5569640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc003ef0 0x7faabc0047c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7faac0009ac0 tx=0x7faabc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8802c7f4bd59a55 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:11.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.088+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac002c6f0 con 0x7faabc003ef0 2023-11-13T11:03:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.088+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac002c850 con 0x7faabc003ef0 2023-11-13T11:03:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.088+0000 7faad4d68640 1 -- 172.21.15.16:0/1169270206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac002cbe0 con 0x7faabc003ef0 2023-11-13T11:03:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.089+0000 7faad6d6c640 1 -- 172.21.15.16:0/1169270206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc003ef0 msgr2=0x7faabc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.089+0000 7faad6d6c640 1 --2- 172.21.15.16:0/1169270206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc003ef0 0x7faabc0047c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7faac0009ac0 tx=0x7faabc0057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.089+0000 7faad656b640 1 -- 172.21.15.16:0/1169270206 reap_dead start 2023-11-13T11:03:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.089+0000 7faad6d6c640 1 -- 172.21.15.16:0/1169270206 shutdown_connections 2023-11-13T11:03:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.089+0000 7faad6d6c640 1 -- 172.21.15.16:0/1169270206 >> 172.21.15.16:0/1169270206 conn(0x7faad008c580 msgr2=0x7faad009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.089+0000 7faad6d6c640 1 -- 172.21.15.16:0/1169270206 shutdown_connections 2023-11-13T11:03:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.089+0000 7faad6d6c640 1 -- 172.21.15.16:0/1169270206 wait complete. 2023-11-13T11:03:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.091+0000 7faad6d6c640 1 Processor -- start 2023-11-13T11:03:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.091+0000 7faad6d6c640 1 -- start start 2023-11-13T11:03:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad6d6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc003ef0 0x7faad01500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad6d6c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad015bf00 0x7faad00b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad6d6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad00c82d0 0x7faad00c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad6d6c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad00c8cc0 con 0x7faad00c82d0 2023-11-13T11:03:11.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad6d6c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad00b6270 con 0x7faabc003ef0 2023-11-13T11:03:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad6d6c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad00b6550 con 0x7faad015bf00 2023-11-13T11:03:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad656b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad00c82d0 0x7faad00c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad656b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad00c82d0 0x7faad00c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58390/0 (socket says 172.21.15.16:58390) 2023-11-13T11:03:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad656b640 1 -- 172.21.15.16:0/1101667542 learned_addr learned my addr 172.21.15.16:0/1101667542 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad5d6a640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc003ef0 0x7faad01500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad656b640 1 -- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad015bf00 msgr2=0x7faad00b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad5569640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad015bf00 0x7faad00b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad656b640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad015bf00 0x7faad00b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad656b640 1 -- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc003ef0 msgr2=0x7faad01500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad656b640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc003ef0 0x7faad01500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad656b640 1 -- 172.21.15.16:0/1101667542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad00b6830 con 0x7faad00c82d0 2023-11-13T11:03:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad5d6a640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc003ef0 0x7faad01500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faad656b640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad00c82d0 0x7faad00c8780 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7faad006a650 tx=0x7faac8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faac67fc640 1 -- 172.21.15.16:0/1101667542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8007cf0 con 0x7faad00c82d0 2023-11-13T11:03:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faac67fc640 1 -- 172.21.15.16:0/1101667542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac8007e70 con 0x7faad00c82d0 2023-11-13T11:03:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.093+0000 7faac67fc640 1 -- 172.21.15.16:0/1101667542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8014620 con 0x7faad00c82d0 2023-11-13T11:03:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.094+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad00b6b10 con 0x7faad00c82d0 2023-11-13T11:03:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.094+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad00b82f0 con 0x7faad00c82d0 2023-11-13T11:03:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.096+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad004e4d0 con 0x7faad00c82d0 2023-11-13T11:03:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.096+0000 7faac67fc640 1 -- 172.21.15.16:0/1101667542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faac801d3b0 con 0x7faad00c82d0 2023-11-13T11:03:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.098+0000 7faac67fc640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faab406ae70 0x7faab406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.098+0000 7faac67fc640 1 -- 172.21.15.16:0/1101667542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7faac8088f20 con 0x7faad00c82d0 2023-11-13T11:03:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.098+0000 7faad5d6a640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faab406ae70 0x7faab406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.100+0000 7faad5d6a640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faab406ae70 0x7faab406d330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faacc007af0 tx=0x7faacc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.102+0000 7faac67fc640 1 -- 172.21.15.16:0/1101667542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faac8054200 con 0x7faad00c82d0 2023-11-13T11:03:11.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.297+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faad01603c0 con 0x7faad00c82d0 2023-11-13T11:03:11.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.299+0000 7faac67fc640 1 -- 172.21.15.16:0/1101667542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7faac8057820 con 0x7faad00c82d0 2023-11-13T11:03:11.304 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:03:11.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faab406ae70 msgr2=0x7faab406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faab406ae70 0x7faab406d330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faacc007af0 tx=0x7faacc009040 comp rx=0 tx=0).stop 2023-11-13T11:03:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad00c82d0 msgr2=0x7faad00c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad00c82d0 0x7faad00c8780 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7faad006a650 tx=0x7faac8009cd0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 shutdown_connections 2023-11-13T11:03:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7faab406ae70 0x7faab406d330 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad00c82d0 0x7faad00c8780 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad015bf00 0x7faad00b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 --2- 172.21.15.16:0/1101667542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faabc003ef0 0x7faad01500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 >> 172.21.15.16:0/1101667542 conn(0x7faad008c580 msgr2=0x7faad00cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:11.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 shutdown_connections 2023-11-13T11:03:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.301+0000 7faad6d6c640 1 -- 172.21.15.16:0/1101667542 wait complete. 2023-11-13T11:03:11.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags' 2023-11-13T11:03:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.554+0000 7fc36a22e640 1 Processor -- start 2023-11-13T11:03:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.554+0000 7fc36a22e640 1 -- start start 2023-11-13T11:03:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.555+0000 7fc36a22e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc36406a8a0 0x7fc364070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.555+0000 7fc36a22e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc364070d50 con 0x7fc3640708a0 2023-11-13T11:03:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.555+0000 7fc36a22e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36415d630 con 0x7fc36415aa90 2023-11-13T11:03:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.555+0000 7fc36a22e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36415d910 con 0x7fc36406a8a0 2023-11-13T11:03:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.555+0000 7fc369a2d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc3640708a0 0x7fc36406c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33376/0 (socket says 172.21.15.16:33376) 2023-11-13T11:03:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.555+0000 7fc369a2d640 1 -- 172.21.15.16:0/686689971 learned_addr learned my addr 172.21.15.16:0/686689971 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.555+0000 7fc368a2b640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc36406a8a0 0x7fc364070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245416642 0 0) 0x7fc364070d50 con 0x7fc3640708a0 2023-11-13T11:03:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc34c003520 con 0x7fc3640708a0 2023-11-13T11:03:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1926713710 0 0) 0x7fc34c003520 con 0x7fc3640708a0 2023-11-13T11:03:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc36406a8a0 msgr2=0x7fc364070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc36406a8a0 0x7fc364070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 >> v1:172.21.15.184:6789/0 conn(0x7fc36415aa90 legacy=0x7fc36406a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36415dbf0 con 0x7fc3640708a0 2023-11-13T11:03:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fc35c001670 con 0x7fc3640708a0 2023-11-13T11:03:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 >> v1:172.21.15.16:6789/0 conn(0x7fc3640708a0 legacy=0x7fc36406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc36406a8a0 0x7fc34c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc34c003f20 0x7fc34c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c004d00 0x7fc34c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36415d910 con 0x7fc34c004d00 2023-11-13T11:03:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c00bb10 con 0x7fc34c003f20 2023-11-13T11:03:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.556+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c00dee0 con 0x7fc36406a8a0 2023-11-13T11:03:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.557+0000 7fc369a2d640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c004d00 0x7fc34c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.557+0000 7fc368a2b640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc34c003f20 0x7fc34c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.557+0000 7fc36922c640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc36406a8a0 0x7fc34c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.557+0000 7fc369a2d640 1 -- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc36406a8a0 msgr2=0x7fc34c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.557+0000 7fc369a2d640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc36406a8a0 0x7fc34c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.557+0000 7fc369a2d640 1 -- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc34c003f20 msgr2=0x7fc34c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.557+0000 7fc369a2d640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc34c003f20 0x7fc34c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.557+0000 7fc369a2d640 1 -- 172.21.15.16:0/686689971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c003520 con 0x7fc34c004d00 2023-11-13T11:03:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.558+0000 7fc369a2d640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c004d00 0x7fc34c005180 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c240 tx=0x7fc34c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6370c20bf1e5a811 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c005ad0 con 0x7fc34c004d00 2023-11-13T11:03:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c005c50 con 0x7fc34c004d00 2023-11-13T11:03:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc363fff640 1 -- 172.21.15.16:0/686689971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c02c8b0 con 0x7fc34c004d00 2023-11-13T11:03:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc36a22e640 1 -- 172.21.15.16:0/686689971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c004d00 msgr2=0x7fc34c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc36a22e640 1 --2- 172.21.15.16:0/686689971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc34c004d00 0x7fc34c005180 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c240 tx=0x7fc34c0057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc369a2d640 1 -- 172.21.15.16:0/686689971 reap_dead start 2023-11-13T11:03:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc36a22e640 1 -- 172.21.15.16:0/686689971 shutdown_connections 2023-11-13T11:03:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc36a22e640 1 -- 172.21.15.16:0/686689971 >> 172.21.15.16:0/686689971 conn(0x7fc3640b04a0 msgr2=0x7fc3640b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc36a22e640 1 -- 172.21.15.16:0/686689971 shutdown_connections 2023-11-13T11:03:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.559+0000 7fc36a22e640 1 -- 172.21.15.16:0/686689971 wait complete. 2023-11-13T11:03:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.562+0000 7fc36a22e640 1 Processor -- start 2023-11-13T11:03:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.562+0000 7fc36a22e640 1 -- start start 2023-11-13T11:03:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.562+0000 7fc36a22e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc34c004d00 0x7fc364154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.562+0000 7fc36a22e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3640ccba0 0x7fc3640a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.562+0000 7fc36a22e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3640cd020 0x7fc3640a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.562+0000 7fc36a22e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc364154a90 con 0x7fc3640ccba0 2023-11-13T11:03:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.562+0000 7fc36a22e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640b2ac0 con 0x7fc3640cd020 2023-11-13T11:03:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.562+0000 7fc36a22e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640b2da0 con 0x7fc34c004d00 2023-11-13T11:03:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc368a2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3640ccba0 0x7fc3640a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc368a2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3640ccba0 0x7fc3640a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58406/0 (socket says 172.21.15.16:58406) 2023-11-13T11:03:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc368a2b640 1 -- 172.21.15.16:0/4262980794 learned_addr learned my addr 172.21.15.16:0/4262980794 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc36922c640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc34c004d00 0x7fc364154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc369a2d640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3640cd020 0x7fc3640a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc368a2b640 1 -- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc34c004d00 msgr2=0x7fc364154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc368a2b640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc34c004d00 0x7fc364154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc368a2b640 1 -- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3640cd020 msgr2=0x7fc3640a93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc368a2b640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3640cd020 0x7fc3640a93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc368a2b640 1 -- 172.21.15.16:0/4262980794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640b3080 con 0x7fc3640ccba0 2023-11-13T11:03:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc369a2d640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3640cd020 0x7fc3640a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.563+0000 7fc368a2b640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3640ccba0 0x7fc3640a8ea0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc3580119a0 tx=0x7fc358011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.564+0000 7fc361ffb640 1 -- 172.21.15.16:0/4262980794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35800faf0 con 0x7fc3640ccba0 2023-11-13T11:03:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.564+0000 7fc361ffb640 1 -- 172.21.15.16:0/4262980794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35800fc50 con 0x7fc3640ccba0 2023-11-13T11:03:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.564+0000 7fc361ffb640 1 -- 172.21.15.16:0/4262980794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358014440 con 0x7fc3640ccba0 2023-11-13T11:03:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.564+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3640b3360 con 0x7fc3640ccba0 2023-11-13T11:03:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.564+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3640b2700 con 0x7fc3640ccba0 2023-11-13T11:03:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.565+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36404e4d0 con 0x7fc3640ccba0 2023-11-13T11:03:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.566+0000 7fc361ffb640 1 -- 172.21.15.16:0/4262980794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3580145a0 con 0x7fc3640ccba0 2023-11-13T11:03:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.569+0000 7fc361ffb640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc35006ae70 0x7fc35006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.569+0000 7fc361ffb640 1 -- 172.21.15.16:0/4262980794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fc358089c80 con 0x7fc3640ccba0 2023-11-13T11:03:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.569+0000 7fc36922c640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc35006ae70 0x7fc35006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.569+0000 7fc36922c640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc35006ae70 0x7fc35006d330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc354007fd0 tx=0x7fc354009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.573+0000 7fc361ffb640 1 -- 172.21.15.16:0/4262980794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc358054ee0 con 0x7fc3640ccba0 2023-11-13T11:03:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:11.839+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags"]} v 0) v1 -- 0x7fc36406d3a0 con 0x7fc3640ccba0 2023-11-13T11:03:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.363+0000 7fc361ffb640 1 -- 172.21.15.16:0/4262980794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags"]}]=0 v349) v1 ==== 154+0+0 (secure 0 0 0) 0x7fc358022020 con 0x7fc3640ccba0 2023-11-13T11:03:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.366+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc35006ae70 msgr2=0x7fc35006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.366+0000 7fc36a22e640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc35006ae70 0x7fc35006d330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc354007fd0 tx=0x7fc354009040 comp rx=0 tx=0).stop 2023-11-13T11:03:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.366+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3640ccba0 msgr2=0x7fc3640a8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:12.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.366+0000 7fc36a22e640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3640ccba0 0x7fc3640a8ea0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc3580119a0 tx=0x7fc358011e60 comp rx=0 tx=0).stop 2023-11-13T11:03:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.367+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 shutdown_connections 2023-11-13T11:03:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.367+0000 7fc36a22e640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc35006ae70 0x7fc35006d330 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.367+0000 7fc36a22e640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3640cd020 0x7fc3640a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.367+0000 7fc36a22e640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3640ccba0 0x7fc3640a8ea0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.367+0000 7fc36a22e640 1 --2- 172.21.15.16:0/4262980794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc34c004d00 0x7fc364154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.367+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 >> 172.21.15.16:0/4262980794 conn(0x7fc3640b04a0 msgr2=0x7fc364158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.367+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 shutdown_connections 2023-11-13T11:03:12.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.367+0000 7fc36a22e640 1 -- 172.21.15.16:0/4262980794 wait complete. 2023-11-13T11:03:12.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-13T11:03:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.629+0000 7f518359e640 1 Processor -- start 2023-11-13T11:03:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.629+0000 7f518359e640 1 -- start start 2023-11-13T11:03:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.630+0000 7f518359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51840cfad0 0x7f5184167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.631+0000 7f518359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51840ce650 con 0x7f5184160240 2023-11-13T11:03:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.631+0000 7f518359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51840ce930 con 0x7f5184160620 2023-11-13T11:03:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.631+0000 7f518359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51840c6670 con 0x7f51840cfad0 2023-11-13T11:03:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.631+0000 7f518259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5184160240 0x7f51840d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33386/0 (socket says 172.21.15.16:33386) 2023-11-13T11:03:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.631+0000 7f518259c640 1 -- 172.21.15.16:0/2920971968 learned_addr learned my addr 172.21.15.16:0/2920971968 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.631+0000 7f5182d9d640 1 --2- 172.21.15.16:0/2920971968 >> v2:172.21.15.16:3300/0 conn(0x7f51840cfad0 0x7f5184167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.631+0000 7f518159a640 1 -- 172.21.15.16:0/2920971968 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636754773 0 0) 0x7f51840ce650 con 0x7f5184160240 2023-11-13T11:03:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.631+0000 7f518159a640 1 -- 172.21.15.16:0/2920971968 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5170003520 con 0x7f5184160240 2023-11-13T11:03:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.631+0000 7f5182d9d640 1 -- 172.21.15.16:0/2920971968 >> v1:172.21.15.184:6789/0 conn(0x7f5184160620 legacy=0x7f51840cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:12.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.632+0000 7f5182d9d640 1 -- 172.21.15.16:0/2920971968 >> v1:172.21.15.16:6789/0 conn(0x7f5184160240 legacy=0x7f51840d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:12.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.632+0000 7f5182d9d640 1 -- 172.21.15.16:0/2920971968 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51840cec10 con 0x7f51840cfad0 2023-11-13T11:03:12.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.633+0000 7f5182d9d640 1 --2- 172.21.15.16:0/2920971968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51840cfad0 0x7f5184167ab0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f517c0097a0 tx=0x7f517c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a90d87ccca8f539 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:12.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.633+0000 7f518159a640 1 -- 172.21.15.16:0/2920971968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c031830 con 0x7f51840cfad0 2023-11-13T11:03:12.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.633+0000 7f518159a640 1 -- 172.21.15.16:0/2920971968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c031990 con 0x7f51840cfad0 2023-11-13T11:03:12.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.633+0000 7f518159a640 1 -- 172.21.15.16:0/2920971968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c031d20 con 0x7f51840cfad0 2023-11-13T11:03:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.634+0000 7f518359e640 1 -- 172.21.15.16:0/2920971968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51840cfad0 msgr2=0x7f5184167ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.634+0000 7f518359e640 1 --2- 172.21.15.16:0/2920971968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51840cfad0 0x7f5184167ab0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f517c0097a0 tx=0x7f517c02b8d0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.634+0000 7f518359e640 1 -- 172.21.15.16:0/2920971968 shutdown_connections 2023-11-13T11:03:12.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.634+0000 7f518359e640 1 --2- 172.21.15.16:0/2920971968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51840cfad0 0x7f5184167ab0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.634+0000 7f518359e640 1 -- 172.21.15.16:0/2920971968 >> 172.21.15.16:0/2920971968 conn(0x7f518409be00 msgr2=0x7f51840d2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.634+0000 7f518359e640 1 -- 172.21.15.16:0/2920971968 shutdown_connections 2023-11-13T11:03:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.634+0000 7f518359e640 1 -- 172.21.15.16:0/2920971968 wait complete. 2023-11-13T11:03:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.636+0000 7f518359e640 1 Processor -- start 2023-11-13T11:03:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.636+0000 7f518359e640 1 -- start start 2023-11-13T11:03:12.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f518359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51840cfad0 0x7f51841f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:12.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f518359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5184160240 0x7f51840b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:12.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f518359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184160620 0x7f518414de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:12.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f518359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5184151be0 con 0x7f5184160240 2023-11-13T11:03:12.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f518359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518414e990 con 0x7f5184160620 2023-11-13T11:03:12.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f518359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518414ec70 con 0x7f51840cfad0 2023-11-13T11:03:12.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f5181d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5184160240 0x7f51840b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:12.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f5181d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5184160240 0x7f51840b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58416/0 (socket says 172.21.15.16:58416) 2023-11-13T11:03:12.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f5181d9b640 1 -- 172.21.15.16:0/2339270147 learned_addr learned my addr 172.21.15.16:0/2339270147 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:12.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.637+0000 7f5182d9d640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184160620 0x7f518414de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:12.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f518259c640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51840cfad0 0x7f51841f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:12.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f5181d9b640 1 -- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51840cfad0 msgr2=0x7f51841f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:12.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f5181d9b640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51840cfad0 0x7f51841f69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f5181d9b640 1 -- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184160620 msgr2=0x7f518414de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:12.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f5181d9b640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184160620 0x7f518414de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f5181d9b640 1 -- 172.21.15.16:0/2339270147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c009400 con 0x7f5184160240 2023-11-13T11:03:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f5182d9d640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184160620 0x7f518414de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f5181d9b640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5184160240 0x7f51840b2ee0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f516c00ba50 tx=0x7f516c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f517affd640 1 -- 172.21.15.16:0/2339270147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c00cbe0 con 0x7f5184160240 2023-11-13T11:03:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f517affd640 1 -- 172.21.15.16:0/2339270147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c00cd80 con 0x7f5184160240 2023-11-13T11:03:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f517affd640 1 -- 172.21.15.16:0/2339270147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c014680 con 0x7f5184160240 2023-11-13T11:03:12.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51840d3a50 con 0x7f5184160240 2023-11-13T11:03:12.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.638+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51841f61c0 con 0x7f5184160240 2023-11-13T11:03:12.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.640+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518404e4d0 con 0x7f5184160240 2023-11-13T11:03:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.641+0000 7f517affd640 1 -- 172.21.15.16:0/2339270147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f516c0147e0 con 0x7f5184160240 2023-11-13T11:03:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.642+0000 7f517affd640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f516406ae70 0x7f516406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.642+0000 7f517affd640 1 -- 172.21.15.16:0/2339270147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f516c089180 con 0x7f5184160240 2023-11-13T11:03:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.642+0000 7f518259c640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f516406ae70 0x7f516406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.643+0000 7f518259c640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f516406ae70 0x7f516406d330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f51740011e0 tx=0x7f5174009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.646+0000 7f517affd640 1 -- 172.21.15.16:0/2339270147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f516c054530 con 0x7f5184160240 2023-11-13T11:03:12.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.839+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f518415e2c0 con 0x7f5184160240 2023-11-13T11:03:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.846+0000 7f517affd640 1 -- 172.21.15.16:0/2339270147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f516c057b50 con 0x7f5184160240 2023-11-13T11:03:12.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f516406ae70 msgr2=0x7f516406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:12.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f516406ae70 0x7f516406d330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f51740011e0 tx=0x7f5174009040 comp rx=0 tx=0).stop 2023-11-13T11:03:12.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5184160240 msgr2=0x7f51840b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:12.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5184160240 0x7f51840b2ee0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f516c00ba50 tx=0x7f516c00bf10 comp rx=0 tx=0).stop 2023-11-13T11:03:12.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 shutdown_connections 2023-11-13T11:03:12.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f516406ae70 0x7f516406d330 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184160620 0x7f518414de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5184160240 0x7f51840b2ee0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 --2- 172.21.15.16:0/2339270147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51840cfad0 0x7f51841f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:12.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 >> 172.21.15.16:0/2339270147 conn(0x7f518409be00 msgr2=0x7f51840c8220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:12.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 shutdown_connections 2023-11-13T11:03:12.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:12.849+0000 7f518359e640 1 -- 172.21.15.16:0/2339270147 wait complete. 2023-11-13T11:03:12.874 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-13T11:03:12.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:03:12.999 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:03:13.125 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-13T11:03:13.126 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:03:13.126 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:13.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:03:13.190 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:03:13.190 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:13.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:03:13.249 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:03:13.250 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:13.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:03:13.313 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:03:13.313 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:13.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:03:13.376 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-13T11:03:13.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.609+0000 7fd57d439640 1 Processor -- start 2023-11-13T11:03:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.609+0000 7fd57d439640 1 -- start start 2023-11-13T11:03:13.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.610+0000 7fd57d439640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5780cb190 0x7fd57815b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:13.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.610+0000 7fd57d439640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 0x7fd57806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:13.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.610+0000 7fd57d439640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd578158c20 con 0x7fd57806a190 2023-11-13T11:03:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.610+0000 7fd57d439640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5780b1830 con 0x7fd5780cb190 2023-11-13T11:03:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.610+0000 7fd57d439640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57815b780 con 0x7fd5780716b0 2023-11-13T11:03:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.611+0000 7fd577fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5780cb190 0x7fd57815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.611+0000 7fd5777fe640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 0x7fd57806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.611+0000 7fd577fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5780cb190 0x7fd57815b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:46990/0 (socket says 172.21.15.16:46990) 2023-11-13T11:03:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.611+0000 7fd577fff640 1 -- 172.21.15.16:0/1209225896 learned_addr learned my addr 172.21.15.16:0/1209225896 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.611+0000 7fd57cc38640 1 --1- 172.21.15.16:0/1209225896 >> v1:172.21.15.184:6789/0 conn(0x7fd57806a190 0x7fd5780b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:53526/0 (socket says 172.21.15.16:53526) 2023-11-13T11:03:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.612+0000 7fd5777fe640 1 -- 172.21.15.16:0/1209225896 >> v2:172.21.15.184:3300/0 conn(0x7fd5780cb190 msgr2=0x7fd57815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.612+0000 7fd5777fe640 1 --2- 172.21.15.16:0/1209225896 >> v2:172.21.15.184:3300/0 conn(0x7fd5780cb190 0x7fd57815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.612+0000 7fd5777fe640 1 -- 172.21.15.16:0/1209225896 >> v1:172.21.15.184:6789/0 conn(0x7fd57806a190 legacy=0x7fd5780b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.612+0000 7fd5777fe640 1 -- 172.21.15.16:0/1209225896 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57815ba60 con 0x7fd5780716b0 2023-11-13T11:03:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.612+0000 7fd577fff640 1 --2- 172.21.15.16:0/1209225896 >> v2:172.21.15.184:3300/0 conn(0x7fd5780cb190 0x7fd57815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T11:03:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.613+0000 7fd5777fe640 1 --2- 172.21.15.16:0/1209225896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 0x7fd57806f970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0097a0 tx=0x7fd56c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d95b378681cb4fd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.614+0000 7fd576ffd640 1 -- 172.21.15.16:0/1209225896 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c02d040 con 0x7fd5780716b0 2023-11-13T11:03:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.614+0000 7fd576ffd640 1 -- 172.21.15.16:0/1209225896 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c031810 con 0x7fd5780716b0 2023-11-13T11:03:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.614+0000 7fd576ffd640 1 -- 172.21.15.16:0/1209225896 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c031ba0 con 0x7fd5780716b0 2023-11-13T11:03:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.614+0000 7fd57d439640 1 -- 172.21.15.16:0/1209225896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 msgr2=0x7fd57806f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.614+0000 7fd57d439640 1 --2- 172.21.15.16:0/1209225896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 0x7fd57806f970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0097a0 tx=0x7fd56c02b8b0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.614+0000 7fd57d439640 1 -- 172.21.15.16:0/1209225896 shutdown_connections 2023-11-13T11:03:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.614+0000 7fd57d439640 1 --2- 172.21.15.16:0/1209225896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 0x7fd57806f970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.614+0000 7fd57d439640 1 --2- 172.21.15.16:0/1209225896 >> v2:172.21.15.184:3300/0 conn(0x7fd5780cb190 0x7fd57815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.614+0000 7fd57d439640 1 -- 172.21.15.16:0/1209225896 >> 172.21.15.16:0/1209225896 conn(0x7fd5780b04a0 msgr2=0x7fd5780b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.615+0000 7fd57d439640 1 -- 172.21.15.16:0/1209225896 shutdown_connections 2023-11-13T11:03:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.615+0000 7fd57d439640 1 -- 172.21.15.16:0/1209225896 wait complete. 2023-11-13T11:03:13.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.616+0000 7fd57d439640 1 Processor -- start 2023-11-13T11:03:13.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.617+0000 7fd57d439640 1 -- start start 2023-11-13T11:03:13.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd57d439640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57806a190 0x7fd5781f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:13.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd57d439640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 0x7fd57814d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:13.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd57d439640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5780cb190 0x7fd5780c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:13.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd57d439640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57814d900 con 0x7fd57806a190 2023-11-13T11:03:13.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd57d439640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57815d3e0 con 0x7fd5780cb190 2023-11-13T11:03:13.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd57d439640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57815c7d0 con 0x7fd5780716b0 2023-11-13T11:03:13.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd577fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57806a190 0x7fd5781f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:13.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd577fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57806a190 0x7fd5781f2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47822/0 (socket says 172.21.15.16:47822) 2023-11-13T11:03:13.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd577fff640 1 -- 172.21.15.16:0/3632474340 learned_addr learned my addr 172.21.15.16:0/3632474340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:13.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd57cc38640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5780cb190 0x7fd5780c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:13.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd5777fe640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 0x7fd57814d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:13.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd577fff640 1 -- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 msgr2=0x7fd57814d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:13.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.618+0000 7fd577fff640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 0x7fd57814d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.619+0000 7fd577fff640 1 -- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5780cb190 msgr2=0x7fd5780c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:13.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.619+0000 7fd577fff640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5780cb190 0x7fd5780c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.619+0000 7fd577fff640 1 -- 172.21.15.16:0/3632474340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680094d0 con 0x7fd57806a190 2023-11-13T11:03:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.619+0000 7fd577fff640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57806a190 0x7fd5781f2830 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd56800e9b0 tx=0x7fd56800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.619+0000 7fd574ff9640 1 -- 172.21.15.16:0/3632474340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56800c9a0 con 0x7fd57806a190 2023-11-13T11:03:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.619+0000 7fd574ff9640 1 -- 172.21.15.16:0/3632474340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56800cb00 con 0x7fd57806a190 2023-11-13T11:03:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.619+0000 7fd574ff9640 1 -- 172.21.15.16:0/3632474340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56800ce70 con 0x7fd57806a190 2023-11-13T11:03:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.619+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c009400 con 0x7fd57806a190 2023-11-13T11:03:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.619+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57815cd90 con 0x7fd57806a190 2023-11-13T11:03:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.620+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57804e4d0 con 0x7fd57806a190 2023-11-13T11:03:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.621+0000 7fd574ff9640 1 -- 172.21.15.16:0/3632474340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd568013470 con 0x7fd57806a190 2023-11-13T11:03:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.623+0000 7fd574ff9640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd56406ae40 0x7fd56406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:13.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.624+0000 7fd574ff9640 1 -- 172.21.15.16:0/3632474340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd568089e90 con 0x7fd57806a190 2023-11-13T11:03:13.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.624+0000 7fd5777fe640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd56406ae40 0x7fd56406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:13.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.625+0000 7fd5777fe640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd56406ae40 0x7fd56406d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd56c036000 tx=0x7fd56c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:13.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.630+0000 7fd574ff9640 1 -- 172.21.15.16:0/3632474340 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd568055060 con 0x7fd57806a190 2023-11-13T11:03:13.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.861+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5780b3100 con 0x7fd57806a190 2023-11-13T11:03:13.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.862+0000 7fd574ff9640 1 -- 172.21.15.16:0/3632474340 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fd56808be90 con 0x7fd57806a190 2023-11-13T11:03:13.867 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:13.867 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3403267614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3403267614},{"type":"v1","addr":"172.21.15.184:6837","nonce":3403267614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1597284197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1597284197},{"type":"v1","addr":"172.21.15.16:6833","nonce":1597284197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1208554242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1208554242},{"type":"v1","addr":"172.21.15.184:6835","nonce":1208554242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":6795,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3561626079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3561626079},{"type":"v1","addr":"172.21.15.16:6835","nonce":3561626079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[]} 2023-11-13T11:03:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 34 2023-11-13T11:03:13.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd56406ae40 msgr2=0x7fd56406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd56406ae40 0x7fd56406d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd56c036000 tx=0x7fd56c03c040 comp rx=0 tx=0).stop 2023-11-13T11:03:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57806a190 msgr2=0x7fd5781f2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57806a190 0x7fd5781f2830 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd56800e9b0 tx=0x7fd56800ee70 comp rx=0 tx=0).stop 2023-11-13T11:03:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 shutdown_connections 2023-11-13T11:03:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd56406ae40 0x7fd56406d300 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5780cb190 0x7fd5780c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5780716b0 0x7fd57814d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 --2- 172.21.15.16:0/3632474340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57806a190 0x7fd5781f2830 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 >> 172.21.15.16:0/3632474340 conn(0x7fd5780b04a0 msgr2=0x7fd5780cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 shutdown_connections 2023-11-13T11:03:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:13.865+0000 7fd57d439640 1 -- 172.21.15.16:0/3632474340 wait complete. 2023-11-13T11:03:13.891 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-13T11:03:13.892 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-13T11:03:13.892 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-13T11:03:13.892 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:03:13.893 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-13T11:03:13.893 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-13T11:03:13.893 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-13T11:03:13.894 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:03:13.894 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-13T11:03:13.894 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-13T11:03:13.895 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-13T11:03:13.895 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:03:13.895 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-13T11:03:13.895 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-13T11:03:13.896 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-13T11:03:13.896 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:03:13.912 INFO:tasks.ceph.mds.c:Stopped 2023-11-13T11:03:13.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-13T11:03:13.913 INFO:tasks.ceph.mds.a:Stopped 2023-11-13T11:03:13.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-13T11:03:13.918 INFO:tasks.ceph.mds.d:Stopped 2023-11-13T11:03:13.918 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-13T11:03:13.919 INFO:tasks.ceph.mds.b:Stopped 2023-11-13T11:03:13.920 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-13T11:03:13.959 INFO:tasks.ceph.mds.c:Started 2023-11-13T11:03:13.959 DEBUG:teuthology.parallel:result is None 2023-11-13T11:03:13.962 INFO:tasks.ceph.mds.a:Started 2023-11-13T11:03:13.963 DEBUG:teuthology.parallel:result is None 2023-11-13T11:03:13.968 INFO:tasks.ceph.mds.d:Started 2023-11-13T11:03:13.968 DEBUG:teuthology.parallel:result is None 2023-11-13T11:03:13.975 INFO:tasks.ceph.mds.b:Started 2023-11-13T11:03:13.975 DEBUG:teuthology.parallel:result is None 2023-11-13T11:03:13.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-13T11:03:14.126 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-13T11:03:14.128 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-13T11:03:14.133 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-11-13T11:03:14.135 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-11-13T11:03:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.167+0000 7ffbd911d640 1 Processor -- start 2023-11-13T11:03:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.167+0000 7ffbd911d640 1 -- start start 2023-11-13T11:03:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.168+0000 7ffbd911d640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffbd415cf20 0x7ffbd406cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.168+0000 7ffbd911d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40cecc0 0x7ffbd4070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.168+0000 7ffbd911d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd40b2900 con 0x7ffbd406bb90 2023-11-13T11:03:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.168+0000 7ffbd911d640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd415d300 con 0x7ffbd415cf20 2023-11-13T11:03:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.168+0000 7ffbd911d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40b1fd0 con 0x7ffbd40cecc0 2023-11-13T11:03:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.168+0000 7ffbcbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffbd406bb90 0x7ffbd40ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59284/0 (socket says 172.21.15.16:59284) 2023-11-13T11:03:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.168+0000 7ffbcbfff640 1 -- 172.21.15.16:0/4135409638 learned_addr learned my addr 172.21.15.16:0/4135409638 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd2d9d640 1 --2- 172.21.15.16:0/4135409638 >> v2:172.21.15.184:3300/0 conn(0x7ffbd415cf20 0x7ffbd406cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd359e640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40cecc0 0x7ffbd4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646521252 0 0) 0x7ffbd40b2900 con 0x7ffbd406bb90 2023-11-13T11:03:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc0003520 con 0x7ffbd406bb90 2023-11-13T11:03:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2798312494 0 0) 0x7ffbc0003520 con 0x7ffbd406bb90 2023-11-13T11:03:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40cecc0 msgr2=0x7ffbd4070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:14.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40cecc0 0x7ffbd4070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:14.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 >> v2:172.21.15.184:3300/0 conn(0x7ffbd415cf20 msgr2=0x7ffbd406cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:14.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 --2- 172.21.15.16:0/4135409638 >> v2:172.21.15.184:3300/0 conn(0x7ffbd415cf20 0x7ffbd406cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:14.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40b22b0 con 0x7ffbd406bb90 2023-11-13T11:03:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7ffbbc001670 con 0x7ffbd406bb90 2023-11-13T11:03:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd2d9d640 1 --2- 172.21.15.16:0/4135409638 >> v2:172.21.15.184:3300/0 conn(0x7ffbd415cf20 0x7ffbd406cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T11:03:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 >> v1:172.21.15.16:6789/0 conn(0x7ffbd406bb90 legacy=0x7ffbd40ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbc0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc00047c0 0x7ffbc0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40cecc0 0x7ffbc0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40b1fd0 con 0x7ffbc0003970 2023-11-13T11:03:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc000b980 con 0x7ffbc00047c0 2023-11-13T11:03:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.169+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc000bac0 con 0x7ffbd40cecc0 2023-11-13T11:03:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.170+0000 7ffbd2d9d640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbc0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.170+0000 7ffbcbfff640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc00047c0 0x7ffbc0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.170+0000 7ffbd359e640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40cecc0 0x7ffbc0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.170+0000 7ffbd2d9d640 1 -- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40cecc0 msgr2=0x7ffbc0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.170+0000 7ffbd2d9d640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40cecc0 0x7ffbc0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.170+0000 7ffbd2d9d640 1 -- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc00047c0 msgr2=0x7ffbc0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.170+0000 7ffbd2d9d640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc00047c0 0x7ffbc0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.170+0000 7ffbd2d9d640 1 -- 172.21.15.16:0/4135409638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc0003520 con 0x7ffbc0003970 2023-11-13T11:03:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.171+0000 7ffbd2d9d640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbc0003d90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009c90 tx=0x7ffbc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77cd3cf6a7b31e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc402c980 con 0x7ffbc0003970 2023-11-13T11:03:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc402cae0 con 0x7ffbc0003970 2023-11-13T11:03:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd259c640 1 -- 172.21.15.16:0/4135409638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc402ce50 con 0x7ffbc0003970 2023-11-13T11:03:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd911d640 1 -- 172.21.15.16:0/4135409638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 msgr2=0x7ffbc0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd911d640 1 --2- 172.21.15.16:0/4135409638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbc0003d90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009c90 tx=0x7ffbc00057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd359e640 1 -- 172.21.15.16:0/4135409638 reap_dead start 2023-11-13T11:03:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd911d640 1 -- 172.21.15.16:0/4135409638 shutdown_connections 2023-11-13T11:03:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd911d640 1 -- 172.21.15.16:0/4135409638 >> 172.21.15.16:0/4135409638 conn(0x7ffbd406a190 msgr2=0x7ffbd406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd911d640 1 -- 172.21.15.16:0/4135409638 shutdown_connections 2023-11-13T11:03:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.172+0000 7ffbd911d640 1 -- 172.21.15.16:0/4135409638 wait complete. 2023-11-13T11:03:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.174+0000 7ffbd911d640 1 Processor -- start 2023-11-13T11:03:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.174+0000 7ffbd911d640 1 -- start start 2023-11-13T11:03:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.175+0000 7ffbd911d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbd40cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.175+0000 7ffbd911d640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40c5060 0x7ffbd415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.175+0000 7ffbd911d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd415f6f0 0x7ffbd414b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.175+0000 7ffbd911d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4149ef0 con 0x7ffbc0003970 2023-11-13T11:03:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.175+0000 7ffbd911d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd415fba0 con 0x7ffbd415f6f0 2023-11-13T11:03:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.175+0000 7ffbd911d640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4149300 con 0x7ffbd40c5060 2023-11-13T11:03:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbd40cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbd40cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47876/0 (socket says 172.21.15.16:47876) 2023-11-13T11:03:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd2d9d640 1 -- 172.21.15.16:0/1899649317 learned_addr learned my addr 172.21.15.16:0/1899649317 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd359e640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd415f6f0 0x7ffbd414b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbcbfff640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40c5060 0x7ffbd415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd2d9d640 1 -- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40c5060 msgr2=0x7ffbd415f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd2d9d640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40c5060 0x7ffbd415f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd2d9d640 1 -- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd415f6f0 msgr2=0x7ffbd414b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd2d9d640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd415f6f0 0x7ffbd414b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd2d9d640 1 -- 172.21.15.16:0/1899649317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd4149550 con 0x7ffbc0003970 2023-11-13T11:03:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.176+0000 7ffbd2d9d640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbd40cd2b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ffbc402c460 tx=0x7ffbc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.177+0000 7ffbcb7fe640 1 -- 172.21.15.16:0/1899649317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4034a80 con 0x7ffbc0003970 2023-11-13T11:03:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.177+0000 7ffbcb7fe640 1 -- 172.21.15.16:0/1899649317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc4034c20 con 0x7ffbc0003970 2023-11-13T11:03:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.177+0000 7ffbcb7fe640 1 -- 172.21.15.16:0/1899649317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4004020 con 0x7ffbc0003970 2023-11-13T11:03:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.177+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd4149830 con 0x7ffbc0003970 2023-11-13T11:03:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.177+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd40cd7f0 con 0x7ffbc0003970 2023-11-13T11:03:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.178+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd414a030 con 0x7ffbc0003970 2023-11-13T11:03:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.179+0000 7ffbcb7fe640 1 -- 172.21.15.16:0/1899649317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffbc4004180 con 0x7ffbc0003970 2023-11-13T11:03:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.180+0000 7ffbcb7fe640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffbb406ae70 0x7ffbb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.180+0000 7ffbcb7fe640 1 -- 172.21.15.16:0/1899649317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ffbc40ab570 con 0x7ffbc0003970 2023-11-13T11:03:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.180+0000 7ffbcbfff640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffbb406ae70 0x7ffbb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.181+0000 7ffbcbfff640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffbb406ae70 0x7ffbb406d330 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc001050 tx=0x7ffbbc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.184+0000 7ffbcb7fe640 1 -- 172.21.15.16:0/1899649317 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffbd414a030 con 0x7ffbc0003970 2023-11-13T11:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:14.397+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffbd414a030 con 0x7ffbc0003970 2023-11-13T11:03:15.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.407+0000 7ffbcb7fe640 1 -- 172.21.15.16:0/1899649317 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffbd414a030 con 0x7ffbc0003970 2023-11-13T11:03:15.412 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-11-13T11:03:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffbb406ae70 msgr2=0x7ffbb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffbb406ae70 0x7ffbb406d330 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc001050 tx=0x7ffbbc009040 comp rx=0 tx=0).stop 2023-11-13T11:03:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 msgr2=0x7ffbd40cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbd40cd2b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ffbc402c460 tx=0x7ffbc4004730 comp rx=0 tx=0).stop 2023-11-13T11:03:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 shutdown_connections 2023-11-13T11:03:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ffbb406ae70 0x7ffbb406d330 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbd415f6f0 0x7ffbd414b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbd40c5060 0x7ffbd415f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 --2- 172.21.15.16:0/1899649317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc0003970 0x7ffbd40cd2b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.411+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 >> 172.21.15.16:0/1899649317 conn(0x7ffbd406a190 msgr2=0x7ffbd40b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.412+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 shutdown_connections 2023-11-13T11:03:15.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.412+0000 7ffbd911d640 1 -- 172.21.15.16:0/1899649317 wait complete. 2023-11-13T11:03:15.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-13T11:03:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.652+0000 7f0b27d41640 1 Processor -- start 2023-11-13T11:03:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.652+0000 7f0b27d41640 1 -- start start 2023-11-13T11:03:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.654+0000 7f0b27d41640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0b2015aaf0 0x7f0b200ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.654+0000 7f0b27d41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b200ce720 0x7f0b200cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.654+0000 7f0b27d41640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b20071ea0 con 0x7f0b200b18f0 2023-11-13T11:03:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.654+0000 7f0b27d41640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b200cf0f0 con 0x7f0b200ce720 2023-11-13T11:03:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.654+0000 7f0b27d41640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b200b1df0 con 0x7f0b2015aaf0 2023-11-13T11:03:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.654+0000 7f0b262de640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b200ce720 0x7f0b200cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.654+0000 7f0b262de640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b200ce720 0x7f0b200cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47892/0 (socket says 172.21.15.16:47892) 2023-11-13T11:03:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.654+0000 7f0b262de640 1 -- 172.21.15.16:0/692686524 learned_addr learned my addr 172.21.15.16:0/692686524 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.655+0000 7f0b25add640 1 --2- 172.21.15.16:0/692686524 >> v2:172.21.15.184:3300/0 conn(0x7f0b2015aaf0 0x7f0b200ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.655+0000 7f0b262de640 1 -- 172.21.15.16:0/692686524 >> v2:172.21.15.184:3300/0 conn(0x7f0b2015aaf0 msgr2=0x7f0b200ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.655+0000 7f0b262de640 1 --2- 172.21.15.16:0/692686524 >> v2:172.21.15.184:3300/0 conn(0x7f0b2015aaf0 0x7f0b200ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.655+0000 7f0b262de640 1 -- 172.21.15.16:0/692686524 >> v1:172.21.15.184:6789/0 conn(0x7f0b200b18f0 legacy=0x7f0b20071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.655+0000 7f0b262de640 1 -- 172.21.15.16:0/692686524 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b200b20d0 con 0x7f0b200ce720 2023-11-13T11:03:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.656+0000 7f0b262de640 1 --2- 172.21.15.16:0/692686524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200ce720 0x7f0b200cebb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0b2006df00 tx=0x7f0b1802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dd5f9541e98b16 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.657+0000 7f0b24adb640 1 -- 172.21.15.16:0/692686524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1802ca80 con 0x7f0b200ce720 2023-11-13T11:03:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.657+0000 7f0b24adb640 1 -- 172.21.15.16:0/692686524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1802cbe0 con 0x7f0b200ce720 2023-11-13T11:03:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.657+0000 7f0b24adb640 1 -- 172.21.15.16:0/692686524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b180343f0 con 0x7f0b200ce720 2023-11-13T11:03:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.657+0000 7f0b27d41640 1 -- 172.21.15.16:0/692686524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200ce720 msgr2=0x7f0b200cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.657+0000 7f0b27d41640 1 --2- 172.21.15.16:0/692686524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200ce720 0x7f0b200cebb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0b2006df00 tx=0x7f0b1802bee0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.657+0000 7f0b27d41640 1 -- 172.21.15.16:0/692686524 shutdown_connections 2023-11-13T11:03:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.657+0000 7f0b27d41640 1 --2- 172.21.15.16:0/692686524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200ce720 0x7f0b200cebb0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.657+0000 7f0b27d41640 1 --2- 172.21.15.16:0/692686524 >> v2:172.21.15.184:3300/0 conn(0x7f0b2015aaf0 0x7f0b200ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.657+0000 7f0b27d41640 1 -- 172.21.15.16:0/692686524 >> 172.21.15.16:0/692686524 conn(0x7f0b2006a190 msgr2=0x7f0b2006a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.658+0000 7f0b27d41640 1 -- 172.21.15.16:0/692686524 shutdown_connections 2023-11-13T11:03:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.658+0000 7f0b27d41640 1 -- 172.21.15.16:0/692686524 wait complete. 2023-11-13T11:03:15.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.659+0000 7f0b27d41640 1 Processor -- start 2023-11-13T11:03:15.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.659+0000 7f0b27d41640 1 -- start start 2023-11-13T11:03:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.660+0000 7f0b27d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200b18f0 0x7f0b200c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.660+0000 7f0b27d41640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b200ce720 0x7f0b2014e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b27d41640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b2015aaf0 0x7f0b200c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b27d41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b20155790 con 0x7f0b200b18f0 2023-11-13T11:03:15.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b27d41640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200c4420 con 0x7f0b2015aaf0 2023-11-13T11:03:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b27d41640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200c4700 con 0x7f0b200ce720 2023-11-13T11:03:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b25add640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200b18f0 0x7f0b200c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b25add640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200b18f0 0x7f0b200c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47894/0 (socket says 172.21.15.16:47894) 2023-11-13T11:03:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b25add640 1 -- 172.21.15.16:0/696447050 learned_addr learned my addr 172.21.15.16:0/696447050 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:15.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b252dc640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b200ce720 0x7f0b2014e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:15.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b262de640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b2015aaf0 0x7f0b200c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:15.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b25add640 1 -- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b200ce720 msgr2=0x7f0b2014e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:15.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b25add640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b200ce720 0x7f0b2014e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b25add640 1 -- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b2015aaf0 msgr2=0x7f0b200c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:15.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b25add640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b2015aaf0 0x7f0b200c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.661+0000 7f0b25add640 1 -- 172.21.15.16:0/696447050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b18009b30 con 0x7f0b200b18f0 2023-11-13T11:03:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.662+0000 7f0b25add640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200b18f0 0x7f0b200c54c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00e870 tx=0x7f0b1c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.662+0000 7f0b167fc640 1 -- 172.21.15.16:0/696447050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0094f0 con 0x7f0b200b18f0 2023-11-13T11:03:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.662+0000 7f0b167fc640 1 -- 172.21.15.16:0/696447050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c00c940 con 0x7f0b200b18f0 2023-11-13T11:03:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.662+0000 7f0b167fc640 1 -- 172.21.15.16:0/696447050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c00ccf0 con 0x7f0b200b18f0 2023-11-13T11:03:15.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.662+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b200c49e0 con 0x7f0b200b18f0 2023-11-13T11:03:15.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.662+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b200c4fa0 con 0x7f0b200b18f0 2023-11-13T11:03:15.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.664+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b200cbc70 con 0x7f0b200b18f0 2023-11-13T11:03:15.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.664+0000 7f0b167fc640 1 -- 172.21.15.16:0/696447050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b1c013470 con 0x7f0b200b18f0 2023-11-13T11:03:15.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.665+0000 7f0b167fc640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b0406ae40 0x7f0b0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:15.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.666+0000 7f0b167fc640 1 -- 172.21.15.16:0/696447050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0b1c0894c0 con 0x7f0b200b18f0 2023-11-13T11:03:15.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.666+0000 7f0b252dc640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b0406ae40 0x7f0b0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.667+0000 7f0b252dc640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b0406ae40 0x7f0b0406d300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0b20155430 tx=0x7f0b10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.670+0000 7f0b167fc640 1 -- 172.21.15.16:0/696447050 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b1c054770 con 0x7f0b200b18f0 2023-11-13T11:03:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.938+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0b20067090 con 0x7f0b200b18f0 2023-11-13T11:03:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.939+0000 7f0b167fc640 1 -- 172.21.15.16:0/696447050 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0b1c057d90 con 0x7f0b200b18f0 2023-11-13T11:03:15.943 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:15.943 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:03:15.944 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-11-13T11:03:15.944 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.944 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-11-13T11:03:15.944 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBcAFJlFf7FKBAAzl2CzFIHt1vexIHicDcMOQ==", 2023-11-13T11:03:15.944 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.945 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:03:15.945 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:03:15.945 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:03:15.945 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.946 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.946 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-11-13T11:03:15.947 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJlxksXERAApz8MD99vmb9+TVqjK4yoHQ==", 2023-11-13T11:03:15.947 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.947 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:03:15.947 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:03:15.947 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:03:15.948 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.948 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.948 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.948 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-11-13T11:03:15.949 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJl3f8tABAAn77Y5rtLX6KXfDt6O64jHw==", 2023-11-13T11:03:15.949 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.949 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:03:15.949 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:03:15.949 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:03:15.949 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.950 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.950 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.950 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-11-13T11:03:15.951 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJliC8UHxAAp4RpLcBhknvkfcD6+u0cWg==", 2023-11-13T11:03:15.951 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.951 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:03:15.951 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:03:15.951 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:03:15.952 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.952 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.952 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.952 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-11-13T11:03:15.953 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBgAFJl6OfsFRAA/pxN6JyHj0g/ZTvQTTBvHg==", 2023-11-13T11:03:15.953 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.953 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:03:15.953 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:03:15.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.954 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.954 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.955 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-11-13T11:03:15.955 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBjAFJlCcthIRAAD9rCdKqLeqILvx+RcVu3aA==", 2023-11-13T11:03:15.955 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.955 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:03:15.956 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:03:15.956 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.956 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.957 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.957 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-11-13T11:03:15.957 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBmAFJldeAFLRAA6BDzhw1Sml5AWptU6G7MEA==", 2023-11-13T11:03:15.957 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.958 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:03:15.958 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:03:15.958 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.959 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.960 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.960 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-11-13T11:03:15.960 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBqAFJlhz82DBAAmRhdLjHKXpAyNNqUkXk5qw==", 2023-11-13T11:03:15.960 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.960 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:03:15.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:03:15.961 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.961 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.962 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-11-13T11:03:15.962 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBxAFJlgXLmHxAAHpM0gY+C/Ionsp0sG9KoKQ==", 2023-11-13T11:03:15.962 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.962 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:03:15.962 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:03:15.963 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.963 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.963 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.963 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-11-13T11:03:15.964 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQB0AFJlfsdPKxAA5Ba0gTAcUVQ6PueL9Ul6qQ==", 2023-11-13T11:03:15.964 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.964 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:03:15.965 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:03:15.965 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.965 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.965 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.966 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-11-13T11:03:15.966 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQB4AFJlD6j5CBAAXuiAkwIbpfUSJ1j1j3QP4w==", 2023-11-13T11:03:15.966 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.966 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:03:15.966 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:03:15.967 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.967 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.967 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.968 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-11-13T11:03:15.968 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQB7AFJlkX9TIhAAPjuDlChUGXjej7ot5x6vDg==", 2023-11-13T11:03:15.968 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.968 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-13T11:03:15.968 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-13T11:03:15.969 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.969 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.969 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.970 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-11-13T11:03:15.970 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJl9p66LRAAYRThccLplmNpsr+W647dOA==", 2023-11-13T11:03:15.970 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.970 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:03:15.971 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-11-13T11:03:15.971 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-13T11:03:15.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.971 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.972 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-11-13T11:03:15.972 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBeAFJlkymOCRAATnUAZgjg1er9vu82gnrxtw==", 2023-11-13T11:03:15.972 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.972 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:03:15.973 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T11:03:15.973 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T11:03:15.973 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T11:03:15.973 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.974 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.974 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-11-13T11:03:15.974 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBdAFJlrk4zORAA7tCfwi0UlPhDcL1GZ6UIIg==", 2023-11-13T11:03:15.974 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.974 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:03:15.975 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T11:03:15.975 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T11:03:15.975 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T11:03:15.975 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.976 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.976 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-11-13T11:03:15.977 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBeAFJlrh6DFBAAtrGSamisl/Jexs9ewdiAkQ==", 2023-11-13T11:03:15.977 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.977 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-13T11:03:15.977 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-13T11:03:15.978 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-13T11:03:15.978 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-13T11:03:15.978 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.979 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.979 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-11-13T11:03:15.979 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBbAFJlhaZyGxAANZe05VzLnzM3KiFjOH93JA==", 2023-11-13T11:03:15.979 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.979 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-13T11:03:15.980 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-13T11:03:15.980 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-13T11:03:15.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.980 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.981 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.981 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-11-13T11:03:15.981 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBcAFJlvFoIHBAAIXc5UQJIKTtHmeVznGZPSA==", 2023-11-13T11:03:15.981 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.981 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-13T11:03:15.982 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-13T11:03:15.982 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.982 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:15.982 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:15.983 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-11-13T11:03:15.983 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBcAFJluuvTERAAn6zxosZEaeBkgENpZDAHqg==", 2023-11-13T11:03:15.983 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-13T11:03:15.983 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-13T11:03:15.984 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-13T11:03:15.984 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-13T11:03:15.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:15.984 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-11-13T11:03:15.984 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:03:15.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b0406ae40 msgr2=0x7f0b0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:15.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b0406ae40 0x7f0b0406d300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0b20155430 tx=0x7f0b10009040 comp rx=0 tx=0).stop 2023-11-13T11:03:15.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200b18f0 msgr2=0x7f0b200c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:15.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200b18f0 0x7f0b200c54c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00e870 tx=0x7f0b1c00ed30 comp rx=0 tx=0).stop 2023-11-13T11:03:15.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 shutdown_connections 2023-11-13T11:03:15.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0b0406ae40 0x7f0b0406d300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b2015aaf0 0x7f0b200c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b200ce720 0x7f0b2014e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 --2- 172.21.15.16:0/696447050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b200b18f0 0x7f0b200c54c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:15.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 >> 172.21.15.16:0/696447050 conn(0x7f0b2006a190 msgr2=0x7f0b200b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:15.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 shutdown_connections 2023-11-13T11:03:15.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:15.942+0000 7f0b27d41640 1 -- 172.21.15.16:0/696447050 wait complete. 2023-11-13T11:03:15.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.218+0000 7f0664600640 1 Processor -- start 2023-11-13T11:03:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.219+0000 7f0664600640 1 -- start start 2023-11-13T11:03:16.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.220+0000 7f0664600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f065c15cf20 0x7f065c06cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.220+0000 7f0664600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c0cecc0 0x7f065c070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f0664600640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c0b2900 con 0x7f065c06bb90 2023-11-13T11:03:16.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f0664600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c15d300 con 0x7f065c15cf20 2023-11-13T11:03:16.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f0664600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0b1fd0 con 0x7f065c0cecc0 2023-11-13T11:03:16.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f066239c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f065c15cf20 0x7f065c06cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f066239c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f065c15cf20 0x7f065c06cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47898/0 (socket says 172.21.15.16:47898) 2023-11-13T11:03:16.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f066239c640 1 -- 172.21.15.16:0/2382097037 learned_addr learned my addr 172.21.15.16:0/2382097037 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:16.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f066239c640 1 -- 172.21.15.16:0/2382097037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c0cecc0 msgr2=0x7f065c070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:16.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f066239c640 1 --2- 172.21.15.16:0/2382097037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c0cecc0 0x7f065c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f066239c640 1 -- 172.21.15.16:0/2382097037 >> v1:172.21.15.184:6789/0 conn(0x7f065c06bb90 legacy=0x7f065c0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:16.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.221+0000 7f066239c640 1 -- 172.21.15.16:0/2382097037 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0b22b0 con 0x7f065c15cf20 2023-11-13T11:03:16.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.222+0000 7f066239c640 1 --2- 172.21.15.16:0/2382097037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c15cf20 0x7f065c06cab0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0658009870 tx=0x7f065802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83944eeced7a85e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:16.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.223+0000 7f066139a640 1 -- 172.21.15.16:0/2382097037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658031870 con 0x7f065c15cf20 2023-11-13T11:03:16.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.223+0000 7f066139a640 1 -- 172.21.15.16:0/2382097037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06580319d0 con 0x7f065c15cf20 2023-11-13T11:03:16.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.223+0000 7f066139a640 1 -- 172.21.15.16:0/2382097037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658031d60 con 0x7f065c15cf20 2023-11-13T11:03:16.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.223+0000 7f0664600640 1 -- 172.21.15.16:0/2382097037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c15cf20 msgr2=0x7f065c06cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.223+0000 7f0664600640 1 --2- 172.21.15.16:0/2382097037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c15cf20 0x7f065c06cab0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0658009870 tx=0x7f065802b910 comp rx=0 tx=0).stop 2023-11-13T11:03:16.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.224+0000 7f0664600640 1 -- 172.21.15.16:0/2382097037 shutdown_connections 2023-11-13T11:03:16.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.224+0000 7f0664600640 1 --2- 172.21.15.16:0/2382097037 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c0cecc0 0x7f065c070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.224+0000 7f0664600640 1 --2- 172.21.15.16:0/2382097037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c15cf20 0x7f065c06cab0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.224+0000 7f0664600640 1 -- 172.21.15.16:0/2382097037 >> 172.21.15.16:0/2382097037 conn(0x7f065c06a190 msgr2=0x7f065c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:16.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.224+0000 7f0664600640 1 -- 172.21.15.16:0/2382097037 shutdown_connections 2023-11-13T11:03:16.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.224+0000 7f0664600640 1 -- 172.21.15.16:0/2382097037 wait complete. 2023-11-13T11:03:16.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.225+0000 7f0664600640 1 Processor -- start 2023-11-13T11:03:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.226+0000 7f0664600640 1 -- start start 2023-11-13T11:03:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0664600640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06bb90 0x7f065c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0664600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c0cecc0 0x7f065c0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0664600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c15cf20 0x7f065c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0664600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0ccf90 con 0x7f065c0cecc0 2023-11-13T11:03:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0664600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0cc0a0 con 0x7f065c15cf20 2023-11-13T11:03:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0664600640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0cc310 con 0x7f065c06bb90 2023-11-13T11:03:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0661b9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c0cecc0 0x7f065c0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0661b9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c0cecc0 0x7f065c0cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47912/0 (socket says 172.21.15.16:47912) 2023-11-13T11:03:16.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0661b9b640 1 -- 172.21.15.16:0/3979247973 learned_addr learned my addr 172.21.15.16:0/3979247973 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:16.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f066239c640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06bb90 0x7f065c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0662b9d640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c15cf20 0x7f065c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0661b9b640 1 -- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06bb90 msgr2=0x7f065c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0661b9b640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06bb90 0x7f065c0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0661b9b640 1 -- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c15cf20 msgr2=0x7f065c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0661b9b640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c15cf20 0x7f065c0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0661b9b640 1 -- 172.21.15.16:0/3979247973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06580094d0 con 0x7f065c0cecc0 2023-11-13T11:03:16.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.227+0000 7f0662b9d640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c15cf20 0x7f065c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:16.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.228+0000 7f0661b9b640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c0cecc0 0x7f065c0cbb60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b3f0 tx=0x7f064c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:16.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.228+0000 7f0652ffd640 1 -- 172.21.15.16:0/3979247973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c011830 con 0x7f065c0cecc0 2023-11-13T11:03:16.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.228+0000 7f0652ffd640 1 -- 172.21.15.16:0/3979247973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c011990 con 0x7f065c0cecc0 2023-11-13T11:03:16.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.228+0000 7f0652ffd640 1 -- 172.21.15.16:0/3979247973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c011d40 con 0x7f065c0cecc0 2023-11-13T11:03:16.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.228+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c0cc5f0 con 0x7f065c0cecc0 2023-11-13T11:03:16.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.228+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c0ccbb0 con 0x7f065c0cecc0 2023-11-13T11:03:16.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.229+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c0cd0d0 con 0x7f065c0cecc0 2023-11-13T11:03:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.230+0000 7f0652ffd640 1 -- 172.21.15.16:0/3979247973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f064c01a400 con 0x7f065c0cecc0 2023-11-13T11:03:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.232+0000 7f0652ffd640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f064006ae40 0x7f064006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.232+0000 7f0652ffd640 1 -- 172.21.15.16:0/3979247973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f064c089290 con 0x7f065c0cecc0 2023-11-13T11:03:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.232+0000 7f066239c640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f064006ae40 0x7f064006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.234+0000 7f066239c640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f064006ae40 0x7f064006d300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0658036000 tx=0x7f065803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:16.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.237+0000 7f0652ffd640 1 -- 172.21.15.16:0/3979247973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f065c0cd0d0 con 0x7f065c0cecc0 2023-11-13T11:03:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.432+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f065c0cd0d0 con 0x7f065c0cecc0 2023-11-13T11:03:16.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.434+0000 7f0652ffd640 1 -- 172.21.15.16:0/3979247973 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13074 (secure 0 0 0) 0x7f065c0cd0d0 con 0x7f065c0cecc0 2023-11-13T11:03:16.439 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:16.439 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":65,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:15.404165+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:16.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f064006ae40 msgr2=0x7f064006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f064006ae40 0x7f064006d300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0658036000 tx=0x7f065803c040 comp rx=0 tx=0).stop 2023-11-13T11:03:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c0cecc0 msgr2=0x7f065c0cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c0cecc0 0x7f065c0cbb60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b3f0 tx=0x7f064c00b8b0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 shutdown_connections 2023-11-13T11:03:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f064006ae40 0x7f064006d300 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f065c15cf20 0x7f065c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f065c0cecc0 0x7f065c0cbb60 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 --2- 172.21.15.16:0/3979247973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f065c06bb90 0x7f065c0cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 >> 172.21.15.16:0/3979247973 conn(0x7f065c06a190 msgr2=0x7f065c0ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.437+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 shutdown_connections 2023-11-13T11:03:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.438+0000 7f0664600640 1 -- 172.21.15.16:0/3979247973 wait complete. 2023-11-13T11:03:16.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:03:16.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:03:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.695+0000 7fb8866c3640 1 Processor -- start 2023-11-13T11:03:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.695+0000 7fb8866c3640 1 -- start start 2023-11-13T11:03:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.697+0000 7fb8866c3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb880070b60 0x7fb8801679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.697+0000 7fb8866c3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88006bca0 con 0x7fb88006b6c0 2023-11-13T11:03:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.697+0000 7fb8866c3640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88006bf80 con 0x7fb8800715b0 2023-11-13T11:03:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.697+0000 7fb8866c3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88006c920 con 0x7fb880070b60 2023-11-13T11:03:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.697+0000 7fb885ec2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb880070b60 0x7fb8801679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.697+0000 7fb885ec2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb880070b60 0x7fb8801679a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47918/0 (socket says 172.21.15.16:47918) 2023-11-13T11:03:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.697+0000 7fb885ec2640 1 -- 172.21.15.16:0/3136899401 learned_addr learned my addr 172.21.15.16:0/3136899401 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.698+0000 7fb885ec2640 1 -- 172.21.15.16:0/3136899401 >> v1:172.21.15.184:6789/0 conn(0x7fb8800715b0 legacy=0x7fb8800b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.698+0000 7fb885ec2640 1 -- 172.21.15.16:0/3136899401 >> v1:172.21.15.16:6789/0 conn(0x7fb88006b6c0 legacy=0x7fb8800ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.698+0000 7fb885ec2640 1 -- 172.21.15.16:0/3136899401 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88006c260 con 0x7fb880070b60 2023-11-13T11:03:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.700+0000 7fb885ec2640 1 --2- 172.21.15.16:0/3136899401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb880070b60 0x7fb8801679a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb878009870 tx=0x7fb87802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773d0c94351762fb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.700+0000 7fb87ffff640 1 -- 172.21.15.16:0/3136899401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878030850 con 0x7fb880070b60 2023-11-13T11:03:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.700+0000 7fb87ffff640 1 -- 172.21.15.16:0/3136899401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8780309b0 con 0x7fb880070b60 2023-11-13T11:03:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.700+0000 7fb87ffff640 1 -- 172.21.15.16:0/3136899401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878030d40 con 0x7fb880070b60 2023-11-13T11:03:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.701+0000 7fb8866c3640 1 -- 172.21.15.16:0/3136899401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb880070b60 msgr2=0x7fb8801679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.701+0000 7fb8866c3640 1 --2- 172.21.15.16:0/3136899401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb880070b60 0x7fb8801679a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb878009870 tx=0x7fb87802b990 comp rx=0 tx=0).stop 2023-11-13T11:03:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.701+0000 7fb8866c3640 1 -- 172.21.15.16:0/3136899401 shutdown_connections 2023-11-13T11:03:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.701+0000 7fb8866c3640 1 --2- 172.21.15.16:0/3136899401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb880070b60 0x7fb8801679a0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.701+0000 7fb8866c3640 1 -- 172.21.15.16:0/3136899401 >> 172.21.15.16:0/3136899401 conn(0x7fb8800b0970 msgr2=0x7fb8800b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.701+0000 7fb8866c3640 1 -- 172.21.15.16:0/3136899401 shutdown_connections 2023-11-13T11:03:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.701+0000 7fb8866c3640 1 -- 172.21.15.16:0/3136899401 wait complete. 2023-11-13T11:03:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.703+0000 7fb8866c3640 1 Processor -- start 2023-11-13T11:03:16.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.703+0000 7fb8866c3640 1 -- start start 2023-11-13T11:03:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.704+0000 7fb8866c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88006b6c0 0x7fb880143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.704+0000 7fb8866c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb880070b60 0x7fb8800c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.704+0000 7fb8866c3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8800715b0 0x7fb8800ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.704+0000 7fb8866c3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800c48e0 con 0x7fb88006b6c0 2023-11-13T11:03:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.704+0000 7fb8866c3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800c4b80 con 0x7fb880070b60 2023-11-13T11:03:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.704+0000 7fb8866c3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800c4e60 con 0x7fb8800715b0 2023-11-13T11:03:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb8856c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88006b6c0 0x7fb880143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb8856c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88006b6c0 0x7fb880143390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47932/0 (socket says 172.21.15.16:47932) 2023-11-13T11:03:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb8856c1640 1 -- 172.21.15.16:0/2736652642 learned_addr learned my addr 172.21.15.16:0/2736652642 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb885ec2640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8800715b0 0x7fb8800ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb884ec0640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb880070b60 0x7fb8800c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb8856c1640 1 -- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8800715b0 msgr2=0x7fb8800ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb8856c1640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8800715b0 0x7fb8800ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb8856c1640 1 -- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb880070b60 msgr2=0x7fb8800c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb8856c1640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb880070b60 0x7fb8800c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb8856c1640 1 -- 172.21.15.16:0/2736652642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8780094d0 con 0x7fb88006b6c0 2023-11-13T11:03:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb884ec0640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb880070b60 0x7fb8800c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.705+0000 7fb885ec2640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8800715b0 0x7fb8800ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.706+0000 7fb8856c1640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88006b6c0 0x7fb880143390 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb87000c730 tx=0x7fb87000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.706+0000 7fb87dffb640 1 -- 172.21.15.16:0/2736652642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870003b70 con 0x7fb88006b6c0 2023-11-13T11:03:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.706+0000 7fb87dffb640 1 -- 172.21.15.16:0/2736652642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb870003cf0 con 0x7fb88006b6c0 2023-11-13T11:03:16.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.706+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8800c5140 con 0x7fb88006b6c0 2023-11-13T11:03:16.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.706+0000 7fb87dffb640 1 -- 172.21.15.16:0/2736652642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870003ea0 con 0x7fb88006b6c0 2023-11-13T11:03:16.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.706+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8801438d0 con 0x7fb88006b6c0 2023-11-13T11:03:16.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.708+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88004e4d0 con 0x7fb88006b6c0 2023-11-13T11:03:16.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.709+0000 7fb87dffb640 1 -- 172.21.15.16:0/2736652642 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb87000aa90 con 0x7fb88006b6c0 2023-11-13T11:03:16.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.710+0000 7fb87dffb640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb86c06ae40 0x7fb86c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.710+0000 7fb87dffb640 1 -- 172.21.15.16:0/2736652642 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb870089240 con 0x7fb88006b6c0 2023-11-13T11:03:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.711+0000 7fb884ec0640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb86c06ae40 0x7fb86c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.712+0000 7fb884ec0640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb86c06ae40 0x7fb86c06d300 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb8801f6ce0 tx=0x7fb874009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:16.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.714+0000 7fb87dffb640 1 -- 172.21.15.16:0/2736652642 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8700544f0 con 0x7fb88006b6c0 2023-11-13T11:03:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.907+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb88006d5a0 con 0x7fb88006b6c0 2023-11-13T11:03:16.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.908+0000 7fb87dffb640 1 -- 172.21.15.16:0/2736652642 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb870057b10 con 0x7fb88006b6c0 2023-11-13T11:03:16.912 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:03:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.910+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb86c06ae40 msgr2=0x7fb86c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.910+0000 7fb8866c3640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb86c06ae40 0x7fb86c06d300 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb8801f6ce0 tx=0x7fb874009040 comp rx=0 tx=0).stop 2023-11-13T11:03:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.910+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88006b6c0 msgr2=0x7fb880143390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.910+0000 7fb8866c3640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88006b6c0 0x7fb880143390 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb87000c730 tx=0x7fb87000cbf0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.911+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 shutdown_connections 2023-11-13T11:03:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.911+0000 7fb8866c3640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb86c06ae40 0x7fb86c06d300 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.911+0000 7fb8866c3640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8800715b0 0x7fb8800ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.911+0000 7fb8866c3640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb880070b60 0x7fb8800c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.911+0000 7fb8866c3640 1 --2- 172.21.15.16:0/2736652642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88006b6c0 0x7fb880143390 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.911+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 >> 172.21.15.16:0/2736652642 conn(0x7fb8800b0970 msgr2=0x7fb880145d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.911+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 shutdown_connections 2023-11-13T11:03:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:16.911+0000 7fb8866c3640 1 -- 172.21.15.16:0/2736652642 wait complete. 2023-11-13T11:03:16.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-13T11:03:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.159+0000 7fa38158e640 1 Processor -- start 2023-11-13T11:03:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.160+0000 7fa38158e640 1 -- start start 2023-11-13T11:03:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.161+0000 7fa38158e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa37c160240 0x7fa37c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.161+0000 7fa38158e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37c06b2a0 con 0x7fa37c0b5970 2023-11-13T11:03:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.161+0000 7fa38158e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37c06b4f0 con 0x7fa37c0d1090 2023-11-13T11:03:17.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.161+0000 7fa38158e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37c06b7d0 con 0x7fa37c160240 2023-11-13T11:03:17.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.162+0000 7fa37b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa37c0b5970 0x7fa37c0d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59314/0 (socket says 172.21.15.16:59314) 2023-11-13T11:03:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.162+0000 7fa37b7fe640 1 -- 172.21.15.16:0/1831025356 learned_addr learned my addr 172.21.15.16:0/1831025356 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.162+0000 7fa37bfff640 1 --2- 172.21.15.16:0/1831025356 >> v2:172.21.15.184:3300/0 conn(0x7fa37c160240 0x7fa37c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.162+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576441725 0 0) 0x7fa37c06b2a0 con 0x7fa37c0b5970 2023-11-13T11:03:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.162+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa364003520 con 0x7fa37c0b5970 2023-11-13T11:03:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3689353972 0 0) 0x7fa364003520 con 0x7fa37c0b5970 2023-11-13T11:03:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 >> v2:172.21.15.184:3300/0 conn(0x7fa37c160240 msgr2=0x7fa37c0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 --2- 172.21.15.16:0/1831025356 >> v2:172.21.15.184:3300/0 conn(0x7fa37c160240 0x7fa37c0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 >> v1:172.21.15.184:6789/0 conn(0x7fa37c0d1090 legacy=0x7fa37c06c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c06bab0 con 0x7fa37c0b5970 2023-11-13T11:03:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fa370001670 con 0x7fa37c0b5970 2023-11-13T11:03:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 >> v1:172.21.15.16:6789/0 conn(0x7fa37c0b5970 legacy=0x7fa37c0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa364003970 0x7fa364003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3640047c0 0x7fa364004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa364005180 0x7fa36400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c06b7d0 con 0x7fa3640047c0 2023-11-13T11:03:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36400c120 con 0x7fa364003970 2023-11-13T11:03:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36400c2e0 con 0x7fa364005180 2023-11-13T11:03:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37b7fe640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3640047c0 0x7fa364004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.163+0000 7fa37bfff640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa364003970 0x7fa364003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.164+0000 7fa380d8d640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa364005180 0x7fa36400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.164+0000 7fa37b7fe640 1 -- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa364005180 msgr2=0x7fa36400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.164+0000 7fa37b7fe640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa364005180 0x7fa36400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.164+0000 7fa37b7fe640 1 -- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa364003970 msgr2=0x7fa364003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.164+0000 7fa37b7fe640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa364003970 0x7fa364003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.164+0000 7fa37b7fe640 1 -- 172.21.15.16:0/1831025356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa364003520 con 0x7fa3640047c0 2023-11-13T11:03:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.164+0000 7fa37bfff640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa364003970 0x7fa364003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:03:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.164+0000 7fa380d8d640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa364005180 0x7fa36400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.164+0000 7fa380d8d640 1 -- 172.21.15.16:0/1831025356 reap_dead start 2023-11-13T11:03:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.165+0000 7fa37b7fe640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3640047c0 0x7fa364004c40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c240 tx=0x7fa3640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a934bac4e75210 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.165+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370005ad0 con 0x7fa3640047c0 2023-11-13T11:03:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.165+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa370005c30 con 0x7fa3640047c0 2023-11-13T11:03:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.165+0000 7fa37affd640 1 -- 172.21.15.16:0/1831025356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37002caf0 con 0x7fa3640047c0 2023-11-13T11:03:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.166+0000 7fa38158e640 1 -- 172.21.15.16:0/1831025356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3640047c0 msgr2=0x7fa364004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.166+0000 7fa38158e640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3640047c0 0x7fa364004c40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c240 tx=0x7fa3640057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.166+0000 7fa38158e640 1 -- 172.21.15.16:0/1831025356 shutdown_connections 2023-11-13T11:03:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.166+0000 7fa38158e640 1 --2- 172.21.15.16:0/1831025356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3640047c0 0x7fa364004c40 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.166+0000 7fa38158e640 1 -- 172.21.15.16:0/1831025356 >> 172.21.15.16:0/1831025356 conn(0x7fa37c0b5f20 msgr2=0x7fa37c0b6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.166+0000 7fa38158e640 1 -- 172.21.15.16:0/1831025356 shutdown_connections 2023-11-13T11:03:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.166+0000 7fa38158e640 1 -- 172.21.15.16:0/1831025356 wait complete. 2023-11-13T11:03:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.168+0000 7fa38158e640 1 Processor -- start 2023-11-13T11:03:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.168+0000 7fa38158e640 1 -- start start 2023-11-13T11:03:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.169+0000 7fa38158e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3640047c0 0x7fa37c1566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.169+0000 7fa38158e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cadd0 0x7fa37c0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.169+0000 7fa38158e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0b0570 0x7fa37c0b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.169+0000 7fa38158e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0b0f60 con 0x7fa37c0cadd0 2023-11-13T11:03:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.169+0000 7fa38158e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0cb790 con 0x7fa37c0b0570 2023-11-13T11:03:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.169+0000 7fa38158e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0d0070 con 0x7fa3640047c0 2023-11-13T11:03:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.169+0000 7fa37b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cadd0 0x7fa37c0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.169+0000 7fa37b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cadd0 0x7fa37c0cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47940/0 (socket says 172.21.15.16:47940) 2023-11-13T11:03:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.169+0000 7fa37b7fe640 1 -- 172.21.15.16:0/670122480 learned_addr learned my addr 172.21.15.16:0/670122480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa37bfff640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3640047c0 0x7fa37c1566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa380d8d640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0b0570 0x7fa37c0b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa37b7fe640 1 -- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3640047c0 msgr2=0x7fa37c1566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa37b7fe640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3640047c0 0x7fa37c1566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa37b7fe640 1 -- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0b0570 msgr2=0x7fa37c0b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa37b7fe640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0b0570 0x7fa37c0b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa37b7fe640 1 -- 172.21.15.16:0/670122480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0b1150 con 0x7fa37c0cadd0 2023-11-13T11:03:17.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa37b7fe640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cadd0 0x7fa37c0cb250 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa370001930 tx=0x7fa37002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:17.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa378ff9640 1 -- 172.21.15.16:0/670122480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37003c040 con 0x7fa37c0cadd0 2023-11-13T11:03:17.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa378ff9640 1 -- 172.21.15.16:0/670122480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37002ccc0 con 0x7fa37c0cadd0 2023-11-13T11:03:17.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa378ff9640 1 -- 172.21.15.16:0/670122480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370007290 con 0x7fa37c0cadd0 2023-11-13T11:03:17.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.170+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c0b1780 con 0x7fa37c0cadd0 2023-11-13T11:03:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.171+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c0b1d40 con 0x7fa37c0cadd0 2023-11-13T11:03:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.172+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c0b1ed0 con 0x7fa37c0cadd0 2023-11-13T11:03:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.173+0000 7fa378ff9640 1 -- 172.21.15.16:0/670122480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa370032070 con 0x7fa37c0cadd0 2023-11-13T11:03:17.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.174+0000 7fa378ff9640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa36806ae40 0x7fa36806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.174+0000 7fa378ff9640 1 -- 172.21.15.16:0/670122480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa3700aa8e0 con 0x7fa37c0cadd0 2023-11-13T11:03:17.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.174+0000 7fa37bfff640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa36806ae40 0x7fa36806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.176+0000 7fa37bfff640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa36806ae40 0x7fa36806d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa37c15eb40 tx=0x7fa36c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.179+0000 7fa378ff9640 1 -- 172.21.15.16:0/670122480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa370075c20 con 0x7fa37c0cadd0 2023-11-13T11:03:17.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.379+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa37c04e4d0 con 0x7fa37c0cadd0 2023-11-13T11:03:17.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.380+0000 7fa378ff9640 1 -- 172.21.15.16:0/670122480 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa370079240 con 0x7fa37c0cadd0 2023-11-13T11:03:17.385 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:17.385 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-13T11:03:17.385 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-13T11:03:17.385 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-13T11:03:17.385 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-13T11:03:17.386 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:17.386 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-13T11:03:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa36806ae40 msgr2=0x7fa36806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa36806ae40 0x7fa36806d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa37c15eb40 tx=0x7fa36c008040 comp rx=0 tx=0).stop 2023-11-13T11:03:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cadd0 msgr2=0x7fa37c0cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cadd0 0x7fa37c0cb250 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa370001930 tx=0x7fa37002e070 comp rx=0 tx=0).stop 2023-11-13T11:03:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 shutdown_connections 2023-11-13T11:03:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa36806ae40 0x7fa36806d300 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa37c0b0570 0x7fa37c0b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa37c0cadd0 0x7fa37c0cb250 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 --2- 172.21.15.16:0/670122480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3640047c0 0x7fa37c1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 >> 172.21.15.16:0/670122480 conn(0x7fa37c0b5f20 msgr2=0x7fa37c06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 shutdown_connections 2023-11-13T11:03:17.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.383+0000 7fa38158e640 1 -- 172.21.15.16:0/670122480 wait complete. 2023-11-13T11:03:17.409 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-13T11:03:17.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-13T11:03:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.636+0000 7f89bbe61640 1 Processor -- start 2023-11-13T11:03:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.637+0000 7f89bbe61640 1 -- start start 2023-11-13T11:03:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.638+0000 7f89bbe61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89b40cdf80 0x7f89b40ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.638+0000 7f89bbe61640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f89b40b2e70 0x7f89b40b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.638+0000 7f89bbe61640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b40cfda0 con 0x7f89b415d0b0 2023-11-13T11:03:17.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.638+0000 7f89bbe61640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40b29e0 con 0x7f89b40cdf80 2023-11-13T11:03:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.638+0000 7f89bbe61640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40b26b0 con 0x7f89b40b2e70 2023-11-13T11:03:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.639+0000 7f89b93fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89b40cdf80 0x7f89b40ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.639+0000 7f89b9bfd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f89b415d0b0 0x7f89b40cd870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59316/0 (socket says 172.21.15.16:59316) 2023-11-13T11:03:17.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.639+0000 7f89b9bfd640 1 -- 172.21.15.16:0/4127308636 learned_addr learned my addr 172.21.15.16:0/4127308636 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:17.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.639+0000 7f89ba3fe640 1 --2- 172.21.15.16:0/4127308636 >> v2:172.21.15.184:3300/0 conn(0x7f89b40b2e70 0x7f89b40b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.639+0000 7f89b93fc640 1 -- 172.21.15.16:0/4127308636 >> v2:172.21.15.184:3300/0 conn(0x7f89b40b2e70 msgr2=0x7f89b40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.639+0000 7f89b93fc640 1 --2- 172.21.15.16:0/4127308636 >> v2:172.21.15.184:3300/0 conn(0x7f89b40b2e70 0x7f89b40b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.639+0000 7f89b93fc640 1 -- 172.21.15.16:0/4127308636 >> v1:172.21.15.16:6789/0 conn(0x7f89b415d0b0 legacy=0x7f89b40cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.639+0000 7f89b93fc640 1 -- 172.21.15.16:0/4127308636 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40cfff0 con 0x7f89b40cdf80 2023-11-13T11:03:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.639+0000 7f89b8bfb640 1 -- 172.21.15.16:0/4127308636 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136594572 0 0) 0x7f89b40cfda0 con 0x7f89b415d0b0 2023-11-13T11:03:17.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.641+0000 7f89b93fc640 1 --2- 172.21.15.16:0/4127308636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 0x7f89b40ce3e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f89b40cd640 tx=0x7f89a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ad6e55cf9afe6b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:17.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.641+0000 7f89b8bfb640 1 -- 172.21.15.16:0/4127308636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8031830 con 0x7f89b40cdf80 2023-11-13T11:03:17.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.641+0000 7f89b8bfb640 1 -- 172.21.15.16:0/4127308636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a8031990 con 0x7f89b40cdf80 2023-11-13T11:03:17.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.641+0000 7f89b8bfb640 1 -- 172.21.15.16:0/4127308636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8031d20 con 0x7f89b40cdf80 2023-11-13T11:03:17.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.641+0000 7f89bbe61640 1 -- 172.21.15.16:0/4127308636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 msgr2=0x7f89b40ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.641+0000 7f89bbe61640 1 --2- 172.21.15.16:0/4127308636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 0x7f89b40ce3e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f89b40cd640 tx=0x7f89a802b8d0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.642+0000 7f89bbe61640 1 -- 172.21.15.16:0/4127308636 shutdown_connections 2023-11-13T11:03:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.642+0000 7f89bbe61640 1 --2- 172.21.15.16:0/4127308636 >> v2:172.21.15.184:3300/0 conn(0x7f89b40b2e70 0x7f89b40b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.642+0000 7f89bbe61640 1 --2- 172.21.15.16:0/4127308636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 0x7f89b40ce3e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.642+0000 7f89bbe61640 1 -- 172.21.15.16:0/4127308636 >> 172.21.15.16:0/4127308636 conn(0x7f89b40d2da0 msgr2=0x7f89b40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:17.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.642+0000 7f89bbe61640 1 -- 172.21.15.16:0/4127308636 shutdown_connections 2023-11-13T11:03:17.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.642+0000 7f89bbe61640 1 -- 172.21.15.16:0/4127308636 wait complete. 2023-11-13T11:03:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.643+0000 7f89bbe61640 1 Processor -- start 2023-11-13T11:03:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.644+0000 7f89bbe61640 1 -- start start 2023-11-13T11:03:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89bbe61640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89b40b2e70 0x7f89b409d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89bbe61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 0x7f89b40c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89bbe61640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b415d0b0 0x7f89b415c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89bbe61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b409de70 con 0x7f89b40cdf80 2023-11-13T11:03:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89bbe61640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40c98d0 con 0x7f89b415d0b0 2023-11-13T11:03:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89bbe61640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40a0300 con 0x7f89b40b2e70 2023-11-13T11:03:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89b93fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 0x7f89b40c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89b93fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 0x7f89b40c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47970/0 (socket says 172.21.15.16:47970) 2023-11-13T11:03:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89b93fc640 1 -- 172.21.15.16:0/3084446116 learned_addr learned my addr 172.21.15.16:0/3084446116 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89b9bfd640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89b40b2e70 0x7f89b409d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89b93fc640 1 -- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89b40b2e70 msgr2=0x7f89b409d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89b93fc640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89b40b2e70 0x7f89b409d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89b93fc640 1 -- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b415d0b0 msgr2=0x7f89b415c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89b93fc640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b415d0b0 0x7f89b415c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.645+0000 7f89b93fc640 1 -- 172.21.15.16:0/3084446116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a8009400 con 0x7f89b40cdf80 2023-11-13T11:03:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.646+0000 7f89b93fc640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 0x7f89b40c91d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009890 tx=0x7f89a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.646+0000 7f89b27fc640 1 -- 172.21.15.16:0/3084446116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a803ba70 con 0x7f89b40cdf80 2023-11-13T11:03:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.646+0000 7f89b27fc640 1 -- 172.21.15.16:0/3084446116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a803bbf0 con 0x7f89b40cdf80 2023-11-13T11:03:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.646+0000 7f89b27fc640 1 -- 172.21.15.16:0/3084446116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8045400 con 0x7f89b40cdf80 2023-11-13T11:03:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.646+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b40a05e0 con 0x7f89b40cdf80 2023-11-13T11:03:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.646+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b415c7f0 con 0x7f89b40cdf80 2023-11-13T11:03:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.647+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b41e7280 con 0x7f89b40cdf80 2023-11-13T11:03:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.648+0000 7f89b27fc640 1 -- 172.21.15.16:0/3084446116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89a804e440 con 0x7f89b40cdf80 2023-11-13T11:03:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.649+0000 7f89b27fc640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89a006ae40 0x7f89a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.649+0000 7f89b27fc640 1 -- 172.21.15.16:0/3084446116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f89a80ab750 con 0x7f89b40cdf80 2023-11-13T11:03:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.650+0000 7f89b9bfd640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89a006ae40 0x7f89a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.651+0000 7f89b9bfd640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89a006ae40 0x7f89a006d300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f89a40012e0 tx=0x7f89a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.654+0000 7f89b27fc640 1 -- 172.21.15.16:0/3084446116 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89a8076970 con 0x7f89b40cdf80 2023-11-13T11:03:17.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:17.848+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f89b4067090 con 0x7f89b40cdf80 2023-11-13T11:03:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.413+0000 7f89b27fc640 1 -- 172.21.15.16:0/3084446116 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) v1 ==== 138+0+0 (secure 0 0 0) 0x7f89a8079f90 con 0x7f89b40cdf80 2023-11-13T11:03:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-13T11:03:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89a006ae40 msgr2=0x7f89a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89a006ae40 0x7f89a006d300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f89a40012e0 tx=0x7f89a4009040 comp rx=0 tx=0).stop 2023-11-13T11:03:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 msgr2=0x7f89b40c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 0x7f89b40c91d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009890 tx=0x7f89a80046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 shutdown_connections 2023-11-13T11:03:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89a006ae40 0x7f89a006d300 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89b415d0b0 0x7f89b415c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89b40cdf80 0x7f89b40c91d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 --2- 172.21.15.16:0/3084446116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89b40b2e70 0x7f89b409d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 >> 172.21.15.16:0/3084446116 conn(0x7f89b40d2da0 msgr2=0x7f89b40d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 shutdown_connections 2023-11-13T11:03:18.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.418+0000 7f89bbe61640 1 -- 172.21.15.16:0/3084446116 wait complete. 2023-11-13T11:03:18.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-13T11:03:18.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.678+0000 7fa8b2e1a640 1 Processor -- start 2023-11-13T11:03:18.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.679+0000 7fa8b2e1a640 1 -- start start 2023-11-13T11:03:18.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.681+0000 7fa8b2e1a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8ac0b32c0 0x7fa8ac0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:18.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.681+0000 7fa8b2e1a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa8ac165640 0x7fa8ac06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:18.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.681+0000 7fa8b2e1a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ac0cd7d0 con 0x7fa8ac0cef40 2023-11-13T11:03:18.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.681+0000 7fa8b2e1a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ac0b19d0 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.681+0000 7fa8b2e1a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ac0ccc50 con 0x7fa8ac165640 2023-11-13T11:03:18.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.681+0000 7fa8b0bb6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8ac0b32c0 0x7fa8ac0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:18.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.681+0000 7fa8b0bb6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8ac0b32c0 0x7fa8ac0cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47980/0 (socket says 172.21.15.16:47980) 2023-11-13T11:03:18.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.681+0000 7fa8b0bb6640 1 -- 172.21.15.16:0/606998749 learned_addr learned my addr 172.21.15.16:0/606998749 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:18.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.682+0000 7fa8b13b7640 1 --2- 172.21.15.16:0/606998749 >> v2:172.21.15.184:3300/0 conn(0x7fa8ac165640 0x7fa8ac06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:18.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.682+0000 7fa8b0bb6640 1 -- 172.21.15.16:0/606998749 >> v2:172.21.15.184:3300/0 conn(0x7fa8ac165640 msgr2=0x7fa8ac06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.682+0000 7fa8b0bb6640 1 --2- 172.21.15.16:0/606998749 >> v2:172.21.15.184:3300/0 conn(0x7fa8ac165640 0x7fa8ac06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.682+0000 7fa8b0bb6640 1 -- 172.21.15.16:0/606998749 >> v1:172.21.15.184:6789/0 conn(0x7fa8ac0cef40 legacy=0x7fa8ac164f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.682+0000 7fa8b0bb6640 1 -- 172.21.15.16:0/606998749 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac0ccf30 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.682+0000 7fa8b13b7640 1 --2- 172.21.15.16:0/606998749 >> v2:172.21.15.184:3300/0 conn(0x7fa8ac165640 0x7fa8ac06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.683+0000 7fa8b0bb6640 1 --2- 172.21.15.16:0/606998749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 0x7fa8ac0cc550 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac06f2b0 tx=0x7fa8a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f0ad609e4d8bef server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.683+0000 7fa8a37fe640 1 -- 172.21.15.16:0/606998749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8031870 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.683+0000 7fa8a37fe640 1 -- 172.21.15.16:0/606998749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a80319d0 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.683+0000 7fa8a37fe640 1 -- 172.21.15.16:0/606998749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8031d60 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.684+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/606998749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 msgr2=0x7fa8ac0cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.684+0000 7fa8b2e1a640 1 --2- 172.21.15.16:0/606998749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 0x7fa8ac0cc550 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac06f2b0 tx=0x7fa8a802b910 comp rx=0 tx=0).stop 2023-11-13T11:03:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.684+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/606998749 shutdown_connections 2023-11-13T11:03:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.684+0000 7fa8b2e1a640 1 --2- 172.21.15.16:0/606998749 >> v2:172.21.15.184:3300/0 conn(0x7fa8ac165640 0x7fa8ac06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.684+0000 7fa8b2e1a640 1 --2- 172.21.15.16:0/606998749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 0x7fa8ac0cc550 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.684+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/606998749 >> 172.21.15.16:0/606998749 conn(0x7fa8ac0b06d0 msgr2=0x7fa8ac0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:18.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.684+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/606998749 shutdown_connections 2023-11-13T11:03:18.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.684+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/606998749 wait complete. 2023-11-13T11:03:18.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.686+0000 7fa8b2e1a640 1 Processor -- start 2023-11-13T11:03:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.686+0000 7fa8b2e1a640 1 -- start start 2023-11-13T11:03:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.687+0000 7fa8b2e1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 0x7fa8ac14ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.687+0000 7fa8b2e1a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ac0cef40 0x7fa8ac1f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.687+0000 7fa8b2e1a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8ac165640 0x7fa8ac1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.687+0000 7fa8b2e1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac14d360 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.687+0000 7fa8b2e1a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac14ebd0 con 0x7fa8ac165640 2023-11-13T11:03:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.687+0000 7fa8b2e1a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac14eeb0 con 0x7fa8ac0cef40 2023-11-13T11:03:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.688+0000 7fa8b0bb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 0x7fa8ac14ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.688+0000 7fa8b0bb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 0x7fa8ac14ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47994/0 (socket says 172.21.15.16:47994) 2023-11-13T11:03:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.688+0000 7fa8b0bb6640 1 -- 172.21.15.16:0/1715660043 learned_addr learned my addr 172.21.15.16:0/1715660043 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.688+0000 7fa8b0bb6640 1 -- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ac0cef40 msgr2=0x7fa8ac1f7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.688+0000 7fa8b0bb6640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ac0cef40 0x7fa8ac1f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.688+0000 7fa8b0bb6640 1 -- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8ac165640 msgr2=0x7fa8ac1513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.688+0000 7fa8b0bb6640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8ac165640 0x7fa8ac1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.688+0000 7fa8b0bb6640 1 -- 172.21.15.16:0/1715660043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a80094d0 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.688+0000 7fa8b0bb6640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 0x7fa8ac14ce20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80090b0 tx=0x7fa8a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.689+0000 7fa8a17fa640 1 -- 172.21.15.16:0/1715660043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a803adf0 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.689+0000 7fa8a17fa640 1 -- 172.21.15.16:0/1715660043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a80443f0 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.689+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac14f190 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.689+0000 7fa8a17fa640 1 -- 172.21.15.16:0/1715660043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80447a0 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.689+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac14f750 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.690+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ac067090 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.691+0000 7fa8a17fa640 1 -- 172.21.15.16:0/1715660043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8a804d450 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.693+0000 7fa8a17fa640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa89806ae40 0x7fa89806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.693+0000 7fa8a17fa640 1 -- 172.21.15.16:0/1715660043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fa8a802d040 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.694+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa89806ae40 0x7fa89806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.695+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa89806ae40 0x7fa89806d300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac14e2c0 tx=0x7fa89c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.698+0000 7fa8a17fa640 1 -- 172.21.15.16:0/1715660043 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8a8076d20 con 0x7fa8ac0b32c0 2023-11-13T11:03:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:18.895+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa8ac0b1530 con 0x7fa8ac0b32c0 2023-11-13T11:03:19.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.431+0000 7fa8a17fa640 1 -- 172.21.15.16:0/1715660043 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa8a807a340 con 0x7fa8ac0b32c0 2023-11-13T11:03:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-13T11:03:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa89806ae40 msgr2=0x7fa89806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa89806ae40 0x7fa89806d300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac14e2c0 tx=0x7fa89c0079b0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 msgr2=0x7fa8ac14ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 0x7fa8ac14ce20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80090b0 tx=0x7fa8a80046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 shutdown_connections 2023-11-13T11:03:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa89806ae40 0x7fa89806d300 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8ac165640 0x7fa8ac1513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa8ac0cef40 0x7fa8ac1f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 --2- 172.21.15.16:0/1715660043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ac0b32c0 0x7fa8ac14ce20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 >> 172.21.15.16:0/1715660043 conn(0x7fa8ac0b06d0 msgr2=0x7fa8ac06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.435+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 shutdown_connections 2023-11-13T11:03:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.436+0000 7fa8b2e1a640 1 -- 172.21.15.16:0/1715660043 wait complete. 2023-11-13T11:03:19.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-13T11:03:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.661+0000 7f6cc9ac2640 1 Processor -- start 2023-11-13T11:03:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.662+0000 7f6cc9ac2640 1 -- start start 2023-11-13T11:03:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.663+0000 7f6cc9ac2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6cc40b3b90 0x7f6cc40b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.663+0000 7f6cc9ac2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc406c3b0 0x7f6cc4070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.663+0000 7f6cc9ac2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc40b2e00 con 0x7f6cc415cf90 2023-11-13T11:03:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.663+0000 7f6cc9ac2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc40b21f0 con 0x7f6cc40b3b90 2023-11-13T11:03:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.663+0000 7f6cc9ac2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc40b24d0 con 0x7f6cc406c3b0 2023-11-13T11:03:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.664+0000 7f6cc2ffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6cc40b3b90 0x7f6cc40b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.664+0000 7f6cc3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc406c3b0 0x7f6cc4070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.664+0000 7f6cc3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc406c3b0 0x7f6cc4070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:53160/0 (socket says 172.21.15.16:53160) 2023-11-13T11:03:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.664+0000 7f6cc3fff640 1 -- 172.21.15.16:0/3176847576 learned_addr learned my addr 172.21.15.16:0/3176847576 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.665+0000 7f6cc3fff640 1 -- 172.21.15.16:0/3176847576 >> v2:172.21.15.184:3300/0 conn(0x7f6cc40b3b90 msgr2=0x7f6cc40b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.665+0000 7f6cc3fff640 1 --2- 172.21.15.16:0/3176847576 >> v2:172.21.15.184:3300/0 conn(0x7f6cc40b3b90 0x7f6cc40b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.665+0000 7f6cc3fff640 1 -- 172.21.15.16:0/3176847576 >> v1:172.21.15.184:6789/0 conn(0x7f6cc415cf90 legacy=0x7f6cc40b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.665+0000 7f6cc3fff640 1 -- 172.21.15.16:0/3176847576 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40b27b0 con 0x7f6cc406c3b0 2023-11-13T11:03:19.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.665+0000 7f6cc2ffd640 1 --2- 172.21.15.16:0/3176847576 >> v2:172.21.15.184:3300/0 conn(0x7f6cc40b3b90 0x7f6cc40b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-13T11:03:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.666+0000 7f6cc3fff640 1 --2- 172.21.15.16:0/3176847576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc406c3b0 0x7f6cc4070440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009870 tx=0x7f6cbc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6565f1367ad0a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc27fc640 1 -- 172.21.15.16:0/3176847576 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc031640 con 0x7f6cc406c3b0 2023-11-13T11:03:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc27fc640 1 -- 172.21.15.16:0/3176847576 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc0317a0 con 0x7f6cc406c3b0 2023-11-13T11:03:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc27fc640 1 -- 172.21.15.16:0/3176847576 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc031b30 con 0x7f6cc406c3b0 2023-11-13T11:03:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/3176847576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc406c3b0 msgr2=0x7f6cc4070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:19.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc9ac2640 1 --2- 172.21.15.16:0/3176847576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc406c3b0 0x7f6cc4070440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009870 tx=0x7f6cbc02b8f0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/3176847576 shutdown_connections 2023-11-13T11:03:19.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc9ac2640 1 --2- 172.21.15.16:0/3176847576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc406c3b0 0x7f6cc4070440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc9ac2640 1 --2- 172.21.15.16:0/3176847576 >> v2:172.21.15.184:3300/0 conn(0x7f6cc40b3b90 0x7f6cc40b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/3176847576 >> 172.21.15.16:0/3176847576 conn(0x7f6cc406a190 msgr2=0x7f6cc406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:19.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.667+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/3176847576 shutdown_connections 2023-11-13T11:03:19.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.668+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/3176847576 wait complete. 2023-11-13T11:03:19.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.669+0000 7f6cc9ac2640 1 Processor -- start 2023-11-13T11:03:19.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.669+0000 7f6cc9ac2640 1 -- start start 2023-11-13T11:03:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.670+0000 7f6cc9ac2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc406c3b0 0x7f6cc41e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.670+0000 7f6cc9ac2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40b3b90 0x7f6cc415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.670+0000 7f6cc9ac2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc415cf90 0x7f6cc414a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.670+0000 7f6cc9ac2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc415f030 con 0x7f6cc40b3b90 2023-11-13T11:03:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.670+0000 7f6cc9ac2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc415f310 con 0x7f6cc406c3b0 2023-11-13T11:03:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.670+0000 7f6cc9ac2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc415f5f0 con 0x7f6cc415cf90 2023-11-13T11:03:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40b3b90 0x7f6cc415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40b3b90 0x7f6cc415e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48004/0 (socket says 172.21.15.16:48004) 2023-11-13T11:03:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc2ffd640 1 -- 172.21.15.16:0/4047205000 learned_addr learned my addr 172.21.15.16:0/4047205000 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc3fff640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc415cf90 0x7f6cc414a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc37fe640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc406c3b0 0x7f6cc41e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc2ffd640 1 -- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc415cf90 msgr2=0x7f6cc414a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc2ffd640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc415cf90 0x7f6cc414a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc2ffd640 1 -- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc406c3b0 msgr2=0x7f6cc41e6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc2ffd640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc406c3b0 0x7f6cc41e6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:19.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.671+0000 7f6cc2ffd640 1 -- 172.21.15.16:0/4047205000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac009400 con 0x7f6cc40b3b90 2023-11-13T11:03:19.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.672+0000 7f6cc2ffd640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40b3b90 0x7f6cc415e7e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00ebe0 tx=0x7f6cac00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.672+0000 7f6cbbfff640 1 -- 172.21.15.16:0/4047205000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac009760 con 0x7f6cc40b3b90 2023-11-13T11:03:19.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.672+0000 7f6cbbfff640 1 -- 172.21.15.16:0/4047205000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac00cc30 con 0x7f6cc40b3b90 2023-11-13T11:03:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.672+0000 7f6cbbfff640 1 -- 172.21.15.16:0/4047205000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00eee0 con 0x7f6cc40b3b90 2023-11-13T11:03:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.672+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc0094d0 con 0x7f6cc40b3b90 2023-11-13T11:03:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.672+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc415fbb0 con 0x7f6cc40b3b90 2023-11-13T11:03:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.673+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc4072520 con 0x7f6cc40b3b90 2023-11-13T11:03:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.675+0000 7f6cbbfff640 1 -- 172.21.15.16:0/4047205000 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6cac013470 con 0x7f6cc40b3b90 2023-11-13T11:03:19.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.676+0000 7f6cbbfff640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ca406ae40 0x7f6ca406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:19.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.676+0000 7f6cbbfff640 1 -- 172.21.15.16:0/4047205000 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f6cac08a1f0 con 0x7f6cc40b3b90 2023-11-13T11:03:19.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.676+0000 7f6cc37fe640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ca406ae40 0x7f6ca406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:19.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.678+0000 7f6cc37fe640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ca406ae40 0x7f6ca406d300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4001ca0 tx=0x7f6cb4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:19.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.680+0000 7f6cbbfff640 1 -- 172.21.15.16:0/4047205000 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6cac0551c0 con 0x7f6cc40b3b90 2023-11-13T11:03:19.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:03:19.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:03:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:19.911+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6cc40adea0 con 0x7f6cc40b3b90 2023-11-13T11:03:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.460+0000 7f6cbbfff640 1 -- 172.21.15.16:0/4047205000 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v37) v1 ==== 173+0+0 (secure 0 0 0) 0x7f6cac0587e0 con 0x7f6cc40b3b90 2023-11-13T11:03:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 9 and data pool 10 2023-11-13T11:03:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.466+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ca406ae40 msgr2=0x7f6ca406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.466+0000 7f6cc9ac2640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ca406ae40 0x7f6ca406d300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4001ca0 tx=0x7f6cb4009040 comp rx=0 tx=0).stop 2023-11-13T11:03:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.466+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40b3b90 msgr2=0x7f6cc415e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:20.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.466+0000 7f6cc9ac2640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40b3b90 0x7f6cc415e7e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00ebe0 tx=0x7f6cac00c520 comp rx=0 tx=0).stop 2023-11-13T11:03:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.467+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 shutdown_connections 2023-11-13T11:03:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.467+0000 7f6cc9ac2640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ca406ae40 0x7f6ca406d300 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.467+0000 7f6cc9ac2640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cc415cf90 0x7f6cc414a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.467+0000 7f6cc9ac2640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc40b3b90 0x7f6cc415e7e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.467+0000 7f6cc9ac2640 1 --2- 172.21.15.16:0/4047205000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cc406c3b0 0x7f6cc41e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.468+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 >> 172.21.15.16:0/4047205000 conn(0x7f6cc406a190 msgr2=0x7f6cc40afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.470+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 shutdown_connections 2023-11-13T11:03:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.471+0000 7f6cc9ac2640 1 -- 172.21.15.16:0/4047205000 wait complete. 2023-11-13T11:03:20.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.690+0000 7f83fb0c9640 1 Processor -- start 2023-11-13T11:03:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.691+0000 7f83fb0c9640 1 -- start start 2023-11-13T11:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.692+0000 7f83fb0c9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f83f40ae030 0x7f83f406c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.692+0000 7f83fb0c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f4070650 0x7f83f406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.692+0000 7f83fb0c9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f40b3a80 con 0x7f83f406fb60 2023-11-13T11:03:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.692+0000 7f83fb0c9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f4161880 con 0x7f83f40ae030 2023-11-13T11:03:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.692+0000 7f83fb0c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4161b60 con 0x7f83f4070650 2023-11-13T11:03:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.693+0000 7f83f9666640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f4070650 0x7f83f406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.693+0000 7f83f3fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f83f40ae030 0x7f83f406c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:20.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.693+0000 7f83f9666640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f4070650 0x7f83f406c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:53178/0 (socket says 172.21.15.16:53178) 2023-11-13T11:03:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.693+0000 7f83f9666640 1 -- 172.21.15.16:0/4202156574 learned_addr learned my addr 172.21.15.16:0/4202156574 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.693+0000 7f83f3fff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f83f40ae030 0x7f83f406c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:47212/0 (socket says 172.21.15.16:47212) 2023-11-13T11:03:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.693+0000 7f83f8e65640 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f83f406fb60 0x7f83f406ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:53568/0 (socket says 172.21.15.16:53568) 2023-11-13T11:03:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.694+0000 7f83f9666640 1 -- 172.21.15.16:0/4202156574 >> v2:172.21.15.184:3300/0 conn(0x7f83f40ae030 msgr2=0x7f83f406c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.694+0000 7f83f9666640 1 --2- 172.21.15.16:0/4202156574 >> v2:172.21.15.184:3300/0 conn(0x7f83f40ae030 0x7f83f406c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.694+0000 7f83f9666640 1 -- 172.21.15.16:0/4202156574 >> v1:172.21.15.184:6789/0 conn(0x7f83f406fb60 legacy=0x7f83f406ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.694+0000 7f83f9666640 1 -- 172.21.15.16:0/4202156574 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f4161e40 con 0x7f83f4070650 2023-11-13T11:03:20.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.695+0000 7f83f9666640 1 --2- 172.21.15.16:0/4202156574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f4070650 0x7f83f406c5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009e50 tx=0x7f83ec02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16e358e8743705d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:20.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.696+0000 7f83f37fe640 1 -- 172.21.15.16:0/4202156574 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec02cb00 con 0x7f83f4070650 2023-11-13T11:03:20.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.696+0000 7f83f37fe640 1 -- 172.21.15.16:0/4202156574 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec02cc60 con 0x7f83f4070650 2023-11-13T11:03:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.696+0000 7f83f37fe640 1 -- 172.21.15.16:0/4202156574 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0343f0 con 0x7f83f4070650 2023-11-13T11:03:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.697+0000 7f83fb0c9640 1 -- 172.21.15.16:0/4202156574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f4070650 msgr2=0x7f83f406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.697+0000 7f83fb0c9640 1 --2- 172.21.15.16:0/4202156574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f4070650 0x7f83f406c5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009e50 tx=0x7f83ec02bf60 comp rx=0 tx=0).stop 2023-11-13T11:03:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.697+0000 7f83fb0c9640 1 -- 172.21.15.16:0/4202156574 shutdown_connections 2023-11-13T11:03:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.697+0000 7f83fb0c9640 1 --2- 172.21.15.16:0/4202156574 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f4070650 0x7f83f406c5a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.697+0000 7f83fb0c9640 1 --2- 172.21.15.16:0/4202156574 >> v2:172.21.15.184:3300/0 conn(0x7f83f40ae030 0x7f83f406c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.697+0000 7f83fb0c9640 1 -- 172.21.15.16:0/4202156574 >> 172.21.15.16:0/4202156574 conn(0x7f83f406a190 msgr2=0x7f83f406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.697+0000 7f83fb0c9640 1 -- 172.21.15.16:0/4202156574 shutdown_connections 2023-11-13T11:03:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.697+0000 7f83fb0c9640 1 -- 172.21.15.16:0/4202156574 wait complete. 2023-11-13T11:03:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.699+0000 7f83fb0c9640 1 Processor -- start 2023-11-13T11:03:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.699+0000 7f83fb0c9640 1 -- start start 2023-11-13T11:03:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.700+0000 7f83fb0c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f406fb60 0x7f83f414ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.700+0000 7f83fb0c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83f4070650 0x7f83f41f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.700+0000 7f83fb0c9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f40ae030 0x7f83f40cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.700+0000 7f83fb0c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f414f150 con 0x7f83f406fb60 2023-11-13T11:03:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.700+0000 7f83fb0c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41f8170 con 0x7f83f4070650 2023-11-13T11:03:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.700+0000 7f83fb0c9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41f78d0 con 0x7f83f40ae030 2023-11-13T11:03:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f8e65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f406fb60 0x7f83f414ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f8e65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f406fb60 0x7f83f414ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48020/0 (socket says 172.21.15.16:48020) 2023-11-13T11:03:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f8e65640 1 -- 172.21.15.16:0/3649838392 learned_addr learned my addr 172.21.15.16:0/3649838392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f3fff640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83f4070650 0x7f83f41f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f9666640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f40ae030 0x7f83f40cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f8e65640 1 -- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f40ae030 msgr2=0x7f83f40cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f8e65640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f40ae030 0x7f83f40cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f8e65640 1 -- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83f4070650 msgr2=0x7f83f41f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f8e65640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83f4070650 0x7f83f41f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f8e65640 1 -- 172.21.15.16:0/3649838392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec009b30 con 0x7f83f406fb60 2023-11-13T11:03:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.701+0000 7f83f9666640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f40ae030 0x7f83f40cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.702+0000 7f83f8e65640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f406fb60 0x7f83f414ec10 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c4d0 tx=0x7f83e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.702+0000 7f83f17fa640 1 -- 172.21.15.16:0/3649838392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8007480 con 0x7f83f406fb60 2023-11-13T11:03:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.702+0000 7f83f17fa640 1 -- 172.21.15.16:0/3649838392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e80075e0 con 0x7f83f406fb60 2023-11-13T11:03:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.702+0000 7f83f17fa640 1 -- 172.21.15.16:0/3649838392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80049e0 con 0x7f83f406fb60 2023-11-13T11:03:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.702+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f41f7bb0 con 0x7f83f406fb60 2023-11-13T11:03:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.702+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f415b1d0 con 0x7f83f406fb60 2023-11-13T11:03:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.703+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f40af500 con 0x7f83f406fb60 2023-11-13T11:03:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.704+0000 7f83f17fa640 1 -- 172.21.15.16:0/3649838392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f83e8004b40 con 0x7f83f406fb60 2023-11-13T11:03:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.705+0000 7f83f17fa640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f83d806ae40 0x7f83d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.705+0000 7f83f17fa640 1 -- 172.21.15.16:0/3649838392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f83e8088840 con 0x7f83f406fb60 2023-11-13T11:03:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.706+0000 7f83f3fff640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f83d806ae40 0x7f83d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.707+0000 7f83f3fff640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f83d806ae40 0x7f83d806d300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f83f40cafc0 tx=0x7f83e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.710+0000 7f83f17fa640 1 -- 172.21.15.16:0/3649838392 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83e8053850 con 0x7f83f406fb60 2023-11-13T11:03:20.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.901+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83f4067090 con 0x7f83f406fb60 2023-11-13T11:03:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.903+0000 7f83f17fa640 1 -- 172.21.15.16:0/3649838392 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16789 (secure 0 0 0) 0x7f83e8056e70 con 0x7f83f406fb60 2023-11-13T11:03:20.908 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:20.908 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":68,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:19.912695+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:03:17.903258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-13T11:03:18.947896+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f83d806ae40 msgr2=0x7f83d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f83d806ae40 0x7f83d806d300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f83f40cafc0 tx=0x7f83e0008040 comp rx=0 tx=0).stop 2023-11-13T11:03:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f406fb60 msgr2=0x7f83f414ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f406fb60 0x7f83f414ec10 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c4d0 tx=0x7f83e800c990 comp rx=0 tx=0).stop 2023-11-13T11:03:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 shutdown_connections 2023-11-13T11:03:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f83d806ae40 0x7f83d806d300 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83f40ae030 0x7f83f40cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83f4070650 0x7f83f41f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 --2- 172.21.15.16:0/3649838392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f406fb60 0x7f83f414ec10 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 >> 172.21.15.16:0/3649838392 conn(0x7f83f406a190 msgr2=0x7f83f415f130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 shutdown_connections 2023-11-13T11:03:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:20.906+0000 7f83fb0c9640 1 -- 172.21.15.16:0/3649838392 wait complete. 2023-11-13T11:03:20.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.142+0000 7efed0810640 1 Processor -- start 2023-11-13T11:03:21.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.143+0000 7efed0810640 1 -- start start 2023-11-13T11:03:21.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.144+0000 7efed0810640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efecc15d150 0x7efecc0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.144+0000 7efed0810640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0cebd0 con 0x7efecc0cfeb0 2023-11-13T11:03:21.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.144+0000 7efed0810640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0cdfc0 con 0x7efecc161520 2023-11-13T11:03:21.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.144+0000 7efed0810640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc0d3ac0 con 0x7efecc15d150 2023-11-13T11:03:21.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.145+0000 7efecad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efecc0cfeb0 0x7efecc0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59330/0 (socket says 172.21.15.16:59330) 2023-11-13T11:03:21.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.145+0000 7efecad9d640 1 -- 172.21.15.16:0/3301518924 learned_addr learned my addr 172.21.15.16:0/3301518924 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:21.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.145+0000 7efeca59c640 1 --2- 172.21.15.16:0/3301518924 >> v2:172.21.15.184:3300/0 conn(0x7efecc15d150 0x7efecc0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:21.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.145+0000 7efec959a640 1 -- 172.21.15.16:0/3301518924 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940992043 0 0) 0x7efecc0cebd0 con 0x7efecc0cfeb0 2023-11-13T11:03:21.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.145+0000 7efec959a640 1 -- 172.21.15.16:0/3301518924 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeb8003520 con 0x7efecc0cfeb0 2023-11-13T11:03:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.145+0000 7efeca59c640 1 -- 172.21.15.16:0/3301518924 >> v1:172.21.15.184:6789/0 conn(0x7efecc161520 legacy=0x7efecc0cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.146+0000 7efeca59c640 1 -- 172.21.15.16:0/3301518924 >> v1:172.21.15.16:6789/0 conn(0x7efecc0cfeb0 legacy=0x7efecc0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.146+0000 7efec959a640 1 -- 172.21.15.16:0/3301518924 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1970933882 0 0) 0x7efeb8003520 con 0x7efecc0cfeb0 2023-11-13T11:03:21.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.146+0000 7efeca59c640 1 -- 172.21.15.16:0/3301518924 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0ce2a0 con 0x7efecc15d150 2023-11-13T11:03:21.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.147+0000 7efeca59c640 1 --2- 172.21.15.16:0/3301518924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efecc15d150 0x7efecc0cd950 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efebc009d90 tx=0x7efebc02bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e3ac36a4efc5cf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:21.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.147+0000 7efec959a640 1 -- 172.21.15.16:0/3301518924 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4001670 con 0x7efecc15d150 2023-11-13T11:03:21.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.147+0000 7efec959a640 1 -- 172.21.15.16:0/3301518924 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc02c820 con 0x7efecc15d150 2023-11-13T11:03:21.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.147+0000 7efec959a640 1 -- 172.21.15.16:0/3301518924 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc02cbb0 con 0x7efecc15d150 2023-11-13T11:03:21.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.148+0000 7efed0810640 1 -- 172.21.15.16:0/3301518924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efecc15d150 msgr2=0x7efecc0cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:21.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.148+0000 7efed0810640 1 --2- 172.21.15.16:0/3301518924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efecc15d150 0x7efecc0cd950 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efebc009d90 tx=0x7efebc02bcf0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.148+0000 7efed0810640 1 -- 172.21.15.16:0/3301518924 shutdown_connections 2023-11-13T11:03:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.148+0000 7efed0810640 1 --2- 172.21.15.16:0/3301518924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efecc15d150 0x7efecc0cd950 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.148+0000 7efed0810640 1 -- 172.21.15.16:0/3301518924 >> 172.21.15.16:0/3301518924 conn(0x7efecc09c030 msgr2=0x7efecc0d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.148+0000 7efed0810640 1 -- 172.21.15.16:0/3301518924 shutdown_connections 2023-11-13T11:03:21.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.148+0000 7efed0810640 1 -- 172.21.15.16:0/3301518924 wait complete. 2023-11-13T11:03:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.150+0000 7efed0810640 1 Processor -- start 2023-11-13T11:03:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.150+0000 7efed0810640 1 -- start start 2023-11-13T11:03:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efed0810640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0cfeb0 0x7efecc154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efed0810640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efecc15d150 0x7efecc1f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efed0810640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efecc161520 0x7efecc1f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efed0810640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1548c0 con 0x7efecc0cfeb0 2023-11-13T11:03:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efed0810640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0c7510 con 0x7efecc161520 2023-11-13T11:03:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efed0810640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0cb7d0 con 0x7efecc15d150 2023-11-13T11:03:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efeca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0cfeb0 0x7efecc154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efeca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0cfeb0 0x7efecc154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48030/0 (socket says 172.21.15.16:48030) 2023-11-13T11:03:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efeca59c640 1 -- 172.21.15.16:0/455984360 learned_addr learned my addr 172.21.15.16:0/455984360 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efeca59c640 1 -- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efecc15d150 msgr2=0x7efecc1f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efeca59c640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efecc15d150 0x7efecc1f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efeca59c640 1 -- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efecc161520 msgr2=0x7efecc1f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efeca59c640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efecc161520 0x7efecc1f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.151+0000 7efeca59c640 1 -- 172.21.15.16:0/455984360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc009a70 con 0x7efecc0cfeb0 2023-11-13T11:03:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.152+0000 7efeca59c640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0cfeb0 0x7efecc154380 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7efebc009d90 tx=0x7efebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.152+0000 7efec2ffd640 1 -- 172.21.15.16:0/455984360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc009450 con 0x7efecc0cfeb0 2023-11-13T11:03:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.152+0000 7efec2ffd640 1 -- 172.21.15.16:0/455984360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc02ce10 con 0x7efecc0cfeb0 2023-11-13T11:03:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.152+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc0cb1f0 con 0x7efecc0cfeb0 2023-11-13T11:03:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.152+0000 7efec2ffd640 1 -- 172.21.15.16:0/455984360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc03e650 con 0x7efecc0cfeb0 2023-11-13T11:03:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.152+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc0cb460 con 0x7efecc0cfeb0 2023-11-13T11:03:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.153+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc067090 con 0x7efecc0cfeb0 2023-11-13T11:03:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.154+0000 7efec2ffd640 1 -- 172.21.15.16:0/455984360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efebc03b050 con 0x7efecc0cfeb0 2023-11-13T11:03:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.155+0000 7efec2ffd640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efeac06ae40 0x7efeac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.155+0000 7efec2ffd640 1 -- 172.21.15.16:0/455984360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efebc0abe60 con 0x7efecc0cfeb0 2023-11-13T11:03:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.156+0000 7efec9d9b640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efeac06ae40 0x7efeac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.157+0000 7efec9d9b640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efeac06ae40 0x7efeac06d300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efeb4006fd0 tx=0x7efeb4006bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.160+0000 7efec2ffd640 1 -- 172.21.15.16:0/455984360 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efebc076d30 con 0x7efecc0cfeb0 2023-11-13T11:03:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.348+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efecc0d1160 con 0x7efecc0cfeb0 2023-11-13T11:03:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.351+0000 7efec2ffd640 1 -- 172.21.15.16:0/455984360 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16789 (secure 0 0 0) 0x7efebc07a350 con 0x7efecc0cfeb0 2023-11-13T11:03:21.356 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:21.356 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":68,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:19.912695+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:03:17.903258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-13T11:03:18.947896+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efeac06ae40 msgr2=0x7efeac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efeac06ae40 0x7efeac06d300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efeb4006fd0 tx=0x7efeb4006bc0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0cfeb0 msgr2=0x7efecc154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0cfeb0 0x7efecc154380 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7efebc009d90 tx=0x7efebc004a40 comp rx=0 tx=0).stop 2023-11-13T11:03:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 shutdown_connections 2023-11-13T11:03:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7efeac06ae40 0x7efeac06d300 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efecc161520 0x7efecc1f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efecc15d150 0x7efecc1f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 --2- 172.21.15.16:0/455984360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc0cfeb0 0x7efecc154380 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 >> 172.21.15.16:0/455984360 conn(0x7efecc09c030 msgr2=0x7efecc15ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 shutdown_connections 2023-11-13T11:03:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.354+0000 7efed0810640 1 -- 172.21.15.16:0/455984360 wait complete. 2023-11-13T11:03:21.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-13T11:03:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.595+0000 7fcdbc8e7640 1 Processor -- start 2023-11-13T11:03:21.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.595+0000 7fcdbc8e7640 1 -- start start 2023-11-13T11:03:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.597+0000 7fcdbc8e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 0x7fcdb80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.597+0000 7fcdbc8e7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdb80b19d0 0x7fcdb80b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.597+0000 7fcdbc8e7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb80ceef0 con 0x7fcdb8144570 2023-11-13T11:03:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.597+0000 7fcdbc8e7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb8144950 con 0x7fcdb80b19d0 2023-11-13T11:03:21.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.597+0000 7fcdbc8e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb80ce300 con 0x7fcdb80ce550 2023-11-13T11:03:21.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.597+0000 7fcdb6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdb80b19d0 0x7fcdb80b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:21.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.597+0000 7fcdb6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdb80b19d0 0x7fcdb80b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48034/0 (socket says 172.21.15.16:48034) 2023-11-13T11:03:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.597+0000 7fcdb6d9d640 1 -- 172.21.15.16:0/1195531273 learned_addr learned my addr 172.21.15.16:0/1195531273 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.598+0000 7fcdaffff640 1 --2- 172.21.15.16:0/1195531273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 0x7fcdb80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.598+0000 7fcdb6d9d640 1 -- 172.21.15.16:0/1195531273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 msgr2=0x7fcdb80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.598+0000 7fcdb6d9d640 1 --2- 172.21.15.16:0/1195531273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 0x7fcdb80ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.598+0000 7fcdb6d9d640 1 -- 172.21.15.16:0/1195531273 >> v1:172.21.15.184:6789/0 conn(0x7fcdb8144570 legacy=0x7fcdb815f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.598+0000 7fcdb6d9d640 1 -- 172.21.15.16:0/1195531273 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb80cd260 con 0x7fcdb80b19d0 2023-11-13T11:03:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.598+0000 7fcdaffff640 1 --2- 172.21.15.16:0/1195531273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 0x7fcdb80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.599+0000 7fcdb6d9d640 1 --2- 172.21.15.16:0/1195531273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b19d0 0x7fcdb80b1e60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009e50 tx=0x7fcdb002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af80c1d19e8b179 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.599+0000 7fcdb5d9b640 1 -- 172.21.15.16:0/1195531273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb002ca80 con 0x7fcdb80b19d0 2023-11-13T11:03:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.599+0000 7fcdb5d9b640 1 -- 172.21.15.16:0/1195531273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb002cbe0 con 0x7fcdb80b19d0 2023-11-13T11:03:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.599+0000 7fcdb5d9b640 1 -- 172.21.15.16:0/1195531273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb00343f0 con 0x7fcdb80b19d0 2023-11-13T11:03:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.600+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/1195531273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b19d0 msgr2=0x7fcdb80b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.600+0000 7fcdbc8e7640 1 --2- 172.21.15.16:0/1195531273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b19d0 0x7fcdb80b1e60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009e50 tx=0x7fcdb002bee0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.600+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/1195531273 shutdown_connections 2023-11-13T11:03:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.600+0000 7fcdbc8e7640 1 --2- 172.21.15.16:0/1195531273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b19d0 0x7fcdb80b1e60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.600+0000 7fcdbc8e7640 1 --2- 172.21.15.16:0/1195531273 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 0x7fcdb80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.600+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/1195531273 >> 172.21.15.16:0/1195531273 conn(0x7fcdb806a190 msgr2=0x7fcdb806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.600+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/1195531273 shutdown_connections 2023-11-13T11:03:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.600+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/1195531273 wait complete. 2023-11-13T11:03:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.602+0000 7fcdbc8e7640 1 Processor -- start 2023-11-13T11:03:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.602+0000 7fcdbc8e7640 1 -- start start 2023-11-13T11:03:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.603+0000 7fcdbc8e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb80b19d0 0x7fcdb81f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.603+0000 7fcdbc8e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 0x7fcdb80c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.603+0000 7fcdbc8e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb8144570 0x7fcdb8138510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.603+0000 7fcdbc8e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb80c8dd0 con 0x7fcdb8144570 2023-11-13T11:03:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.603+0000 7fcdbc8e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb80c9c10 con 0x7fcdb80b19d0 2023-11-13T11:03:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.603+0000 7fcdbc8e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb8138a50 con 0x7fcdb80ce550 2023-11-13T11:03:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb8144570 0x7fcdb8138510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb8144570 0x7fcdb8138510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48052/0 (socket says 172.21.15.16:48052) 2023-11-13T11:03:21.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb6d9d640 1 -- 172.21.15.16:0/2197512839 learned_addr learned my addr 172.21.15.16:0/2197512839 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:21.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb6d9d640 1 -- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 msgr2=0x7fcdb80c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:21.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb6d9d640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 0x7fcdb80c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb6d9d640 1 -- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb80b19d0 msgr2=0x7fcdb81f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb659c640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb80b19d0 0x7fcdb81f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb6d9d640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb80b19d0 0x7fcdb81f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb6d9d640 1 -- 172.21.15.16:0/2197512839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0009b30 con 0x7fcdb8144570 2023-11-13T11:03:21.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb659c640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb80b19d0 0x7fcdb81f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:21.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.604+0000 7fcdb6d9d640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb8144570 0x7fcdb8138510 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0036040 tx=0x7fcdb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:21.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.605+0000 7fcdaeffd640 1 -- 172.21.15.16:0/2197512839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb003e450 con 0x7fcdb8144570 2023-11-13T11:03:21.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.605+0000 7fcdaeffd640 1 -- 172.21.15.16:0/2197512839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb003e5b0 con 0x7fcdb8144570 2023-11-13T11:03:21.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.605+0000 7fcdaeffd640 1 -- 172.21.15.16:0/2197512839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb003e960 con 0x7fcdb8144570 2023-11-13T11:03:21.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.605+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb80c9000 con 0x7fcdb8144570 2023-11-13T11:03:21.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.605+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb80c95c0 con 0x7fcdb8144570 2023-11-13T11:03:21.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.606+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb80c9dc0 con 0x7fcdb8144570 2023-11-13T11:03:21.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.607+0000 7fcdaeffd640 1 -- 172.21.15.16:0/2197512839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcdb002cd50 con 0x7fcdb8144570 2023-11-13T11:03:21.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.608+0000 7fcdaeffd640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd9806ae40 0x7fcd9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:21.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.608+0000 7fcdaeffd640 1 -- 172.21.15.16:0/2197512839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcdb00ac320 con 0x7fcdb8144570 2023-11-13T11:03:21.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.609+0000 7fcdb659c640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd9806ae40 0x7fcd9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:21.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.610+0000 7fcdb659c640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd9806ae40 0x7fcd9806d300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8141140 tx=0x7fcda8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:21.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.612+0000 7fcdaeffd640 1 -- 172.21.15.16:0/2197512839 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcdb80c9dc0 con 0x7fcdb8144570 2023-11-13T11:03:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:21.848+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcdb80c9d80 con 0x7fcdb8144570 2023-11-13T11:03:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.476+0000 7fcdaeffd640 1 -- 172.21.15.16:0/2197512839 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcdb0077340 con 0x7fcdb8144570 2023-11-13T11:03:22.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd9806ae40 msgr2=0x7fcd9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd9806ae40 0x7fcd9806d300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8141140 tx=0x7fcda8008040 comp rx=0 tx=0).stop 2023-11-13T11:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb8144570 msgr2=0x7fcdb8138510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb8144570 0x7fcdb8138510 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0036040 tx=0x7fcdb0004a40 comp rx=0 tx=0).stop 2023-11-13T11:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 shutdown_connections 2023-11-13T11:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcd9806ae40 0x7fcd9806d300 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb8144570 0x7fcdb8138510 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdb80ce550 0x7fcdb80c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 --2- 172.21.15.16:0/2197512839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb80b19d0 0x7fcdb81f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 >> 172.21.15.16:0/2197512839 conn(0x7fcdb806a190 msgr2=0x7fcdb8141e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 shutdown_connections 2023-11-13T11:03:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.480+0000 7fcdbc8e7640 1 -- 172.21.15.16:0/2197512839 wait complete. 2023-11-13T11:03:22.505 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-13T11:03:22.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-13T11:03:22.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.732+0000 7f7a4b3d0640 1 Processor -- start 2023-11-13T11:03:22.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.732+0000 7f7a4b3d0640 1 -- start start 2023-11-13T11:03:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4b3d0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7a4415ce70 0x7f7a440cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4b3d0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440cdbb0 0x7f7a440d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4b3d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a440d1850 0x7f7a440d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4b3d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a440d3240 con 0x7f7a440d1850 2023-11-13T11:03:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4b3d0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a440ce0e0 con 0x7f7a4415ce70 2023-11-13T11:03:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4b3d0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440ce3c0 con 0x7f7a440cdbb0 2023-11-13T11:03:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4abcf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a440d1850 0x7f7a440d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4abcf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a440d1850 0x7f7a440d2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48058/0 (socket says 172.21.15.16:48058) 2023-11-13T11:03:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4abcf640 1 -- 172.21.15.16:0/3944173255 learned_addr learned my addr 172.21.15.16:0/3944173255 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.734+0000 7f7a4a3ce640 1 --2- 172.21.15.16:0/3944173255 >> v2:172.21.15.184:3300/0 conn(0x7f7a4415ce70 0x7f7a440cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.735+0000 7f7a49bcd640 1 --2- 172.21.15.16:0/3944173255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440cdbb0 0x7f7a440d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:22.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.735+0000 7f7a4abcf640 1 -- 172.21.15.16:0/3944173255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440cdbb0 msgr2=0x7f7a440d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:22.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.735+0000 7f7a4abcf640 1 --2- 172.21.15.16:0/3944173255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440cdbb0 0x7f7a440d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.735+0000 7f7a4abcf640 1 -- 172.21.15.16:0/3944173255 >> v2:172.21.15.184:3300/0 conn(0x7f7a4415ce70 msgr2=0x7f7a440cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:22.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.735+0000 7f7a4abcf640 1 --2- 172.21.15.16:0/3944173255 >> v2:172.21.15.184:3300/0 conn(0x7f7a4415ce70 0x7f7a440cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.735+0000 7f7a4abcf640 1 -- 172.21.15.16:0/3944173255 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a440ce6a0 con 0x7f7a440d1850 2023-11-13T11:03:22.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.736+0000 7f7a4abcf640 1 --2- 172.21.15.16:0/3944173255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440d1850 0x7f7a440d2d00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009e50 tx=0x7f7a3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8de3ea68a5903c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:22.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.737+0000 7f7a493cc640 1 -- 172.21.15.16:0/3944173255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c02ca70 con 0x7f7a440d1850 2023-11-13T11:03:22.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.737+0000 7f7a493cc640 1 -- 172.21.15.16:0/3944173255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c02cbd0 con 0x7f7a440d1850 2023-11-13T11:03:22.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.737+0000 7f7a493cc640 1 -- 172.21.15.16:0/3944173255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0343f0 con 0x7f7a440d1850 2023-11-13T11:03:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.737+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/3944173255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440d1850 msgr2=0x7f7a440d2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.737+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/3944173255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440d1850 0x7f7a440d2d00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009e50 tx=0x7f7a3c02bed0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.738+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/3944173255 shutdown_connections 2023-11-13T11:03:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.738+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/3944173255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440d1850 0x7f7a440d2d00 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.738+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/3944173255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440cdbb0 0x7f7a440d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.738+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/3944173255 >> v2:172.21.15.184:3300/0 conn(0x7f7a4415ce70 0x7f7a440cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.738+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/3944173255 >> 172.21.15.16:0/3944173255 conn(0x7f7a4408c580 msgr2=0x7f7a4409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:22.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.738+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/3944173255 shutdown_connections 2023-11-13T11:03:22.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.738+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/3944173255 wait complete. 2023-11-13T11:03:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.740+0000 7f7a4b3d0640 1 Processor -- start 2023-11-13T11:03:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.740+0000 7f7a4b3d0640 1 -- start start 2023-11-13T11:03:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4b3d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440cdbb0 0x7f7a44158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4b3d0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440d1850 0x7f7a441f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4b3d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a4415ce70 0x7f7a44156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4b3d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a441f4950 con 0x7f7a440cdbb0 2023-11-13T11:03:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4b3d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440b8790 con 0x7f7a4415ce70 2023-11-13T11:03:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4b3d0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a441f3680 con 0x7f7a440d1850 2023-11-13T11:03:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4a3ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440cdbb0 0x7f7a44158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4a3ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440cdbb0 0x7f7a44158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48066/0 (socket says 172.21.15.16:48066) 2023-11-13T11:03:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4a3ce640 1 -- 172.21.15.16:0/246717238 learned_addr learned my addr 172.21.15.16:0/246717238 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a49bcd640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440d1850 0x7f7a441f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.741+0000 7f7a4abcf640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a4415ce70 0x7f7a44156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a4a3ce640 1 -- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440d1850 msgr2=0x7f7a441f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a4a3ce640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440d1850 0x7f7a441f3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a4a3ce640 1 -- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a4415ce70 msgr2=0x7f7a44156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a4a3ce640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a4415ce70 0x7f7a44156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a4a3ce640 1 -- 172.21.15.16:0/246717238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c009b30 con 0x7f7a440cdbb0 2023-11-13T11:03:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a49bcd640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440d1850 0x7f7a441f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a4a3ce640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440cdbb0 0x7f7a44158ed0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000e9e0 tx=0x7f7a4000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a3affd640 1 -- 172.21.15.16:0/246717238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40009670 con 0x7f7a440cdbb0 2023-11-13T11:03:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a3affd640 1 -- 172.21.15.16:0/246717238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4000c9d0 con 0x7f7a440cdbb0 2023-11-13T11:03:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a3affd640 1 -- 172.21.15.16:0/246717238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000cd80 con 0x7f7a440cdbb0 2023-11-13T11:03:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a441f3d40 con 0x7f7a440cdbb0 2023-11-13T11:03:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.742+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a441f4300 con 0x7f7a440cdbb0 2023-11-13T11:03:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.743+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4404e4d0 con 0x7f7a440cdbb0 2023-11-13T11:03:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.745+0000 7f7a3affd640 1 -- 172.21.15.16:0/246717238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a40013470 con 0x7f7a440cdbb0 2023-11-13T11:03:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.745+0000 7f7a3affd640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7a2806ae40 0x7f7a2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.746+0000 7f7a3affd640 1 -- 172.21.15.16:0/246717238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7a40089950 con 0x7f7a440cdbb0 2023-11-13T11:03:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.746+0000 7f7a49bcd640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7a2806ae40 0x7f7a2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.747+0000 7f7a49bcd640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7a2806ae40 0x7f7a2806d300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7a441600c0 tx=0x7f7a34008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.750+0000 7f7a3affd640 1 -- 172.21.15.16:0/246717238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a400548b0 con 0x7f7a440cdbb0 2023-11-13T11:03:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:22.987+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7a440cc400 con 0x7f7a440cdbb0 2023-11-13T11:03:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.480+0000 7f7a3affd640 1 -- 172.21.15.16:0/246717238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a400170c0 con 0x7f7a440cdbb0 2023-11-13T11:03:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7a2806ae40 msgr2=0x7f7a2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7a2806ae40 0x7f7a2806d300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7a441600c0 tx=0x7f7a34008040 comp rx=0 tx=0).stop 2023-11-13T11:03:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440cdbb0 msgr2=0x7f7a44158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440cdbb0 0x7f7a44158ed0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000e9e0 tx=0x7f7a4000eea0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 shutdown_connections 2023-11-13T11:03:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7a2806ae40 0x7f7a2806d300 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a4415ce70 0x7f7a44156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a440d1850 0x7f7a441f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 --2- 172.21.15.16:0/246717238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a440cdbb0 0x7f7a44158ed0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.484+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 >> 172.21.15.16:0/246717238 conn(0x7f7a4408c580 msgr2=0x7f7a440cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:23.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.485+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 shutdown_connections 2023-11-13T11:03:23.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.485+0000 7f7a4b3d0640 1 -- 172.21.15.16:0/246717238 wait complete. 2023-11-13T11:03:23.509 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-13T11:03:23.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:23.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.729+0000 7f8a057dc640 1 Processor -- start 2023-11-13T11:03:23.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.729+0000 7f8a057dc640 1 -- start start 2023-11-13T11:03:23.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.731+0000 7f8a057dc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a00159170 0x7f8a0006cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:23.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.731+0000 7f8a057dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000adf90 0x7f8a0015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.731+0000 7f8a057dc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0006d700 con 0x7f8a0015d330 2023-11-13T11:03:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.731+0000 7f8a057dc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0006df10 con 0x7f8a00159170 2023-11-13T11:03:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.731+0000 7f8a057dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0015d760 con 0x7f8a000adf90 2023-11-13T11:03:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.732+0000 7f89ff7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8a0015d330 0x7f8a000b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41048/0 (socket says 172.21.15.16:41048) 2023-11-13T11:03:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.732+0000 7f89ff7fe640 1 -- 172.21.15.16:0/284049327 learned_addr learned my addr 172.21.15.16:0/284049327 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:23.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.732+0000 7f89feffd640 1 --2- 172.21.15.16:0/284049327 >> v2:172.21.15.184:3300/0 conn(0x7f8a00159170 0x7f8a0006cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:23.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.732+0000 7f89f7fff640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000adf90 0x7f8a0015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:23.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.732+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751323355 0 0) 0x7f8a0006d700 con 0x7f8a0015d330 2023-11-13T11:03:23.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.732+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89ec003520 con 0x7f8a0015d330 2023-11-13T11:03:23.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.732+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58119751 0 0) 0x7f89ec003520 con 0x7f8a0015d330 2023-11-13T11:03:23.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000adf90 msgr2=0x7f8a0015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:23.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000adf90 0x7f8a0015cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 >> v2:172.21.15.184:3300/0 conn(0x7f8a00159170 msgr2=0x7f8a0006cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:23.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 --2- 172.21.15.16:0/284049327 >> v2:172.21.15.184:3300/0 conn(0x7f8a00159170 0x7f8a0006cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0006d950 con 0x7f8a0015d330 2023-11-13T11:03:23.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f89f80019f0 con 0x7f8a0015d330 2023-11-13T11:03:23.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 >> v1:172.21.15.16:6789/0 conn(0x7f8a0015d330 legacy=0x7f8a000b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:23.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec003970 0x7f89ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:23.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec0047c0 0x7f89ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:23.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000adf90 0x7f89ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:23.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0006df10 con 0x7f89ec0047c0 2023-11-13T11:03:23.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0015d760 con 0x7f89ec003970 2023-11-13T11:03:23.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec00bab0 con 0x7f8a000adf90 2023-11-13T11:03:23.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89f7fff640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec0047c0 0x7f89ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:23.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89f7fff640 1 -- 172.21.15.16:0/284049327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000adf90 msgr2=0x7f89ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:23.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89f7fff640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000adf90 0x7f89ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89f7fff640 1 -- 172.21.15.16:0/284049327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec003970 msgr2=0x7f89ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:23.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89f7fff640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89ec003970 0x7f89ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.733+0000 7f89f7fff640 1 -- 172.21.15.16:0/284049327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec003520 con 0x7f89ec0047c0 2023-11-13T11:03:23.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.735+0000 7f89f7fff640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec0047c0 0x7f89ec004c40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009650 tx=0x7f89ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8269a36fc168993 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:23.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.735+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e802e8a0 con 0x7f89ec0047c0 2023-11-13T11:03:23.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.735+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e802ea00 con 0x7f89ec0047c0 2023-11-13T11:03:23.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.735+0000 7f89fe7fc640 1 -- 172.21.15.16:0/284049327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e802ed90 con 0x7f89ec0047c0 2023-11-13T11:03:23.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.735+0000 7f8a057dc640 1 -- 172.21.15.16:0/284049327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec0047c0 msgr2=0x7f89ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:23.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.735+0000 7f8a057dc640 1 --2- 172.21.15.16:0/284049327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89ec0047c0 0x7f89ec004c40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009650 tx=0x7f89ec0057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.735+0000 7f89ff7fe640 1 -- 172.21.15.16:0/284049327 reap_dead start 2023-11-13T11:03:23.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.736+0000 7f8a057dc640 1 -- 172.21.15.16:0/284049327 shutdown_connections 2023-11-13T11:03:23.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.736+0000 7f8a057dc640 1 -- 172.21.15.16:0/284049327 >> 172.21.15.16:0/284049327 conn(0x7f8a0006a190 msgr2=0x7f8a0006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:23.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.736+0000 7f8a057dc640 1 -- 172.21.15.16:0/284049327 shutdown_connections 2023-11-13T11:03:23.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.736+0000 7f8a057dc640 1 -- 172.21.15.16:0/284049327 wait complete. 2023-11-13T11:03:23.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.737+0000 7f8a057dc640 1 Processor -- start 2023-11-13T11:03:23.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.738+0000 7f8a057dc640 1 -- start start 2023-11-13T11:03:23.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f8a057dc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0047c0 0x7f8a00152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:23.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f8a057dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a000ce090 0x7f8a000cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:23.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f8a057dc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a000ac8e0 0x7f8a000acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:23.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f8a057dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000ad2a0 con 0x7f8a000ce090 2023-11-13T11:03:23.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f8a057dc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000cc840 con 0x7f8a000ac8e0 2023-11-13T11:03:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f8a057dc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000cccd0 con 0x7f89ec0047c0 2023-11-13T11:03:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a000ce090 0x7f8a000cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a000ce090 0x7f8a000cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55326/0 (socket says 172.21.15.16:55326) 2023-11-13T11:03:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89f7fff640 1 -- 172.21.15.16:0/1484048922 learned_addr learned my addr 172.21.15.16:0/1484048922 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89feffd640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0047c0 0x7f8a00152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89ff7fe640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a000ac8e0 0x7f8a000acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89f7fff640 1 -- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0047c0 msgr2=0x7f8a00152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89f7fff640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0047c0 0x7f8a00152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89f7fff640 1 -- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a000ac8e0 msgr2=0x7f8a000acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89f7fff640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a000ac8e0 0x7f8a000acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.739+0000 7f89f7fff640 1 -- 172.21.15.16:0/1484048922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a00152540 con 0x7f8a000ce090 2023-11-13T11:03:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.740+0000 7f89f7fff640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a000ce090 0x7f8a000cb3e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009b80 tx=0x7f89e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.740+0000 7f89f77fe640 1 -- 172.21.15.16:0/1484048922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8003610 con 0x7f8a000ce090 2023-11-13T11:03:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.740+0000 7f89f77fe640 1 -- 172.21.15.16:0/1484048922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8003770 con 0x7f8a000ce090 2023-11-13T11:03:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.740+0000 7f89f77fe640 1 -- 172.21.15.16:0/1484048922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8003b00 con 0x7f8a000ce090 2023-11-13T11:03:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.740+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a000cbc30 con 0x7f8a000ce090 2023-11-13T11:03:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.740+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a000cc1f0 con 0x7f8a000ce090 2023-11-13T11:03:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.741+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a000cc9f0 con 0x7f8a000ce090 2023-11-13T11:03:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.742+0000 7f89f77fe640 1 -- 172.21.15.16:0/1484048922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89e803d070 con 0x7f8a000ce090 2023-11-13T11:03:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.743+0000 7f89f77fe640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89e006ae70 0x7f89e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.743+0000 7f89f77fe640 1 -- 172.21.15.16:0/1484048922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f89e80aada0 con 0x7f8a000ce090 2023-11-13T11:03:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.744+0000 7f89feffd640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89e006ae70 0x7f89e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.745+0000 7f89feffd640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89e006ae70 0x7f89e006d330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a0015ca70 tx=0x7f89f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.748+0000 7f89f77fe640 1 -- 172.21.15.16:0/1484048922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a000cc9f0 con 0x7f8a000ce090 2023-11-13T11:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.982+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a000cc9f0 con 0x7f8a000ce090 2023-11-13T11:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.984+0000 7f89f77fe640 1 -- 172.21.15.16:0/1484048922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f8a000cc9f0 con 0x7f8a000ce090 2023-11-13T11:03:23.988 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1247955601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1247955601},{"type":"v1","addr":"172.21.15.184:6836","nonce":1247955601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3721806669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3721806669},{"type":"v1","addr":"172.21.15.16:6834","nonce":3721806669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2700274237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2700274237},{"type":"v1","addr":"172.21.15.16:6835","nonce":2700274237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:22.988900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/60343961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":60343961},{"type":"v1","addr":"172.21.15.184:6837","nonce":60343961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:03:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 40 2023-11-13T11:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89e006ae70 msgr2=0x7f89e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89e006ae70 0x7f89e006d330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a0015ca70 tx=0x7f89f0008040 comp rx=0 tx=0).stop 2023-11-13T11:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a000ce090 msgr2=0x7f8a000cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a000ce090 0x7f8a000cb3e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009b80 tx=0x7f89e8004730 comp rx=0 tx=0).stop 2023-11-13T11:03:23.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 shutdown_connections 2023-11-13T11:03:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89e006ae70 0x7f89e006d330 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a000ac8e0 0x7f8a000acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a000ce090 0x7f8a000cb3e0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 --2- 172.21.15.16:0/1484048922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89ec0047c0 0x7f8a00152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:23.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 >> 172.21.15.16:0/1484048922 conn(0x7f8a0006a190 msgr2=0x7f8a00071040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:23.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.986+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 shutdown_connections 2023-11-13T11:03:23.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:23.987+0000 7f8a057dc640 1 -- 172.21.15.16:0/1484048922 wait complete. 2023-11-13T11:03:24.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.219+0000 7fb3bb59e640 1 Processor -- start 2023-11-13T11:03:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.219+0000 7fb3bb59e640 1 -- start start 2023-11-13T11:03:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.221+0000 7fb3bb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0d2ae0 0x7fb3bc0d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.221+0000 7fb3bb59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3bc0b3cf0 con 0x7fb3bc15cfc0 2023-11-13T11:03:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.221+0000 7fb3bb59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3bc0d3480 con 0x7fb3bc0b29d0 2023-11-13T11:03:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.221+0000 7fb3bb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0cde80 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.221+0000 7fb3ba59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb3bc15cfc0 0x7fb3bc0cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41050/0 (socket says 172.21.15.16:41050) 2023-11-13T11:03:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.221+0000 7fb3ba59c640 1 -- 172.21.15.16:0/3397389768 learned_addr learned my addr 172.21.15.16:0/3397389768 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.221+0000 7fb3b9d9b640 1 --2- 172.21.15.16:0/3397389768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0d2ae0 0x7fb3bc0d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.222+0000 7fb3b959a640 1 -- 172.21.15.16:0/3397389768 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224888705 0 0) 0x7fb3bc0b3cf0 con 0x7fb3bc15cfc0 2023-11-13T11:03:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.222+0000 7fb3b959a640 1 -- 172.21.15.16:0/3397389768 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3a4003520 con 0x7fb3bc15cfc0 2023-11-13T11:03:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.222+0000 7fb3b959a640 1 -- 172.21.15.16:0/3397389768 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3235594181 0 0) 0x7fb3a4003520 con 0x7fb3bc15cfc0 2023-11-13T11:03:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.222+0000 7fb3b9d9b640 1 -- 172.21.15.16:0/3397389768 >> v1:172.21.15.184:6789/0 conn(0x7fb3bc0b29d0 legacy=0x7fb3bc0b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.222+0000 7fb3b9d9b640 1 -- 172.21.15.16:0/3397389768 >> v1:172.21.15.16:6789/0 conn(0x7fb3bc15cfc0 legacy=0x7fb3bc0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.222+0000 7fb3b9d9b640 1 -- 172.21.15.16:0/3397389768 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc0b3730 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.224+0000 7fb3b9d9b640 1 --2- 172.21.15.16:0/3397389768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0d2ae0 0x7fb3bc0d2f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00097d0 tx=0x7fb3b002b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea4798585be279a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.224+0000 7fb3b959a640 1 -- 172.21.15.16:0/3397389768 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac001670 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.224+0000 7fb3b959a640 1 -- 172.21.15.16:0/3397389768 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b002e050 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.224+0000 7fb3b959a640 1 -- 172.21.15.16:0/3397389768 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0030710 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.225+0000 7fb3bb59e640 1 -- 172.21.15.16:0/3397389768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0d2ae0 msgr2=0x7fb3bc0d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.225+0000 7fb3bb59e640 1 --2- 172.21.15.16:0/3397389768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0d2ae0 0x7fb3bc0d2f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00097d0 tx=0x7fb3b002b770 comp rx=0 tx=0).stop 2023-11-13T11:03:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.225+0000 7fb3bb59e640 1 -- 172.21.15.16:0/3397389768 shutdown_connections 2023-11-13T11:03:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.225+0000 7fb3bb59e640 1 --2- 172.21.15.16:0/3397389768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0d2ae0 0x7fb3bc0d2f40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.225+0000 7fb3bb59e640 1 -- 172.21.15.16:0/3397389768 >> 172.21.15.16:0/3397389768 conn(0x7fb3bc08c570 msgr2=0x7fb3bc09be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.225+0000 7fb3bb59e640 1 -- 172.21.15.16:0/3397389768 shutdown_connections 2023-11-13T11:03:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.225+0000 7fb3bb59e640 1 -- 172.21.15.16:0/3397389768 wait complete. 2023-11-13T11:03:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.227+0000 7fb3bb59e640 1 Processor -- start 2023-11-13T11:03:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.227+0000 7fb3bb59e640 1 -- start start 2023-11-13T11:03:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3bb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0b29d0 0x7fb3bc15e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3bb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc0d2ae0 0x7fb3bc151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3bb59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3bc15cfc0 0x7fb3bc15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3bb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0b7cd0 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3bb59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0b70c0 con 0x7fb3bc15cfc0 2023-11-13T11:03:24.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3bb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0b73a0 con 0x7fb3bc0b29d0 2023-11-13T11:03:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3b9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc0d2ae0 0x7fb3bc151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3b9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc0d2ae0 0x7fb3bc151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55340/0 (socket says 172.21.15.16:55340) 2023-11-13T11:03:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3b9d9b640 1 -- 172.21.15.16:0/1536945382 learned_addr learned my addr 172.21.15.16:0/1536945382 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.228+0000 7fb3ba59c640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0b29d0 0x7fb3bc15e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3bad9d640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3bc15cfc0 0x7fb3bc15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3b9d9b640 1 -- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0b29d0 msgr2=0x7fb3bc15e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3b9d9b640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0b29d0 0x7fb3bc15e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3b9d9b640 1 -- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3bc15cfc0 msgr2=0x7fb3bc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3b9d9b640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3bc15cfc0 0x7fb3bc15b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3b9d9b640 1 -- 172.21.15.16:0/1536945382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b0009400 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3bad9d640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3bc15cfc0 0x7fb3bc15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3b9d9b640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc0d2ae0 0x7fb3bc151f10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0000c00 tx=0x7fb3b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3aaffd640 1 -- 172.21.15.16:0/1536945382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b003c620 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3aaffd640 1 -- 172.21.15.16:0/1536945382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b003c7a0 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3aaffd640 1 -- 172.21.15.16:0/1536945382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b003cb50 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.229+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3bc0b7680 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.230+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3bc0b8160 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.231+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3bc04e4d0 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.231+0000 7fb3aaffd640 1 -- 172.21.15.16:0/1536945382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb3b0039070 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.232+0000 7fb3aaffd640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb39c06ae40 0x7fb39c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.233+0000 7fb3aaffd640 1 -- 172.21.15.16:0/1536945382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb3b00aae80 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.233+0000 7fb3ba59c640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb39c06ae40 0x7fb39c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.234+0000 7fb3ba59c640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb39c06ae40 0x7fb39c06d300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac005880 tx=0x7fb3ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.237+0000 7fb3aaffd640 1 -- 172.21.15.16:0/1536945382 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3b0075df0 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.424+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3bc0c5df0 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.426+0000 7fb3aaffd640 1 -- 172.21.15.16:0/1536945382 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16776 (secure 0 0 0) 0x7fb3b0037090 con 0x7fb3bc0d2ae0 2023-11-13T11:03:24.431 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:24.431 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":69,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:21.435542+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:03:17.903258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-13T11:03:18.947896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:24.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb39c06ae40 msgr2=0x7fb39c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb39c06ae40 0x7fb39c06d300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac005880 tx=0x7fb3ac009040 comp rx=0 tx=0).stop 2023-11-13T11:03:24.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc0d2ae0 msgr2=0x7fb3bc151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc0d2ae0 0x7fb3bc151f10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0000c00 tx=0x7fb3b00046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 shutdown_connections 2023-11-13T11:03:24.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb39c06ae40 0x7fb39c06d300 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3bc15cfc0 0x7fb3bc15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3bc0d2ae0 0x7fb3bc151f10 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 --2- 172.21.15.16:0/1536945382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3bc0b29d0 0x7fb3bc15e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 >> 172.21.15.16:0/1536945382 conn(0x7fb3bc08c570 msgr2=0x7fb3bc06b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 shutdown_connections 2023-11-13T11:03:24.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.429+0000 7fb3bb59e640 1 -- 172.21.15.16:0/1536945382 wait complete. 2023-11-13T11:03:24.452 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-13T11:03:24.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:03:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.661+0000 7f0d197c4640 1 Processor -- start 2023-11-13T11:03:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.661+0000 7f0d197c4640 1 -- start start 2023-11-13T11:03:24.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.662+0000 7f0d197c4640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0d1406a190 0x7f0d1406a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.662+0000 7f0d197c4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d14158c20 con 0x7f0d1415b190 2023-11-13T11:03:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.663+0000 7f0d197c4640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140b1830 con 0x7f0d140cb190 2023-11-13T11:03:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.663+0000 7f0d197c4640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1406ab40 con 0x7f0d1406a190 2023-11-13T11:03:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.663+0000 7f0d137fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0d1415b190 0x7f0d1406f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41052/0 (socket says 172.21.15.16:41052) 2023-11-13T11:03:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.663+0000 7f0d137fe640 1 -- 172.21.15.16:0/3752732778 learned_addr learned my addr 172.21.15.16:0/3752732778 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.663+0000 7f0d18fc3640 1 --2- 172.21.15.16:0/3752732778 >> v2:172.21.15.184:3300/0 conn(0x7f0d1406a190 0x7f0d1406a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.663+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349340291 0 0) 0x7f0d14158c20 con 0x7f0d1415b190 2023-11-13T11:03:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.663+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cfc003520 con 0x7f0d1415b190 2023-11-13T11:03:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1539581706 0 0) 0x7f0cfc003520 con 0x7f0d1415b190 2023-11-13T11:03:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 >> v2:172.21.15.184:3300/0 conn(0x7f0d1406a190 msgr2=0x7f0d1406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 --2- 172.21.15.16:0/3752732778 >> v2:172.21.15.184:3300/0 conn(0x7f0d1406a190 0x7f0d1406a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 >> v1:172.21.15.184:6789/0 conn(0x7f0d140cb190 legacy=0x7f0d140716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1415b780 con 0x7f0d1415b190 2023-11-13T11:03:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f0d08001670 con 0x7f0d1415b190 2023-11-13T11:03:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 >> v1:172.21.15.16:6789/0 conn(0x7f0d1415b190 legacy=0x7f0d1406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0039c0 0x7f0cfc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cfc0047c0 0x7f0cfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cfc005250 0x7f0cfc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1406ab40 con 0x7f0cfc0047c0 2023-11-13T11:03:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00c1a0 con 0x7f0cfc0039c0 2023-11-13T11:03:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00c360 con 0x7f0cfc005250 2023-11-13T11:03:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cfc0047c0 0x7f0cfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d13fff640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0039c0 0x7f0cfc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d137fe640 1 -- 172.21.15.16:0/3752732778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cfc005250 msgr2=0x7f0cfc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cfc005250 0x7f0cfc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d137fe640 1 -- 172.21.15.16:0/3752732778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0039c0 msgr2=0x7f0cfc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.664+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0039c0 0x7f0cfc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.665+0000 7f0d137fe640 1 -- 172.21.15.16:0/3752732778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc003520 con 0x7f0cfc0047c0 2023-11-13T11:03:24.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.665+0000 7f0d13fff640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0039c0 0x7f0cfc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:24.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.665+0000 7f0d18fc3640 1 -- 172.21.15.16:0/3752732778 reap_dead start 2023-11-13T11:03:24.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.666+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cfc0047c0 0x7f0cfc004c40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c190 tx=0x7f0cfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73af7a9fbc7b1129 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.666+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08005ad0 con 0x7f0cfc0047c0 2023-11-13T11:03:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.666+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d08005c30 con 0x7f0cfc0047c0 2023-11-13T11:03:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.666+0000 7f0d12ffd640 1 -- 172.21.15.16:0/3752732778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0802c7f0 con 0x7f0cfc0047c0 2023-11-13T11:03:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.666+0000 7f0d197c4640 1 -- 172.21.15.16:0/3752732778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cfc0047c0 msgr2=0x7f0cfc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.666+0000 7f0d197c4640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cfc0047c0 0x7f0cfc004c40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c190 tx=0x7f0cfc0057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.666+0000 7f0d197c4640 1 -- 172.21.15.16:0/3752732778 shutdown_connections 2023-11-13T11:03:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.666+0000 7f0d197c4640 1 --2- 172.21.15.16:0/3752732778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cfc0047c0 0x7f0cfc004c40 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.667+0000 7f0d197c4640 1 -- 172.21.15.16:0/3752732778 >> 172.21.15.16:0/3752732778 conn(0x7f0d140b04a0 msgr2=0x7f0d140b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.667+0000 7f0d197c4640 1 -- 172.21.15.16:0/3752732778 shutdown_connections 2023-11-13T11:03:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.667+0000 7f0d197c4640 1 -- 172.21.15.16:0/3752732778 wait complete. 2023-11-13T11:03:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.668+0000 7f0d197c4640 1 Processor -- start 2023-11-13T11:03:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.668+0000 7f0d197c4640 1 -- start start 2023-11-13T11:03:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.669+0000 7f0d197c4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0047c0 0x7f0d14153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.669+0000 7f0d197c4640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d14153ff0 0x7f0d14140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.669+0000 7f0d197c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14140e50 0x7f0d141f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.669+0000 7f0d197c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14141300 con 0x7f0d14140e50 2023-11-13T11:03:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.669+0000 7f0d197c4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14154470 con 0x7f0cfc0047c0 2023-11-13T11:03:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.669+0000 7f0d197c4640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14156820 con 0x7f0d14153ff0 2023-11-13T11:03:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d18fc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14140e50 0x7f0d141f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d18fc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14140e50 0x7f0d141f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55354/0 (socket says 172.21.15.16:55354) 2023-11-13T11:03:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d18fc3640 1 -- 172.21.15.16:0/3217944701 learned_addr learned my addr 172.21.15.16:0/3217944701 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d137fe640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d14153ff0 0x7f0d14140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d18fc3640 1 -- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d14153ff0 msgr2=0x7f0d14140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d18fc3640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d14153ff0 0x7f0d14140910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d13fff640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0047c0 0x7f0d14153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d18fc3640 1 -- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0047c0 msgr2=0x7f0d14153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d18fc3640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0047c0 0x7f0d14153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.670+0000 7f0d18fc3640 1 -- 172.21.15.16:0/3217944701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d14155ef0 con 0x7f0d14140e50 2023-11-13T11:03:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.671+0000 7f0d18fc3640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14140e50 0x7f0d141f2a30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0d140b2c80 tx=0x7f0d0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.671+0000 7f0d10ff9640 1 -- 172.21.15.16:0/3217944701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00ed90 con 0x7f0d14140e50 2023-11-13T11:03:24.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.671+0000 7f0d10ff9640 1 -- 172.21.15.16:0/3217944701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c014420 con 0x7f0d14140e50 2023-11-13T11:03:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.671+0000 7f0d10ff9640 1 -- 172.21.15.16:0/3217944701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0147d0 con 0x7f0d14140e50 2023-11-13T11:03:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.671+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d141561d0 con 0x7f0d14140e50 2023-11-13T11:03:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.671+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d14155bc0 con 0x7f0d14140e50 2023-11-13T11:03:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.672+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d141569b0 con 0x7f0d14140e50 2023-11-13T11:03:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.673+0000 7f0d10ff9640 1 -- 172.21.15.16:0/3217944701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d0c014930 con 0x7f0d14140e50 2023-11-13T11:03:24.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.674+0000 7f0d10ff9640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0d0006ae70 0x7f0d0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:24.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.674+0000 7f0d10ff9640 1 -- 172.21.15.16:0/3217944701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0d0c089c90 con 0x7f0d14140e50 2023-11-13T11:03:24.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.675+0000 7f0d13fff640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0d0006ae70 0x7f0d0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:24.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.676+0000 7f0d13fff640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0d0006ae70 0x7f0d0006d330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0d14157960 tx=0x7f0d04009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:24.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.679+0000 7f0d10ff9640 1 -- 172.21.15.16:0/3217944701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d0c054c90 con 0x7f0d14140e50 2023-11-13T11:03:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.874+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d1415c020 con 0x7f0d0006ae70 2023-11-13T11:03:24.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.899+0000 7f0d10ff9640 1 -- 172.21.15.16:0/3217944701 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275714 (secure 0 0 0) 0x7f0d0400d040 con 0x7f0d0006ae70 2023-11-13T11:03:24.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:24.908 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:03:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.904+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0d0006ae70 msgr2=0x7f0d0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.904+0000 7f0d197c4640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0d0006ae70 0x7f0d0006d330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0d14157960 tx=0x7f0d04009040 comp rx=0 tx=0).stop 2023-11-13T11:03:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14140e50 msgr2=0x7f0d141f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14140e50 0x7f0d141f2a30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0d140b2c80 tx=0x7f0d0c00e4b0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 shutdown_connections 2023-11-13T11:03:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f0d0006ae70 0x7f0d0006d330 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d14140e50 0x7f0d141f2a30 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d14153ff0 0x7f0d14140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 --2- 172.21.15.16:0/3217944701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cfc0047c0 0x7f0d14153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:24.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 >> 172.21.15.16:0/3217944701 conn(0x7f0d140b04a0 msgr2=0x7f0d140cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:24.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 shutdown_connections 2023-11-13T11:03:24.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:24.905+0000 7f0d197c4640 1 -- 172.21.15.16:0/3217944701 wait complete. 2023-11-13T11:03:24.929 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-11-13T11:03:24.156436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":70,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":352564,"kb_used_data":7372,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754622156,"statfs":{"total":773094113280,"available":772733087744,"internally_reserved":0,"allocated":7548928,"data_stored":4012902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":353357361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.737938"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058741+0000","last_change":"2023-11-13T11:03:20.473726+0000","last_active":"2023-11-13T11:03:22.058741+0000","last_peered":"2023-11-13T11:03:22.058741+0000","last_clean":"2023-11-13T11:03:22.058741+0000","last_became_active":"2023-11-13T11:03:20.473463+0000","last_became_peered":"2023-11-13T11:03:20.473463+0000","last_unstale":"2023-11-13T11:03:22.058741+0000","last_undegraded":"2023-11-13T11:03:22.058741+0000","last_fullsized":"2023-11-13T11:03:22.058741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:24:37.146283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058815+0000","last_change":"2023-11-13T11:03:20.471989+0000","last_active":"2023-11-13T11:03:22.058815+0000","last_peered":"2023-11-13T11:03:22.058815+0000","last_clean":"2023-11-13T11:03:22.058815+0000","last_became_active":"2023-11-13T11:03:20.470759+0000","last_became_peered":"2023-11-13T11:03:20.470759+0000","last_unstale":"2023-11-13T11:03:22.058815+0000","last_undegraded":"2023-11-13T11:03:22.058815+0000","last_fullsized":"2023-11-13T11:03:22.058815+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:33.483394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.432660+0000","last_change":"2023-11-13T11:03:18.432660+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.432660+0000","last_undegraded":"2023-11-13T11:03:18.432660+0000","last_fullsized":"2023-11-13T11:03:18.432660+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.434034+0000","last_change":"2023-11-13T11:03:18.434034+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.434034+0000","last_undegraded":"2023-11-13T11:03:18.434034+0000","last_fullsized":"2023-11-13T11:03:18.434034+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.432731+0000","last_change":"2023-11-13T11:03:18.432731+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.432731+0000","last_undegraded":"2023-11-13T11:03:18.432731+0000","last_fullsized":"2023-11-13T11:03:18.432731+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.426974+0000","last_change":"2023-11-13T11:03:18.426974+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.426974+0000","last_undegraded":"2023-11-13T11:03:18.426974+0000","last_fullsized":"2023-11-13T11:03:18.426974+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.423052+0000","last_change":"2023-11-13T11:03:18.423052+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.423052+0000","last_undegraded":"2023-11-13T11:03:18.423052+0000","last_fullsized":"2023-11-13T11:03:18.423052+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.424577+0000","last_change":"2023-11-13T11:03:18.424577+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.424577+0000","last_undegraded":"2023-11-13T11:03:18.424577+0000","last_fullsized":"2023-11-13T11:03:18.424577+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.431321+0000","last_change":"2023-11-13T11:03:18.431321+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.431321+0000","last_undegraded":"2023-11-13T11:03:18.431321+0000","last_fullsized":"2023-11-13T11:03:18.431321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428573+0000","last_change":"2023-11-13T11:03:18.428573+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428573+0000","last_undegraded":"2023-11-13T11:03:18.428573+0000","last_fullsized":"2023-11-13T11:03:18.428573+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429859+0000","last_change":"2023-11-13T11:03:18.429859+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429859+0000","last_undegraded":"2023-11-13T11:03:18.429859+0000","last_fullsized":"2023-11-13T11:03:18.429859+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.421232+0000","last_change":"2023-11-13T11:03:18.421232+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.421232+0000","last_undegraded":"2023-11-13T11:03:18.421232+0000","last_fullsized":"2023-11-13T11:03:18.421232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428911+0000","last_change":"2023-11-13T11:03:18.428911+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428911+0000","last_undegraded":"2023-11-13T11:03:18.428911+0000","last_fullsized":"2023-11-13T11:03:18.428911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.425857+0000","last_change":"2023-11-13T11:03:18.425857+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.425857+0000","last_undegraded":"2023-11-13T11:03:18.425857+0000","last_fullsized":"2023-11-13T11:03:18.425857+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429650+0000","last_change":"2023-11-13T11:03:18.429650+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429650+0000","last_undegraded":"2023-11-13T11:03:18.429650+0000","last_fullsized":"2023-11-13T11:03:18.429650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.433832+0000","last_change":"2023-11-13T11:03:18.433832+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.433832+0000","last_undegraded":"2023-11-13T11:03:18.433832+0000","last_fullsized":"2023-11-13T11:03:18.433832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428094+0000","last_change":"2023-11-13T11:03:18.428094+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428094+0000","last_undegraded":"2023-11-13T11:03:18.428094+0000","last_fullsized":"2023-11-13T11:03:18.428094+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.433356+0000","last_change":"2023-11-13T11:03:18.433356+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.433356+0000","last_undegraded":"2023-11-13T11:03:18.433356+0000","last_fullsized":"2023-11-13T11:03:18.433356+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429355+0000","last_change":"2023-11-13T11:03:18.429355+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429355+0000","last_undegraded":"2023-11-13T11:03:18.429355+0000","last_fullsized":"2023-11-13T11:03:18.429355+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.430231+0000","last_change":"2023-11-13T11:03:18.430231+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.430231+0000","last_undegraded":"2023-11-13T11:03:18.430231+0000","last_fullsized":"2023-11-13T11:03:18.430231+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428157+0000","last_change":"2023-11-13T11:03:18.428157+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428157+0000","last_undegraded":"2023-11-13T11:03:18.428157+0000","last_fullsized":"2023-11-13T11:03:18.428157+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059038+0000","last_change":"2023-11-13T11:03:19.469381+0000","last_active":"2023-11-13T11:03:22.059038+0000","last_peered":"2023-11-13T11:03:22.059038+0000","last_clean":"2023-11-13T11:03:22.059038+0000","last_became_active":"2023-11-13T11:03:19.468768+0000","last_became_peered":"2023-11-13T11:03:19.468768+0000","last_unstale":"2023-11-13T11:03:22.059038+0000","last_undegraded":"2023-11-13T11:03:22.059038+0000","last_fullsized":"2023-11-13T11:03:22.059038+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:37:23.154386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.424388+0000","last_change":"2023-11-13T11:03:18.424388+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.424388+0000","last_undegraded":"2023-11-13T11:03:18.424388+0000","last_fullsized":"2023-11-13T11:03:18.424388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.433296+0000","last_change":"2023-11-13T11:03:18.433296+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.433296+0000","last_undegraded":"2023-11-13T11:03:18.433296+0000","last_fullsized":"2023-11-13T11:03:18.433296+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.423667+0000","last_change":"2023-11-13T11:03:18.423667+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.423667+0000","last_undegraded":"2023-11-13T11:03:18.423667+0000","last_fullsized":"2023-11-13T11:03:18.423667+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428757+0000","last_change":"2023-11-13T11:03:18.428757+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428757+0000","last_undegraded":"2023-11-13T11:03:18.428757+0000","last_fullsized":"2023-11-13T11:03:18.428757+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429504+0000","last_change":"2023-11-13T11:03:18.429504+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429504+0000","last_undegraded":"2023-11-13T11:03:18.429504+0000","last_fullsized":"2023-11-13T11:03:18.429504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.426053+0000","last_change":"2023-11-13T11:03:18.426053+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.426053+0000","last_undegraded":"2023-11-13T11:03:18.426053+0000","last_fullsized":"2023-11-13T11:03:18.426053+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059174+0000","last_change":"2023-11-13T11:03:19.468809+0000","last_active":"2023-11-13T11:03:22.059174+0000","last_peered":"2023-11-13T11:03:22.059174+0000","last_clean":"2023-11-13T11:03:22.059174+0000","last_became_active":"2023-11-13T11:03:19.467750+0000","last_became_peered":"2023-11-13T11:03:19.467750+0000","last_unstale":"2023-11-13T11:03:22.059174+0000","last_undegraded":"2023-11-13T11:03:22.059174+0000","last_fullsized":"2023-11-13T11:03:22.059174+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:45:50.416913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.421233+0000","last_change":"2023-11-13T11:03:18.421233+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.421233+0000","last_undegraded":"2023-11-13T11:03:18.421233+0000","last_fullsized":"2023-11-13T11:03:18.421233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.431809+0000","last_change":"2023-11-13T11:03:18.431809+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.431809+0000","last_undegraded":"2023-11-13T11:03:18.431809+0000","last_fullsized":"2023-11-13T11:03:18.431809+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058922+0000","last_change":"2023-11-13T11:03:19.468080+0000","last_active":"2023-11-13T11:03:22.058922+0000","last_peered":"2023-11-13T11:03:22.058922+0000","last_clean":"2023-11-13T11:03:22.058922+0000","last_became_active":"2023-11-13T11:03:19.467716+0000","last_became_peered":"2023-11-13T11:03:19.467716+0000","last_unstale":"2023-11-13T11:03:22.058922+0000","last_undegraded":"2023-11-13T11:03:22.058922+0000","last_fullsized":"2023-11-13T11:03:22.058922+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:57:39.710883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.424719+0000","last_change":"2023-11-13T11:03:18.424719+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.424719+0000","last_undegraded":"2023-11-13T11:03:18.424719+0000","last_fullsized":"2023-11-13T11:03:18.424719+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.423491+0000","last_change":"2023-11-13T11:03:18.423491+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.423491+0000","last_undegraded":"2023-11-13T11:03:18.423491+0000","last_fullsized":"2023-11-13T11:03:18.423491+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":205,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-13T11:03:18.434428+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:03:18.434428+0000","last_peered":"2023-11-13T11:03:18.434428+0000","last_clean":"2023-11-13T11:03:18.434428+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:03:18.434428+0000","last_undegraded":"2023-11-13T11:03:18.434428+0000","last_fullsized":"2023-11-13T11:03:18.434428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44432,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327408,"statfs":{"total":96636764160,"available":96591265792,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1060000000000001,"15min":1.1259999999999999},"min":{"1min":0.73899999999999999,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.706,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.2250000000000001,"15min":1.1379999999999999},"min":{"1min":0.82199999999999995,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.498,"5min":1.865,"15min":2.2879999999999998},"last":1.014}]},{"osd":1,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.252,"15min":1.1799999999999999},"min":{"1min":0.90300000000000002,"5min":0.64000000000000001,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.089},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.2889999999999999},"min":{"1min":0.67700000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.649,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.1679999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.319,"15min":1.276},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.534,"5min":2.153,"15min":2.1680000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.383,"15min":1.3460000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.6100000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.3700000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.429,"15min":1.3779999999999999},"min":{"1min":0.81799999999999995,"5min":0.76300000000000001,"15min":0.63800000000000001},"max":{"1min":1.657,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.458,"15min":1.4379999999999999},"min":{"1min":0.99299999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.762,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.21}]},{"osd":4,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.456,"15min":1.552},"min":{"1min":0.94899999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.782,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.286},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.512,"15min":1.5740000000000001},"min":{"1min":1.1599999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.9419999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.448}]},{"osd":5,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.6060000000000001,"15min":1.6399999999999999},"min":{"1min":0.94499999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.8200000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.498},{"interface":"front","average":{"1min":1.625,"5min":1.75,"15min":1.7490000000000001},"min":{"1min":1.4650000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":1.865,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.6859999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.6930000000000001,"15min":1.645},"min":{"1min":1.2849999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0640000000000001,"5min":2.657,"15min":2.657},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.641,"15min":1.6379999999999999},"min":{"1min":1.1399999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1349999999999998,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.575}]}]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45012,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326828,"statfs":{"total":96636764160,"available":96590671872,"internally_reserved":0,"allocated":1384448,"data_stored":941850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.093,"15min":1.016},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7529999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.284},{"interface":"front","average":{"1min":1.179,"5min":1.2190000000000001,"15min":1.202},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67800000000000005},"max":{"1min":1.639,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.0760000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.306,"15min":1.2350000000000001},"min":{"1min":0.79100000000000004,"5min":0.72399999999999998,"15min":0.65400000000000003},"max":{"1min":1.73,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.282,"15min":1.2450000000000001},"min":{"1min":0.875,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.115}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.389,"15min":1.2649999999999999},"min":{"1min":0.81200000000000006,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":4.4500000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.262},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.3100000000000001,"15min":1.2370000000000001},"min":{"1min":0.78900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5760000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.0089999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.387,"15min":1.391},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8300000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.23},{"interface":"front","average":{"1min":1.53,"5min":1.5169999999999999,"15min":1.4870000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.80700000000000005},"max":{"1min":4.6959999999999997,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":1.212}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3700000000000001,"15min":1.329},"min":{"1min":0.93500000000000005,"5min":0.86599999999999999,"15min":0.55500000000000005},"max":{"1min":1.5800000000000001,"5min":3.484,"15min":3.484},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.518,"15min":1.53},"min":{"1min":1.093,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.4430000000000001,"5min":3.653,"15min":3.653},"last":0.95299999999999996}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.5800000000000001,"15min":1.6040000000000001},"min":{"1min":1.0800000000000001,"5min":0.92000000000000004,"15min":0.78000000000000003},"max":{"1min":4.8760000000000003,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.538,"15min":1.5189999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.61799999999999999},"max":{"1min":1.7609999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.177}]},{"osd":7,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.645,"15min":1.6659999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.1349999999999998,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.744,"5min":1.7330000000000001,"15min":1.714},"min":{"1min":1.2490000000000001,"5min":1.1200000000000001,"15min":0.69499999999999995},"max":{"1min":4.7919999999999998,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":1.377}]}]},{"osd":4,"up_from":15,"seq":64424509541,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54160,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317680,"statfs":{"total":96636764160,"available":96581304320,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.153,"15min":1.089},"min":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":1.986,"15min":1.986},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.145,"15min":1.085},"min":{"1min":0.69199999999999995,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.83799999999999997}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2470000000000001,"15min":1.163},"min":{"1min":0.76200000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.774,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.05},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.26},"min":{"1min":0.79300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":1.877,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.4650000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.298,"15min":1.242},"min":{"1min":0.93300000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.575,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.347,"15min":1.349},"min":{"1min":0.91100000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.72,"5min":1.952,"15min":1.952},"last":1.637}]},{"osd":3,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.488,"15min":1.421},"min":{"1min":1.1080000000000001,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.786,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.841},{"interface":"front","average":{"1min":1.379,"5min":1.3149999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.841,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.5740000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.478,"15min":1.4910000000000001},"min":{"1min":0.96399999999999997,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.778,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.996},{"interface":"front","average":{"1min":1.323,"5min":1.371,"15min":1.3939999999999999},"min":{"1min":0.81599999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":2.0529999999999999,"5min":2.089,"15min":2.089},"last":1.7809999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.621,"15min":1.6220000000000001},"min":{"1min":1.0600000000000001,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":1.7769999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":2.1030000000000002},{"interface":"front","average":{"1min":1.401,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.92400000000000004,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":1.873,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":2.0310000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.5369999999999999,"15min":1.5820000000000001},"min":{"1min":1.05,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":2.1480000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.9039999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.599,"15min":1.609},"min":{"1min":1.0189999999999999,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":1.931,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":2.1499999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40272,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331568,"statfs":{"total":96636764160,"available":96595525632,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.628,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.282,"5min":1.867,"15min":1.867},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.022,"15min":1.026},"min":{"1min":0.65800000000000003,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.819,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.032,"15min":1.0489999999999999},"min":{"1min":0.625,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.8089999999999999,"5min":1.966,"15min":3.71},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.296,"5min":1.1240000000000001,"15min":1.103},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.179,"15min":1.1040000000000001},"min":{"1min":0.72899999999999998,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":2.0019999999999998,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.24,"5min":1.171,"15min":1.2829999999999999},"min":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7809999999999999,"5min":1.952,"15min":3.5710000000000002},"last":1.143}]},{"osd":4,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.89200000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.7509999999999999,"5min":2.411,"15min":3.5030000000000001},"last":1.278},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3879999999999999,"15min":1.355},"min":{"1min":1.1259999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.831,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.5840000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4179999999999999,"15min":1.4750000000000001},"min":{"1min":1.151,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.99,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.454,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0649999999999999,"5min":2.673,"15min":3.3809999999999998},"last":1.45}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.49},"min":{"1min":1.1479999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.944,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.206},{"interface":"front","average":{"1min":1.546,"5min":1.4870000000000001,"15min":1.5840000000000001},"min":{"1min":1.1060000000000001,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":2.093,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.3380000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5609999999999999,"15min":1.609},"min":{"1min":0.84799999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1339999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.631},{"interface":"front","average":{"1min":1.649,"5min":1.6040000000000001,"15min":1.5620000000000001},"min":{"1min":1.1359999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.4089999999999998,"5min":2.75,"15min":2.75},"last":1.391}]}]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50004,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321836,"statfs":{"total":96636764160,"available":96585560064,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94199999999999995,"15min":0.93100000000000005},"min":{"1min":0.55700000000000005,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.089,"5min":1.0229999999999999,"15min":0.93400000000000005},"min":{"1min":0.54400000000000004,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":2.9460000000000002,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":1.7330000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0309999999999999,"15min":0.96299999999999997},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.376},"max":{"1min":3.0720000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.361,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.68100000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.3380000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.0920000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.216,"15min":1.2430000000000001},"min":{"1min":0.71599999999999997,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2549999999999999,"15min":1.2749999999999999},"min":{"1min":0.72699999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.4060000000000001,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.456}]},{"osd":4,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4359999999999999,"15min":1.4730000000000001},"min":{"1min":0.89900000000000002,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":2.633,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.393,"15min":1.409},"min":{"1min":0.80000000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":1.5189999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.4690000000000001,"15min":1.482},"min":{"1min":1.034,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.1680000000000001,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":1.165},{"interface":"front","average":{"1min":1.623,"5min":1.591,"15min":1.5680000000000001},"min":{"1min":0.97099999999999997,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.7509999999999999,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.6699999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5189999999999999,"15min":1.55},"min":{"1min":0.89500000000000002,"5min":0.80900000000000005,"15min":0.76500000000000001},"max":{"1min":3.395,"5min":3.395,"15min":4.7569999999999997},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6120000000000001,"15min":1.599},"min":{"1min":1.2130000000000001,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.4039999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.5700000000000001,"15min":1.607},"min":{"1min":1.0569999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":3.2429999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.694,"5min":1.6479999999999999,"15min":1.6259999999999999},"min":{"1min":1.2090000000000001,"5min":0.91200000000000003,"15min":0.82799999999999996},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":1.5760000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28372,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343468,"statfs":{"total":96636764160,"available":96607711232,"internally_reserved":0,"allocated":1384448,"data_stored":941850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85699999999999998,"15min":0.94899999999999995},"min":{"1min":0.495,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.4910000000000001,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":1.03},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.97399999999999998,"15min":0.95699999999999996},"min":{"1min":0.623,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2689999999999999,"5min":1.869,"15min":1.869},"last":0.98899999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.0509999999999999,"15min":1},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.129},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0680000000000001,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":1.111}]},{"osd":3,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.204,"15min":1.175},"min":{"1min":0.68600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.177},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.57799999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.0669999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.321,"15min":1.3129999999999999},"min":{"1min":0.76200000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":1.212},{"interface":"front","average":{"1min":1.536,"5min":1.4179999999999999,"15min":1.3859999999999999},"min":{"1min":0.69599999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.0299999999999998,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.8620000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4159999999999999,"15min":1.4139999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.0750000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.9330000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.387,"15min":1.4079999999999999},"min":{"1min":1.0309999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9410000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.8120000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.341,"15min":1.407},"min":{"1min":0.90900000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.153},{"interface":"front","average":{"1min":1.649,"5min":1.504,"15min":1.5069999999999999},"min":{"1min":1.0640000000000001,"5min":0.79400000000000004,"15min":0.752},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3879999999999999},"last":1.776}]},{"osd":7,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.466,"15min":1.494},"min":{"1min":1.228,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.0609999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4399999999999999,"15min":1.4570000000000001},"min":{"1min":0.96899999999999997,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.419}]}]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44432,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327408,"statfs":{"total":96636764160,"available":96591265792,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.872,"15min":0.90100000000000002},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.32100000000000001},"max":{"1min":1.7010000000000001,"5min":1.74,"15min":1.74},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0449999999999999,"15min":1.048},"min":{"1min":0.46700000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.736,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.80500000000000005}]},{"osd":2,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.1890000000000001,"15min":1.151},"min":{"1min":0.55300000000000005,"5min":0.441,"15min":0.441},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.0700000000000001,"15min":1.0269999999999999},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.371},"max":{"1min":1.968,"5min":1.968,"15min":2.1669999999999998},"last":1.1519999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.1439999999999999,"15min":1.111},"min":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.45200000000000001},"max":{"1min":1.9259999999999999,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.59799999999999998,"5min":0.497,"15min":0.497},"max":{"1min":2.2240000000000002,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.258}]},{"osd":4,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.3879999999999999,"15min":1.3660000000000001},"min":{"1min":0.82199999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":4.4279999999999999,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.284},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.3580000000000001,"15min":1.4319999999999999},"min":{"1min":0.72699999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":1.4810000000000001,"15min":1.55},"min":{"1min":1.0149999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.391},{"interface":"front","average":{"1min":1.6779999999999999,"5min":1.4590000000000001,"15min":1.4930000000000001},"min":{"1min":0.85199999999999998,"5min":0.78600000000000003,"15min":0.747},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":1.464}]},{"osd":6,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.5569999999999999,"15min":1.54},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":4.617,"5min":4.617,"15min":4.617},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.744,"5min":1.595,"15min":1.5840000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.72899999999999998},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.5840000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.694,"5min":1.629,"15min":1.6759999999999999},"min":{"1min":0.89100000000000001,"5min":0.78900000000000003,"15min":0.76700000000000002},"max":{"1min":2.4710000000000001,"5min":2.742,"15min":3.3540000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.6619999999999999,"15min":1.631},"min":{"1min":0.90600000000000003,"5min":0.88100000000000001,"15min":0.77400000000000002},"max":{"1min":3.976,"5min":3.976,"15min":3.976},"last":1.5449999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542245,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45880,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325960,"statfs":{"total":96636764160,"available":96589783040,"internally_reserved":0,"allocated":827392,"data_stored":371792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.079,"15min":1.1120000000000001},"min":{"1min":0.67200000000000004,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6020000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.0740000000000001,"15min":1.0720000000000001},"min":{"1min":0.57599999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4810000000000001,"5min":1.6080000000000001,"15min":2.266},"last":1.167}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1519999999999999,"15min":1.0669999999999999},"min":{"1min":0.65900000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.397,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2330000000000001,"15min":1.2290000000000001},"min":{"1min":0.79800000000000004,"5min":0.70099999999999996,"15min":0.64700000000000002},"max":{"1min":1.6080000000000001,"5min":1.893,"15min":2.1899999999999999},"last":1.2609999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2809999999999999,"15min":1.298},"min":{"1min":0.76000000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.859,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.355},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.278,"15min":1.2230000000000001},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.736,"5min":1.79,"15min":1.79},"last":1.4610000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.3320000000000001,"15min":1.262},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7210000000000001,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.647},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5049999999999999,"15min":1.4610000000000001},"min":{"1min":0.96299999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0459999999999998,"5min":2.2639999999999998,"15min":2.8159999999999998},"last":1.514}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3440000000000001,"15min":1.3240000000000001},"min":{"1min":0.629,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.6240000000000001,"5min":1.984,"15min":3.0190000000000001},"last":1.054},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.405,"15min":1.3759999999999999},"min":{"1min":0.70699999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.756,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.2130000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.504,"15min":1.5309999999999999},"min":{"1min":0.88700000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9550000000000001,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.506,"15min":1.48},"min":{"1min":0.73999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8540000000000001,"5min":2.3079999999999998,"15min":3.3620000000000001},"last":1.3080000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6100000000000001,"15min":1.5720000000000001},"min":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":1.8140000000000001,"5min":2.379,"15min":2.9020000000000001},"last":1.605},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.5620000000000001,"15min":1.512},"min":{"1min":0.76700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.9179999999999999,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:03:24.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:03:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.200+0000 7fd864589640 1 Processor -- start 2023-11-13T11:03:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.200+0000 7fd864589640 1 -- start start 2023-11-13T11:03:25.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd864589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15d9f0 0x7fd85c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd864589640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd85c15ddd0 0x7fd85c15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd864589640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd85c15a610 0x7fd85c0b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd864589640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c15b070 con 0x7fd85c15a610 2023-11-13T11:03:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd864589640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c15aab0 con 0x7fd85c15ddd0 2023-11-13T11:03:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd864589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c15ad90 con 0x7fd85c15d9f0 2023-11-13T11:03:25.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd862b26640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd85c15a610 0x7fd85c0b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd862b26640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd85c15a610 0x7fd85c0b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55358/0 (socket says 172.21.15.16:55358) 2023-11-13T11:03:25.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd862b26640 1 -- 172.21.15.16:0/2460827633 learned_addr learned my addr 172.21.15.16:0/2460827633 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:25.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd861b24640 1 --2- 172.21.15.16:0/2460827633 >> v2:172.21.15.184:3300/0 conn(0x7fd85c15ddd0 0x7fd85c15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.202+0000 7fd862325640 1 --2- 172.21.15.16:0/2460827633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15d9f0 0x7fd85c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.203+0000 7fd862b26640 1 -- 172.21.15.16:0/2460827633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15d9f0 msgr2=0x7fd85c06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.203+0000 7fd862b26640 1 --2- 172.21.15.16:0/2460827633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15d9f0 0x7fd85c06d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.203+0000 7fd862b26640 1 -- 172.21.15.16:0/2460827633 >> v2:172.21.15.184:3300/0 conn(0x7fd85c15ddd0 msgr2=0x7fd85c15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:25.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.203+0000 7fd862b26640 1 --2- 172.21.15.16:0/2460827633 >> v2:172.21.15.184:3300/0 conn(0x7fd85c15ddd0 0x7fd85c15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.203+0000 7fd862b26640 1 -- 172.21.15.16:0/2460827633 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c06cca0 con 0x7fd85c15a610 2023-11-13T11:03:25.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.203+0000 7fd862325640 1 --2- 172.21.15.16:0/2460827633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15d9f0 0x7fd85c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:25.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.204+0000 7fd862b26640 1 --2- 172.21.15.16:0/2460827633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 0x7fd85c0b2630 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd85c071d60 tx=0x7fd85402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30c5201cf8a238d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:25.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.204+0000 7fd861323640 1 -- 172.21.15.16:0/2460827633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85402c9f0 con 0x7fd85c15a610 2023-11-13T11:03:25.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.204+0000 7fd861323640 1 -- 172.21.15.16:0/2460827633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85402cb50 con 0x7fd85c15a610 2023-11-13T11:03:25.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.204+0000 7fd861323640 1 -- 172.21.15.16:0/2460827633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8540343f0 con 0x7fd85c15a610 2023-11-13T11:03:25.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.205+0000 7fd864589640 1 -- 172.21.15.16:0/2460827633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 msgr2=0x7fd85c0b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.205+0000 7fd864589640 1 --2- 172.21.15.16:0/2460827633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 0x7fd85c0b2630 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd85c071d60 tx=0x7fd85402be50 comp rx=0 tx=0).stop 2023-11-13T11:03:25.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.205+0000 7fd864589640 1 -- 172.21.15.16:0/2460827633 shutdown_connections 2023-11-13T11:03:25.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.205+0000 7fd864589640 1 --2- 172.21.15.16:0/2460827633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 0x7fd85c0b2630 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.205+0000 7fd864589640 1 --2- 172.21.15.16:0/2460827633 >> v2:172.21.15.184:3300/0 conn(0x7fd85c15ddd0 0x7fd85c15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.205+0000 7fd864589640 1 --2- 172.21.15.16:0/2460827633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15d9f0 0x7fd85c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.205+0000 7fd864589640 1 -- 172.21.15.16:0/2460827633 >> 172.21.15.16:0/2460827633 conn(0x7fd85c06a190 msgr2=0x7fd85c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.205+0000 7fd864589640 1 -- 172.21.15.16:0/2460827633 shutdown_connections 2023-11-13T11:03:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.205+0000 7fd864589640 1 -- 172.21.15.16:0/2460827633 wait complete. 2023-11-13T11:03:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.207+0000 7fd864589640 1 Processor -- start 2023-11-13T11:03:25.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.207+0000 7fd864589640 1 -- start start 2023-11-13T11:03:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd864589640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 0x7fd85c0b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd864589640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c15d9f0 0x7fd85c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd864589640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15ddd0 0x7fd85c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd864589640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c14e870 con 0x7fd85c15a610 2023-11-13T11:03:25.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd864589640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0b2a90 con 0x7fd85c15d9f0 2023-11-13T11:03:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd864589640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0ca640 con 0x7fd85c15ddd0 2023-11-13T11:03:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd862325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 0x7fd85c0b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd862325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 0x7fd85c0b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55362/0 (socket says 172.21.15.16:55362) 2023-11-13T11:03:25.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd862325640 1 -- 172.21.15.16:0/2122133340 learned_addr learned my addr 172.21.15.16:0/2122133340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:25.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd861b24640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c15d9f0 0x7fd85c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd862325640 1 -- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15ddd0 msgr2=0x7fd85c1471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd862b26640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15ddd0 0x7fd85c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd862325640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15ddd0 0x7fd85c1471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd862325640 1 -- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c15d9f0 msgr2=0x7fd85c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd862325640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c15d9f0 0x7fd85c0ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.208+0000 7fd862325640 1 -- 172.21.15.16:0/2122133340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd854009b30 con 0x7fd85c15a610 2023-11-13T11:03:25.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.209+0000 7fd862325640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 0x7fd85c0b2550 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c7f0 tx=0x7fd85800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:25.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.209+0000 7fd852ffd640 1 -- 172.21.15.16:0/2122133340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85800cea0 con 0x7fd85c15a610 2023-11-13T11:03:25.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.209+0000 7fd852ffd640 1 -- 172.21.15.16:0/2122133340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858007a30 con 0x7fd85c15a610 2023-11-13T11:03:25.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.209+0000 7fd852ffd640 1 -- 172.21.15.16:0/2122133340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858007dc0 con 0x7fd85c15a610 2023-11-13T11:03:25.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.209+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c15d5b0 con 0x7fd85c15a610 2023-11-13T11:03:25.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.209+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c0c9900 con 0x7fd85c15a610 2023-11-13T11:03:25.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.211+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c067090 con 0x7fd85c15a610 2023-11-13T11:03:25.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.211+0000 7fd852ffd640 1 -- 172.21.15.16:0/2122133340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8580113f0 con 0x7fd85c15a610 2023-11-13T11:03:25.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.213+0000 7fd852ffd640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd84006ae40 0x7fd84006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.213+0000 7fd852ffd640 1 -- 172.21.15.16:0/2122133340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd8580898e0 con 0x7fd85c15a610 2023-11-13T11:03:25.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.213+0000 7fd861b24640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd84006ae40 0x7fd84006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.215+0000 7fd861b24640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd84006ae40 0x7fd84006d300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0b15d0 tx=0x7fd84c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:25.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.217+0000 7fd852ffd640 1 -- 172.21.15.16:0/2122133340 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8580547b0 con 0x7fd85c15a610 2023-11-13T11:03:25.407 INFO:tasks.ceph.mds.b.smithi184.stderr:2023-11-13T11:03:25.403+0000 7f8122271640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-13T11:03:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.407+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd85c0c94a0 con 0x7fd84006ae40 2023-11-13T11:03:25.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.428+0000 7fd852ffd640 1 -- 172.21.15.16:0/2122133340 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275714 (secure 0 0 0) 0x7fd84c009d60 con 0x7fd84006ae40 2023-11-13T11:03:25.436 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:03:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd84006ae40 msgr2=0x7fd84006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd84006ae40 0x7fd84006d300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0b15d0 tx=0x7fd84c008040 comp rx=0 tx=0).stop 2023-11-13T11:03:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 msgr2=0x7fd85c0b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 0x7fd85c0b2550 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c7f0 tx=0x7fd85800ccb0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 shutdown_connections 2023-11-13T11:03:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd84006ae40 0x7fd84006d300 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd85c15ddd0 0x7fd85c1471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd85c15d9f0 0x7fd85c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 --2- 172.21.15.16:0/2122133340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd85c15a610 0x7fd85c0b2550 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 >> 172.21.15.16:0/2122133340 conn(0x7fd85c06a190 msgr2=0x7fd85c0caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 shutdown_connections 2023-11-13T11:03:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.434+0000 7fd864589640 1 -- 172.21.15.16:0/2122133340 wait complete. 2023-11-13T11:03:25.458 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-11-13T11:03:24.156436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":70,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":352564,"kb_used_data":7372,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754622156,"statfs":{"total":773094113280,"available":772733087744,"internally_reserved":0,"allocated":7548928,"data_stored":4012902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":353357361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.737938"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058741+0000","last_change":"2023-11-13T11:03:20.473726+0000","last_active":"2023-11-13T11:03:22.058741+0000","last_peered":"2023-11-13T11:03:22.058741+0000","last_clean":"2023-11-13T11:03:22.058741+0000","last_became_active":"2023-11-13T11:03:20.473463+0000","last_became_peered":"2023-11-13T11:03:20.473463+0000","last_unstale":"2023-11-13T11:03:22.058741+0000","last_undegraded":"2023-11-13T11:03:22.058741+0000","last_fullsized":"2023-11-13T11:03:22.058741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:24:37.146283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058815+0000","last_change":"2023-11-13T11:03:20.471989+0000","last_active":"2023-11-13T11:03:22.058815+0000","last_peered":"2023-11-13T11:03:22.058815+0000","last_clean":"2023-11-13T11:03:22.058815+0000","last_became_active":"2023-11-13T11:03:20.470759+0000","last_became_peered":"2023-11-13T11:03:20.470759+0000","last_unstale":"2023-11-13T11:03:22.058815+0000","last_undegraded":"2023-11-13T11:03:22.058815+0000","last_fullsized":"2023-11-13T11:03:22.058815+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:33.483394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.432660+0000","last_change":"2023-11-13T11:03:18.432660+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.432660+0000","last_undegraded":"2023-11-13T11:03:18.432660+0000","last_fullsized":"2023-11-13T11:03:18.432660+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.434034+0000","last_change":"2023-11-13T11:03:18.434034+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.434034+0000","last_undegraded":"2023-11-13T11:03:18.434034+0000","last_fullsized":"2023-11-13T11:03:18.434034+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.432731+0000","last_change":"2023-11-13T11:03:18.432731+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.432731+0000","last_undegraded":"2023-11-13T11:03:18.432731+0000","last_fullsized":"2023-11-13T11:03:18.432731+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.426974+0000","last_change":"2023-11-13T11:03:18.426974+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.426974+0000","last_undegraded":"2023-11-13T11:03:18.426974+0000","last_fullsized":"2023-11-13T11:03:18.426974+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.423052+0000","last_change":"2023-11-13T11:03:18.423052+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.423052+0000","last_undegraded":"2023-11-13T11:03:18.423052+0000","last_fullsized":"2023-11-13T11:03:18.423052+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.424577+0000","last_change":"2023-11-13T11:03:18.424577+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.424577+0000","last_undegraded":"2023-11-13T11:03:18.424577+0000","last_fullsized":"2023-11-13T11:03:18.424577+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.431321+0000","last_change":"2023-11-13T11:03:18.431321+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.431321+0000","last_undegraded":"2023-11-13T11:03:18.431321+0000","last_fullsized":"2023-11-13T11:03:18.431321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428573+0000","last_change":"2023-11-13T11:03:18.428573+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428573+0000","last_undegraded":"2023-11-13T11:03:18.428573+0000","last_fullsized":"2023-11-13T11:03:18.428573+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429859+0000","last_change":"2023-11-13T11:03:18.429859+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429859+0000","last_undegraded":"2023-11-13T11:03:18.429859+0000","last_fullsized":"2023-11-13T11:03:18.429859+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.421232+0000","last_change":"2023-11-13T11:03:18.421232+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.421232+0000","last_undegraded":"2023-11-13T11:03:18.421232+0000","last_fullsized":"2023-11-13T11:03:18.421232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428911+0000","last_change":"2023-11-13T11:03:18.428911+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428911+0000","last_undegraded":"2023-11-13T11:03:18.428911+0000","last_fullsized":"2023-11-13T11:03:18.428911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.425857+0000","last_change":"2023-11-13T11:03:18.425857+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.425857+0000","last_undegraded":"2023-11-13T11:03:18.425857+0000","last_fullsized":"2023-11-13T11:03:18.425857+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429650+0000","last_change":"2023-11-13T11:03:18.429650+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429650+0000","last_undegraded":"2023-11-13T11:03:18.429650+0000","last_fullsized":"2023-11-13T11:03:18.429650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.433832+0000","last_change":"2023-11-13T11:03:18.433832+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.433832+0000","last_undegraded":"2023-11-13T11:03:18.433832+0000","last_fullsized":"2023-11-13T11:03:18.433832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428094+0000","last_change":"2023-11-13T11:03:18.428094+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428094+0000","last_undegraded":"2023-11-13T11:03:18.428094+0000","last_fullsized":"2023-11-13T11:03:18.428094+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.433356+0000","last_change":"2023-11-13T11:03:18.433356+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.433356+0000","last_undegraded":"2023-11-13T11:03:18.433356+0000","last_fullsized":"2023-11-13T11:03:18.433356+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429355+0000","last_change":"2023-11-13T11:03:18.429355+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429355+0000","last_undegraded":"2023-11-13T11:03:18.429355+0000","last_fullsized":"2023-11-13T11:03:18.429355+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.430231+0000","last_change":"2023-11-13T11:03:18.430231+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.430231+0000","last_undegraded":"2023-11-13T11:03:18.430231+0000","last_fullsized":"2023-11-13T11:03:18.430231+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428157+0000","last_change":"2023-11-13T11:03:18.428157+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428157+0000","last_undegraded":"2023-11-13T11:03:18.428157+0000","last_fullsized":"2023-11-13T11:03:18.428157+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059038+0000","last_change":"2023-11-13T11:03:19.469381+0000","last_active":"2023-11-13T11:03:22.059038+0000","last_peered":"2023-11-13T11:03:22.059038+0000","last_clean":"2023-11-13T11:03:22.059038+0000","last_became_active":"2023-11-13T11:03:19.468768+0000","last_became_peered":"2023-11-13T11:03:19.468768+0000","last_unstale":"2023-11-13T11:03:22.059038+0000","last_undegraded":"2023-11-13T11:03:22.059038+0000","last_fullsized":"2023-11-13T11:03:22.059038+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:37:23.154386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.424388+0000","last_change":"2023-11-13T11:03:18.424388+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.424388+0000","last_undegraded":"2023-11-13T11:03:18.424388+0000","last_fullsized":"2023-11-13T11:03:18.424388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.433296+0000","last_change":"2023-11-13T11:03:18.433296+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.433296+0000","last_undegraded":"2023-11-13T11:03:18.433296+0000","last_fullsized":"2023-11-13T11:03:18.433296+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.423667+0000","last_change":"2023-11-13T11:03:18.423667+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.423667+0000","last_undegraded":"2023-11-13T11:03:18.423667+0000","last_fullsized":"2023-11-13T11:03:18.423667+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428757+0000","last_change":"2023-11-13T11:03:18.428757+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428757+0000","last_undegraded":"2023-11-13T11:03:18.428757+0000","last_fullsized":"2023-11-13T11:03:18.428757+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429504+0000","last_change":"2023-11-13T11:03:18.429504+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429504+0000","last_undegraded":"2023-11-13T11:03:18.429504+0000","last_fullsized":"2023-11-13T11:03:18.429504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.426053+0000","last_change":"2023-11-13T11:03:18.426053+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.426053+0000","last_undegraded":"2023-11-13T11:03:18.426053+0000","last_fullsized":"2023-11-13T11:03:18.426053+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059174+0000","last_change":"2023-11-13T11:03:19.468809+0000","last_active":"2023-11-13T11:03:22.059174+0000","last_peered":"2023-11-13T11:03:22.059174+0000","last_clean":"2023-11-13T11:03:22.059174+0000","last_became_active":"2023-11-13T11:03:19.467750+0000","last_became_peered":"2023-11-13T11:03:19.467750+0000","last_unstale":"2023-11-13T11:03:22.059174+0000","last_undegraded":"2023-11-13T11:03:22.059174+0000","last_fullsized":"2023-11-13T11:03:22.059174+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:45:50.416913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.421233+0000","last_change":"2023-11-13T11:03:18.421233+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.421233+0000","last_undegraded":"2023-11-13T11:03:18.421233+0000","last_fullsized":"2023-11-13T11:03:18.421233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.431809+0000","last_change":"2023-11-13T11:03:18.431809+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.431809+0000","last_undegraded":"2023-11-13T11:03:18.431809+0000","last_fullsized":"2023-11-13T11:03:18.431809+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058922+0000","last_change":"2023-11-13T11:03:19.468080+0000","last_active":"2023-11-13T11:03:22.058922+0000","last_peered":"2023-11-13T11:03:22.058922+0000","last_clean":"2023-11-13T11:03:22.058922+0000","last_became_active":"2023-11-13T11:03:19.467716+0000","last_became_peered":"2023-11-13T11:03:19.467716+0000","last_unstale":"2023-11-13T11:03:22.058922+0000","last_undegraded":"2023-11-13T11:03:22.058922+0000","last_fullsized":"2023-11-13T11:03:22.058922+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:57:39.710883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.424719+0000","last_change":"2023-11-13T11:03:18.424719+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.424719+0000","last_undegraded":"2023-11-13T11:03:18.424719+0000","last_fullsized":"2023-11-13T11:03:18.424719+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.423491+0000","last_change":"2023-11-13T11:03:18.423491+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.423491+0000","last_undegraded":"2023-11-13T11:03:18.423491+0000","last_fullsized":"2023-11-13T11:03:18.423491+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":205,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-13T11:03:18.434428+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:03:18.434428+0000","last_peered":"2023-11-13T11:03:18.434428+0000","last_clean":"2023-11-13T11:03:18.434428+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:03:18.434428+0000","last_undegraded":"2023-11-13T11:03:18.434428+0000","last_fullsized":"2023-11-13T11:03:18.434428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44432,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327408,"statfs":{"total":96636764160,"available":96591265792,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1060000000000001,"15min":1.1259999999999999},"min":{"1min":0.73899999999999999,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.706,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.2250000000000001,"15min":1.1379999999999999},"min":{"1min":0.82199999999999995,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.498,"5min":1.865,"15min":2.2879999999999998},"last":1.014}]},{"osd":1,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.252,"15min":1.1799999999999999},"min":{"1min":0.90300000000000002,"5min":0.64000000000000001,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.089},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.2889999999999999},"min":{"1min":0.67700000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.649,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.1679999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.319,"15min":1.276},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.534,"5min":2.153,"15min":2.1680000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.383,"15min":1.3460000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.6100000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.3700000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.429,"15min":1.3779999999999999},"min":{"1min":0.81799999999999995,"5min":0.76300000000000001,"15min":0.63800000000000001},"max":{"1min":1.657,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.458,"15min":1.4379999999999999},"min":{"1min":0.99299999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.762,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.21}]},{"osd":4,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.456,"15min":1.552},"min":{"1min":0.94899999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.782,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.286},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.512,"15min":1.5740000000000001},"min":{"1min":1.1599999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.9419999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.448}]},{"osd":5,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.6060000000000001,"15min":1.6399999999999999},"min":{"1min":0.94499999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.8200000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.498},{"interface":"front","average":{"1min":1.625,"5min":1.75,"15min":1.7490000000000001},"min":{"1min":1.4650000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":1.865,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.6859999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.6930000000000001,"15min":1.645},"min":{"1min":1.2849999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0640000000000001,"5min":2.657,"15min":2.657},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.641,"15min":1.6379999999999999},"min":{"1min":1.1399999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1349999999999998,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.575}]}]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45012,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326828,"statfs":{"total":96636764160,"available":96590671872,"internally_reserved":0,"allocated":1384448,"data_stored":941850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.093,"15min":1.016},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7529999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.284},{"interface":"front","average":{"1min":1.179,"5min":1.2190000000000001,"15min":1.202},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67800000000000005},"max":{"1min":1.639,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.0760000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.306,"15min":1.2350000000000001},"min":{"1min":0.79100000000000004,"5min":0.72399999999999998,"15min":0.65400000000000003},"max":{"1min":1.73,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.282,"15min":1.2450000000000001},"min":{"1min":0.875,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.115}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.389,"15min":1.2649999999999999},"min":{"1min":0.81200000000000006,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":4.4500000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.262},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.3100000000000001,"15min":1.2370000000000001},"min":{"1min":0.78900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5760000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.0089999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.387,"15min":1.391},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8300000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.23},{"interface":"front","average":{"1min":1.53,"5min":1.5169999999999999,"15min":1.4870000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.80700000000000005},"max":{"1min":4.6959999999999997,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":1.212}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3700000000000001,"15min":1.329},"min":{"1min":0.93500000000000005,"5min":0.86599999999999999,"15min":0.55500000000000005},"max":{"1min":1.5800000000000001,"5min":3.484,"15min":3.484},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.518,"15min":1.53},"min":{"1min":1.093,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.4430000000000001,"5min":3.653,"15min":3.653},"last":0.95299999999999996}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.5800000000000001,"15min":1.6040000000000001},"min":{"1min":1.0800000000000001,"5min":0.92000000000000004,"15min":0.78000000000000003},"max":{"1min":4.8760000000000003,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.538,"15min":1.5189999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.61799999999999999},"max":{"1min":1.7609999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.177}]},{"osd":7,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.645,"15min":1.6659999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.1349999999999998,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.744,"5min":1.7330000000000001,"15min":1.714},"min":{"1min":1.2490000000000001,"5min":1.1200000000000001,"15min":0.69499999999999995},"max":{"1min":4.7919999999999998,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":1.377}]}]},{"osd":4,"up_from":15,"seq":64424509541,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54160,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317680,"statfs":{"total":96636764160,"available":96581304320,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.153,"15min":1.089},"min":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":1.986,"15min":1.986},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.145,"15min":1.085},"min":{"1min":0.69199999999999995,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.83799999999999997}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2470000000000001,"15min":1.163},"min":{"1min":0.76200000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.774,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.05},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.26},"min":{"1min":0.79300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":1.877,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.4650000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.298,"15min":1.242},"min":{"1min":0.93300000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.575,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.347,"15min":1.349},"min":{"1min":0.91100000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.72,"5min":1.952,"15min":1.952},"last":1.637}]},{"osd":3,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.488,"15min":1.421},"min":{"1min":1.1080000000000001,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.786,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.841},{"interface":"front","average":{"1min":1.379,"5min":1.3149999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.841,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.5740000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.478,"15min":1.4910000000000001},"min":{"1min":0.96399999999999997,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.778,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.996},{"interface":"front","average":{"1min":1.323,"5min":1.371,"15min":1.3939999999999999},"min":{"1min":0.81599999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":2.0529999999999999,"5min":2.089,"15min":2.089},"last":1.7809999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.621,"15min":1.6220000000000001},"min":{"1min":1.0600000000000001,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":1.7769999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":2.1030000000000002},{"interface":"front","average":{"1min":1.401,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.92400000000000004,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":1.873,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":2.0310000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.5369999999999999,"15min":1.5820000000000001},"min":{"1min":1.05,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":2.1480000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.9039999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.599,"15min":1.609},"min":{"1min":1.0189999999999999,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":1.931,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":2.1499999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40272,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331568,"statfs":{"total":96636764160,"available":96595525632,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.628,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.282,"5min":1.867,"15min":1.867},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.022,"15min":1.026},"min":{"1min":0.65800000000000003,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.819,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.032,"15min":1.0489999999999999},"min":{"1min":0.625,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.8089999999999999,"5min":1.966,"15min":3.71},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.296,"5min":1.1240000000000001,"15min":1.103},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.179,"15min":1.1040000000000001},"min":{"1min":0.72899999999999998,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":2.0019999999999998,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.24,"5min":1.171,"15min":1.2829999999999999},"min":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7809999999999999,"5min":1.952,"15min":3.5710000000000002},"last":1.143}]},{"osd":4,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.89200000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.7509999999999999,"5min":2.411,"15min":3.5030000000000001},"last":1.278},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3879999999999999,"15min":1.355},"min":{"1min":1.1259999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.831,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.5840000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4179999999999999,"15min":1.4750000000000001},"min":{"1min":1.151,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.99,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.454,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0649999999999999,"5min":2.673,"15min":3.3809999999999998},"last":1.45}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.49},"min":{"1min":1.1479999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.944,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.206},{"interface":"front","average":{"1min":1.546,"5min":1.4870000000000001,"15min":1.5840000000000001},"min":{"1min":1.1060000000000001,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":2.093,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.3380000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5609999999999999,"15min":1.609},"min":{"1min":0.84799999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1339999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.631},{"interface":"front","average":{"1min":1.649,"5min":1.6040000000000001,"15min":1.5620000000000001},"min":{"1min":1.1359999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.4089999999999998,"5min":2.75,"15min":2.75},"last":1.391}]}]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50004,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321836,"statfs":{"total":96636764160,"available":96585560064,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94199999999999995,"15min":0.93100000000000005},"min":{"1min":0.55700000000000005,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.089,"5min":1.0229999999999999,"15min":0.93400000000000005},"min":{"1min":0.54400000000000004,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":2.9460000000000002,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":1.7330000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0309999999999999,"15min":0.96299999999999997},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.376},"max":{"1min":3.0720000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.361,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.68100000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.3380000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.0920000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.216,"15min":1.2430000000000001},"min":{"1min":0.71599999999999997,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2549999999999999,"15min":1.2749999999999999},"min":{"1min":0.72699999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.4060000000000001,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.456}]},{"osd":4,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4359999999999999,"15min":1.4730000000000001},"min":{"1min":0.89900000000000002,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":2.633,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.393,"15min":1.409},"min":{"1min":0.80000000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":1.5189999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.4690000000000001,"15min":1.482},"min":{"1min":1.034,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.1680000000000001,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":1.165},{"interface":"front","average":{"1min":1.623,"5min":1.591,"15min":1.5680000000000001},"min":{"1min":0.97099999999999997,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.7509999999999999,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.6699999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5189999999999999,"15min":1.55},"min":{"1min":0.89500000000000002,"5min":0.80900000000000005,"15min":0.76500000000000001},"max":{"1min":3.395,"5min":3.395,"15min":4.7569999999999997},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6120000000000001,"15min":1.599},"min":{"1min":1.2130000000000001,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.4039999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.5700000000000001,"15min":1.607},"min":{"1min":1.0569999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":3.2429999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.694,"5min":1.6479999999999999,"15min":1.6259999999999999},"min":{"1min":1.2090000000000001,"5min":0.91200000000000003,"15min":0.82799999999999996},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":1.5760000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28372,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343468,"statfs":{"total":96636764160,"available":96607711232,"internally_reserved":0,"allocated":1384448,"data_stored":941850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85699999999999998,"15min":0.94899999999999995},"min":{"1min":0.495,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.4910000000000001,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":1.03},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.97399999999999998,"15min":0.95699999999999996},"min":{"1min":0.623,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2689999999999999,"5min":1.869,"15min":1.869},"last":0.98899999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.0509999999999999,"15min":1},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.129},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0680000000000001,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":1.111}]},{"osd":3,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.204,"15min":1.175},"min":{"1min":0.68600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.177},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.57799999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.0669999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.321,"15min":1.3129999999999999},"min":{"1min":0.76200000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":1.212},{"interface":"front","average":{"1min":1.536,"5min":1.4179999999999999,"15min":1.3859999999999999},"min":{"1min":0.69599999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.0299999999999998,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.8620000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4159999999999999,"15min":1.4139999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.0750000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.9330000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.387,"15min":1.4079999999999999},"min":{"1min":1.0309999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9410000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.8120000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.341,"15min":1.407},"min":{"1min":0.90900000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.153},{"interface":"front","average":{"1min":1.649,"5min":1.504,"15min":1.5069999999999999},"min":{"1min":1.0640000000000001,"5min":0.79400000000000004,"15min":0.752},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3879999999999999},"last":1.776}]},{"osd":7,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.466,"15min":1.494},"min":{"1min":1.228,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.0609999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4399999999999999,"15min":1.4570000000000001},"min":{"1min":0.96899999999999997,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.419}]}]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44432,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327408,"statfs":{"total":96636764160,"available":96591265792,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.872,"15min":0.90100000000000002},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.32100000000000001},"max":{"1min":1.7010000000000001,"5min":1.74,"15min":1.74},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0449999999999999,"15min":1.048},"min":{"1min":0.46700000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.736,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.80500000000000005}]},{"osd":2,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.1890000000000001,"15min":1.151},"min":{"1min":0.55300000000000005,"5min":0.441,"15min":0.441},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.0700000000000001,"15min":1.0269999999999999},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.371},"max":{"1min":1.968,"5min":1.968,"15min":2.1669999999999998},"last":1.1519999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.1439999999999999,"15min":1.111},"min":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.45200000000000001},"max":{"1min":1.9259999999999999,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.59799999999999998,"5min":0.497,"15min":0.497},"max":{"1min":2.2240000000000002,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.258}]},{"osd":4,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.3879999999999999,"15min":1.3660000000000001},"min":{"1min":0.82199999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":4.4279999999999999,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.284},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.3580000000000001,"15min":1.4319999999999999},"min":{"1min":0.72699999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":1.4810000000000001,"15min":1.55},"min":{"1min":1.0149999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.391},{"interface":"front","average":{"1min":1.6779999999999999,"5min":1.4590000000000001,"15min":1.4930000000000001},"min":{"1min":0.85199999999999998,"5min":0.78600000000000003,"15min":0.747},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":1.464}]},{"osd":6,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.5569999999999999,"15min":1.54},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":4.617,"5min":4.617,"15min":4.617},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.744,"5min":1.595,"15min":1.5840000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.72899999999999998},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.5840000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.694,"5min":1.629,"15min":1.6759999999999999},"min":{"1min":0.89100000000000001,"5min":0.78900000000000003,"15min":0.76700000000000002},"max":{"1min":2.4710000000000001,"5min":2.742,"15min":3.3540000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.6619999999999999,"15min":1.631},"min":{"1min":0.90600000000000003,"5min":0.88100000000000001,"15min":0.77400000000000002},"max":{"1min":3.976,"5min":3.976,"15min":3.976},"last":1.5449999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542245,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45880,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325960,"statfs":{"total":96636764160,"available":96589783040,"internally_reserved":0,"allocated":827392,"data_stored":371792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.079,"15min":1.1120000000000001},"min":{"1min":0.67200000000000004,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6020000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.0740000000000001,"15min":1.0720000000000001},"min":{"1min":0.57599999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4810000000000001,"5min":1.6080000000000001,"15min":2.266},"last":1.167}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1519999999999999,"15min":1.0669999999999999},"min":{"1min":0.65900000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.397,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2330000000000001,"15min":1.2290000000000001},"min":{"1min":0.79800000000000004,"5min":0.70099999999999996,"15min":0.64700000000000002},"max":{"1min":1.6080000000000001,"5min":1.893,"15min":2.1899999999999999},"last":1.2609999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2809999999999999,"15min":1.298},"min":{"1min":0.76000000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.859,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.355},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.278,"15min":1.2230000000000001},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.736,"5min":1.79,"15min":1.79},"last":1.4610000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.3320000000000001,"15min":1.262},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7210000000000001,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.647},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5049999999999999,"15min":1.4610000000000001},"min":{"1min":0.96299999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0459999999999998,"5min":2.2639999999999998,"15min":2.8159999999999998},"last":1.514}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3440000000000001,"15min":1.3240000000000001},"min":{"1min":0.629,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.6240000000000001,"5min":1.984,"15min":3.0190000000000001},"last":1.054},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.405,"15min":1.3759999999999999},"min":{"1min":0.70699999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.756,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.2130000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.504,"15min":1.5309999999999999},"min":{"1min":0.88700000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9550000000000001,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.506,"15min":1.48},"min":{"1min":0.73999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8540000000000001,"5min":2.3079999999999998,"15min":3.3620000000000001},"last":1.3080000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6100000000000001,"15min":1.5720000000000001},"min":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":1.8140000000000001,"5min":2.379,"15min":2.9020000000000001},"last":1.605},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.5620000000000001,"15min":1.512},"min":{"1min":0.76700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.9179999999999999,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:03:25.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:03:25.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.721+0000 7f21d0b33640 1 Processor -- start 2023-11-13T11:03:25.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.721+0000 7f21d0b33640 1 -- start start 2023-11-13T11:03:25.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.723+0000 7f21d0b33640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21cc15d9f0 0x7f21cc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.723+0000 7f21d0b33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc15e240 0x7f21cc0b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.723+0000 7f21d0b33640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc15a7e0 con 0x7f21cc15ddd0 2023-11-13T11:03:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.723+0000 7f21d0b33640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc15b070 con 0x7f21cc15d9f0 2023-11-13T11:03:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.723+0000 7f21d0b33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc15aab0 con 0x7f21cc15e240 2023-11-13T11:03:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.723+0000 7f21c3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f21cc15ddd0 0x7f21cc15a0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41066/0 (socket says 172.21.15.16:41066) 2023-11-13T11:03:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.724+0000 7f21c3fff640 1 -- 172.21.15.16:0/3954947822 learned_addr learned my addr 172.21.15.16:0/3954947822 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:25.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.724+0000 7f21ca59c640 1 --2- 172.21.15.16:0/3954947822 >> v2:172.21.15.184:3300/0 conn(0x7f21cc15d9f0 0x7f21cc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.724+0000 7f21cad9d640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc15e240 0x7f21cc0b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.724+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479903502 0 0) 0x7f21cc15a7e0 con 0x7f21cc15ddd0 2023-11-13T11:03:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.724+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21b8003520 con 0x7f21cc15ddd0 2023-11-13T11:03:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.724+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2669335159 0 0) 0x7f21b8003520 con 0x7f21cc15ddd0 2023-11-13T11:03:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.724+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc15e240 msgr2=0x7f21cc0b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:25.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc15e240 0x7f21cc0b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 >> v2:172.21.15.184:3300/0 conn(0x7f21cc15d9f0 msgr2=0x7f21cc06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:25.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 --2- 172.21.15.16:0/3954947822 >> v2:172.21.15.184:3300/0 conn(0x7f21cc15d9f0 0x7f21cc06d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc15ad90 con 0x7f21cc15ddd0 2023-11-13T11:03:25.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f21b4001670 con 0x7f21cc15ddd0 2023-11-13T11:03:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 >> v1:172.21.15.16:6789/0 conn(0x7f21cc15ddd0 legacy=0x7f21cc15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc15e240 0x7f21b8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21b8003ef0 0x7f21b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b8004d00 0x7f21b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc15b070 con 0x7f21b8004d00 2023-11-13T11:03:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc15aab0 con 0x7f21b8003ef0 2023-11-13T11:03:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b800b9d0 con 0x7f21cc15e240 2023-11-13T11:03:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21cad9d640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b8004d00 0x7f21b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21ca59c640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc15e240 0x7f21b8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21cad9d640 1 -- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc15e240 msgr2=0x7f21b8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21cad9d640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc15e240 0x7f21b8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21cad9d640 1 -- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21b8003ef0 msgr2=0x7f21b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21cad9d640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21b8003ef0 0x7f21b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21cad9d640 1 -- 172.21.15.16:0/3954947822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b8003520 con 0x7f21b8004d00 2023-11-13T11:03:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.725+0000 7f21ca59c640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc15e240 0x7f21b8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.726+0000 7f21cad9d640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b8004d00 0x7f21b8005180 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009fc0 tx=0x7f21b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a544c4f5fa1881 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.727+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c402cbd0 con 0x7f21b8004d00 2023-11-13T11:03:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.727+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c400f040 con 0x7f21b8004d00 2023-11-13T11:03:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.727+0000 7f21c9d9b640 1 -- 172.21.15.16:0/3954947822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4034440 con 0x7f21b8004d00 2023-11-13T11:03:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.727+0000 7f21d0b33640 1 -- 172.21.15.16:0/3954947822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b8004d00 msgr2=0x7f21b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.727+0000 7f21d0b33640 1 --2- 172.21.15.16:0/3954947822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b8004d00 0x7f21b8005180 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009fc0 tx=0x7f21b80057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.727+0000 7f21cad9d640 1 -- 172.21.15.16:0/3954947822 reap_dead start 2023-11-13T11:03:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.727+0000 7f21d0b33640 1 -- 172.21.15.16:0/3954947822 shutdown_connections 2023-11-13T11:03:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.727+0000 7f21d0b33640 1 -- 172.21.15.16:0/3954947822 >> 172.21.15.16:0/3954947822 conn(0x7f21cc06a190 msgr2=0x7f21cc06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.728+0000 7f21d0b33640 1 -- 172.21.15.16:0/3954947822 shutdown_connections 2023-11-13T11:03:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.728+0000 7f21d0b33640 1 -- 172.21.15.16:0/3954947822 wait complete. 2023-11-13T11:03:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.729+0000 7f21d0b33640 1 Processor -- start 2023-11-13T11:03:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.729+0000 7f21d0b33640 1 -- start start 2023-11-13T11:03:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.730+0000 7f21d0b33640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21b8004d00 0x7f21cc15f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.730+0000 7f21d0b33640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc160820 0x7f21cc165100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.730+0000 7f21d0b33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc161230 0x7f21cc152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.730+0000 7f21d0b33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc15f780 con 0x7f21cc161230 2023-11-13T11:03:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.730+0000 7f21d0b33640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc15fa60 con 0x7f21b8004d00 2023-11-13T11:03:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.730+0000 7f21d0b33640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc15fd40 con 0x7f21cc160820 2023-11-13T11:03:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc161230 0x7f21cc152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc161230 0x7f21cc152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55388/0 (socket says 172.21.15.16:55388) 2023-11-13T11:03:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21cad9d640 1 -- 172.21.15.16:0/4284651596 learned_addr learned my addr 172.21.15.16:0/4284651596 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21ca59c640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21b8004d00 0x7f21cc15f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21c3fff640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc160820 0x7f21cc165100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21cad9d640 1 -- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc160820 msgr2=0x7f21cc165100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21cad9d640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc160820 0x7f21cc165100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21cad9d640 1 -- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21b8004d00 msgr2=0x7f21cc15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21cad9d640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21b8004d00 0x7f21cc15f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.731+0000 7f21cad9d640 1 -- 172.21.15.16:0/4284651596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc160020 con 0x7f21cc161230 2023-11-13T11:03:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.732+0000 7f21cad9d640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc161230 0x7f21cc152f70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f21c4002230 tx=0x7f21c4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.732+0000 7f21c2ffd640 1 -- 172.21.15.16:0/4284651596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4034d50 con 0x7f21cc161230 2023-11-13T11:03:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.732+0000 7f21c2ffd640 1 -- 172.21.15.16:0/4284651596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c402de60 con 0x7f21cc161230 2023-11-13T11:03:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.732+0000 7f21c2ffd640 1 -- 172.21.15.16:0/4284651596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c403e680 con 0x7f21cc161230 2023-11-13T11:03:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.732+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc160300 con 0x7f21cc161230 2023-11-13T11:03:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.732+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc161690 con 0x7f21cc161230 2023-11-13T11:03:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.734+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc1604f0 con 0x7f21cc161230 2023-11-13T11:03:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.734+0000 7f21c2ffd640 1 -- 172.21.15.16:0/4284651596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21c403e7e0 con 0x7f21cc161230 2023-11-13T11:03:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.736+0000 7f21c2ffd640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f21ac06ae70 0x7f21ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.736+0000 7f21c2ffd640 1 -- 172.21.15.16:0/4284651596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f21c40ab930 con 0x7f21cc161230 2023-11-13T11:03:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.737+0000 7f21ca59c640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f21ac06ae70 0x7f21ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.738+0000 7f21ca59c640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f21ac06ae70 0x7f21ac06d330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009b20 tx=0x7f21bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.741+0000 7f21c2ffd640 1 -- 172.21.15.16:0/4284651596 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21c4076800 con 0x7f21cc161230 2023-11-13T11:03:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.934+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21cc15fbd0 con 0x7f21ac06ae70 2023-11-13T11:03:25.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.956+0000 7f21c2ffd640 1 -- 172.21.15.16:0/4284651596 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275714 (secure 0 0 0) 0x7f21cc15fbd0 con 0x7f21ac06ae70 2023-11-13T11:03:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:25.966 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:03:25.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f21ac06ae70 msgr2=0x7f21ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f21ac06ae70 0x7f21ac06d330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009b20 tx=0x7f21bc008040 comp rx=0 tx=0).stop 2023-11-13T11:03:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc161230 msgr2=0x7f21cc152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc161230 0x7f21cc152f70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f21c4002230 tx=0x7f21c4009750 comp rx=0 tx=0).stop 2023-11-13T11:03:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 shutdown_connections 2023-11-13T11:03:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f21ac06ae70 0x7f21ac06d330 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc161230 0x7f21cc152f70 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21cc160820 0x7f21cc165100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 --2- 172.21.15.16:0/4284651596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21b8004d00 0x7f21cc15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:25.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 >> 172.21.15.16:0/4284651596 conn(0x7f21cc06a190 msgr2=0x7f21cc15c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:25.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 shutdown_connections 2023-11-13T11:03:25.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:25.962+0000 7f21d0b33640 1 -- 172.21.15.16:0/4284651596 wait complete. 2023-11-13T11:03:25.988 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-11-13T11:03:24.156436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":70,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":352564,"kb_used_data":7372,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754622156,"statfs":{"total":773094113280,"available":772733087744,"internally_reserved":0,"allocated":7548928,"data_stored":4012902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":353357361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.737938"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058741+0000","last_change":"2023-11-13T11:03:20.473726+0000","last_active":"2023-11-13T11:03:22.058741+0000","last_peered":"2023-11-13T11:03:22.058741+0000","last_clean":"2023-11-13T11:03:22.058741+0000","last_became_active":"2023-11-13T11:03:20.473463+0000","last_became_peered":"2023-11-13T11:03:20.473463+0000","last_unstale":"2023-11-13T11:03:22.058741+0000","last_undegraded":"2023-11-13T11:03:22.058741+0000","last_fullsized":"2023-11-13T11:03:22.058741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:24:37.146283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058815+0000","last_change":"2023-11-13T11:03:20.471989+0000","last_active":"2023-11-13T11:03:22.058815+0000","last_peered":"2023-11-13T11:03:22.058815+0000","last_clean":"2023-11-13T11:03:22.058815+0000","last_became_active":"2023-11-13T11:03:20.470759+0000","last_became_peered":"2023-11-13T11:03:20.470759+0000","last_unstale":"2023-11-13T11:03:22.058815+0000","last_undegraded":"2023-11-13T11:03:22.058815+0000","last_fullsized":"2023-11-13T11:03:22.058815+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:33.483394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.432660+0000","last_change":"2023-11-13T11:03:18.432660+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.432660+0000","last_undegraded":"2023-11-13T11:03:18.432660+0000","last_fullsized":"2023-11-13T11:03:18.432660+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.434034+0000","last_change":"2023-11-13T11:03:18.434034+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.434034+0000","last_undegraded":"2023-11-13T11:03:18.434034+0000","last_fullsized":"2023-11-13T11:03:18.434034+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.432731+0000","last_change":"2023-11-13T11:03:18.432731+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.432731+0000","last_undegraded":"2023-11-13T11:03:18.432731+0000","last_fullsized":"2023-11-13T11:03:18.432731+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.426974+0000","last_change":"2023-11-13T11:03:18.426974+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.426974+0000","last_undegraded":"2023-11-13T11:03:18.426974+0000","last_fullsized":"2023-11-13T11:03:18.426974+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.423052+0000","last_change":"2023-11-13T11:03:18.423052+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.423052+0000","last_undegraded":"2023-11-13T11:03:18.423052+0000","last_fullsized":"2023-11-13T11:03:18.423052+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.424577+0000","last_change":"2023-11-13T11:03:18.424577+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.424577+0000","last_undegraded":"2023-11-13T11:03:18.424577+0000","last_fullsized":"2023-11-13T11:03:18.424577+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.431321+0000","last_change":"2023-11-13T11:03:18.431321+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.431321+0000","last_undegraded":"2023-11-13T11:03:18.431321+0000","last_fullsized":"2023-11-13T11:03:18.431321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428573+0000","last_change":"2023-11-13T11:03:18.428573+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428573+0000","last_undegraded":"2023-11-13T11:03:18.428573+0000","last_fullsized":"2023-11-13T11:03:18.428573+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429859+0000","last_change":"2023-11-13T11:03:18.429859+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429859+0000","last_undegraded":"2023-11-13T11:03:18.429859+0000","last_fullsized":"2023-11-13T11:03:18.429859+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.421232+0000","last_change":"2023-11-13T11:03:18.421232+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.421232+0000","last_undegraded":"2023-11-13T11:03:18.421232+0000","last_fullsized":"2023-11-13T11:03:18.421232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428911+0000","last_change":"2023-11-13T11:03:18.428911+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428911+0000","last_undegraded":"2023-11-13T11:03:18.428911+0000","last_fullsized":"2023-11-13T11:03:18.428911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.425857+0000","last_change":"2023-11-13T11:03:18.425857+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.425857+0000","last_undegraded":"2023-11-13T11:03:18.425857+0000","last_fullsized":"2023-11-13T11:03:18.425857+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429650+0000","last_change":"2023-11-13T11:03:18.429650+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429650+0000","last_undegraded":"2023-11-13T11:03:18.429650+0000","last_fullsized":"2023-11-13T11:03:18.429650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.433832+0000","last_change":"2023-11-13T11:03:18.433832+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.433832+0000","last_undegraded":"2023-11-13T11:03:18.433832+0000","last_fullsized":"2023-11-13T11:03:18.433832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428094+0000","last_change":"2023-11-13T11:03:18.428094+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428094+0000","last_undegraded":"2023-11-13T11:03:18.428094+0000","last_fullsized":"2023-11-13T11:03:18.428094+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.433356+0000","last_change":"2023-11-13T11:03:18.433356+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.433356+0000","last_undegraded":"2023-11-13T11:03:18.433356+0000","last_fullsized":"2023-11-13T11:03:18.433356+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429355+0000","last_change":"2023-11-13T11:03:18.429355+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429355+0000","last_undegraded":"2023-11-13T11:03:18.429355+0000","last_fullsized":"2023-11-13T11:03:18.429355+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.430231+0000","last_change":"2023-11-13T11:03:18.430231+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.430231+0000","last_undegraded":"2023-11-13T11:03:18.430231+0000","last_fullsized":"2023-11-13T11:03:18.430231+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428157+0000","last_change":"2023-11-13T11:03:18.428157+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428157+0000","last_undegraded":"2023-11-13T11:03:18.428157+0000","last_fullsized":"2023-11-13T11:03:18.428157+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059038+0000","last_change":"2023-11-13T11:03:19.469381+0000","last_active":"2023-11-13T11:03:22.059038+0000","last_peered":"2023-11-13T11:03:22.059038+0000","last_clean":"2023-11-13T11:03:22.059038+0000","last_became_active":"2023-11-13T11:03:19.468768+0000","last_became_peered":"2023-11-13T11:03:19.468768+0000","last_unstale":"2023-11-13T11:03:22.059038+0000","last_undegraded":"2023-11-13T11:03:22.059038+0000","last_fullsized":"2023-11-13T11:03:22.059038+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:37:23.154386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.424388+0000","last_change":"2023-11-13T11:03:18.424388+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.424388+0000","last_undegraded":"2023-11-13T11:03:18.424388+0000","last_fullsized":"2023-11-13T11:03:18.424388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.433296+0000","last_change":"2023-11-13T11:03:18.433296+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.433296+0000","last_undegraded":"2023-11-13T11:03:18.433296+0000","last_fullsized":"2023-11-13T11:03:18.433296+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.423667+0000","last_change":"2023-11-13T11:03:18.423667+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.423667+0000","last_undegraded":"2023-11-13T11:03:18.423667+0000","last_fullsized":"2023-11-13T11:03:18.423667+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.428757+0000","last_change":"2023-11-13T11:03:18.428757+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.428757+0000","last_undegraded":"2023-11-13T11:03:18.428757+0000","last_fullsized":"2023-11-13T11:03:18.428757+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.429504+0000","last_change":"2023-11-13T11:03:18.429504+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.429504+0000","last_undegraded":"2023-11-13T11:03:18.429504+0000","last_fullsized":"2023-11-13T11:03:18.429504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.426053+0000","last_change":"2023-11-13T11:03:18.426053+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.426053+0000","last_undegraded":"2023-11-13T11:03:18.426053+0000","last_fullsized":"2023-11-13T11:03:18.426053+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059174+0000","last_change":"2023-11-13T11:03:19.468809+0000","last_active":"2023-11-13T11:03:22.059174+0000","last_peered":"2023-11-13T11:03:22.059174+0000","last_clean":"2023-11-13T11:03:22.059174+0000","last_became_active":"2023-11-13T11:03:19.467750+0000","last_became_peered":"2023-11-13T11:03:19.467750+0000","last_unstale":"2023-11-13T11:03:22.059174+0000","last_undegraded":"2023-11-13T11:03:22.059174+0000","last_fullsized":"2023-11-13T11:03:22.059174+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:45:50.416913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.421233+0000","last_change":"2023-11-13T11:03:18.421233+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.421233+0000","last_undegraded":"2023-11-13T11:03:18.421233+0000","last_fullsized":"2023-11-13T11:03:18.421233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.431809+0000","last_change":"2023-11-13T11:03:18.431809+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.431809+0000","last_undegraded":"2023-11-13T11:03:18.431809+0000","last_fullsized":"2023-11-13T11:03:18.431809+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058922+0000","last_change":"2023-11-13T11:03:19.468080+0000","last_active":"2023-11-13T11:03:22.058922+0000","last_peered":"2023-11-13T11:03:22.058922+0000","last_clean":"2023-11-13T11:03:22.058922+0000","last_became_active":"2023-11-13T11:03:19.467716+0000","last_became_peered":"2023-11-13T11:03:19.467716+0000","last_unstale":"2023-11-13T11:03:22.058922+0000","last_undegraded":"2023-11-13T11:03:22.058922+0000","last_fullsized":"2023-11-13T11:03:22.058922+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:57:39.710883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.424719+0000","last_change":"2023-11-13T11:03:18.424719+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.424719+0000","last_undegraded":"2023-11-13T11:03:18.424719+0000","last_fullsized":"2023-11-13T11:03:18.424719+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-13T11:03:19.414992+0000","last_change":"2023-11-13T11:03:19.414992+0000","last_active":"2023-11-13T11:03:19.414992+0000","last_peered":"2023-11-13T11:03:19.414992+0000","last_clean":"2023-11-13T11:03:19.414992+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:19.414992+0000","last_undegraded":"2023-11-13T11:03:19.414992+0000","last_fullsized":"2023-11-13T11:03:19.414992+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2023-11-13T11:03:18.423491+0000","last_change":"2023-11-13T11:03:18.423491+0000","last_active":"2023-11-13T11:03:18.397583+0000","last_peered":"2023-11-13T11:03:18.397583+0000","last_clean":"2023-11-13T11:03:18.397583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-13T11:03:18.423491+0000","last_undegraded":"2023-11-13T11:03:18.423491+0000","last_fullsized":"2023-11-13T11:03:18.423491+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":205,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-13T11:03:18.434428+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:03:18.434428+0000","last_peered":"2023-11-13T11:03:18.434428+0000","last_clean":"2023-11-13T11:03:18.434428+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:03:18.434428+0000","last_undegraded":"2023-11-13T11:03:18.434428+0000","last_fullsized":"2023-11-13T11:03:18.434428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44432,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327408,"statfs":{"total":96636764160,"available":96591265792,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1060000000000001,"15min":1.1259999999999999},"min":{"1min":0.73899999999999999,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.706,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.2250000000000001,"15min":1.1379999999999999},"min":{"1min":0.82199999999999995,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.498,"5min":1.865,"15min":2.2879999999999998},"last":1.014}]},{"osd":1,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.252,"15min":1.1799999999999999},"min":{"1min":0.90300000000000002,"5min":0.64000000000000001,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.089},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.2889999999999999},"min":{"1min":0.67700000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.649,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.1679999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.319,"15min":1.276},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.534,"5min":2.153,"15min":2.1680000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.383,"15min":1.3460000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.6100000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.3700000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.429,"15min":1.3779999999999999},"min":{"1min":0.81799999999999995,"5min":0.76300000000000001,"15min":0.63800000000000001},"max":{"1min":1.657,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.458,"15min":1.4379999999999999},"min":{"1min":0.99299999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.762,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.21}]},{"osd":4,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.456,"15min":1.552},"min":{"1min":0.94899999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.782,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.286},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.512,"15min":1.5740000000000001},"min":{"1min":1.1599999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.9419999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.448}]},{"osd":5,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.6060000000000001,"15min":1.6399999999999999},"min":{"1min":0.94499999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.8200000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.498},{"interface":"front","average":{"1min":1.625,"5min":1.75,"15min":1.7490000000000001},"min":{"1min":1.4650000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":1.865,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.6859999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.6930000000000001,"15min":1.645},"min":{"1min":1.2849999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0640000000000001,"5min":2.657,"15min":2.657},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.641,"15min":1.6379999999999999},"min":{"1min":1.1399999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1349999999999998,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.575}]}]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45012,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326828,"statfs":{"total":96636764160,"available":96590671872,"internally_reserved":0,"allocated":1384448,"data_stored":941850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.093,"15min":1.016},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7529999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.284},{"interface":"front","average":{"1min":1.179,"5min":1.2190000000000001,"15min":1.202},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67800000000000005},"max":{"1min":1.639,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.0760000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.306,"15min":1.2350000000000001},"min":{"1min":0.79100000000000004,"5min":0.72399999999999998,"15min":0.65400000000000003},"max":{"1min":1.73,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.282,"15min":1.2450000000000001},"min":{"1min":0.875,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.115}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.389,"15min":1.2649999999999999},"min":{"1min":0.81200000000000006,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":4.4500000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.262},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.3100000000000001,"15min":1.2370000000000001},"min":{"1min":0.78900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5760000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.0089999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.387,"15min":1.391},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8300000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.23},{"interface":"front","average":{"1min":1.53,"5min":1.5169999999999999,"15min":1.4870000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.80700000000000005},"max":{"1min":4.6959999999999997,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":1.212}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3700000000000001,"15min":1.329},"min":{"1min":0.93500000000000005,"5min":0.86599999999999999,"15min":0.55500000000000005},"max":{"1min":1.5800000000000001,"5min":3.484,"15min":3.484},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.518,"15min":1.53},"min":{"1min":1.093,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.4430000000000001,"5min":3.653,"15min":3.653},"last":0.95299999999999996}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.5800000000000001,"15min":1.6040000000000001},"min":{"1min":1.0800000000000001,"5min":0.92000000000000004,"15min":0.78000000000000003},"max":{"1min":4.8760000000000003,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.538,"15min":1.5189999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.61799999999999999},"max":{"1min":1.7609999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.177}]},{"osd":7,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.645,"15min":1.6659999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.1349999999999998,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.744,"5min":1.7330000000000001,"15min":1.714},"min":{"1min":1.2490000000000001,"5min":1.1200000000000001,"15min":0.69499999999999995},"max":{"1min":4.7919999999999998,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":1.377}]}]},{"osd":4,"up_from":15,"seq":64424509541,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54160,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317680,"statfs":{"total":96636764160,"available":96581304320,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.153,"15min":1.089},"min":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":1.986,"15min":1.986},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.145,"15min":1.085},"min":{"1min":0.69199999999999995,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.83799999999999997}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2470000000000001,"15min":1.163},"min":{"1min":0.76200000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.774,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.05},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.26},"min":{"1min":0.79300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":1.877,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.4650000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.298,"15min":1.242},"min":{"1min":0.93300000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.575,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.347,"15min":1.349},"min":{"1min":0.91100000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.72,"5min":1.952,"15min":1.952},"last":1.637}]},{"osd":3,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.488,"15min":1.421},"min":{"1min":1.1080000000000001,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.786,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.841},{"interface":"front","average":{"1min":1.379,"5min":1.3149999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.841,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.5740000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.478,"15min":1.4910000000000001},"min":{"1min":0.96399999999999997,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.778,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.996},{"interface":"front","average":{"1min":1.323,"5min":1.371,"15min":1.3939999999999999},"min":{"1min":0.81599999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":2.0529999999999999,"5min":2.089,"15min":2.089},"last":1.7809999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.621,"15min":1.6220000000000001},"min":{"1min":1.0600000000000001,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":1.7769999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":2.1030000000000002},{"interface":"front","average":{"1min":1.401,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.92400000000000004,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":1.873,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":2.0310000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.5369999999999999,"15min":1.5820000000000001},"min":{"1min":1.05,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":2.1480000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.9039999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.599,"15min":1.609},"min":{"1min":1.0189999999999999,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":1.931,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":2.1499999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40272,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331568,"statfs":{"total":96636764160,"available":96595525632,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.628,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.282,"5min":1.867,"15min":1.867},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.022,"15min":1.026},"min":{"1min":0.65800000000000003,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.819,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.032,"15min":1.0489999999999999},"min":{"1min":0.625,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.8089999999999999,"5min":1.966,"15min":3.71},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.296,"5min":1.1240000000000001,"15min":1.103},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.179,"15min":1.1040000000000001},"min":{"1min":0.72899999999999998,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":2.0019999999999998,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.24,"5min":1.171,"15min":1.2829999999999999},"min":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7809999999999999,"5min":1.952,"15min":3.5710000000000002},"last":1.143}]},{"osd":4,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.89200000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.7509999999999999,"5min":2.411,"15min":3.5030000000000001},"last":1.278},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3879999999999999,"15min":1.355},"min":{"1min":1.1259999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.831,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.5840000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4179999999999999,"15min":1.4750000000000001},"min":{"1min":1.151,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.99,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.454,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0649999999999999,"5min":2.673,"15min":3.3809999999999998},"last":1.45}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.49},"min":{"1min":1.1479999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.944,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.206},{"interface":"front","average":{"1min":1.546,"5min":1.4870000000000001,"15min":1.5840000000000001},"min":{"1min":1.1060000000000001,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":2.093,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.3380000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5609999999999999,"15min":1.609},"min":{"1min":0.84799999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1339999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.631},{"interface":"front","average":{"1min":1.649,"5min":1.6040000000000001,"15min":1.5620000000000001},"min":{"1min":1.1359999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.4089999999999998,"5min":2.75,"15min":2.75},"last":1.391}]}]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50004,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321836,"statfs":{"total":96636764160,"available":96585560064,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94199999999999995,"15min":0.93100000000000005},"min":{"1min":0.55700000000000005,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.089,"5min":1.0229999999999999,"15min":0.93400000000000005},"min":{"1min":0.54400000000000004,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":2.9460000000000002,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":1.7330000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0309999999999999,"15min":0.96299999999999997},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.376},"max":{"1min":3.0720000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.361,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.68100000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.3380000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.0920000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.216,"15min":1.2430000000000001},"min":{"1min":0.71599999999999997,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2549999999999999,"15min":1.2749999999999999},"min":{"1min":0.72699999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.4060000000000001,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.456}]},{"osd":4,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4359999999999999,"15min":1.4730000000000001},"min":{"1min":0.89900000000000002,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":2.633,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.393,"15min":1.409},"min":{"1min":0.80000000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":1.5189999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.4690000000000001,"15min":1.482},"min":{"1min":1.034,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.1680000000000001,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":1.165},{"interface":"front","average":{"1min":1.623,"5min":1.591,"15min":1.5680000000000001},"min":{"1min":0.97099999999999997,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.7509999999999999,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.6699999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5189999999999999,"15min":1.55},"min":{"1min":0.89500000000000002,"5min":0.80900000000000005,"15min":0.76500000000000001},"max":{"1min":3.395,"5min":3.395,"15min":4.7569999999999997},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6120000000000001,"15min":1.599},"min":{"1min":1.2130000000000001,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.4039999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.5700000000000001,"15min":1.607},"min":{"1min":1.0569999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":3.2429999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.694,"5min":1.6479999999999999,"15min":1.6259999999999999},"min":{"1min":1.2090000000000001,"5min":0.91200000000000003,"15min":0.82799999999999996},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":1.5760000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28372,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343468,"statfs":{"total":96636764160,"available":96607711232,"internally_reserved":0,"allocated":1384448,"data_stored":941850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85699999999999998,"15min":0.94899999999999995},"min":{"1min":0.495,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.4910000000000001,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":1.03},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.97399999999999998,"15min":0.95699999999999996},"min":{"1min":0.623,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2689999999999999,"5min":1.869,"15min":1.869},"last":0.98899999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.0509999999999999,"15min":1},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.129},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0680000000000001,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":1.111}]},{"osd":3,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.204,"15min":1.175},"min":{"1min":0.68600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.177},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.57799999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.0669999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.321,"15min":1.3129999999999999},"min":{"1min":0.76200000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":1.212},{"interface":"front","average":{"1min":1.536,"5min":1.4179999999999999,"15min":1.3859999999999999},"min":{"1min":0.69599999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.0299999999999998,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.8620000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4159999999999999,"15min":1.4139999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.0750000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.9330000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.387,"15min":1.4079999999999999},"min":{"1min":1.0309999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9410000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.8120000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.341,"15min":1.407},"min":{"1min":0.90900000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.153},{"interface":"front","average":{"1min":1.649,"5min":1.504,"15min":1.5069999999999999},"min":{"1min":1.0640000000000001,"5min":0.79400000000000004,"15min":0.752},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3879999999999999},"last":1.776}]},{"osd":7,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.466,"15min":1.494},"min":{"1min":1.228,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.0609999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4399999999999999,"15min":1.4570000000000001},"min":{"1min":0.96899999999999997,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.419}]}]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44432,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327408,"statfs":{"total":96636764160,"available":96591265792,"internally_reserved":0,"allocated":790528,"data_stored":351482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.872,"15min":0.90100000000000002},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.32100000000000001},"max":{"1min":1.7010000000000001,"5min":1.74,"15min":1.74},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0449999999999999,"15min":1.048},"min":{"1min":0.46700000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.736,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.80500000000000005}]},{"osd":2,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.1890000000000001,"15min":1.151},"min":{"1min":0.55300000000000005,"5min":0.441,"15min":0.441},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.0700000000000001,"15min":1.0269999999999999},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.371},"max":{"1min":1.968,"5min":1.968,"15min":2.1669999999999998},"last":1.1519999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.1439999999999999,"15min":1.111},"min":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.45200000000000001},"max":{"1min":1.9259999999999999,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.59799999999999998,"5min":0.497,"15min":0.497},"max":{"1min":2.2240000000000002,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.258}]},{"osd":4,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.3879999999999999,"15min":1.3660000000000001},"min":{"1min":0.82199999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":4.4279999999999999,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.284},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.3580000000000001,"15min":1.4319999999999999},"min":{"1min":0.72699999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":1.4810000000000001,"15min":1.55},"min":{"1min":1.0149999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.391},{"interface":"front","average":{"1min":1.6779999999999999,"5min":1.4590000000000001,"15min":1.4930000000000001},"min":{"1min":0.85199999999999998,"5min":0.78600000000000003,"15min":0.747},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":1.464}]},{"osd":6,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.5569999999999999,"15min":1.54},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":4.617,"5min":4.617,"15min":4.617},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.744,"5min":1.595,"15min":1.5840000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.72899999999999998},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.5840000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.694,"5min":1.629,"15min":1.6759999999999999},"min":{"1min":0.89100000000000001,"5min":0.78900000000000003,"15min":0.76700000000000002},"max":{"1min":2.4710000000000001,"5min":2.742,"15min":3.3540000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.6619999999999999,"15min":1.631},"min":{"1min":0.90600000000000003,"5min":0.88100000000000001,"15min":0.77400000000000002},"max":{"1min":3.976,"5min":3.976,"15min":3.976},"last":1.5449999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542245,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45880,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325960,"statfs":{"total":96636764160,"available":96589783040,"internally_reserved":0,"allocated":827392,"data_stored":371792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.079,"15min":1.1120000000000001},"min":{"1min":0.67200000000000004,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6020000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.0740000000000001,"15min":1.0720000000000001},"min":{"1min":0.57599999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4810000000000001,"5min":1.6080000000000001,"15min":2.266},"last":1.167}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1519999999999999,"15min":1.0669999999999999},"min":{"1min":0.65900000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.397,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2330000000000001,"15min":1.2290000000000001},"min":{"1min":0.79800000000000004,"5min":0.70099999999999996,"15min":0.64700000000000002},"max":{"1min":1.6080000000000001,"5min":1.893,"15min":2.1899999999999999},"last":1.2609999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2809999999999999,"15min":1.298},"min":{"1min":0.76000000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.859,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.355},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.278,"15min":1.2230000000000001},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.736,"5min":1.79,"15min":1.79},"last":1.4610000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.3320000000000001,"15min":1.262},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7210000000000001,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.647},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5049999999999999,"15min":1.4610000000000001},"min":{"1min":0.96299999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0459999999999998,"5min":2.2639999999999998,"15min":2.8159999999999998},"last":1.514}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3440000000000001,"15min":1.3240000000000001},"min":{"1min":0.629,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.6240000000000001,"5min":1.984,"15min":3.0190000000000001},"last":1.054},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.405,"15min":1.3759999999999999},"min":{"1min":0.70699999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.756,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.2130000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.504,"15min":1.5309999999999999},"min":{"1min":0.88700000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9550000000000001,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.506,"15min":1.48},"min":{"1min":0.73999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8540000000000001,"5min":2.3079999999999998,"15min":3.3620000000000001},"last":1.3080000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6100000000000001,"15min":1.5720000000000001},"min":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":1.8140000000000001,"5min":2.379,"15min":2.9020000000000001},"last":1.605},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.5620000000000001,"15min":1.512},"min":{"1min":0.76700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.9179999999999999,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:03:25.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2023-11-13T11:03:25.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:25.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2023-11-13T11:03:25.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:25.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2023-11-13T11:03:25.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:25.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2023-11-13T11:03:25.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:25.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2023-11-13T11:03:25.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:25.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2023-11-13T11:03:25.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2023-11-13T11:03:26.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-11-13T11:03:26.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2023-11-13T11:03:26.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2023-11-13T11:03:26.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2023-11-13T11:03:26.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-11-13T11:03:26.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-11-13T11:03:26.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2023-11-13T11:03:26.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2023-11-13T11:03:26.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2023-11-13T11:03:26.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2023-11-13T11:03:26.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2023-11-13T11:03:26.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2023-11-13T11:03:26.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-11-13T11:03:26.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2023-11-13T11:03:26.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2023-11-13T11:03:26.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-11-13T11:03:26.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-11-13T11:03:26.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2023-11-13T11:03:26.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2023-11-13T11:03:26.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2023-11-13T11:03:26.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-11-13T11:03:26.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2023-11-13T11:03:26.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2023-11-13T11:03:26.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-11-13T11:03:26.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-11-13T11:03:26.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.432660+0000', 'last_change': '2023-11-13T11:03:18.432660+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.432660+0000', 'last_undegraded': '2023-11-13T11:03:18.432660+0000', 'last_fullsized': '2023-11-13T11:03:18.432660+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-13T11:03:26.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-11-13T11:03:26.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-11-13T11:03:26.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.434034+0000', 'last_change': '2023-11-13T11:03:18.434034+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.434034+0000', 'last_undegraded': '2023-11-13T11:03:18.434034+0000', 'last_fullsized': '2023-11-13T11:03:18.434034+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-13T11:03:26.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-11-13T11:03:26.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-11-13T11:03:26.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.432731+0000', 'last_change': '2023-11-13T11:03:18.432731+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.432731+0000', 'last_undegraded': '2023-11-13T11:03:18.432731+0000', 'last_fullsized': '2023-11-13T11:03:18.432731+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-13T11:03:26.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-11-13T11:03:26.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-11-13T11:03:26.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.426974+0000', 'last_change': '2023-11-13T11:03:18.426974+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.426974+0000', 'last_undegraded': '2023-11-13T11:03:18.426974+0000', 'last_fullsized': '2023-11-13T11:03:18.426974+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-13T11:03:26.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-11-13T11:03:26.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-11-13T11:03:26.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.423052+0000', 'last_change': '2023-11-13T11:03:18.423052+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.423052+0000', 'last_undegraded': '2023-11-13T11:03:18.423052+0000', 'last_fullsized': '2023-11-13T11:03:18.423052+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-13T11:03:26.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-11-13T11:03:26.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-11-13T11:03:26.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.424577+0000', 'last_change': '2023-11-13T11:03:18.424577+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.424577+0000', 'last_undegraded': '2023-11-13T11:03:18.424577+0000', 'last_fullsized': '2023-11-13T11:03:18.424577+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-13T11:03:26.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-11-13T11:03:26.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-11-13T11:03:26.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.431321+0000', 'last_change': '2023-11-13T11:03:18.431321+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.431321+0000', 'last_undegraded': '2023-11-13T11:03:18.431321+0000', 'last_fullsized': '2023-11-13T11:03:18.431321+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-13T11:03:26.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-11-13T11:03:26.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-11-13T11:03:26.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.428573+0000', 'last_change': '2023-11-13T11:03:18.428573+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.428573+0000', 'last_undegraded': '2023-11-13T11:03:18.428573+0000', 'last_fullsized': '2023-11-13T11:03:18.428573+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-13T11:03:26.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-11-13T11:03:26.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-11-13T11:03:26.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.429859+0000', 'last_change': '2023-11-13T11:03:18.429859+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.429859+0000', 'last_undegraded': '2023-11-13T11:03:18.429859+0000', 'last_fullsized': '2023-11-13T11:03:18.429859+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-13T11:03:26.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-11-13T11:03:26.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-11-13T11:03:26.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.421232+0000', 'last_change': '2023-11-13T11:03:18.421232+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.421232+0000', 'last_undegraded': '2023-11-13T11:03:18.421232+0000', 'last_fullsized': '2023-11-13T11:03:18.421232+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-13T11:03:26.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-11-13T11:03:26.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-11-13T11:03:26.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.428911+0000', 'last_change': '2023-11-13T11:03:18.428911+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.428911+0000', 'last_undegraded': '2023-11-13T11:03:18.428911+0000', 'last_fullsized': '2023-11-13T11:03:18.428911+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-13T11:03:26.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-11-13T11:03:26.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-11-13T11:03:26.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.425857+0000', 'last_change': '2023-11-13T11:03:18.425857+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.425857+0000', 'last_undegraded': '2023-11-13T11:03:18.425857+0000', 'last_fullsized': '2023-11-13T11:03:18.425857+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-13T11:03:26.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-11-13T11:03:26.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-11-13T11:03:26.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.429650+0000', 'last_change': '2023-11-13T11:03:18.429650+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.429650+0000', 'last_undegraded': '2023-11-13T11:03:18.429650+0000', 'last_fullsized': '2023-11-13T11:03:18.429650+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-13T11:03:26.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-11-13T11:03:26.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-11-13T11:03:26.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.433832+0000', 'last_change': '2023-11-13T11:03:18.433832+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.433832+0000', 'last_undegraded': '2023-11-13T11:03:18.433832+0000', 'last_fullsized': '2023-11-13T11:03:18.433832+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-13T11:03:26.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-11-13T11:03:26.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-11-13T11:03:26.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.428094+0000', 'last_change': '2023-11-13T11:03:18.428094+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.428094+0000', 'last_undegraded': '2023-11-13T11:03:18.428094+0000', 'last_fullsized': '2023-11-13T11:03:18.428094+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-13T11:03:26.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-11-13T11:03:26.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-11-13T11:03:26.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.433356+0000', 'last_change': '2023-11-13T11:03:18.433356+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.433356+0000', 'last_undegraded': '2023-11-13T11:03:18.433356+0000', 'last_fullsized': '2023-11-13T11:03:18.433356+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-13T11:03:26.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-11-13T11:03:26.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-11-13T11:03:26.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.429355+0000', 'last_change': '2023-11-13T11:03:18.429355+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.429355+0000', 'last_undegraded': '2023-11-13T11:03:18.429355+0000', 'last_fullsized': '2023-11-13T11:03:18.429355+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-13T11:03:26.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-11-13T11:03:26.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-11-13T11:03:26.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.430231+0000', 'last_change': '2023-11-13T11:03:18.430231+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.430231+0000', 'last_undegraded': '2023-11-13T11:03:18.430231+0000', 'last_fullsized': '2023-11-13T11:03:18.430231+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-13T11:03:26.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-11-13T11:03:26.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-11-13T11:03:26.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.428157+0000', 'last_change': '2023-11-13T11:03:18.428157+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.428157+0000', 'last_undegraded': '2023-11-13T11:03:18.428157+0000', 'last_fullsized': '2023-11-13T11:03:18.428157+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-13T11:03:26.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-11-13T11:03:26.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-11-13T11:03:26.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-11-13T11:03:26.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.424388+0000', 'last_change': '2023-11-13T11:03:18.424388+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.424388+0000', 'last_undegraded': '2023-11-13T11:03:18.424388+0000', 'last_fullsized': '2023-11-13T11:03:18.424388+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-13T11:03:26.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-11-13T11:03:26.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-11-13T11:03:26.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.433296+0000', 'last_change': '2023-11-13T11:03:18.433296+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.433296+0000', 'last_undegraded': '2023-11-13T11:03:18.433296+0000', 'last_fullsized': '2023-11-13T11:03:18.433296+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-13T11:03:26.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-11-13T11:03:26.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-11-13T11:03:26.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.423667+0000', 'last_change': '2023-11-13T11:03:18.423667+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.423667+0000', 'last_undegraded': '2023-11-13T11:03:18.423667+0000', 'last_fullsized': '2023-11-13T11:03:18.423667+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-13T11:03:26.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-11-13T11:03:26.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-11-13T11:03:26.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.428757+0000', 'last_change': '2023-11-13T11:03:18.428757+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.428757+0000', 'last_undegraded': '2023-11-13T11:03:18.428757+0000', 'last_fullsized': '2023-11-13T11:03:18.428757+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-13T11:03:26.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-11-13T11:03:26.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-11-13T11:03:26.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.429504+0000', 'last_change': '2023-11-13T11:03:18.429504+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.429504+0000', 'last_undegraded': '2023-11-13T11:03:18.429504+0000', 'last_fullsized': '2023-11-13T11:03:18.429504+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-13T11:03:26.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-11-13T11:03:26.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-11-13T11:03:26.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.426053+0000', 'last_change': '2023-11-13T11:03:18.426053+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.426053+0000', 'last_undegraded': '2023-11-13T11:03:18.426053+0000', 'last_fullsized': '2023-11-13T11:03:18.426053+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-13T11:03:26.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-11-13T11:03:26.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-11-13T11:03:26.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-11-13T11:03:26.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.421233+0000', 'last_change': '2023-11-13T11:03:18.421233+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.421233+0000', 'last_undegraded': '2023-11-13T11:03:18.421233+0000', 'last_fullsized': '2023-11-13T11:03:18.421233+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-13T11:03:26.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-11-13T11:03:26.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-11-13T11:03:26.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.431809+0000', 'last_change': '2023-11-13T11:03:18.431809+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.431809+0000', 'last_undegraded': '2023-11-13T11:03:18.431809+0000', 'last_fullsized': '2023-11-13T11:03:18.431809+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-13T11:03:26.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-11-13T11:03:26.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-11-13T11:03:26.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-11-13T11:03:26.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.424719+0000', 'last_change': '2023-11-13T11:03:18.424719+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.424719+0000', 'last_undegraded': '2023-11-13T11:03:18.424719+0000', 'last_fullsized': '2023-11-13T11:03:18.424719+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-13T11:03:26.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-11-13T11:03:26.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-13T11:03:19.414992+0000', 'last_change': '2023-11-13T11:03:19.414992+0000', 'last_active': '2023-11-13T11:03:19.414992+0000', 'last_peered': '2023-11-13T11:03:19.414992+0000', 'last_clean': '2023-11-13T11:03:19.414992+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:19.414992+0000', 'last_undegraded': '2023-11-13T11:03:19.414992+0000', 'last_fullsized': '2023-11-13T11:03:19.414992+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:19.414992+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-13T11:03:26.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-11-13T11:03:26.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2023-11-13T11:03:18.423491+0000', 'last_change': '2023-11-13T11:03:18.423491+0000', 'last_active': '2023-11-13T11:03:18.397583+0000', 'last_peered': '2023-11-13T11:03:18.397583+0000', 'last_clean': '2023-11-13T11:03:18.397583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-13T11:03:18.423491+0000', 'last_undegraded': '2023-11-13T11:03:18.423491+0000', 'last_fullsized': '2023-11-13T11:03:18.423491+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'last_clean_scrub_stamp': '2023-11-13T11:03:18.397583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-13T11:03:26.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-13T11:03:26.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.301+0000 7f0912452640 1 Processor -- start 2023-11-13T11:03:26.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.302+0000 7f0912452640 1 -- start start 2023-11-13T11:03:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.303+0000 7f0912452640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f090c0b6d90 0x7f090c15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.303+0000 7f0912452640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c0d1b80 con 0x7f090c1608c0 2023-11-13T11:03:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.303+0000 7f0912452640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c0d1000 con 0x7f090c160ca0 2023-11-13T11:03:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.303+0000 7f0912452640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c0d12e0 con 0x7f090c0b6d90 2023-11-13T11:03:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.303+0000 7f0911450640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f090c1608c0 0x7f090c0ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41074/0 (socket says 172.21.15.16:41074) 2023-11-13T11:03:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.303+0000 7f0911450640 1 -- 172.21.15.16:0/1060623415 learned_addr learned my addr 172.21.15.16:0/1060623415 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.303+0000 7f0911c51640 1 --2- 172.21.15.16:0/1060623415 >> v2:172.21.15.16:3300/0 conn(0x7f090c0b6d90 0x7f090c15c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:26.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.304+0000 7f090bfff640 1 -- 172.21.15.16:0/1060623415 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029568773 0 0) 0x7f08fc001490 con 0x7f090c1608c0 2023-11-13T11:03:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.304+0000 7f090bfff640 1 -- 172.21.15.16:0/1060623415 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f4003520 con 0x7f090c1608c0 2023-11-13T11:03:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.304+0000 7f0911c51640 1 -- 172.21.15.16:0/1060623415 >> v1:172.21.15.184:6789/0 conn(0x7f090c160ca0 legacy=0x7f090c0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.304+0000 7f0911c51640 1 -- 172.21.15.16:0/1060623415 >> v1:172.21.15.16:6789/0 conn(0x7f090c1608c0 legacy=0x7f090c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.304+0000 7f0911c51640 1 -- 172.21.15.16:0/1060623415 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c0d15c0 con 0x7f090c0b6d90 2023-11-13T11:03:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.305+0000 7f0911c51640 1 --2- 172.21.15.16:0/1060623415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 0x7f090c15c7a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f09040097a0 tx=0x7f090402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e5287e5ff92b87 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.305+0000 7f090bfff640 1 -- 172.21.15.16:0/1060623415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904031830 con 0x7f090c0b6d90 2023-11-13T11:03:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.305+0000 7f090bfff640 1 -- 172.21.15.16:0/1060623415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904031990 con 0x7f090c0b6d90 2023-11-13T11:03:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.305+0000 7f090bfff640 1 -- 172.21.15.16:0/1060623415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904031d20 con 0x7f090c0b6d90 2023-11-13T11:03:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.306+0000 7f0912452640 1 -- 172.21.15.16:0/1060623415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 msgr2=0x7f090c15c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.306+0000 7f0912452640 1 --2- 172.21.15.16:0/1060623415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 0x7f090c15c7a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f09040097a0 tx=0x7f090402b8d0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.306+0000 7f0912452640 1 -- 172.21.15.16:0/1060623415 shutdown_connections 2023-11-13T11:03:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.306+0000 7f0912452640 1 --2- 172.21.15.16:0/1060623415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 0x7f090c15c7a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.306+0000 7f0912452640 1 -- 172.21.15.16:0/1060623415 >> 172.21.15.16:0/1060623415 conn(0x7f090c0d2b50 msgr2=0x7f090c0d2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.306+0000 7f0912452640 1 -- 172.21.15.16:0/1060623415 shutdown_connections 2023-11-13T11:03:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.306+0000 7f0912452640 1 -- 172.21.15.16:0/1060623415 wait complete. 2023-11-13T11:03:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.308+0000 7f0912452640 1 Processor -- start 2023-11-13T11:03:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.308+0000 7f0912452640 1 -- start start 2023-11-13T11:03:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0912452640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 0x7f090c0c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0912452640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f090c1608c0 0x7f090c147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0912452640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f090c160ca0 0x7f090c06a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0912452640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0c6410 con 0x7f090c0b6d90 2023-11-13T11:03:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0912452640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0c66f0 con 0x7f090c160ca0 2023-11-13T11:03:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0912452640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0c69d0 con 0x7f090c1608c0 2023-11-13T11:03:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0911450640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 0x7f090c0c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0911450640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 0x7f090c0c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55412/0 (socket says 172.21.15.16:55412) 2023-11-13T11:03:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0911450640 1 -- 172.21.15.16:0/3041992861 learned_addr learned my addr 172.21.15.16:0/3041992861 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0911c51640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f090c160ca0 0x7f090c06a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.309+0000 7f0910c4f640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f090c1608c0 0x7f090c147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0911450640 1 -- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f090c1608c0 msgr2=0x7f090c147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0911450640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f090c1608c0 0x7f090c147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0911450640 1 -- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f090c160ca0 msgr2=0x7f090c06a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0911450640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f090c160ca0 0x7f090c06a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0911450640 1 -- 172.21.15.16:0/3041992861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904009400 con 0x7f090c0b6d90 2023-11-13T11:03:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0911450640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 0x7f090c0c71d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00cbc0 tx=0x7f08fc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0909ffb640 1 -- 172.21.15.16:0/3041992861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00ad40 con 0x7f090c0b6d90 2023-11-13T11:03:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0909ffb640 1 -- 172.21.15.16:0/3041992861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc012400 con 0x7f090c0b6d90 2023-11-13T11:03:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0909ffb640 1 -- 172.21.15.16:0/3041992861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00aea0 con 0x7f090c0b6d90 2023-11-13T11:03:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.310+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c0c6cb0 con 0x7f090c0b6d90 2023-11-13T11:03:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.311+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c06ae20 con 0x7f090c0b6d90 2023-11-13T11:03:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.312+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c0cdd50 con 0x7f090c0b6d90 2023-11-13T11:03:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.313+0000 7f0909ffb640 1 -- 172.21.15.16:0/3041992861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08fc007070 con 0x7f090c0b6d90 2023-11-13T11:03:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.314+0000 7f0909ffb640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f08f806ae40 0x7f08f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.314+0000 7f0909ffb640 1 -- 172.21.15.16:0/3041992861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f08fc089570 con 0x7f090c0b6d90 2023-11-13T11:03:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.315+0000 7f0910c4f640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f08f806ae40 0x7f08f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.316+0000 7f0910c4f640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f08f806ae40 0x7f08f806d300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0900006fd0 tx=0x7f0900008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.319+0000 7f0909ffb640 1 -- 172.21.15.16:0/3041992861 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08fc054590 con 0x7f090c0b6d90 2023-11-13T11:03:26.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.584+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f090c0c6550 con 0x7f090c0b6d90 2023-11-13T11:03:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.585+0000 7f0909ffb640 1 -- 172.21.15.16:0/3041992861 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1109 (secure 0 0 0) 0x7f090c0c6550 con 0x7f090c0b6d90 2023-11-13T11:03:26.589 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:26.589 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":505,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":69,"num_osds":8,"num_up_osds":8,"osd_up_since":1699872913,"num_in_osds":8,"osd_in_since":1699872904,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":62},{"state_name":"creating+peering","count":29},{"state_name":"active+clean","count":6}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":361025536,"bytes_avail":772733087744,"bytes_total":773094113280,"unknown_pgs_ratio":0.63917523622512817,"inactive_pgs_ratio":0.29896906018257141},"fsmap":{"epoch":40,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":6885}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2023-11-13T11:03:06.151110+0000","services":{}},"progress_events":{}} 2023-11-13T11:03:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f08f806ae40 msgr2=0x7f08f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f08f806ae40 0x7f08f806d300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0900006fd0 tx=0x7f0900008040 comp rx=0 tx=0).stop 2023-11-13T11:03:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 msgr2=0x7f090c0c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 0x7f090c0c71d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00cbc0 tx=0x7f08fc00a430 comp rx=0 tx=0).stop 2023-11-13T11:03:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 shutdown_connections 2023-11-13T11:03:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f08f806ae40 0x7f08f806d300 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f090c160ca0 0x7f090c06a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f090c1608c0 0x7f090c147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 --2- 172.21.15.16:0/3041992861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090c0b6d90 0x7f090c0c71d0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.587+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 >> 172.21.15.16:0/3041992861 conn(0x7f090c0d2b50 msgr2=0x7f090c0d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:26.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.588+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 shutdown_connections 2023-11-13T11:03:26.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.588+0000 7f0912452640 1 -- 172.21.15.16:0/3041992861 wait complete. 2023-11-13T11:03:26.612 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-13T11:03:26.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:03:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.837+0000 7f89dffff640 1 Processor -- start 2023-11-13T11:03:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.837+0000 7f89dffff640 1 -- start start 2023-11-13T11:03:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.839+0000 7f89dffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89e015aa90 0x7f89e006a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.839+0000 7f89dffff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f89e006a6d0 0x7f89e0070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.839+0000 7f89dffff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e006d680 con 0x7f89e00708a0 2023-11-13T11:03:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.839+0000 7f89dffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e006ab50 con 0x7f89e015aa90 2023-11-13T11:03:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.839+0000 7f89dffff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e0070d30 con 0x7f89e006a6d0 2023-11-13T11:03:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.839+0000 7f89deffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89e015aa90 0x7f89e006a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.840+0000 7f89deffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89e015aa90 0x7f89e006a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55426/0 (socket says 172.21.15.16:55426) 2023-11-13T11:03:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.840+0000 7f89deffd640 1 -- 172.21.15.16:0/2121909513 learned_addr learned my addr 172.21.15.16:0/2121909513 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.840+0000 7f89de7fc640 1 --2- 172.21.15.16:0/2121909513 >> v2:172.21.15.184:3300/0 conn(0x7f89e006a6d0 0x7f89e0070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.840+0000 7f89deffd640 1 -- 172.21.15.16:0/2121909513 >> v2:172.21.15.184:3300/0 conn(0x7f89e006a6d0 msgr2=0x7f89e0070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.840+0000 7f89deffd640 1 --2- 172.21.15.16:0/2121909513 >> v2:172.21.15.184:3300/0 conn(0x7f89e006a6d0 0x7f89e0070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.840+0000 7f89deffd640 1 -- 172.21.15.16:0/2121909513 >> v1:172.21.15.184:6789/0 conn(0x7f89e00708a0 legacy=0x7f89e00b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.840+0000 7f89deffd640 1 -- 172.21.15.16:0/2121909513 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e015d630 con 0x7f89e015aa90 2023-11-13T11:03:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.841+0000 7f89deffd640 1 --2- 172.21.15.16:0/2121909513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e015aa90 0x7f89e006a190 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f89e006b320 tx=0x7f89d002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213cc1b9b2b29183 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.841+0000 7f89ddffb640 1 -- 172.21.15.16:0/2121909513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d00317e0 con 0x7f89e015aa90 2023-11-13T11:03:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.841+0000 7f89ddffb640 1 -- 172.21.15.16:0/2121909513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d0031940 con 0x7f89e015aa90 2023-11-13T11:03:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.841+0000 7f89ddffb640 1 -- 172.21.15.16:0/2121909513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0031cd0 con 0x7f89e015aa90 2023-11-13T11:03:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.842+0000 7f89dffff640 1 -- 172.21.15.16:0/2121909513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e015aa90 msgr2=0x7f89e006a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.842+0000 7f89dffff640 1 --2- 172.21.15.16:0/2121909513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e015aa90 0x7f89e006a190 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f89e006b320 tx=0x7f89d002b880 comp rx=0 tx=0).stop 2023-11-13T11:03:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.842+0000 7f89dffff640 1 -- 172.21.15.16:0/2121909513 shutdown_connections 2023-11-13T11:03:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.842+0000 7f89dffff640 1 --2- 172.21.15.16:0/2121909513 >> v2:172.21.15.184:3300/0 conn(0x7f89e006a6d0 0x7f89e0070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.842+0000 7f89dffff640 1 --2- 172.21.15.16:0/2121909513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e015aa90 0x7f89e006a190 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.842+0000 7f89dffff640 1 -- 172.21.15.16:0/2121909513 >> 172.21.15.16:0/2121909513 conn(0x7f89e00b04a0 msgr2=0x7f89e00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.842+0000 7f89dffff640 1 -- 172.21.15.16:0/2121909513 shutdown_connections 2023-11-13T11:03:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.842+0000 7f89dffff640 1 -- 172.21.15.16:0/2121909513 wait complete. 2023-11-13T11:03:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.844+0000 7f89dffff640 1 Processor -- start 2023-11-13T11:03:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.844+0000 7f89dffff640 1 -- start start 2023-11-13T11:03:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.845+0000 7f89dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e006a6d0 0x7f89e015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.845+0000 7f89dffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00708a0 0x7f89e014d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.845+0000 7f89dffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e015aa90 0x7f89e00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.845+0000 7f89dffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e01577a0 con 0x7f89e006a6d0 2023-11-13T11:03:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.845+0000 7f89dffff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e0157a80 con 0x7f89e015aa90 2023-11-13T11:03:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.845+0000 7f89dffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e0157d60 con 0x7f89e00708a0 2023-11-13T11:03:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.846+0000 7f89deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e006a6d0 0x7f89e015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:26.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.846+0000 7f89deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e006a6d0 0x7f89e015bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55442/0 (socket says 172.21.15.16:55442) 2023-11-13T11:03:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.846+0000 7f89deffd640 1 -- 172.21.15.16:0/2087901887 learned_addr learned my addr 172.21.15.16:0/2087901887 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.846+0000 7f89deffd640 1 -- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00708a0 msgr2=0x7f89e014d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.846+0000 7f89deffd640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00708a0 0x7f89e014d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.846+0000 7f89deffd640 1 -- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e015aa90 msgr2=0x7f89e00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.846+0000 7f89deffd640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e015aa90 0x7f89e00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.846+0000 7f89deffd640 1 -- 172.21.15.16:0/2087901887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d00094d0 con 0x7f89e006a6d0 2023-11-13T11:03:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.846+0000 7f89deffd640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e006a6d0 0x7f89e015bf30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f89d002b8c0 tx=0x7f89d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.847+0000 7f89c77fe640 1 -- 172.21.15.16:0/2087901887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0032880 con 0x7f89e006a6d0 2023-11-13T11:03:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.847+0000 7f89c77fe640 1 -- 172.21.15.16:0/2087901887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d0032a20 con 0x7f89e006a6d0 2023-11-13T11:03:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.847+0000 7f89c77fe640 1 -- 172.21.15.16:0/2087901887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0032dd0 con 0x7f89e006a6d0 2023-11-13T11:03:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.847+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e0158040 con 0x7f89e006a6d0 2023-11-13T11:03:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.847+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e01f2620 con 0x7f89e006a6d0 2023-11-13T11:03:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.848+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e0158560 con 0x7f89e006a6d0 2023-11-13T11:03:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.849+0000 7f89c77fe640 1 -- 172.21.15.16:0/2087901887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89d004e450 con 0x7f89e006a6d0 2023-11-13T11:03:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.850+0000 7f89c77fe640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89cc06ae40 0x7f89cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.850+0000 7f89c77fe640 1 -- 172.21.15.16:0/2087901887 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f89d00ab990 con 0x7f89e006a6d0 2023-11-13T11:03:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.850+0000 7f89de7fc640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89cc06ae40 0x7f89cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.852+0000 7f89de7fc640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89cc06ae40 0x7f89cc06d300 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f89d4006fd0 tx=0x7f89d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:26.854+0000 7f89c77fe640 1 -- 172.21.15.16:0/2087901887 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89e0158560 con 0x7f89e006a6d0 2023-11-13T11:03:27.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.046+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89e015ded0 con 0x7f89cc06ae40 2023-11-13T11:03:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.068+0000 7f89c77fe640 1 -- 172.21.15.16:0/2087901887 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286885 (secure 0 0 0) 0x7f89d400be20 con 0x7f89cc06ae40 2023-11-13T11:03:27.076 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:27.078 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:03:27.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89cc06ae40 msgr2=0x7f89cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:27.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89cc06ae40 0x7f89cc06d300 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f89d4006fd0 tx=0x7f89d4008040 comp rx=0 tx=0).stop 2023-11-13T11:03:27.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e006a6d0 msgr2=0x7f89e015bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e006a6d0 0x7f89e015bf30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f89d002b8c0 tx=0x7f89d00046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 shutdown_connections 2023-11-13T11:03:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f89cc06ae40 0x7f89cc06d300 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e015aa90 0x7f89e00aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:27.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89e00708a0 0x7f89e014d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 --2- 172.21.15.16:0/2087901887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e006a6d0 0x7f89e015bf30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 >> 172.21.15.16:0/2087901887 conn(0x7f89e00b04a0 msgr2=0x7f89e00afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 shutdown_connections 2023-11-13T11:03:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:27.075+0000 7f89dffff640 1 -- 172.21.15.16:0/2087901887 wait complete. 2023-11-13T11:03:27.100 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2023-11-13T11:03:26.157210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":352892,"kb_used_data":7688,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754621828,"statfs":{"total":773094113280,"available":772732751872,"internally_reserved":0,"allocated":7872512,"data_stored":4159892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":353357361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.723030"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.447492+0000","last_change":"2023-11-13T11:03:20.481234+0000","last_active":"2023-11-13T11:03:21.447492+0000","last_peered":"2023-11-13T11:03:21.447492+0000","last_clean":"2023-11-13T11:03:21.447492+0000","last_became_active":"2023-11-13T11:03:20.480949+0000","last_became_peered":"2023-11-13T11:03:20.480949+0000","last_unstale":"2023-11-13T11:03:21.447492+0000","last_undegraded":"2023-11-13T11:03:21.447492+0000","last_fullsized":"2023-11-13T11:03:21.447492+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:15.570305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.447386+0000","last_change":"2023-11-13T11:03:20.473909+0000","last_active":"2023-11-13T11:03:21.447386+0000","last_peered":"2023-11-13T11:03:21.447386+0000","last_clean":"2023-11-13T11:03:21.447386+0000","last_became_active":"2023-11-13T11:03:20.473638+0000","last_became_peered":"2023-11-13T11:03:20.473638+0000","last_unstale":"2023-11-13T11:03:21.447386+0000","last_undegraded":"2023-11-13T11:03:21.447386+0000","last_fullsized":"2023-11-13T11:03:21.447386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:53:50.663729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299244+0000","last_change":"2023-11-13T11:03:20.468089+0000","last_active":"2023-11-13T11:03:22.299244+0000","last_peered":"2023-11-13T11:03:22.299244+0000","last_clean":"2023-11-13T11:03:22.299244+0000","last_became_active":"2023-11-13T11:03:20.467527+0000","last_became_peered":"2023-11-13T11:03:20.467527+0000","last_unstale":"2023-11-13T11:03:22.299244+0000","last_undegraded":"2023-11-13T11:03:22.299244+0000","last_fullsized":"2023-11-13T11:03:22.299244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:21:31.064220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441904+0000","last_change":"2023-11-13T11:03:20.473876+0000","last_active":"2023-11-13T11:03:21.441904+0000","last_peered":"2023-11-13T11:03:21.441904+0000","last_clean":"2023-11-13T11:03:21.441904+0000","last_became_active":"2023-11-13T11:03:20.473496+0000","last_became_peered":"2023-11-13T11:03:20.473496+0000","last_unstale":"2023-11-13T11:03:21.441904+0000","last_undegraded":"2023-11-13T11:03:21.441904+0000","last_fullsized":"2023-11-13T11:03:21.441904+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:35:11.804201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297643+0000","last_change":"2023-11-13T11:03:20.481275+0000","last_active":"2023-11-13T11:03:22.297643+0000","last_peered":"2023-11-13T11:03:22.297643+0000","last_clean":"2023-11-13T11:03:22.297643+0000","last_became_active":"2023-11-13T11:03:20.480960+0000","last_became_peered":"2023-11-13T11:03:20.480960+0000","last_unstale":"2023-11-13T11:03:22.297643+0000","last_undegraded":"2023-11-13T11:03:22.297643+0000","last_fullsized":"2023-11-13T11:03:22.297643+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:17:06.846928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.446428+0000","last_change":"2023-11-13T11:03:20.476263+0000","last_active":"2023-11-13T11:03:21.446428+0000","last_peered":"2023-11-13T11:03:21.446428+0000","last_clean":"2023-11-13T11:03:21.446428+0000","last_became_active":"2023-11-13T11:03:20.472373+0000","last_became_peered":"2023-11-13T11:03:20.472373+0000","last_unstale":"2023-11-13T11:03:21.446428+0000","last_undegraded":"2023-11-13T11:03:21.446428+0000","last_fullsized":"2023-11-13T11:03:21.446428+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:02:35.586086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.444426+0000","last_change":"2023-11-13T11:03:20.466101+0000","last_active":"2023-11-13T11:03:21.444426+0000","last_peered":"2023-11-13T11:03:21.444426+0000","last_clean":"2023-11-13T11:03:21.444426+0000","last_became_active":"2023-11-13T11:03:20.465531+0000","last_became_peered":"2023-11-13T11:03:20.465531+0000","last_unstale":"2023-11-13T11:03:21.444426+0000","last_undegraded":"2023-11-13T11:03:21.444426+0000","last_fullsized":"2023-11-13T11:03:21.444426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:38:54.348922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297273+0000","last_change":"2023-11-13T11:03:20.482233+0000","last_active":"2023-11-13T11:03:22.297273+0000","last_peered":"2023-11-13T11:03:22.297273+0000","last_clean":"2023-11-13T11:03:22.297273+0000","last_became_active":"2023-11-13T11:03:20.481971+0000","last_became_peered":"2023-11-13T11:03:20.481971+0000","last_unstale":"2023-11-13T11:03:22.297273+0000","last_undegraded":"2023-11-13T11:03:22.297273+0000","last_fullsized":"2023-11-13T11:03:22.297273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:17:24.322051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442399+0000","last_change":"2023-11-13T11:03:20.475387+0000","last_active":"2023-11-13T11:03:21.442399+0000","last_peered":"2023-11-13T11:03:21.442399+0000","last_clean":"2023-11-13T11:03:21.442399+0000","last_became_active":"2023-11-13T11:03:20.475063+0000","last_became_peered":"2023-11-13T11:03:20.475063+0000","last_unstale":"2023-11-13T11:03:21.442399+0000","last_undegraded":"2023-11-13T11:03:21.442399+0000","last_fullsized":"2023-11-13T11:03:21.442399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:58:33.061768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.837119+0000","last_change":"2023-11-13T11:03:20.463396+0000","last_active":"2023-11-13T11:03:21.837119+0000","last_peered":"2023-11-13T11:03:21.837119+0000","last_clean":"2023-11-13T11:03:21.837119+0000","last_became_active":"2023-11-13T11:03:20.463113+0000","last_became_peered":"2023-11-13T11:03:20.463113+0000","last_unstale":"2023-11-13T11:03:21.837119+0000","last_undegraded":"2023-11-13T11:03:21.837119+0000","last_fullsized":"2023-11-13T11:03:21.837119+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:32.831255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442465+0000","last_change":"2023-11-13T11:03:20.477366+0000","last_active":"2023-11-13T11:03:21.442465+0000","last_peered":"2023-11-13T11:03:21.442465+0000","last_clean":"2023-11-13T11:03:21.442465+0000","last_became_active":"2023-11-13T11:03:20.476702+0000","last_became_peered":"2023-11-13T11:03:20.476702+0000","last_unstale":"2023-11-13T11:03:21.442465+0000","last_undegraded":"2023-11-13T11:03:21.442465+0000","last_fullsized":"2023-11-13T11:03:21.442465+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:57:36.112404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442044+0000","last_change":"2023-11-13T11:03:20.487960+0000","last_active":"2023-11-13T11:03:21.442044+0000","last_peered":"2023-11-13T11:03:21.442044+0000","last_clean":"2023-11-13T11:03:21.442044+0000","last_became_active":"2023-11-13T11:03:20.486438+0000","last_became_peered":"2023-11-13T11:03:20.486438+0000","last_unstale":"2023-11-13T11:03:21.442044+0000","last_undegraded":"2023-11-13T11:03:21.442044+0000","last_fullsized":"2023-11-13T11:03:21.442044+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:48.755047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297506+0000","last_change":"2023-11-13T11:03:20.477742+0000","last_active":"2023-11-13T11:03:22.297506+0000","last_peered":"2023-11-13T11:03:22.297506+0000","last_clean":"2023-11-13T11:03:22.297506+0000","last_became_active":"2023-11-13T11:03:20.477473+0000","last_became_peered":"2023-11-13T11:03:20.477473+0000","last_unstale":"2023-11-13T11:03:22.297506+0000","last_undegraded":"2023-11-13T11:03:22.297506+0000","last_fullsized":"2023-11-13T11:03:22.297506+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:11:29.076081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297105+0000","last_change":"2023-11-13T11:03:20.469272+0000","last_active":"2023-11-13T11:03:22.297105+0000","last_peered":"2023-11-13T11:03:22.297105+0000","last_clean":"2023-11-13T11:03:22.297105+0000","last_became_active":"2023-11-13T11:03:20.468953+0000","last_became_peered":"2023-11-13T11:03:20.468953+0000","last_unstale":"2023-11-13T11:03:22.297105+0000","last_undegraded":"2023-11-13T11:03:22.297105+0000","last_fullsized":"2023-11-13T11:03:22.297105+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:28:15.701895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836278+0000","last_change":"2023-11-13T11:03:20.467960+0000","last_active":"2023-11-13T11:03:21.836278+0000","last_peered":"2023-11-13T11:03:21.836278+0000","last_clean":"2023-11-13T11:03:21.836278+0000","last_became_active":"2023-11-13T11:03:20.466799+0000","last_became_peered":"2023-11-13T11:03:20.466799+0000","last_unstale":"2023-11-13T11:03:21.836278+0000","last_undegraded":"2023-11-13T11:03:21.836278+0000","last_fullsized":"2023-11-13T11:03:21.836278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:11:29.076081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058741+0000","last_change":"2023-11-13T11:03:20.473726+0000","last_active":"2023-11-13T11:03:22.058741+0000","last_peered":"2023-11-13T11:03:22.058741+0000","last_clean":"2023-11-13T11:03:22.058741+0000","last_became_active":"2023-11-13T11:03:20.473463+0000","last_became_peered":"2023-11-13T11:03:20.473463+0000","last_unstale":"2023-11-13T11:03:22.058741+0000","last_undegraded":"2023-11-13T11:03:22.058741+0000","last_fullsized":"2023-11-13T11:03:22.058741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:24:37.146283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297642+0000","last_change":"2023-11-13T11:03:20.478944+0000","last_active":"2023-11-13T11:03:22.297642+0000","last_peered":"2023-11-13T11:03:22.297642+0000","last_clean":"2023-11-13T11:03:22.297642+0000","last_became_active":"2023-11-13T11:03:20.478624+0000","last_became_peered":"2023-11-13T11:03:20.478624+0000","last_unstale":"2023-11-13T11:03:22.297642+0000","last_undegraded":"2023-11-13T11:03:22.297642+0000","last_fullsized":"2023-11-13T11:03:22.297642+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:32:53.662105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441902+0000","last_change":"2023-11-13T11:03:20.479029+0000","last_active":"2023-11-13T11:03:21.441902+0000","last_peered":"2023-11-13T11:03:21.441902+0000","last_clean":"2023-11-13T11:03:21.441902+0000","last_became_active":"2023-11-13T11:03:20.478776+0000","last_became_peered":"2023-11-13T11:03:20.478776+0000","last_unstale":"2023-11-13T11:03:21.441902+0000","last_undegraded":"2023-11-13T11:03:21.441902+0000","last_fullsized":"2023-11-13T11:03:21.441902+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:53:50.663729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442651+0000","last_change":"2023-11-13T11:03:20.477561+0000","last_active":"2023-11-13T11:03:21.442651+0000","last_peered":"2023-11-13T11:03:21.442651+0000","last_clean":"2023-11-13T11:03:21.442651+0000","last_became_active":"2023-11-13T11:03:20.477087+0000","last_became_peered":"2023-11-13T11:03:20.477087+0000","last_unstale":"2023-11-13T11:03:21.442651+0000","last_undegraded":"2023-11-13T11:03:21.442651+0000","last_fullsized":"2023-11-13T11:03:21.442651+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:32:18.758861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296990+0000","last_change":"2023-11-13T11:03:20.478838+0000","last_active":"2023-11-13T11:03:22.296990+0000","last_peered":"2023-11-13T11:03:22.296990+0000","last_clean":"2023-11-13T11:03:22.296990+0000","last_became_active":"2023-11-13T11:03:20.478275+0000","last_became_peered":"2023-11-13T11:03:20.478275+0000","last_unstale":"2023-11-13T11:03:22.296990+0000","last_undegraded":"2023-11-13T11:03:22.296990+0000","last_fullsized":"2023-11-13T11:03:22.296990+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:45:14.457719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297712+0000","last_change":"2023-11-13T11:03:20.482167+0000","last_active":"2023-11-13T11:03:22.297712+0000","last_peered":"2023-11-13T11:03:22.297712+0000","last_clean":"2023-11-13T11:03:22.297712+0000","last_became_active":"2023-11-13T11:03:20.481889+0000","last_became_peered":"2023-11-13T11:03:20.481889+0000","last_unstale":"2023-11-13T11:03:22.297712+0000","last_undegraded":"2023-11-13T11:03:22.297712+0000","last_fullsized":"2023-11-13T11:03:22.297712+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:41.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.447215+0000","last_change":"2023-11-13T11:03:20.472937+0000","last_active":"2023-11-13T11:03:21.447215+0000","last_peered":"2023-11-13T11:03:21.447215+0000","last_clean":"2023-11-13T11:03:21.447215+0000","last_became_active":"2023-11-13T11:03:20.472646+0000","last_became_peered":"2023-11-13T11:03:20.472646+0000","last_unstale":"2023-11-13T11:03:21.447215+0000","last_undegraded":"2023-11-13T11:03:21.447215+0000","last_fullsized":"2023-11-13T11:03:21.447215+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:10:29.987250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058815+0000","last_change":"2023-11-13T11:03:20.471989+0000","last_active":"2023-11-13T11:03:22.058815+0000","last_peered":"2023-11-13T11:03:22.058815+0000","last_clean":"2023-11-13T11:03:22.058815+0000","last_became_active":"2023-11-13T11:03:20.470759+0000","last_became_peered":"2023-11-13T11:03:20.470759+0000","last_unstale":"2023-11-13T11:03:22.058815+0000","last_undegraded":"2023-11-13T11:03:22.058815+0000","last_fullsized":"2023-11-13T11:03:22.058815+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:33.483394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297281+0000","last_change":"2023-11-13T11:03:20.466775+0000","last_active":"2023-11-13T11:03:22.297281+0000","last_peered":"2023-11-13T11:03:22.297281+0000","last_clean":"2023-11-13T11:03:22.297281+0000","last_became_active":"2023-11-13T11:03:20.466504+0000","last_became_peered":"2023-11-13T11:03:20.466504+0000","last_unstale":"2023-11-13T11:03:22.297281+0000","last_undegraded":"2023-11-13T11:03:22.297281+0000","last_fullsized":"2023-11-13T11:03:22.297281+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:48:04.143689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299765+0000","last_change":"2023-11-13T11:03:20.472580+0000","last_active":"2023-11-13T11:03:22.299765+0000","last_peered":"2023-11-13T11:03:22.299765+0000","last_clean":"2023-11-13T11:03:22.299765+0000","last_became_active":"2023-11-13T11:03:20.472176+0000","last_became_peered":"2023-11-13T11:03:20.472176+0000","last_unstale":"2023-11-13T11:03:22.299765+0000","last_undegraded":"2023-11-13T11:03:22.299765+0000","last_fullsized":"2023-11-13T11:03:22.299765+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:36:22.298361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298183+0000","last_change":"2023-11-13T11:03:20.466179+0000","last_active":"2023-11-13T11:03:22.298183+0000","last_peered":"2023-11-13T11:03:22.298183+0000","last_clean":"2023-11-13T11:03:22.298183+0000","last_became_active":"2023-11-13T11:03:20.464657+0000","last_became_peered":"2023-11-13T11:03:20.464657+0000","last_unstale":"2023-11-13T11:03:22.298183+0000","last_undegraded":"2023-11-13T11:03:22.298183+0000","last_fullsized":"2023-11-13T11:03:22.298183+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:28:07.308881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299015+0000","last_change":"2023-11-13T11:03:20.470580+0000","last_active":"2023-11-13T11:03:22.299015+0000","last_peered":"2023-11-13T11:03:22.299015+0000","last_clean":"2023-11-13T11:03:22.299015+0000","last_became_active":"2023-11-13T11:03:20.470263+0000","last_became_peered":"2023-11-13T11:03:20.470263+0000","last_unstale":"2023-11-13T11:03:22.299015+0000","last_undegraded":"2023-11-13T11:03:22.299015+0000","last_fullsized":"2023-11-13T11:03:22.299015+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:32.831255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.446417+0000","last_change":"2023-11-13T11:03:20.477205+0000","last_active":"2023-11-13T11:03:21.446417+0000","last_peered":"2023-11-13T11:03:21.446417+0000","last_clean":"2023-11-13T11:03:21.446417+0000","last_became_active":"2023-11-13T11:03:20.476849+0000","last_became_peered":"2023-11-13T11:03:20.476849+0000","last_unstale":"2023-11-13T11:03:21.446417+0000","last_undegraded":"2023-11-13T11:03:21.446417+0000","last_fullsized":"2023-11-13T11:03:21.446417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:27:37.299626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441724+0000","last_change":"2023-11-13T11:03:20.485616+0000","last_active":"2023-11-13T11:03:21.441724+0000","last_peered":"2023-11-13T11:03:21.441724+0000","last_clean":"2023-11-13T11:03:21.441724+0000","last_became_active":"2023-11-13T11:03:20.482900+0000","last_became_peered":"2023-11-13T11:03:20.482900+0000","last_unstale":"2023-11-13T11:03:21.441724+0000","last_undegraded":"2023-11-13T11:03:21.441724+0000","last_fullsized":"2023-11-13T11:03:21.441724+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:10:29.987250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441309+0000","last_change":"2023-11-13T11:03:20.476615+0000","last_active":"2023-11-13T11:03:21.441309+0000","last_peered":"2023-11-13T11:03:21.441309+0000","last_clean":"2023-11-13T11:03:21.441309+0000","last_became_active":"2023-11-13T11:03:20.474970+0000","last_became_peered":"2023-11-13T11:03:20.474970+0000","last_unstale":"2023-11-13T11:03:21.441309+0000","last_undegraded":"2023-11-13T11:03:21.441309+0000","last_fullsized":"2023-11-13T11:03:21.441309+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:02:35.586086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297973+0000","last_change":"2023-11-13T11:03:20.465386+0000","last_active":"2023-11-13T11:03:22.297973+0000","last_peered":"2023-11-13T11:03:22.297973+0000","last_clean":"2023-11-13T11:03:22.297973+0000","last_became_active":"2023-11-13T11:03:20.465100+0000","last_became_peered":"2023-11-13T11:03:20.465100+0000","last_unstale":"2023-11-13T11:03:22.297973+0000","last_undegraded":"2023-11-13T11:03:22.297973+0000","last_fullsized":"2023-11-13T11:03:22.297973+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:32:53.662105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.444374+0000","last_change":"2023-11-13T11:03:20.467138+0000","last_active":"2023-11-13T11:03:21.444374+0000","last_peered":"2023-11-13T11:03:21.444374+0000","last_clean":"2023-11-13T11:03:21.444374+0000","last_became_active":"2023-11-13T11:03:20.466692+0000","last_became_peered":"2023-11-13T11:03:20.466692+0000","last_unstale":"2023-11-13T11:03:21.444374+0000","last_undegraded":"2023-11-13T11:03:21.444374+0000","last_fullsized":"2023-11-13T11:03:21.444374+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.869476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442092+0000","last_change":"2023-11-13T11:03:20.478001+0000","last_active":"2023-11-13T11:03:21.442092+0000","last_peered":"2023-11-13T11:03:21.442092+0000","last_clean":"2023-11-13T11:03:21.442092+0000","last_became_active":"2023-11-13T11:03:20.476026+0000","last_became_peered":"2023-11-13T11:03:20.476026+0000","last_unstale":"2023-11-13T11:03:21.442092+0000","last_undegraded":"2023-11-13T11:03:21.442092+0000","last_fullsized":"2023-11-13T11:03:21.442092+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:36:46.335523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"68'4","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.486577+0000","last_change":"2023-11-13T11:03:19.462104+0000","last_active":"2023-11-13T11:03:20.486577+0000","last_peered":"2023-11-13T11:03:20.486577+0000","last_clean":"2023-11-13T11:03:20.486577+0000","last_became_active":"2023-11-13T11:03:19.461813+0000","last_became_peered":"2023-11-13T11:03:19.461813+0000","last_unstale":"2023-11-13T11:03:20.486577+0000","last_undegraded":"2023-11-13T11:03:20.486577+0000","last_fullsized":"2023-11-13T11:03:20.486577+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:11:28.058672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298818+0000","last_change":"2023-11-13T11:03:20.481842+0000","last_active":"2023-11-13T11:03:22.298818+0000","last_peered":"2023-11-13T11:03:22.298818+0000","last_clean":"2023-11-13T11:03:22.298818+0000","last_became_active":"2023-11-13T11:03:20.481508+0000","last_became_peered":"2023-11-13T11:03:20.481508+0000","last_unstale":"2023-11-13T11:03:22.298818+0000","last_undegraded":"2023-11-13T11:03:22.298818+0000","last_fullsized":"2023-11-13T11:03:22.298818+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:57:06.652730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.486648+0000","last_change":"2023-11-13T11:03:19.467329+0000","last_active":"2023-11-13T11:03:20.486648+0000","last_peered":"2023-11-13T11:03:20.486648+0000","last_clean":"2023-11-13T11:03:20.486648+0000","last_became_active":"2023-11-13T11:03:19.465815+0000","last_became_peered":"2023-11-13T11:03:19.465815+0000","last_unstale":"2023-11-13T11:03:20.486648+0000","last_undegraded":"2023-11-13T11:03:20.486648+0000","last_fullsized":"2023-11-13T11:03:20.486648+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:31.813846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.447727+0000","last_change":"2023-11-13T11:03:20.471019+0000","last_active":"2023-11-13T11:03:21.447727+0000","last_peered":"2023-11-13T11:03:21.447727+0000","last_clean":"2023-11-13T11:03:21.447727+0000","last_became_active":"2023-11-13T11:03:20.470352+0000","last_became_peered":"2023-11-13T11:03:20.470352+0000","last_unstale":"2023-11-13T11:03:21.447727+0000","last_undegraded":"2023-11-13T11:03:21.447727+0000","last_fullsized":"2023-11-13T11:03:21.447727+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:35:11.804201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.485670+0000","last_change":"2023-11-13T11:03:19.465871+0000","last_active":"2023-11-13T11:03:20.485670+0000","last_peered":"2023-11-13T11:03:20.485670+0000","last_clean":"2023-11-13T11:03:20.485670+0000","last_became_active":"2023-11-13T11:03:19.465541+0000","last_became_peered":"2023-11-13T11:03:19.465541+0000","last_unstale":"2023-11-13T11:03:20.485670+0000","last_undegraded":"2023-11-13T11:03:20.485670+0000","last_fullsized":"2023-11-13T11:03:20.485670+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:40.173268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442595+0000","last_change":"2023-11-13T11:03:20.479413+0000","last_active":"2023-11-13T11:03:21.442595+0000","last_peered":"2023-11-13T11:03:21.442595+0000","last_clean":"2023-11-13T11:03:21.442595+0000","last_became_active":"2023-11-13T11:03:20.479139+0000","last_became_peered":"2023-11-13T11:03:20.479139+0000","last_unstale":"2023-11-13T11:03:21.442595+0000","last_undegraded":"2023-11-13T11:03:21.442595+0000","last_fullsized":"2023-11-13T11:03:21.442595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:41:06.292346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.485628+0000","last_change":"2023-11-13T11:03:19.471284+0000","last_active":"2023-11-13T11:03:20.485628+0000","last_peered":"2023-11-13T11:03:20.485628+0000","last_clean":"2023-11-13T11:03:20.485628+0000","last_became_active":"2023-11-13T11:03:19.470979+0000","last_became_peered":"2023-11-13T11:03:19.470979+0000","last_unstale":"2023-11-13T11:03:20.485628+0000","last_undegraded":"2023-11-13T11:03:20.485628+0000","last_fullsized":"2023-11-13T11:03:20.485628+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:38:53.331513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296423+0000","last_change":"2023-11-13T11:03:20.463568+0000","last_active":"2023-11-13T11:03:22.296423+0000","last_peered":"2023-11-13T11:03:22.296423+0000","last_clean":"2023-11-13T11:03:22.296423+0000","last_became_active":"2023-11-13T11:03:20.462426+0000","last_became_peered":"2023-11-13T11:03:20.462426+0000","last_unstale":"2023-11-13T11:03:22.296423+0000","last_undegraded":"2023-11-13T11:03:22.296423+0000","last_fullsized":"2023-11-13T11:03:22.296423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:05:18.302427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.482354+0000","last_change":"2023-11-13T11:03:19.461169+0000","last_active":"2023-11-13T11:03:20.482354+0000","last_peered":"2023-11-13T11:03:20.482354+0000","last_clean":"2023-11-13T11:03:20.482354+0000","last_became_active":"2023-11-13T11:03:19.460849+0000","last_became_peered":"2023-11-13T11:03:19.460849+0000","last_unstale":"2023-11-13T11:03:20.482354+0000","last_undegraded":"2023-11-13T11:03:20.482354+0000","last_fullsized":"2023-11-13T11:03:20.482354+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:45:54.847751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.444937+0000","last_change":"2023-11-13T11:03:20.476919+0000","last_active":"2023-11-13T11:03:21.444937+0000","last_peered":"2023-11-13T11:03:21.444937+0000","last_clean":"2023-11-13T11:03:21.444937+0000","last_became_active":"2023-11-13T11:03:20.476599+0000","last_became_peered":"2023-11-13T11:03:20.476599+0000","last_unstale":"2023-11-13T11:03:21.444937+0000","last_undegraded":"2023-11-13T11:03:21.444937+0000","last_fullsized":"2023-11-13T11:03:21.444937+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:36:22.298361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.485821+0000","last_change":"2023-11-13T11:03:19.470566+0000","last_active":"2023-11-13T11:03:20.485821+0000","last_peered":"2023-11-13T11:03:20.485821+0000","last_clean":"2023-11-13T11:03:20.485821+0000","last_became_active":"2023-11-13T11:03:19.470250+0000","last_became_peered":"2023-11-13T11:03:19.470250+0000","last_unstale":"2023-11-13T11:03:20.485821+0000","last_undegraded":"2023-11-13T11:03:20.485821+0000","last_fullsized":"2023-11-13T11:03:20.485821+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:15.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836870+0000","last_change":"2023-11-13T11:03:20.478266+0000","last_active":"2023-11-13T11:03:21.836870+0000","last_peered":"2023-11-13T11:03:21.836870+0000","last_clean":"2023-11-13T11:03:21.836870+0000","last_became_active":"2023-11-13T11:03:20.477220+0000","last_became_peered":"2023-11-13T11:03:20.477220+0000","last_unstale":"2023-11-13T11:03:21.836870+0000","last_undegraded":"2023-11-13T11:03:21.836870+0000","last_fullsized":"2023-11-13T11:03:21.836870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:06.214101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.458907+0000","last_change":"2023-11-13T11:03:19.464829+0000","last_active":"2023-11-13T11:03:20.458907+0000","last_peered":"2023-11-13T11:03:20.458907+0000","last_clean":"2023-11-13T11:03:20.458907+0000","last_became_active":"2023-11-13T11:03:19.463540+0000","last_became_peered":"2023-11-13T11:03:19.463540+0000","last_unstale":"2023-11-13T11:03:20.458907+0000","last_undegraded":"2023-11-13T11:03:20.458907+0000","last_fullsized":"2023-11-13T11:03:20.458907+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:21.547257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296745+0000","last_change":"2023-11-13T11:03:20.479489+0000","last_active":"2023-11-13T11:03:22.296745+0000","last_peered":"2023-11-13T11:03:22.296745+0000","last_clean":"2023-11-13T11:03:22.296745+0000","last_became_active":"2023-11-13T11:03:20.478574+0000","last_became_peered":"2023-11-13T11:03:20.478574+0000","last_unstale":"2023-11-13T11:03:22.296745+0000","last_undegraded":"2023-11-13T11:03:22.296745+0000","last_fullsized":"2023-11-13T11:03:22.296745+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:10:13.749773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.472241+0000","last_change":"2023-11-13T11:03:19.453306+0000","last_active":"2023-11-13T11:03:20.472241+0000","last_peered":"2023-11-13T11:03:20.472241+0000","last_clean":"2023-11-13T11:03:20.472241+0000","last_became_active":"2023-11-13T11:03:19.451270+0000","last_became_peered":"2023-11-13T11:03:19.451270+0000","last_unstale":"2023-11-13T11:03:20.472241+0000","last_undegraded":"2023-11-13T11:03:20.472241+0000","last_fullsized":"2023-11-13T11:03:20.472241+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:21.547257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.446774+0000","last_change":"2023-11-13T11:03:20.467546+0000","last_active":"2023-11-13T11:03:21.446774+0000","last_peered":"2023-11-13T11:03:21.446774+0000","last_clean":"2023-11-13T11:03:21.446774+0000","last_became_active":"2023-11-13T11:03:20.466353+0000","last_became_peered":"2023-11-13T11:03:20.466353+0000","last_unstale":"2023-11-13T11:03:21.446774+0000","last_undegraded":"2023-11-13T11:03:21.446774+0000","last_fullsized":"2023-11-13T11:03:21.446774+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:02:35.586086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.473152+0000","last_change":"2023-11-13T11:03:19.465601+0000","last_active":"2023-11-13T11:03:20.473152+0000","last_peered":"2023-11-13T11:03:20.473152+0000","last_clean":"2023-11-13T11:03:20.473152+0000","last_became_active":"2023-11-13T11:03:19.465326+0000","last_became_peered":"2023-11-13T11:03:19.465326+0000","last_unstale":"2023-11-13T11:03:20.473152+0000","last_undegraded":"2023-11-13T11:03:20.473152+0000","last_fullsized":"2023-11-13T11:03:20.473152+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:31.813846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297069+0000","last_change":"2023-11-13T11:03:20.459160+0000","last_active":"2023-11-13T11:03:22.297069+0000","last_peered":"2023-11-13T11:03:22.297069+0000","last_clean":"2023-11-13T11:03:22.297069+0000","last_became_active":"2023-11-13T11:03:20.458674+0000","last_became_peered":"2023-11-13T11:03:20.458674+0000","last_unstale":"2023-11-13T11:03:22.297069+0000","last_undegraded":"2023-11-13T11:03:22.297069+0000","last_fullsized":"2023-11-13T11:03:22.297069+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:17:24.322051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.464388+0000","last_change":"2023-11-13T11:03:19.458487+0000","last_active":"2023-11-13T11:03:20.464388+0000","last_peered":"2023-11-13T11:03:20.464388+0000","last_clean":"2023-11-13T11:03:20.464388+0000","last_became_active":"2023-11-13T11:03:19.457021+0000","last_became_peered":"2023-11-13T11:03:19.457021+0000","last_unstale":"2023-11-13T11:03:20.464388+0000","last_undegraded":"2023-11-13T11:03:20.464388+0000","last_fullsized":"2023-11-13T11:03:20.464388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:58:49.422056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298690+0000","last_change":"2023-11-13T11:03:20.479550+0000","last_active":"2023-11-13T11:03:22.298690+0000","last_peered":"2023-11-13T11:03:22.298690+0000","last_clean":"2023-11-13T11:03:22.298690+0000","last_became_active":"2023-11-13T11:03:20.478812+0000","last_became_peered":"2023-11-13T11:03:20.478812+0000","last_unstale":"2023-11-13T11:03:22.298690+0000","last_undegraded":"2023-11-13T11:03:22.298690+0000","last_fullsized":"2023-11-13T11:03:22.298690+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:06.214101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"68'3","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492480+0000","last_change":"2023-11-13T11:03:19.467377+0000","last_active":"2023-11-13T11:03:20.492480+0000","last_peered":"2023-11-13T11:03:20.492480+0000","last_clean":"2023-11-13T11:03:20.492480+0000","last_became_active":"2023-11-13T11:03:19.467124+0000","last_became_peered":"2023-11-13T11:03:19.467124+0000","last_unstale":"2023-11-13T11:03:20.492480+0000","last_undegraded":"2023-11-13T11:03:20.492480+0000","last_fullsized":"2023-11-13T11:03:20.492480+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:58:15.706470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298181+0000","last_change":"2023-11-13T11:03:20.480420+0000","last_active":"2023-11-13T11:03:22.298181+0000","last_peered":"2023-11-13T11:03:22.298181+0000","last_clean":"2023-11-13T11:03:22.298181+0000","last_became_active":"2023-11-13T11:03:20.480145+0000","last_became_peered":"2023-11-13T11:03:20.480145+0000","last_unstale":"2023-11-13T11:03:22.298181+0000","last_undegraded":"2023-11-13T11:03:22.298181+0000","last_fullsized":"2023-11-13T11:03:22.298181+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:52:21.636883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"68'6","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.487942+0000","last_change":"2023-11-13T11:03:19.465606+0000","last_active":"2023-11-13T11:03:20.487942+0000","last_peered":"2023-11-13T11:03:20.487942+0000","last_clean":"2023-11-13T11:03:20.487942+0000","last_became_active":"2023-11-13T11:03:19.465133+0000","last_became_peered":"2023-11-13T11:03:19.465133+0000","last_unstale":"2023-11-13T11:03:20.487942+0000","last_undegraded":"2023-11-13T11:03:20.487942+0000","last_fullsized":"2023-11-13T11:03:20.487942+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:36:21.280952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836272+0000","last_change":"2023-11-13T11:03:20.476387+0000","last_active":"2023-11-13T11:03:21.836272+0000","last_peered":"2023-11-13T11:03:21.836272+0000","last_clean":"2023-11-13T11:03:21.836272+0000","last_became_active":"2023-11-13T11:03:20.476116+0000","last_became_peered":"2023-11-13T11:03:20.476116+0000","last_unstale":"2023-11-13T11:03:21.836272+0000","last_undegraded":"2023-11-13T11:03:21.836272+0000","last_fullsized":"2023-11-13T11:03:21.836272+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:28:15.701895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.493013+0000","last_change":"2023-11-13T11:03:19.463340+0000","last_active":"2023-11-13T11:03:20.493013+0000","last_peered":"2023-11-13T11:03:20.493013+0000","last_clean":"2023-11-13T11:03:20.493013+0000","last_became_active":"2023-11-13T11:03:19.463065+0000","last_became_peered":"2023-11-13T11:03:19.463065+0000","last_unstale":"2023-11-13T11:03:20.493013+0000","last_undegraded":"2023-11-13T11:03:20.493013+0000","last_fullsized":"2023-11-13T11:03:20.493013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:05.196692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836185+0000","last_change":"2023-11-13T11:03:20.462404+0000","last_active":"2023-11-13T11:03:21.836185+0000","last_peered":"2023-11-13T11:03:21.836185+0000","last_clean":"2023-11-13T11:03:21.836185+0000","last_became_active":"2023-11-13T11:03:20.462106+0000","last_became_peered":"2023-11-13T11:03:20.462106+0000","last_unstale":"2023-11-13T11:03:21.836185+0000","last_undegraded":"2023-11-13T11:03:21.836185+0000","last_fullsized":"2023-11-13T11:03:21.836185+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.593902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.441386+0000","last_change":"2023-11-13T11:03:19.462210+0000","last_active":"2023-11-13T11:03:20.441386+0000","last_peered":"2023-11-13T11:03:20.441386+0000","last_clean":"2023-11-13T11:03:20.441386+0000","last_became_active":"2023-11-13T11:03:19.461775+0000","last_became_peered":"2023-11-13T11:03:19.461775+0000","last_unstale":"2023-11-13T11:03:20.441386+0000","last_undegraded":"2023-11-13T11:03:20.441386+0000","last_fullsized":"2023-11-13T11:03:20.441386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:52:49.857765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836506+0000","last_change":"2023-11-13T11:03:20.456308+0000","last_active":"2023-11-13T11:03:21.836506+0000","last_peered":"2023-11-13T11:03:21.836506+0000","last_clean":"2023-11-13T11:03:21.836506+0000","last_became_active":"2023-11-13T11:03:20.454264+0000","last_became_peered":"2023-11-13T11:03:20.454264+0000","last_unstale":"2023-11-13T11:03:21.836506+0000","last_undegraded":"2023-11-13T11:03:21.836506+0000","last_fullsized":"2023-11-13T11:03:21.836506+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:41.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.457139+0000","last_change":"2023-11-13T11:03:19.457132+0000","last_active":"2023-11-13T11:03:20.457139+0000","last_peered":"2023-11-13T11:03:20.457139+0000","last_clean":"2023-11-13T11:03:20.457139+0000","last_became_active":"2023-11-13T11:03:19.455974+0000","last_became_peered":"2023-11-13T11:03:19.455974+0000","last_unstale":"2023-11-13T11:03:20.457139+0000","last_undegraded":"2023-11-13T11:03:20.457139+0000","last_fullsized":"2023-11-13T11:03:20.457139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:10:12.732364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.445033+0000","last_change":"2023-11-13T11:03:20.468314+0000","last_active":"2023-11-13T11:03:21.445033+0000","last_peered":"2023-11-13T11:03:21.445033+0000","last_clean":"2023-11-13T11:03:21.445033+0000","last_became_active":"2023-11-13T11:03:20.468045+0000","last_became_peered":"2023-11-13T11:03:20.468045+0000","last_unstale":"2023-11-13T11:03:21.445033+0000","last_undegraded":"2023-11-13T11:03:21.445033+0000","last_fullsized":"2023-11-13T11:03:21.445033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:41:18.051038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"68'8","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492661+0000","last_change":"2023-11-13T11:03:19.461004+0000","last_active":"2023-11-13T11:03:20.492661+0000","last_peered":"2023-11-13T11:03:20.492661+0000","last_clean":"2023-11-13T11:03:20.492661+0000","last_became_active":"2023-11-13T11:03:19.460772+0000","last_became_peered":"2023-11-13T11:03:19.460772+0000","last_unstale":"2023-11-13T11:03:20.492661+0000","last_undegraded":"2023-11-13T11:03:20.492661+0000","last_fullsized":"2023-11-13T11:03:20.492661+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:43:02.704304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442230+0000","last_change":"2023-11-13T11:03:20.487082+0000","last_active":"2023-11-13T11:03:21.442230+0000","last_peered":"2023-11-13T11:03:21.442230+0000","last_clean":"2023-11-13T11:03:21.442230+0000","last_became_active":"2023-11-13T11:03:20.486346+0000","last_became_peered":"2023-11-13T11:03:20.486346+0000","last_unstale":"2023-11-13T11:03:21.442230+0000","last_undegraded":"2023-11-13T11:03:21.442230+0000","last_fullsized":"2023-11-13T11:03:21.442230+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:38:34.204968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492262+0000","last_change":"2023-11-13T11:03:19.457153+0000","last_active":"2023-11-13T11:03:20.492262+0000","last_peered":"2023-11-13T11:03:20.492262+0000","last_clean":"2023-11-13T11:03:20.492262+0000","last_became_active":"2023-11-13T11:03:19.456874+0000","last_became_peered":"2023-11-13T11:03:19.456874+0000","last_unstale":"2023-11-13T11:03:20.492262+0000","last_undegraded":"2023-11-13T11:03:20.492262+0000","last_fullsized":"2023-11-13T11:03:20.492262+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:36:21.280952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299367+0000","last_change":"2023-11-13T11:03:20.470055+0000","last_active":"2023-11-13T11:03:22.299367+0000","last_peered":"2023-11-13T11:03:22.299367+0000","last_clean":"2023-11-13T11:03:22.299367+0000","last_became_active":"2023-11-13T11:03:20.469745+0000","last_became_peered":"2023-11-13T11:03:20.469745+0000","last_unstale":"2023-11-13T11:03:22.299367+0000","last_undegraded":"2023-11-13T11:03:22.299367+0000","last_fullsized":"2023-11-13T11:03:22.299367+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:38:54.348922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492118+0000","last_change":"2023-11-13T11:03:19.460723+0000","last_active":"2023-11-13T11:03:20.492118+0000","last_peered":"2023-11-13T11:03:20.492118+0000","last_clean":"2023-11-13T11:03:20.492118+0000","last_became_active":"2023-11-13T11:03:19.460398+0000","last_became_peered":"2023-11-13T11:03:19.460398+0000","last_unstale":"2023-11-13T11:03:20.492118+0000","last_undegraded":"2023-11-13T11:03:20.492118+0000","last_fullsized":"2023-11-13T11:03:20.492118+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:40.173268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299033+0000","last_change":"2023-11-13T11:03:20.469995+0000","last_active":"2023-11-13T11:03:22.299033+0000","last_peered":"2023-11-13T11:03:22.299033+0000","last_clean":"2023-11-13T11:03:22.299033+0000","last_became_active":"2023-11-13T11:03:20.469673+0000","last_became_peered":"2023-11-13T11:03:20.469673+0000","last_unstale":"2023-11-13T11:03:22.299033+0000","last_undegraded":"2023-11-13T11:03:22.299033+0000","last_fullsized":"2023-11-13T11:03:22.299033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.869476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492197+0000","last_change":"2023-11-13T11:03:19.462642+0000","last_active":"2023-11-13T11:03:20.492197+0000","last_peered":"2023-11-13T11:03:20.492197+0000","last_clean":"2023-11-13T11:03:20.492197+0000","last_became_active":"2023-11-13T11:03:19.462365+0000","last_became_peered":"2023-11-13T11:03:19.462365+0000","last_unstale":"2023-11-13T11:03:20.492197+0000","last_undegraded":"2023-11-13T11:03:20.492197+0000","last_fullsized":"2023-11-13T11:03:20.492197+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:05:17.285018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299164+0000","last_change":"2023-11-13T11:03:20.474321+0000","last_active":"2023-11-13T11:03:22.299164+0000","last_peered":"2023-11-13T11:03:22.299164+0000","last_clean":"2023-11-13T11:03:22.299164+0000","last_became_active":"2023-11-13T11:03:20.468744+0000","last_became_peered":"2023-11-13T11:03:20.468744+0000","last_unstale":"2023-11-13T11:03:22.299164+0000","last_undegraded":"2023-11-13T11:03:22.299164+0000","last_fullsized":"2023-11-13T11:03:22.299164+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:59:35.919204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059038+0000","last_change":"2023-11-13T11:03:19.469381+0000","last_active":"2023-11-13T11:03:22.059038+0000","last_peered":"2023-11-13T11:03:22.059038+0000","last_clean":"2023-11-13T11:03:22.059038+0000","last_became_active":"2023-11-13T11:03:19.468768+0000","last_became_peered":"2023-11-13T11:03:19.468768+0000","last_unstale":"2023-11-13T11:03:22.059038+0000","last_undegraded":"2023-11-13T11:03:22.059038+0000","last_fullsized":"2023-11-13T11:03:22.059038+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:37:23.154386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.445436+0000","last_change":"2023-11-13T11:03:20.473666+0000","last_active":"2023-11-13T11:03:21.445436+0000","last_peered":"2023-11-13T11:03:21.445436+0000","last_clean":"2023-11-13T11:03:21.445436+0000","last_became_active":"2023-11-13T11:03:20.472677+0000","last_became_peered":"2023-11-13T11:03:20.472677+0000","last_unstale":"2023-11-13T11:03:21.445436+0000","last_undegraded":"2023-11-13T11:03:21.445436+0000","last_fullsized":"2023-11-13T11:03:21.445436+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:27:37.299626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.488193+0000","last_change":"2023-11-13T11:03:19.470541+0000","last_active":"2023-11-13T11:03:20.488193+0000","last_peered":"2023-11-13T11:03:20.488193+0000","last_clean":"2023-11-13T11:03:20.488193+0000","last_became_active":"2023-11-13T11:03:19.470025+0000","last_became_peered":"2023-11-13T11:03:19.470025+0000","last_unstale":"2023-11-13T11:03:20.488193+0000","last_undegraded":"2023-11-13T11:03:20.488193+0000","last_fullsized":"2023-11-13T11:03:20.488193+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:41:17.033629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296924+0000","last_change":"2023-11-13T11:03:20.467427+0000","last_active":"2023-11-13T11:03:22.296924+0000","last_peered":"2023-11-13T11:03:22.296924+0000","last_clean":"2023-11-13T11:03:22.296924+0000","last_became_active":"2023-11-13T11:03:20.467133+0000","last_became_peered":"2023-11-13T11:03:20.467133+0000","last_unstale":"2023-11-13T11:03:22.296924+0000","last_undegraded":"2023-11-13T11:03:22.296924+0000","last_fullsized":"2023-11-13T11:03:22.296924+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.593902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.442840+0000","last_change":"2023-11-13T11:03:19.460424+0000","last_active":"2023-11-13T11:03:20.442840+0000","last_peered":"2023-11-13T11:03:20.442840+0000","last_clean":"2023-11-13T11:03:20.442840+0000","last_became_active":"2023-11-13T11:03:19.460137+0000","last_became_peered":"2023-11-13T11:03:19.460137+0000","last_unstale":"2023-11-13T11:03:20.442840+0000","last_undegraded":"2023-11-13T11:03:20.442840+0000","last_fullsized":"2023-11-13T11:03:20.442840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:42:27.040835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441389+0000","last_change":"2023-11-13T11:03:20.472596+0000","last_active":"2023-11-13T11:03:21.441389+0000","last_peered":"2023-11-13T11:03:21.441389+0000","last_clean":"2023-11-13T11:03:21.441389+0000","last_became_active":"2023-11-13T11:03:20.472243+0000","last_became_peered":"2023-11-13T11:03:20.472243+0000","last_unstale":"2023-11-13T11:03:21.441389+0000","last_undegraded":"2023-11-13T11:03:21.441389+0000","last_fullsized":"2023-11-13T11:03:21.441389+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:15.570305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.445684+0000","last_change":"2023-11-13T11:03:19.462649+0000","last_active":"2023-11-13T11:03:20.445684+0000","last_peered":"2023-11-13T11:03:20.445684+0000","last_clean":"2023-11-13T11:03:20.445684+0000","last_became_active":"2023-11-13T11:03:19.462308+0000","last_became_peered":"2023-11-13T11:03:19.462308+0000","last_unstale":"2023-11-13T11:03:20.445684+0000","last_undegraded":"2023-11-13T11:03:20.445684+0000","last_fullsized":"2023-11-13T11:03:20.445684+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:21.547257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297182+0000","last_change":"2023-11-13T11:03:20.454510+0000","last_active":"2023-11-13T11:03:22.297182+0000","last_peered":"2023-11-13T11:03:22.297182+0000","last_clean":"2023-11-13T11:03:22.297182+0000","last_became_active":"2023-11-13T11:03:20.454165+0000","last_became_peered":"2023-11-13T11:03:20.454165+0000","last_unstale":"2023-11-13T11:03:22.297182+0000","last_undegraded":"2023-11-13T11:03:22.297182+0000","last_fullsized":"2023-11-13T11:03:22.297182+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:48:04.143689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492509+0000","last_change":"2023-11-13T11:03:19.456513+0000","last_active":"2023-11-13T11:03:20.492509+0000","last_peered":"2023-11-13T11:03:20.492509+0000","last_clean":"2023-11-13T11:03:20.492509+0000","last_became_active":"2023-11-13T11:03:19.455795+0000","last_became_peered":"2023-11-13T11:03:19.455795+0000","last_unstale":"2023-11-13T11:03:20.492509+0000","last_undegraded":"2023-11-13T11:03:20.492509+0000","last_fullsized":"2023-11-13T11:03:20.492509+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:38:53.331513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297194+0000","last_change":"2023-11-13T11:03:20.464089+0000","last_active":"2023-11-13T11:03:22.297194+0000","last_peered":"2023-11-13T11:03:22.297194+0000","last_clean":"2023-11-13T11:03:22.297194+0000","last_became_active":"2023-11-13T11:03:20.463759+0000","last_became_peered":"2023-11-13T11:03:20.463759+0000","last_unstale":"2023-11-13T11:03:22.297194+0000","last_undegraded":"2023-11-13T11:03:22.297194+0000","last_fullsized":"2023-11-13T11:03:22.297194+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:17:06.846928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442339+0000","last_change":"2023-11-13T11:03:19.460123+0000","last_active":"2023-11-13T11:03:21.442339+0000","last_peered":"2023-11-13T11:03:21.442339+0000","last_clean":"2023-11-13T11:03:21.442339+0000","last_became_active":"2023-11-13T11:03:19.459856+0000","last_became_peered":"2023-11-13T11:03:19.459856+0000","last_unstale":"2023-11-13T11:03:21.442339+0000","last_undegraded":"2023-11-13T11:03:21.442339+0000","last_fullsized":"2023-11-13T11:03:21.442339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:15.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298176+0000","last_change":"2023-11-13T11:03:20.469332+0000","last_active":"2023-11-13T11:03:22.298176+0000","last_peered":"2023-11-13T11:03:22.298176+0000","last_clean":"2023-11-13T11:03:22.298176+0000","last_became_active":"2023-11-13T11:03:20.469074+0000","last_became_peered":"2023-11-13T11:03:20.469074+0000","last_unstale":"2023-11-13T11:03:22.298176+0000","last_undegraded":"2023-11-13T11:03:22.298176+0000","last_fullsized":"2023-11-13T11:03:22.298176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:15:47.761631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.445273+0000","last_change":"2023-11-13T11:03:19.468701+0000","last_active":"2023-11-13T11:03:20.445273+0000","last_peered":"2023-11-13T11:03:20.445273+0000","last_clean":"2023-11-13T11:03:20.445273+0000","last_became_active":"2023-11-13T11:03:19.468415+0000","last_became_peered":"2023-11-13T11:03:19.468415+0000","last_unstale":"2023-11-13T11:03:20.445273+0000","last_undegraded":"2023-11-13T11:03:20.445273+0000","last_fullsized":"2023-11-13T11:03:20.445273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:11:28.058672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298633+0000","last_change":"2023-11-13T11:03:20.469925+0000","last_active":"2023-11-13T11:03:22.298633+0000","last_peered":"2023-11-13T11:03:22.298633+0000","last_clean":"2023-11-13T11:03:22.298633+0000","last_became_active":"2023-11-13T11:03:20.469129+0000","last_became_peered":"2023-11-13T11:03:20.469129+0000","last_unstale":"2023-11-13T11:03:22.298633+0000","last_undegraded":"2023-11-13T11:03:22.298633+0000","last_fullsized":"2023-11-13T11:03:22.298633+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:30:08.514257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059174+0000","last_change":"2023-11-13T11:03:19.468809+0000","last_active":"2023-11-13T11:03:22.059174+0000","last_peered":"2023-11-13T11:03:22.059174+0000","last_clean":"2023-11-13T11:03:22.059174+0000","last_became_active":"2023-11-13T11:03:19.467750+0000","last_became_peered":"2023-11-13T11:03:19.467750+0000","last_unstale":"2023-11-13T11:03:22.059174+0000","last_undegraded":"2023-11-13T11:03:22.059174+0000","last_fullsized":"2023-11-13T11:03:22.059174+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:45:50.416913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441062+0000","last_change":"2023-11-13T11:03:20.466160+0000","last_active":"2023-11-13T11:03:21.441062+0000","last_peered":"2023-11-13T11:03:21.441062+0000","last_clean":"2023-11-13T11:03:21.441062+0000","last_became_active":"2023-11-13T11:03:20.465675+0000","last_became_peered":"2023-11-13T11:03:20.465675+0000","last_unstale":"2023-11-13T11:03:21.441062+0000","last_undegraded":"2023-11-13T11:03:21.441062+0000","last_fullsized":"2023-11-13T11:03:21.441062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:27:37.299626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.461189+0000","last_change":"2023-11-13T11:03:19.465809+0000","last_active":"2023-11-13T11:03:20.461189+0000","last_peered":"2023-11-13T11:03:20.461189+0000","last_clean":"2023-11-13T11:03:20.461189+0000","last_became_active":"2023-11-13T11:03:19.465063+0000","last_became_peered":"2023-11-13T11:03:19.465063+0000","last_unstale":"2023-11-13T11:03:20.461189+0000","last_undegraded":"2023-11-13T11:03:20.461189+0000","last_fullsized":"2023-11-13T11:03:20.461189+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:00:41.592546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298363+0000","last_change":"2023-11-13T11:03:20.476368+0000","last_active":"2023-11-13T11:03:22.298363+0000","last_peered":"2023-11-13T11:03:22.298363+0000","last_clean":"2023-11-13T11:03:22.298363+0000","last_became_active":"2023-11-13T11:03:20.476109+0000","last_became_peered":"2023-11-13T11:03:20.476109+0000","last_unstale":"2023-11-13T11:03:22.298363+0000","last_undegraded":"2023-11-13T11:03:22.298363+0000","last_fullsized":"2023-11-13T11:03:22.298363+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:22.564666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.458999+0000","last_change":"2023-11-13T11:03:19.463189+0000","last_active":"2023-11-13T11:03:20.458999+0000","last_peered":"2023-11-13T11:03:20.458999+0000","last_clean":"2023-11-13T11:03:20.458999+0000","last_became_active":"2023-11-13T11:03:19.461814+0000","last_became_peered":"2023-11-13T11:03:19.461814+0000","last_unstale":"2023-11-13T11:03:20.458999+0000","last_undegraded":"2023-11-13T11:03:20.458999+0000","last_fullsized":"2023-11-13T11:03:20.458999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:05.196692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.446764+0000","last_change":"2023-11-13T11:03:20.474978+0000","last_active":"2023-11-13T11:03:21.446764+0000","last_peered":"2023-11-13T11:03:21.446764+0000","last_clean":"2023-11-13T11:03:21.446764+0000","last_became_active":"2023-11-13T11:03:20.474677+0000","last_became_peered":"2023-11-13T11:03:20.474677+0000","last_unstale":"2023-11-13T11:03:21.446764+0000","last_undegraded":"2023-11-13T11:03:21.446764+0000","last_fullsized":"2023-11-13T11:03:21.446764+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:15.570305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058922+0000","last_change":"2023-11-13T11:03:19.468080+0000","last_active":"2023-11-13T11:03:22.058922+0000","last_peered":"2023-11-13T11:03:22.058922+0000","last_clean":"2023-11-13T11:03:22.058922+0000","last_became_active":"2023-11-13T11:03:19.467716+0000","last_became_peered":"2023-11-13T11:03:19.467716+0000","last_unstale":"2023-11-13T11:03:22.058922+0000","last_undegraded":"2023-11-13T11:03:22.058922+0000","last_fullsized":"2023-11-13T11:03:22.058922+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:57:39.710883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296906+0000","last_change":"2023-11-13T11:03:20.475905+0000","last_active":"2023-11-13T11:03:22.296906+0000","last_peered":"2023-11-13T11:03:22.296906+0000","last_clean":"2023-11-13T11:03:22.296906+0000","last_became_active":"2023-11-13T11:03:20.474090+0000","last_became_peered":"2023-11-13T11:03:20.474090+0000","last_unstale":"2023-11-13T11:03:22.296906+0000","last_undegraded":"2023-11-13T11:03:22.296906+0000","last_fullsized":"2023-11-13T11:03:22.296906+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:57:32.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.446020+0000","last_change":"2023-11-13T11:03:19.459075+0000","last_active":"2023-11-13T11:03:20.446020+0000","last_peered":"2023-11-13T11:03:20.446020+0000","last_clean":"2023-11-13T11:03:20.446020+0000","last_became_active":"2023-11-13T11:03:19.458782+0000","last_became_peered":"2023-11-13T11:03:19.458782+0000","last_unstale":"2023-11-13T11:03:20.446020+0000","last_undegraded":"2023-11-13T11:03:20.446020+0000","last_fullsized":"2023-11-13T11:03:20.446020+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:05.196692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836855+0000","last_change":"2023-11-13T11:03:20.465840+0000","last_active":"2023-11-13T11:03:21.836855+0000","last_peered":"2023-11-13T11:03:21.836855+0000","last_clean":"2023-11-13T11:03:21.836855+0000","last_became_active":"2023-11-13T11:03:20.465501+0000","last_became_peered":"2023-11-13T11:03:20.465501+0000","last_unstale":"2023-11-13T11:03:21.836855+0000","last_undegraded":"2023-11-13T11:03:21.836855+0000","last_fullsized":"2023-11-13T11:03:21.836855+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:22.564666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.446370+0000","last_change":"2023-11-13T11:03:19.464113+0000","last_active":"2023-11-13T11:03:20.446370+0000","last_peered":"2023-11-13T11:03:20.446370+0000","last_clean":"2023-11-13T11:03:20.446370+0000","last_became_active":"2023-11-13T11:03:19.463678+0000","last_became_peered":"2023-11-13T11:03:19.463678+0000","last_unstale":"2023-11-13T11:03:20.446370+0000","last_undegraded":"2023-11-13T11:03:20.446370+0000","last_fullsized":"2023-11-13T11:03:20.446370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:31.813846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":208,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.440483+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:03:20.440483+0000","last_peered":"2023-11-13T11:03:20.440483+0000","last_clean":"2023-11-13T11:03:20.440483+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:03:20.440483+0000","last_undegraded":"2023-11-13T11:03:20.440483+0000","last_fullsized":"2023-11-13T11:03:20.440483+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44488,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327352,"statfs":{"total":96636764160,"available":96591208448,"internally_reserved":0,"allocated":847872,"data_stored":374056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1060000000000001,"15min":1.1259999999999999},"min":{"1min":0.73899999999999999,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.706,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.87},{"interface":"front","average":{"1min":1.153,"5min":1.2250000000000001,"15min":1.1379999999999999},"min":{"1min":0.82199999999999995,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.498,"5min":1.865,"15min":2.2879999999999998},"last":0.97599999999999998}]},{"osd":1,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.252,"15min":1.1799999999999999},"min":{"1min":0.90300000000000002,"5min":0.64000000000000001,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.2889999999999999},"min":{"1min":0.67700000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.649,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.125}]},{"osd":2,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.319,"15min":1.276},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.534,"5min":2.153,"15min":2.1680000000000001},"last":1.177},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.383,"15min":1.3460000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.6100000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.5680000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.429,"15min":1.3779999999999999},"min":{"1min":0.81799999999999995,"5min":0.76300000000000001,"15min":0.63800000000000001},"max":{"1min":1.657,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.53},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.458,"15min":1.4379999999999999},"min":{"1min":0.99299999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.762,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.423}]},{"osd":4,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.456,"15min":1.552},"min":{"1min":0.94899999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.782,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.355},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.512,"15min":1.5740000000000001},"min":{"1min":1.1599999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.9419999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.2649999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.6060000000000001,"15min":1.6399999999999999},"min":{"1min":0.94499999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.8200000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.625,"5min":1.75,"15min":1.7490000000000001},"min":{"1min":1.4650000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":1.865,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.746}]},{"osd":7,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.6930000000000001,"15min":1.645},"min":{"1min":1.2849999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0640000000000001,"5min":2.657,"15min":2.657},"last":1.639},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.641,"15min":1.6379999999999999},"min":{"1min":1.1399999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1349999999999998,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.589}]}]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45068,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326772,"statfs":{"total":96636764160,"available":96590614528,"internally_reserved":0,"allocated":1441792,"data_stored":963884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.093,"15min":1.016},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7529999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.2190000000000001,"15min":1.202},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67800000000000005},"max":{"1min":1.639,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.419}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.306,"15min":1.2350000000000001},"min":{"1min":0.79100000000000004,"5min":0.72399999999999998,"15min":0.65400000000000003},"max":{"1min":1.73,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.228},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.282,"15min":1.2450000000000001},"min":{"1min":0.875,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.94499999999999995}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.389,"15min":1.2649999999999999},"min":{"1min":0.81200000000000006,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":4.4500000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.3100000000000001,"15min":1.2370000000000001},"min":{"1min":0.78900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5760000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.3660000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.387,"15min":1.391},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8300000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.615},{"interface":"front","average":{"1min":1.53,"5min":1.5169999999999999,"15min":1.4870000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.80700000000000005},"max":{"1min":4.6959999999999997,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":1.1539999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3700000000000001,"15min":1.329},"min":{"1min":0.93500000000000005,"5min":0.86599999999999999,"15min":0.55500000000000005},"max":{"1min":1.5800000000000001,"5min":3.484,"15min":3.484},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.518,"15min":1.53},"min":{"1min":1.093,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.4430000000000001,"5min":3.653,"15min":3.653},"last":1.5009999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.5800000000000001,"15min":1.6040000000000001},"min":{"1min":1.0800000000000001,"5min":0.92000000000000004,"15min":0.78000000000000003},"max":{"1min":4.8760000000000003,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.538,"15min":1.5189999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.61799999999999999},"max":{"1min":1.7609999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.6419999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.645,"15min":1.6659999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.1349999999999998,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.744,"5min":1.7330000000000001,"15min":1.714},"min":{"1min":1.2490000000000001,"5min":1.1200000000000001,"15min":0.69499999999999995},"max":{"1min":4.7919999999999998,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":1.5309999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54200,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317640,"statfs":{"total":96636764160,"available":96581263360,"internally_reserved":0,"allocated":831488,"data_stored":372374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.153,"15min":1.089},"min":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":1.986,"15min":1.986},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.145,"15min":1.085},"min":{"1min":0.69199999999999995,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.83099999999999996}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2470000000000001,"15min":1.163},"min":{"1min":0.76200000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.774,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.26},"min":{"1min":0.79300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":1.877,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.2390000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.298,"15min":1.242},"min":{"1min":0.93300000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.575,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.347,"15min":1.349},"min":{"1min":0.91100000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.72,"5min":1.952,"15min":1.952},"last":1.6479999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.488,"15min":1.421},"min":{"1min":1.1080000000000001,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.786,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.345},{"interface":"front","average":{"1min":1.379,"5min":1.3149999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.841,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.2909999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.478,"15min":1.4910000000000001},"min":{"1min":0.96399999999999997,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.778,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.371,"15min":1.3939999999999999},"min":{"1min":0.81599999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":2.0529999999999999,"5min":2.089,"15min":2.089},"last":1.0429999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.621,"15min":1.6220000000000001},"min":{"1min":1.0600000000000001,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":1.7769999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.524},{"interface":"front","average":{"1min":1.401,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.92400000000000004,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":1.873,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.327}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.5369999999999999,"15min":1.5820000000000001},"min":{"1min":1.05,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":2.1480000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.599,"15min":1.609},"min":{"1min":1.0189999999999999,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":1.931,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.381}]}]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40316,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331524,"statfs":{"total":96636764160,"available":96595480576,"internally_reserved":0,"allocated":835584,"data_stored":371952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.628,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.282,"5min":1.867,"15min":1.867},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.022,"15min":1.026},"min":{"1min":0.65800000000000003,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.819,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":1.0569999999999999}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.032,"15min":1.0489999999999999},"min":{"1min":0.625,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.8089999999999999,"5min":1.966,"15min":3.71},"last":1.534},{"interface":"front","average":{"1min":1.296,"5min":1.1240000000000001,"15min":1.103},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":1.26}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.179,"15min":1.1040000000000001},"min":{"1min":0.72899999999999998,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":2.0019999999999998,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.24,"5min":1.171,"15min":1.2829999999999999},"min":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7809999999999999,"5min":1.952,"15min":3.5710000000000002},"last":0.80800000000000005}]},{"osd":4,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.89200000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.7509999999999999,"5min":2.411,"15min":3.5030000000000001},"last":2.0779999999999998},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3879999999999999,"15min":1.355},"min":{"1min":1.1259999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.831,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.21}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4179999999999999,"15min":1.4750000000000001},"min":{"1min":1.151,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.99,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.357},{"interface":"front","average":{"1min":1.46,"5min":1.454,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0649999999999999,"5min":2.673,"15min":3.3809999999999998},"last":2.21}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.49},"min":{"1min":1.1479999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.944,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.546,"5min":1.4870000000000001,"15min":1.5840000000000001},"min":{"1min":1.1060000000000001,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":2.093,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.1539999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5609999999999999,"15min":1.609},"min":{"1min":0.84799999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1339999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":2.1579999999999999},{"interface":"front","average":{"1min":1.649,"5min":1.6040000000000001,"15min":1.5620000000000001},"min":{"1min":1.1359999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.4089999999999998,"5min":2.75,"15min":2.75},"last":1.4750000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509541,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50052,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321788,"statfs":{"total":96636764160,"available":96585510912,"internally_reserved":0,"allocated":835584,"data_stored":371848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94199999999999995,"15min":0.93100000000000005},"min":{"1min":0.55700000000000005,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":1.089,"5min":1.0229999999999999,"15min":0.93400000000000005},"min":{"1min":0.54400000000000004,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":2.9460000000000002,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":0.629}]},{"osd":1,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0309999999999999,"15min":0.96299999999999997},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.376},"max":{"1min":3.0720000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.68100000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.3380000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":0.81699999999999995}]},{"osd":3,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.216,"15min":1.2430000000000001},"min":{"1min":0.71599999999999997,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2549999999999999,"15min":1.2749999999999999},"min":{"1min":0.72699999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.4060000000000001,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.1080000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4359999999999999,"15min":1.4730000000000001},"min":{"1min":0.89900000000000002,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":2.633,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.393,"15min":1.409},"min":{"1min":0.80000000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":1.1970000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.4690000000000001,"15min":1.482},"min":{"1min":1.034,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.1680000000000001,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":1.534},{"interface":"front","average":{"1min":1.623,"5min":1.591,"15min":1.5680000000000001},"min":{"1min":0.97099999999999997,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.7509999999999999,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.256}]},{"osd":6,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5189999999999999,"15min":1.55},"min":{"1min":0.89500000000000002,"5min":0.80900000000000005,"15min":0.76500000000000001},"max":{"1min":3.395,"5min":3.395,"15min":4.7569999999999997},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6120000000000001,"15min":1.599},"min":{"1min":1.2130000000000001,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.603}]},{"osd":7,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.5700000000000001,"15min":1.607},"min":{"1min":1.0569999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":3.2429999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.694,"5min":1.6479999999999999,"15min":1.6259999999999999},"min":{"1min":1.2090000000000001,"5min":0.91200000000000003,"15min":0.82799999999999996},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":1.6579999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509540,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28416,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343424,"statfs":{"total":96636764160,"available":96607666176,"internally_reserved":0,"allocated":1425408,"data_stored":962194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85699999999999998,"15min":0.94899999999999995},"min":{"1min":0.495,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.4910000000000001,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":1.127},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.97399999999999998,"15min":0.95699999999999996},"min":{"1min":0.623,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2689999999999999,"5min":1.869,"15min":1.869},"last":1.075}]},{"osd":2,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.0509999999999999,"15min":1},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0680000000000001,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":1.167}]},{"osd":3,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.204,"15min":1.175},"min":{"1min":0.68600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.577},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.57799999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":0.72699999999999998}]},{"osd":4,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.321,"15min":1.3129999999999999},"min":{"1min":0.76200000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.536,"5min":1.4179999999999999,"15min":1.3859999999999999},"min":{"1min":0.69599999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.0299999999999998,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.4119999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4159999999999999,"15min":1.4139999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.0750000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.387,"15min":1.4079999999999999},"min":{"1min":1.0309999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9410000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.474}]},{"osd":6,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.341,"15min":1.407},"min":{"1min":0.90900000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.204},{"interface":"front","average":{"1min":1.649,"5min":1.504,"15min":1.5069999999999999},"min":{"1min":1.0640000000000001,"5min":0.79400000000000004,"15min":0.752},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3879999999999999},"last":1.24}]},{"osd":7,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.466,"15min":1.494},"min":{"1min":1.228,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.0609999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.292},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4399999999999999,"15min":1.4570000000000001},"min":{"1min":0.96899999999999997,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.3360000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44472,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327368,"statfs":{"total":96636764160,"available":96591224832,"internally_reserved":0,"allocated":827392,"data_stored":371792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.872,"15min":0.90100000000000002},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.32100000000000001},"max":{"1min":1.7010000000000001,"5min":1.74,"15min":1.74},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0449999999999999,"15min":1.048},"min":{"1min":0.46700000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.736,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.748}]},{"osd":2,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.1890000000000001,"15min":1.151},"min":{"1min":0.55300000000000005,"5min":0.441,"15min":0.441},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.262},{"interface":"front","average":{"1min":1.161,"5min":1.0700000000000001,"15min":1.0269999999999999},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.371},"max":{"1min":1.968,"5min":1.968,"15min":2.1669999999999998},"last":0.80300000000000005}]},{"osd":3,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.1439999999999999,"15min":1.111},"min":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.45200000000000001},"max":{"1min":1.9259999999999999,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.59799999999999998,"5min":0.497,"15min":0.497},"max":{"1min":2.2240000000000002,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":0.88100000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.3879999999999999,"15min":1.3660000000000001},"min":{"1min":0.82199999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":4.4279999999999999,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.3580000000000001,"15min":1.4319999999999999},"min":{"1min":0.72699999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.139}]},{"osd":5,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":1.4810000000000001,"15min":1.55},"min":{"1min":1.0149999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.611},{"interface":"front","average":{"1min":1.6779999999999999,"5min":1.4590000000000001,"15min":1.4930000000000001},"min":{"1min":0.85199999999999998,"5min":0.78600000000000003,"15min":0.747},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":1.2430000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.5569999999999999,"15min":1.54},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":4.617,"5min":4.617,"15min":4.617},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.744,"5min":1.595,"15min":1.5840000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.72899999999999998},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.091}]},{"osd":7,"last update":"Mon Nov 13 11:02:25 2023","interfaces":[{"interface":"back","average":{"1min":1.694,"5min":1.629,"15min":1.6759999999999999},"min":{"1min":0.89100000000000001,"5min":0.78900000000000003,"15min":0.76700000000000002},"max":{"1min":2.4710000000000001,"5min":2.742,"15min":3.3540000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.6619999999999999,"15min":1.631},"min":{"1min":0.90600000000000003,"5min":0.88100000000000001,"15min":0.77400000000000002},"max":{"1min":3.976,"5min":3.976,"15min":3.976},"last":1.673}]}]},{"osd":7,"up_from":14,"seq":60129542245,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45880,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325960,"statfs":{"total":96636764160,"available":96589783040,"internally_reserved":0,"allocated":827392,"data_stored":371792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.079,"15min":1.1120000000000001},"min":{"1min":0.67200000000000004,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6020000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.0740000000000001,"15min":1.0720000000000001},"min":{"1min":0.57599999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4810000000000001,"5min":1.6080000000000001,"15min":2.266},"last":1.167}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1519999999999999,"15min":1.0669999999999999},"min":{"1min":0.65900000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.397,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2330000000000001,"15min":1.2290000000000001},"min":{"1min":0.79800000000000004,"5min":0.70099999999999996,"15min":0.64700000000000002},"max":{"1min":1.6080000000000001,"5min":1.893,"15min":2.1899999999999999},"last":1.2609999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2809999999999999,"15min":1.298},"min":{"1min":0.76000000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.859,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.355},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.278,"15min":1.2230000000000001},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.736,"5min":1.79,"15min":1.79},"last":1.4610000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.3320000000000001,"15min":1.262},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7210000000000001,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.647},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5049999999999999,"15min":1.4610000000000001},"min":{"1min":0.96299999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0459999999999998,"5min":2.2639999999999998,"15min":2.8159999999999998},"last":1.514}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3440000000000001,"15min":1.3240000000000001},"min":{"1min":0.629,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.6240000000000001,"5min":1.984,"15min":3.0190000000000001},"last":1.054},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.405,"15min":1.3759999999999999},"min":{"1min":0.70699999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.756,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.2130000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.504,"15min":1.5309999999999999},"min":{"1min":0.88700000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9550000000000001,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.506,"15min":1.48},"min":{"1min":0.73999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8540000000000001,"5min":2.3079999999999998,"15min":3.3620000000000001},"last":1.3080000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6100000000000001,"15min":1.5720000000000001},"min":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":1.8140000000000001,"5min":2.379,"15min":2.9020000000000001},"last":1.605},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.5620000000000001,"15min":1.512},"min":{"1min":0.76700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.9179999999999999,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:03:30.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:03:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.333+0000 7f1849535640 1 Processor -- start 2023-11-13T11:03:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.333+0000 7f1849535640 1 -- start start 2023-11-13T11:03:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.335+0000 7f1849535640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f184415aa90 0x7f184406a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.335+0000 7f1849535640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f184406a6d0 0x7f1844070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.335+0000 7f1849535640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18440708a0 0x7f18440b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.335+0000 7f1849535640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184406ab50 con 0x7f184415aa90 2023-11-13T11:03:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.335+0000 7f1849535640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1844070d30 con 0x7f184406a6d0 2023-11-13T11:03:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.335+0000 7f1849535640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184415d630 con 0x7f18440708a0 2023-11-13T11:03:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.335+0000 7f1843fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f184415aa90 0x7f184406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.335+0000 7f1843fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f184415aa90 0x7f184406a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55458/0 (socket says 172.21.15.16:55458) 2023-11-13T11:03:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.335+0000 7f1843fff640 1 -- 172.21.15.16:0/4003062265 learned_addr learned my addr 172.21.15.16:0/4003062265 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.336+0000 7f1848d34640 1 --2- 172.21.15.16:0/4003062265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18440708a0 0x7f18440b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.336+0000 7f183b7fe640 1 --2- 172.21.15.16:0/4003062265 >> v2:172.21.15.184:3300/0 conn(0x7f184406a6d0 0x7f1844070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.336+0000 7f1843fff640 1 -- 172.21.15.16:0/4003062265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18440708a0 msgr2=0x7f18440b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.336+0000 7f1843fff640 1 --2- 172.21.15.16:0/4003062265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18440708a0 0x7f18440b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.336+0000 7f1843fff640 1 -- 172.21.15.16:0/4003062265 >> v2:172.21.15.184:3300/0 conn(0x7f184406a6d0 msgr2=0x7f1844070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.336+0000 7f1843fff640 1 --2- 172.21.15.16:0/4003062265 >> v2:172.21.15.184:3300/0 conn(0x7f184406a6d0 0x7f1844070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.336+0000 7f1843fff640 1 -- 172.21.15.16:0/4003062265 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184415d910 con 0x7f184415aa90 2023-11-13T11:03:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.336+0000 7f183b7fe640 1 --2- 172.21.15.16:0/4003062265 >> v2:172.21.15.184:3300/0 conn(0x7f184406a6d0 0x7f1844070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.337+0000 7f1843fff640 1 --2- 172.21.15.16:0/4003062265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184415aa90 0x7f184406a190 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f184406b320 tx=0x7f183402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4231445558368a10 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.337+0000 7f18437fe640 1 -- 172.21.15.16:0/4003062265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183402c9f0 con 0x7f184415aa90 2023-11-13T11:03:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f18437fe640 1 -- 172.21.15.16:0/4003062265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183402cb50 con 0x7f184415aa90 2023-11-13T11:03:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f18437fe640 1 -- 172.21.15.16:0/4003062265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18340343f0 con 0x7f184415aa90 2023-11-13T11:03:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f1849535640 1 -- 172.21.15.16:0/4003062265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184415aa90 msgr2=0x7f184406a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f1849535640 1 --2- 172.21.15.16:0/4003062265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184415aa90 0x7f184406a190 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f184406b320 tx=0x7f183402be50 comp rx=0 tx=0).stop 2023-11-13T11:03:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f1849535640 1 -- 172.21.15.16:0/4003062265 shutdown_connections 2023-11-13T11:03:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f1849535640 1 --2- 172.21.15.16:0/4003062265 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18440708a0 0x7f18440b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f1849535640 1 --2- 172.21.15.16:0/4003062265 >> v2:172.21.15.184:3300/0 conn(0x7f184406a6d0 0x7f1844070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f1849535640 1 --2- 172.21.15.16:0/4003062265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f184415aa90 0x7f184406a190 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f1849535640 1 -- 172.21.15.16:0/4003062265 >> 172.21.15.16:0/4003062265 conn(0x7f18440b04a0 msgr2=0x7f18440b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f1849535640 1 -- 172.21.15.16:0/4003062265 shutdown_connections 2023-11-13T11:03:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.338+0000 7f1849535640 1 -- 172.21.15.16:0/4003062265 wait complete. 2023-11-13T11:03:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.340+0000 7f1849535640 1 Processor -- start 2023-11-13T11:03:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.340+0000 7f1849535640 1 -- start start 2023-11-13T11:03:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.341+0000 7f1849535640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f184406a6d0 0x7f1844153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.341+0000 7f1849535640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18440708a0 0x7f18440b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.341+0000 7f1849535640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f184415aa90 0x7f184414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.341+0000 7f1849535640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1844153d70 con 0x7f18440708a0 2023-11-13T11:03:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.341+0000 7f1849535640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440b38f0 con 0x7f184406a6d0 2023-11-13T11:03:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.341+0000 7f1849535640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18440aff50 con 0x7f184415aa90 2023-11-13T11:03:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f183b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18440708a0 0x7f18440b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f183b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18440708a0 0x7f18440b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55462/0 (socket says 172.21.15.16:55462) 2023-11-13T11:03:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f183b7fe640 1 -- 172.21.15.16:0/2577938025 learned_addr learned my addr 172.21.15.16:0/2577938025 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f1848d34640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f184415aa90 0x7f184414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f1843fff640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f184406a6d0 0x7f1844153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f183b7fe640 1 -- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f184415aa90 msgr2=0x7f184414f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f183b7fe640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f184415aa90 0x7f184414f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f183b7fe640 1 -- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f184406a6d0 msgr2=0x7f1844153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f183b7fe640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f184406a6d0 0x7f1844153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f183b7fe640 1 -- 172.21.15.16:0/2577938025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834009b30 con 0x7f18440708a0 2023-11-13T11:03:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.342+0000 7f183b7fe640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18440708a0 0x7f18440b33b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c7f0 tx=0x7f182c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.343+0000 7f18417fa640 1 -- 172.21.15.16:0/2577938025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c007820 con 0x7f18440708a0 2023-11-13T11:03:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.343+0000 7f18417fa640 1 -- 172.21.15.16:0/2577938025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c007980 con 0x7f18440708a0 2023-11-13T11:03:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.343+0000 7f18417fa640 1 -- 172.21.15.16:0/2577938025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c007d30 con 0x7f18440708a0 2023-11-13T11:03:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.343+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18440af660 con 0x7f18440708a0 2023-11-13T11:03:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.343+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18440af890 con 0x7f18440708a0 2023-11-13T11:03:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.344+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1844153550 con 0x7f18440708a0 2023-11-13T11:03:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.345+0000 7f18417fa640 1 -- 172.21.15.16:0/2577938025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f182c0113f0 con 0x7f18440708a0 2023-11-13T11:03:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.346+0000 7f18417fa640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f182406ae40 0x7f182406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.346+0000 7f18417fa640 1 -- 172.21.15.16:0/2577938025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f182c089a80 con 0x7f18440708a0 2023-11-13T11:03:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.347+0000 7f1843fff640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f182406ae40 0x7f182406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.348+0000 7f1843fff640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f182406ae40 0x7f182406d300 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1834035000 tx=0x7f1834030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.350+0000 7f18417fa640 1 -- 172.21.15.16:0/2577938025 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1844153550 con 0x7f18440708a0 2023-11-13T11:03:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.541+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f184415ded0 con 0x7f182406ae40 2023-11-13T11:03:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.564+0000 7f18417fa640 1 -- 172.21.15.16:0/2577938025 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286807 (secure 0 0 0) 0x7f1834009510 con 0x7f182406ae40 2023-11-13T11:03:30.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:03:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.569+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f182406ae40 msgr2=0x7f182406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.569+0000 7f1849535640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f182406ae40 0x7f182406d300 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1834035000 tx=0x7f1834030000 comp rx=0 tx=0).stop 2023-11-13T11:03:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.569+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18440708a0 msgr2=0x7f18440b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.569+0000 7f1849535640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18440708a0 0x7f18440b33b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f182c00c7f0 tx=0x7f182c00ccb0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.569+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 shutdown_connections 2023-11-13T11:03:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.569+0000 7f1849535640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f182406ae40 0x7f182406d300 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.569+0000 7f1849535640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f184415aa90 0x7f184414f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.570+0000 7f1849535640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18440708a0 0x7f18440b33b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.570+0000 7f1849535640 1 --2- 172.21.15.16:0/2577938025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f184406a6d0 0x7f1844153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.570+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 >> 172.21.15.16:0/2577938025 conn(0x7f18440b04a0 msgr2=0x7f1844157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.570+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 shutdown_connections 2023-11-13T11:03:30.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.570+0000 7f1849535640 1 -- 172.21.15.16:0/2577938025 wait complete. 2023-11-13T11:03:30.593 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2023-11-13T11:03:30.159025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":352892,"kb_used_data":7688,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754621828,"statfs":{"total":773094113280,"available":772732751872,"internally_reserved":0,"allocated":7872512,"data_stored":4159892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":353357361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.720474"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.447492+0000","last_change":"2023-11-13T11:03:20.481234+0000","last_active":"2023-11-13T11:03:21.447492+0000","last_peered":"2023-11-13T11:03:21.447492+0000","last_clean":"2023-11-13T11:03:21.447492+0000","last_became_active":"2023-11-13T11:03:20.480949+0000","last_became_peered":"2023-11-13T11:03:20.480949+0000","last_unstale":"2023-11-13T11:03:21.447492+0000","last_undegraded":"2023-11-13T11:03:21.447492+0000","last_fullsized":"2023-11-13T11:03:21.447492+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:15.570305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.447386+0000","last_change":"2023-11-13T11:03:20.473909+0000","last_active":"2023-11-13T11:03:21.447386+0000","last_peered":"2023-11-13T11:03:21.447386+0000","last_clean":"2023-11-13T11:03:21.447386+0000","last_became_active":"2023-11-13T11:03:20.473638+0000","last_became_peered":"2023-11-13T11:03:20.473638+0000","last_unstale":"2023-11-13T11:03:21.447386+0000","last_undegraded":"2023-11-13T11:03:21.447386+0000","last_fullsized":"2023-11-13T11:03:21.447386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:53:50.663729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299244+0000","last_change":"2023-11-13T11:03:20.468089+0000","last_active":"2023-11-13T11:03:22.299244+0000","last_peered":"2023-11-13T11:03:22.299244+0000","last_clean":"2023-11-13T11:03:22.299244+0000","last_became_active":"2023-11-13T11:03:20.467527+0000","last_became_peered":"2023-11-13T11:03:20.467527+0000","last_unstale":"2023-11-13T11:03:22.299244+0000","last_undegraded":"2023-11-13T11:03:22.299244+0000","last_fullsized":"2023-11-13T11:03:22.299244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:21:31.064220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441904+0000","last_change":"2023-11-13T11:03:20.473876+0000","last_active":"2023-11-13T11:03:21.441904+0000","last_peered":"2023-11-13T11:03:21.441904+0000","last_clean":"2023-11-13T11:03:21.441904+0000","last_became_active":"2023-11-13T11:03:20.473496+0000","last_became_peered":"2023-11-13T11:03:20.473496+0000","last_unstale":"2023-11-13T11:03:21.441904+0000","last_undegraded":"2023-11-13T11:03:21.441904+0000","last_fullsized":"2023-11-13T11:03:21.441904+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:35:11.804201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297643+0000","last_change":"2023-11-13T11:03:20.481275+0000","last_active":"2023-11-13T11:03:22.297643+0000","last_peered":"2023-11-13T11:03:22.297643+0000","last_clean":"2023-11-13T11:03:22.297643+0000","last_became_active":"2023-11-13T11:03:20.480960+0000","last_became_peered":"2023-11-13T11:03:20.480960+0000","last_unstale":"2023-11-13T11:03:22.297643+0000","last_undegraded":"2023-11-13T11:03:22.297643+0000","last_fullsized":"2023-11-13T11:03:22.297643+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:17:06.846928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.446428+0000","last_change":"2023-11-13T11:03:20.476263+0000","last_active":"2023-11-13T11:03:21.446428+0000","last_peered":"2023-11-13T11:03:21.446428+0000","last_clean":"2023-11-13T11:03:21.446428+0000","last_became_active":"2023-11-13T11:03:20.472373+0000","last_became_peered":"2023-11-13T11:03:20.472373+0000","last_unstale":"2023-11-13T11:03:21.446428+0000","last_undegraded":"2023-11-13T11:03:21.446428+0000","last_fullsized":"2023-11-13T11:03:21.446428+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:02:35.586086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.444426+0000","last_change":"2023-11-13T11:03:20.466101+0000","last_active":"2023-11-13T11:03:21.444426+0000","last_peered":"2023-11-13T11:03:21.444426+0000","last_clean":"2023-11-13T11:03:21.444426+0000","last_became_active":"2023-11-13T11:03:20.465531+0000","last_became_peered":"2023-11-13T11:03:20.465531+0000","last_unstale":"2023-11-13T11:03:21.444426+0000","last_undegraded":"2023-11-13T11:03:21.444426+0000","last_fullsized":"2023-11-13T11:03:21.444426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:38:54.348922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297273+0000","last_change":"2023-11-13T11:03:20.482233+0000","last_active":"2023-11-13T11:03:22.297273+0000","last_peered":"2023-11-13T11:03:22.297273+0000","last_clean":"2023-11-13T11:03:22.297273+0000","last_became_active":"2023-11-13T11:03:20.481971+0000","last_became_peered":"2023-11-13T11:03:20.481971+0000","last_unstale":"2023-11-13T11:03:22.297273+0000","last_undegraded":"2023-11-13T11:03:22.297273+0000","last_fullsized":"2023-11-13T11:03:22.297273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:17:24.322051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442399+0000","last_change":"2023-11-13T11:03:20.475387+0000","last_active":"2023-11-13T11:03:21.442399+0000","last_peered":"2023-11-13T11:03:21.442399+0000","last_clean":"2023-11-13T11:03:21.442399+0000","last_became_active":"2023-11-13T11:03:20.475063+0000","last_became_peered":"2023-11-13T11:03:20.475063+0000","last_unstale":"2023-11-13T11:03:21.442399+0000","last_undegraded":"2023-11-13T11:03:21.442399+0000","last_fullsized":"2023-11-13T11:03:21.442399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:58:33.061768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.837119+0000","last_change":"2023-11-13T11:03:20.463396+0000","last_active":"2023-11-13T11:03:21.837119+0000","last_peered":"2023-11-13T11:03:21.837119+0000","last_clean":"2023-11-13T11:03:21.837119+0000","last_became_active":"2023-11-13T11:03:20.463113+0000","last_became_peered":"2023-11-13T11:03:20.463113+0000","last_unstale":"2023-11-13T11:03:21.837119+0000","last_undegraded":"2023-11-13T11:03:21.837119+0000","last_fullsized":"2023-11-13T11:03:21.837119+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:32.831255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442465+0000","last_change":"2023-11-13T11:03:20.477366+0000","last_active":"2023-11-13T11:03:21.442465+0000","last_peered":"2023-11-13T11:03:21.442465+0000","last_clean":"2023-11-13T11:03:21.442465+0000","last_became_active":"2023-11-13T11:03:20.476702+0000","last_became_peered":"2023-11-13T11:03:20.476702+0000","last_unstale":"2023-11-13T11:03:21.442465+0000","last_undegraded":"2023-11-13T11:03:21.442465+0000","last_fullsized":"2023-11-13T11:03:21.442465+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:57:36.112404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442044+0000","last_change":"2023-11-13T11:03:20.487960+0000","last_active":"2023-11-13T11:03:21.442044+0000","last_peered":"2023-11-13T11:03:21.442044+0000","last_clean":"2023-11-13T11:03:21.442044+0000","last_became_active":"2023-11-13T11:03:20.486438+0000","last_became_peered":"2023-11-13T11:03:20.486438+0000","last_unstale":"2023-11-13T11:03:21.442044+0000","last_undegraded":"2023-11-13T11:03:21.442044+0000","last_fullsized":"2023-11-13T11:03:21.442044+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:18:48.755047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297506+0000","last_change":"2023-11-13T11:03:20.477742+0000","last_active":"2023-11-13T11:03:22.297506+0000","last_peered":"2023-11-13T11:03:22.297506+0000","last_clean":"2023-11-13T11:03:22.297506+0000","last_became_active":"2023-11-13T11:03:20.477473+0000","last_became_peered":"2023-11-13T11:03:20.477473+0000","last_unstale":"2023-11-13T11:03:22.297506+0000","last_undegraded":"2023-11-13T11:03:22.297506+0000","last_fullsized":"2023-11-13T11:03:22.297506+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:11:29.076081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297105+0000","last_change":"2023-11-13T11:03:20.469272+0000","last_active":"2023-11-13T11:03:22.297105+0000","last_peered":"2023-11-13T11:03:22.297105+0000","last_clean":"2023-11-13T11:03:22.297105+0000","last_became_active":"2023-11-13T11:03:20.468953+0000","last_became_peered":"2023-11-13T11:03:20.468953+0000","last_unstale":"2023-11-13T11:03:22.297105+0000","last_undegraded":"2023-11-13T11:03:22.297105+0000","last_fullsized":"2023-11-13T11:03:22.297105+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:28:15.701895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836278+0000","last_change":"2023-11-13T11:03:20.467960+0000","last_active":"2023-11-13T11:03:21.836278+0000","last_peered":"2023-11-13T11:03:21.836278+0000","last_clean":"2023-11-13T11:03:21.836278+0000","last_became_active":"2023-11-13T11:03:20.466799+0000","last_became_peered":"2023-11-13T11:03:20.466799+0000","last_unstale":"2023-11-13T11:03:21.836278+0000","last_undegraded":"2023-11-13T11:03:21.836278+0000","last_fullsized":"2023-11-13T11:03:21.836278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:11:29.076081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058741+0000","last_change":"2023-11-13T11:03:20.473726+0000","last_active":"2023-11-13T11:03:22.058741+0000","last_peered":"2023-11-13T11:03:22.058741+0000","last_clean":"2023-11-13T11:03:22.058741+0000","last_became_active":"2023-11-13T11:03:20.473463+0000","last_became_peered":"2023-11-13T11:03:20.473463+0000","last_unstale":"2023-11-13T11:03:22.058741+0000","last_undegraded":"2023-11-13T11:03:22.058741+0000","last_fullsized":"2023-11-13T11:03:22.058741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:24:37.146283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297642+0000","last_change":"2023-11-13T11:03:20.478944+0000","last_active":"2023-11-13T11:03:22.297642+0000","last_peered":"2023-11-13T11:03:22.297642+0000","last_clean":"2023-11-13T11:03:22.297642+0000","last_became_active":"2023-11-13T11:03:20.478624+0000","last_became_peered":"2023-11-13T11:03:20.478624+0000","last_unstale":"2023-11-13T11:03:22.297642+0000","last_undegraded":"2023-11-13T11:03:22.297642+0000","last_fullsized":"2023-11-13T11:03:22.297642+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:32:53.662105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441902+0000","last_change":"2023-11-13T11:03:20.479029+0000","last_active":"2023-11-13T11:03:21.441902+0000","last_peered":"2023-11-13T11:03:21.441902+0000","last_clean":"2023-11-13T11:03:21.441902+0000","last_became_active":"2023-11-13T11:03:20.478776+0000","last_became_peered":"2023-11-13T11:03:20.478776+0000","last_unstale":"2023-11-13T11:03:21.441902+0000","last_undegraded":"2023-11-13T11:03:21.441902+0000","last_fullsized":"2023-11-13T11:03:21.441902+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:53:50.663729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442651+0000","last_change":"2023-11-13T11:03:20.477561+0000","last_active":"2023-11-13T11:03:21.442651+0000","last_peered":"2023-11-13T11:03:21.442651+0000","last_clean":"2023-11-13T11:03:21.442651+0000","last_became_active":"2023-11-13T11:03:20.477087+0000","last_became_peered":"2023-11-13T11:03:20.477087+0000","last_unstale":"2023-11-13T11:03:21.442651+0000","last_undegraded":"2023-11-13T11:03:21.442651+0000","last_fullsized":"2023-11-13T11:03:21.442651+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:32:18.758861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296990+0000","last_change":"2023-11-13T11:03:20.478838+0000","last_active":"2023-11-13T11:03:22.296990+0000","last_peered":"2023-11-13T11:03:22.296990+0000","last_clean":"2023-11-13T11:03:22.296990+0000","last_became_active":"2023-11-13T11:03:20.478275+0000","last_became_peered":"2023-11-13T11:03:20.478275+0000","last_unstale":"2023-11-13T11:03:22.296990+0000","last_undegraded":"2023-11-13T11:03:22.296990+0000","last_fullsized":"2023-11-13T11:03:22.296990+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:45:14.457719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297712+0000","last_change":"2023-11-13T11:03:20.482167+0000","last_active":"2023-11-13T11:03:22.297712+0000","last_peered":"2023-11-13T11:03:22.297712+0000","last_clean":"2023-11-13T11:03:22.297712+0000","last_became_active":"2023-11-13T11:03:20.481889+0000","last_became_peered":"2023-11-13T11:03:20.481889+0000","last_unstale":"2023-11-13T11:03:22.297712+0000","last_undegraded":"2023-11-13T11:03:22.297712+0000","last_fullsized":"2023-11-13T11:03:22.297712+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:41.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.447215+0000","last_change":"2023-11-13T11:03:20.472937+0000","last_active":"2023-11-13T11:03:21.447215+0000","last_peered":"2023-11-13T11:03:21.447215+0000","last_clean":"2023-11-13T11:03:21.447215+0000","last_became_active":"2023-11-13T11:03:20.472646+0000","last_became_peered":"2023-11-13T11:03:20.472646+0000","last_unstale":"2023-11-13T11:03:21.447215+0000","last_undegraded":"2023-11-13T11:03:21.447215+0000","last_fullsized":"2023-11-13T11:03:21.447215+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:10:29.987250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058815+0000","last_change":"2023-11-13T11:03:20.471989+0000","last_active":"2023-11-13T11:03:22.058815+0000","last_peered":"2023-11-13T11:03:22.058815+0000","last_clean":"2023-11-13T11:03:22.058815+0000","last_became_active":"2023-11-13T11:03:20.470759+0000","last_became_peered":"2023-11-13T11:03:20.470759+0000","last_unstale":"2023-11-13T11:03:22.058815+0000","last_undegraded":"2023-11-13T11:03:22.058815+0000","last_fullsized":"2023-11-13T11:03:22.058815+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:33.483394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297281+0000","last_change":"2023-11-13T11:03:20.466775+0000","last_active":"2023-11-13T11:03:22.297281+0000","last_peered":"2023-11-13T11:03:22.297281+0000","last_clean":"2023-11-13T11:03:22.297281+0000","last_became_active":"2023-11-13T11:03:20.466504+0000","last_became_peered":"2023-11-13T11:03:20.466504+0000","last_unstale":"2023-11-13T11:03:22.297281+0000","last_undegraded":"2023-11-13T11:03:22.297281+0000","last_fullsized":"2023-11-13T11:03:22.297281+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:48:04.143689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299765+0000","last_change":"2023-11-13T11:03:20.472580+0000","last_active":"2023-11-13T11:03:22.299765+0000","last_peered":"2023-11-13T11:03:22.299765+0000","last_clean":"2023-11-13T11:03:22.299765+0000","last_became_active":"2023-11-13T11:03:20.472176+0000","last_became_peered":"2023-11-13T11:03:20.472176+0000","last_unstale":"2023-11-13T11:03:22.299765+0000","last_undegraded":"2023-11-13T11:03:22.299765+0000","last_fullsized":"2023-11-13T11:03:22.299765+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:36:22.298361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298183+0000","last_change":"2023-11-13T11:03:20.466179+0000","last_active":"2023-11-13T11:03:22.298183+0000","last_peered":"2023-11-13T11:03:22.298183+0000","last_clean":"2023-11-13T11:03:22.298183+0000","last_became_active":"2023-11-13T11:03:20.464657+0000","last_became_peered":"2023-11-13T11:03:20.464657+0000","last_unstale":"2023-11-13T11:03:22.298183+0000","last_undegraded":"2023-11-13T11:03:22.298183+0000","last_fullsized":"2023-11-13T11:03:22.298183+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:28:07.308881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299015+0000","last_change":"2023-11-13T11:03:20.470580+0000","last_active":"2023-11-13T11:03:22.299015+0000","last_peered":"2023-11-13T11:03:22.299015+0000","last_clean":"2023-11-13T11:03:22.299015+0000","last_became_active":"2023-11-13T11:03:20.470263+0000","last_became_peered":"2023-11-13T11:03:20.470263+0000","last_unstale":"2023-11-13T11:03:22.299015+0000","last_undegraded":"2023-11-13T11:03:22.299015+0000","last_fullsized":"2023-11-13T11:03:22.299015+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:32.831255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.446417+0000","last_change":"2023-11-13T11:03:20.477205+0000","last_active":"2023-11-13T11:03:21.446417+0000","last_peered":"2023-11-13T11:03:21.446417+0000","last_clean":"2023-11-13T11:03:21.446417+0000","last_became_active":"2023-11-13T11:03:20.476849+0000","last_became_peered":"2023-11-13T11:03:20.476849+0000","last_unstale":"2023-11-13T11:03:21.446417+0000","last_undegraded":"2023-11-13T11:03:21.446417+0000","last_fullsized":"2023-11-13T11:03:21.446417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:27:37.299626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441724+0000","last_change":"2023-11-13T11:03:20.485616+0000","last_active":"2023-11-13T11:03:21.441724+0000","last_peered":"2023-11-13T11:03:21.441724+0000","last_clean":"2023-11-13T11:03:21.441724+0000","last_became_active":"2023-11-13T11:03:20.482900+0000","last_became_peered":"2023-11-13T11:03:20.482900+0000","last_unstale":"2023-11-13T11:03:21.441724+0000","last_undegraded":"2023-11-13T11:03:21.441724+0000","last_fullsized":"2023-11-13T11:03:21.441724+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:10:29.987250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441309+0000","last_change":"2023-11-13T11:03:20.476615+0000","last_active":"2023-11-13T11:03:21.441309+0000","last_peered":"2023-11-13T11:03:21.441309+0000","last_clean":"2023-11-13T11:03:21.441309+0000","last_became_active":"2023-11-13T11:03:20.474970+0000","last_became_peered":"2023-11-13T11:03:20.474970+0000","last_unstale":"2023-11-13T11:03:21.441309+0000","last_undegraded":"2023-11-13T11:03:21.441309+0000","last_fullsized":"2023-11-13T11:03:21.441309+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:02:35.586086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297973+0000","last_change":"2023-11-13T11:03:20.465386+0000","last_active":"2023-11-13T11:03:22.297973+0000","last_peered":"2023-11-13T11:03:22.297973+0000","last_clean":"2023-11-13T11:03:22.297973+0000","last_became_active":"2023-11-13T11:03:20.465100+0000","last_became_peered":"2023-11-13T11:03:20.465100+0000","last_unstale":"2023-11-13T11:03:22.297973+0000","last_undegraded":"2023-11-13T11:03:22.297973+0000","last_fullsized":"2023-11-13T11:03:22.297973+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:32:53.662105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.444374+0000","last_change":"2023-11-13T11:03:20.467138+0000","last_active":"2023-11-13T11:03:21.444374+0000","last_peered":"2023-11-13T11:03:21.444374+0000","last_clean":"2023-11-13T11:03:21.444374+0000","last_became_active":"2023-11-13T11:03:20.466692+0000","last_became_peered":"2023-11-13T11:03:20.466692+0000","last_unstale":"2023-11-13T11:03:21.444374+0000","last_undegraded":"2023-11-13T11:03:21.444374+0000","last_fullsized":"2023-11-13T11:03:21.444374+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.869476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442092+0000","last_change":"2023-11-13T11:03:20.478001+0000","last_active":"2023-11-13T11:03:21.442092+0000","last_peered":"2023-11-13T11:03:21.442092+0000","last_clean":"2023-11-13T11:03:21.442092+0000","last_became_active":"2023-11-13T11:03:20.476026+0000","last_became_peered":"2023-11-13T11:03:20.476026+0000","last_unstale":"2023-11-13T11:03:21.442092+0000","last_undegraded":"2023-11-13T11:03:21.442092+0000","last_fullsized":"2023-11-13T11:03:21.442092+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:36:46.335523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"68'4","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.486577+0000","last_change":"2023-11-13T11:03:19.462104+0000","last_active":"2023-11-13T11:03:20.486577+0000","last_peered":"2023-11-13T11:03:20.486577+0000","last_clean":"2023-11-13T11:03:20.486577+0000","last_became_active":"2023-11-13T11:03:19.461813+0000","last_became_peered":"2023-11-13T11:03:19.461813+0000","last_unstale":"2023-11-13T11:03:20.486577+0000","last_undegraded":"2023-11-13T11:03:20.486577+0000","last_fullsized":"2023-11-13T11:03:20.486577+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:11:28.058672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298818+0000","last_change":"2023-11-13T11:03:20.481842+0000","last_active":"2023-11-13T11:03:22.298818+0000","last_peered":"2023-11-13T11:03:22.298818+0000","last_clean":"2023-11-13T11:03:22.298818+0000","last_became_active":"2023-11-13T11:03:20.481508+0000","last_became_peered":"2023-11-13T11:03:20.481508+0000","last_unstale":"2023-11-13T11:03:22.298818+0000","last_undegraded":"2023-11-13T11:03:22.298818+0000","last_fullsized":"2023-11-13T11:03:22.298818+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:57:06.652730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.486648+0000","last_change":"2023-11-13T11:03:19.467329+0000","last_active":"2023-11-13T11:03:20.486648+0000","last_peered":"2023-11-13T11:03:20.486648+0000","last_clean":"2023-11-13T11:03:20.486648+0000","last_became_active":"2023-11-13T11:03:19.465815+0000","last_became_peered":"2023-11-13T11:03:19.465815+0000","last_unstale":"2023-11-13T11:03:20.486648+0000","last_undegraded":"2023-11-13T11:03:20.486648+0000","last_fullsized":"2023-11-13T11:03:20.486648+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:31.813846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.447727+0000","last_change":"2023-11-13T11:03:20.471019+0000","last_active":"2023-11-13T11:03:21.447727+0000","last_peered":"2023-11-13T11:03:21.447727+0000","last_clean":"2023-11-13T11:03:21.447727+0000","last_became_active":"2023-11-13T11:03:20.470352+0000","last_became_peered":"2023-11-13T11:03:20.470352+0000","last_unstale":"2023-11-13T11:03:21.447727+0000","last_undegraded":"2023-11-13T11:03:21.447727+0000","last_fullsized":"2023-11-13T11:03:21.447727+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:35:11.804201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.485670+0000","last_change":"2023-11-13T11:03:19.465871+0000","last_active":"2023-11-13T11:03:20.485670+0000","last_peered":"2023-11-13T11:03:20.485670+0000","last_clean":"2023-11-13T11:03:20.485670+0000","last_became_active":"2023-11-13T11:03:19.465541+0000","last_became_peered":"2023-11-13T11:03:19.465541+0000","last_unstale":"2023-11-13T11:03:20.485670+0000","last_undegraded":"2023-11-13T11:03:20.485670+0000","last_fullsized":"2023-11-13T11:03:20.485670+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:40.173268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442595+0000","last_change":"2023-11-13T11:03:20.479413+0000","last_active":"2023-11-13T11:03:21.442595+0000","last_peered":"2023-11-13T11:03:21.442595+0000","last_clean":"2023-11-13T11:03:21.442595+0000","last_became_active":"2023-11-13T11:03:20.479139+0000","last_became_peered":"2023-11-13T11:03:20.479139+0000","last_unstale":"2023-11-13T11:03:21.442595+0000","last_undegraded":"2023-11-13T11:03:21.442595+0000","last_fullsized":"2023-11-13T11:03:21.442595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:41:06.292346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.485628+0000","last_change":"2023-11-13T11:03:19.471284+0000","last_active":"2023-11-13T11:03:20.485628+0000","last_peered":"2023-11-13T11:03:20.485628+0000","last_clean":"2023-11-13T11:03:20.485628+0000","last_became_active":"2023-11-13T11:03:19.470979+0000","last_became_peered":"2023-11-13T11:03:19.470979+0000","last_unstale":"2023-11-13T11:03:20.485628+0000","last_undegraded":"2023-11-13T11:03:20.485628+0000","last_fullsized":"2023-11-13T11:03:20.485628+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:38:53.331513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296423+0000","last_change":"2023-11-13T11:03:20.463568+0000","last_active":"2023-11-13T11:03:22.296423+0000","last_peered":"2023-11-13T11:03:22.296423+0000","last_clean":"2023-11-13T11:03:22.296423+0000","last_became_active":"2023-11-13T11:03:20.462426+0000","last_became_peered":"2023-11-13T11:03:20.462426+0000","last_unstale":"2023-11-13T11:03:22.296423+0000","last_undegraded":"2023-11-13T11:03:22.296423+0000","last_fullsized":"2023-11-13T11:03:22.296423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:05:18.302427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.482354+0000","last_change":"2023-11-13T11:03:19.461169+0000","last_active":"2023-11-13T11:03:20.482354+0000","last_peered":"2023-11-13T11:03:20.482354+0000","last_clean":"2023-11-13T11:03:20.482354+0000","last_became_active":"2023-11-13T11:03:19.460849+0000","last_became_peered":"2023-11-13T11:03:19.460849+0000","last_unstale":"2023-11-13T11:03:20.482354+0000","last_undegraded":"2023-11-13T11:03:20.482354+0000","last_fullsized":"2023-11-13T11:03:20.482354+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T16:45:54.847751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.444937+0000","last_change":"2023-11-13T11:03:20.476919+0000","last_active":"2023-11-13T11:03:21.444937+0000","last_peered":"2023-11-13T11:03:21.444937+0000","last_clean":"2023-11-13T11:03:21.444937+0000","last_became_active":"2023-11-13T11:03:20.476599+0000","last_became_peered":"2023-11-13T11:03:20.476599+0000","last_unstale":"2023-11-13T11:03:21.444937+0000","last_undegraded":"2023-11-13T11:03:21.444937+0000","last_fullsized":"2023-11-13T11:03:21.444937+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:36:22.298361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.485821+0000","last_change":"2023-11-13T11:03:19.470566+0000","last_active":"2023-11-13T11:03:20.485821+0000","last_peered":"2023-11-13T11:03:20.485821+0000","last_clean":"2023-11-13T11:03:20.485821+0000","last_became_active":"2023-11-13T11:03:19.470250+0000","last_became_peered":"2023-11-13T11:03:19.470250+0000","last_unstale":"2023-11-13T11:03:20.485821+0000","last_undegraded":"2023-11-13T11:03:20.485821+0000","last_fullsized":"2023-11-13T11:03:20.485821+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:15.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836870+0000","last_change":"2023-11-13T11:03:20.478266+0000","last_active":"2023-11-13T11:03:21.836870+0000","last_peered":"2023-11-13T11:03:21.836870+0000","last_clean":"2023-11-13T11:03:21.836870+0000","last_became_active":"2023-11-13T11:03:20.477220+0000","last_became_peered":"2023-11-13T11:03:20.477220+0000","last_unstale":"2023-11-13T11:03:21.836870+0000","last_undegraded":"2023-11-13T11:03:21.836870+0000","last_fullsized":"2023-11-13T11:03:21.836870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:06.214101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.458907+0000","last_change":"2023-11-13T11:03:19.464829+0000","last_active":"2023-11-13T11:03:20.458907+0000","last_peered":"2023-11-13T11:03:20.458907+0000","last_clean":"2023-11-13T11:03:20.458907+0000","last_became_active":"2023-11-13T11:03:19.463540+0000","last_became_peered":"2023-11-13T11:03:19.463540+0000","last_unstale":"2023-11-13T11:03:20.458907+0000","last_undegraded":"2023-11-13T11:03:20.458907+0000","last_fullsized":"2023-11-13T11:03:20.458907+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:21.547257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296745+0000","last_change":"2023-11-13T11:03:20.479489+0000","last_active":"2023-11-13T11:03:22.296745+0000","last_peered":"2023-11-13T11:03:22.296745+0000","last_clean":"2023-11-13T11:03:22.296745+0000","last_became_active":"2023-11-13T11:03:20.478574+0000","last_became_peered":"2023-11-13T11:03:20.478574+0000","last_unstale":"2023-11-13T11:03:22.296745+0000","last_undegraded":"2023-11-13T11:03:22.296745+0000","last_fullsized":"2023-11-13T11:03:22.296745+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:10:13.749773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.472241+0000","last_change":"2023-11-13T11:03:19.453306+0000","last_active":"2023-11-13T11:03:20.472241+0000","last_peered":"2023-11-13T11:03:20.472241+0000","last_clean":"2023-11-13T11:03:20.472241+0000","last_became_active":"2023-11-13T11:03:19.451270+0000","last_became_peered":"2023-11-13T11:03:19.451270+0000","last_unstale":"2023-11-13T11:03:20.472241+0000","last_undegraded":"2023-11-13T11:03:20.472241+0000","last_fullsized":"2023-11-13T11:03:20.472241+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:21.547257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.446774+0000","last_change":"2023-11-13T11:03:20.467546+0000","last_active":"2023-11-13T11:03:21.446774+0000","last_peered":"2023-11-13T11:03:21.446774+0000","last_clean":"2023-11-13T11:03:21.446774+0000","last_became_active":"2023-11-13T11:03:20.466353+0000","last_became_peered":"2023-11-13T11:03:20.466353+0000","last_unstale":"2023-11-13T11:03:21.446774+0000","last_undegraded":"2023-11-13T11:03:21.446774+0000","last_fullsized":"2023-11-13T11:03:21.446774+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:02:35.586086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.473152+0000","last_change":"2023-11-13T11:03:19.465601+0000","last_active":"2023-11-13T11:03:20.473152+0000","last_peered":"2023-11-13T11:03:20.473152+0000","last_clean":"2023-11-13T11:03:20.473152+0000","last_became_active":"2023-11-13T11:03:19.465326+0000","last_became_peered":"2023-11-13T11:03:19.465326+0000","last_unstale":"2023-11-13T11:03:20.473152+0000","last_undegraded":"2023-11-13T11:03:20.473152+0000","last_fullsized":"2023-11-13T11:03:20.473152+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:31.813846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297069+0000","last_change":"2023-11-13T11:03:20.459160+0000","last_active":"2023-11-13T11:03:22.297069+0000","last_peered":"2023-11-13T11:03:22.297069+0000","last_clean":"2023-11-13T11:03:22.297069+0000","last_became_active":"2023-11-13T11:03:20.458674+0000","last_became_peered":"2023-11-13T11:03:20.458674+0000","last_unstale":"2023-11-13T11:03:22.297069+0000","last_undegraded":"2023-11-13T11:03:22.297069+0000","last_fullsized":"2023-11-13T11:03:22.297069+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:17:24.322051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.464388+0000","last_change":"2023-11-13T11:03:19.458487+0000","last_active":"2023-11-13T11:03:20.464388+0000","last_peered":"2023-11-13T11:03:20.464388+0000","last_clean":"2023-11-13T11:03:20.464388+0000","last_became_active":"2023-11-13T11:03:19.457021+0000","last_became_peered":"2023-11-13T11:03:19.457021+0000","last_unstale":"2023-11-13T11:03:20.464388+0000","last_undegraded":"2023-11-13T11:03:20.464388+0000","last_fullsized":"2023-11-13T11:03:20.464388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:58:49.422056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298690+0000","last_change":"2023-11-13T11:03:20.479550+0000","last_active":"2023-11-13T11:03:22.298690+0000","last_peered":"2023-11-13T11:03:22.298690+0000","last_clean":"2023-11-13T11:03:22.298690+0000","last_became_active":"2023-11-13T11:03:20.478812+0000","last_became_peered":"2023-11-13T11:03:20.478812+0000","last_unstale":"2023-11-13T11:03:22.298690+0000","last_undegraded":"2023-11-13T11:03:22.298690+0000","last_fullsized":"2023-11-13T11:03:22.298690+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:06.214101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"68'3","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492480+0000","last_change":"2023-11-13T11:03:19.467377+0000","last_active":"2023-11-13T11:03:20.492480+0000","last_peered":"2023-11-13T11:03:20.492480+0000","last_clean":"2023-11-13T11:03:20.492480+0000","last_became_active":"2023-11-13T11:03:19.467124+0000","last_became_peered":"2023-11-13T11:03:19.467124+0000","last_unstale":"2023-11-13T11:03:20.492480+0000","last_undegraded":"2023-11-13T11:03:20.492480+0000","last_fullsized":"2023-11-13T11:03:20.492480+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:58:15.706470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298181+0000","last_change":"2023-11-13T11:03:20.480420+0000","last_active":"2023-11-13T11:03:22.298181+0000","last_peered":"2023-11-13T11:03:22.298181+0000","last_clean":"2023-11-13T11:03:22.298181+0000","last_became_active":"2023-11-13T11:03:20.480145+0000","last_became_peered":"2023-11-13T11:03:20.480145+0000","last_unstale":"2023-11-13T11:03:22.298181+0000","last_undegraded":"2023-11-13T11:03:22.298181+0000","last_fullsized":"2023-11-13T11:03:22.298181+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:52:21.636883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"68'6","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.487942+0000","last_change":"2023-11-13T11:03:19.465606+0000","last_active":"2023-11-13T11:03:20.487942+0000","last_peered":"2023-11-13T11:03:20.487942+0000","last_clean":"2023-11-13T11:03:20.487942+0000","last_became_active":"2023-11-13T11:03:19.465133+0000","last_became_peered":"2023-11-13T11:03:19.465133+0000","last_unstale":"2023-11-13T11:03:20.487942+0000","last_undegraded":"2023-11-13T11:03:20.487942+0000","last_fullsized":"2023-11-13T11:03:20.487942+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:36:21.280952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836272+0000","last_change":"2023-11-13T11:03:20.476387+0000","last_active":"2023-11-13T11:03:21.836272+0000","last_peered":"2023-11-13T11:03:21.836272+0000","last_clean":"2023-11-13T11:03:21.836272+0000","last_became_active":"2023-11-13T11:03:20.476116+0000","last_became_peered":"2023-11-13T11:03:20.476116+0000","last_unstale":"2023-11-13T11:03:21.836272+0000","last_undegraded":"2023-11-13T11:03:21.836272+0000","last_fullsized":"2023-11-13T11:03:21.836272+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:28:15.701895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.493013+0000","last_change":"2023-11-13T11:03:19.463340+0000","last_active":"2023-11-13T11:03:20.493013+0000","last_peered":"2023-11-13T11:03:20.493013+0000","last_clean":"2023-11-13T11:03:20.493013+0000","last_became_active":"2023-11-13T11:03:19.463065+0000","last_became_peered":"2023-11-13T11:03:19.463065+0000","last_unstale":"2023-11-13T11:03:20.493013+0000","last_undegraded":"2023-11-13T11:03:20.493013+0000","last_fullsized":"2023-11-13T11:03:20.493013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:05.196692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836185+0000","last_change":"2023-11-13T11:03:20.462404+0000","last_active":"2023-11-13T11:03:21.836185+0000","last_peered":"2023-11-13T11:03:21.836185+0000","last_clean":"2023-11-13T11:03:21.836185+0000","last_became_active":"2023-11-13T11:03:20.462106+0000","last_became_peered":"2023-11-13T11:03:20.462106+0000","last_unstale":"2023-11-13T11:03:21.836185+0000","last_undegraded":"2023-11-13T11:03:21.836185+0000","last_fullsized":"2023-11-13T11:03:21.836185+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.593902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.441386+0000","last_change":"2023-11-13T11:03:19.462210+0000","last_active":"2023-11-13T11:03:20.441386+0000","last_peered":"2023-11-13T11:03:20.441386+0000","last_clean":"2023-11-13T11:03:20.441386+0000","last_became_active":"2023-11-13T11:03:19.461775+0000","last_became_peered":"2023-11-13T11:03:19.461775+0000","last_unstale":"2023-11-13T11:03:20.441386+0000","last_undegraded":"2023-11-13T11:03:20.441386+0000","last_fullsized":"2023-11-13T11:03:20.441386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:52:49.857765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836506+0000","last_change":"2023-11-13T11:03:20.456308+0000","last_active":"2023-11-13T11:03:21.836506+0000","last_peered":"2023-11-13T11:03:21.836506+0000","last_clean":"2023-11-13T11:03:21.836506+0000","last_became_active":"2023-11-13T11:03:20.454264+0000","last_became_peered":"2023-11-13T11:03:20.454264+0000","last_unstale":"2023-11-13T11:03:21.836506+0000","last_undegraded":"2023-11-13T11:03:21.836506+0000","last_fullsized":"2023-11-13T11:03:21.836506+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:41.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.457139+0000","last_change":"2023-11-13T11:03:19.457132+0000","last_active":"2023-11-13T11:03:20.457139+0000","last_peered":"2023-11-13T11:03:20.457139+0000","last_clean":"2023-11-13T11:03:20.457139+0000","last_became_active":"2023-11-13T11:03:19.455974+0000","last_became_peered":"2023-11-13T11:03:19.455974+0000","last_unstale":"2023-11-13T11:03:20.457139+0000","last_undegraded":"2023-11-13T11:03:20.457139+0000","last_fullsized":"2023-11-13T11:03:20.457139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:10:12.732364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.445033+0000","last_change":"2023-11-13T11:03:20.468314+0000","last_active":"2023-11-13T11:03:21.445033+0000","last_peered":"2023-11-13T11:03:21.445033+0000","last_clean":"2023-11-13T11:03:21.445033+0000","last_became_active":"2023-11-13T11:03:20.468045+0000","last_became_peered":"2023-11-13T11:03:20.468045+0000","last_unstale":"2023-11-13T11:03:21.445033+0000","last_undegraded":"2023-11-13T11:03:21.445033+0000","last_fullsized":"2023-11-13T11:03:21.445033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:41:18.051038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"68'8","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492661+0000","last_change":"2023-11-13T11:03:19.461004+0000","last_active":"2023-11-13T11:03:20.492661+0000","last_peered":"2023-11-13T11:03:20.492661+0000","last_clean":"2023-11-13T11:03:20.492661+0000","last_became_active":"2023-11-13T11:03:19.460772+0000","last_became_peered":"2023-11-13T11:03:19.460772+0000","last_unstale":"2023-11-13T11:03:20.492661+0000","last_undegraded":"2023-11-13T11:03:20.492661+0000","last_fullsized":"2023-11-13T11:03:20.492661+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:43:02.704304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442230+0000","last_change":"2023-11-13T11:03:20.487082+0000","last_active":"2023-11-13T11:03:21.442230+0000","last_peered":"2023-11-13T11:03:21.442230+0000","last_clean":"2023-11-13T11:03:21.442230+0000","last_became_active":"2023-11-13T11:03:20.486346+0000","last_became_peered":"2023-11-13T11:03:20.486346+0000","last_unstale":"2023-11-13T11:03:21.442230+0000","last_undegraded":"2023-11-13T11:03:21.442230+0000","last_fullsized":"2023-11-13T11:03:21.442230+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:38:34.204968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492262+0000","last_change":"2023-11-13T11:03:19.457153+0000","last_active":"2023-11-13T11:03:20.492262+0000","last_peered":"2023-11-13T11:03:20.492262+0000","last_clean":"2023-11-13T11:03:20.492262+0000","last_became_active":"2023-11-13T11:03:19.456874+0000","last_became_peered":"2023-11-13T11:03:19.456874+0000","last_unstale":"2023-11-13T11:03:20.492262+0000","last_undegraded":"2023-11-13T11:03:20.492262+0000","last_fullsized":"2023-11-13T11:03:20.492262+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T13:36:21.280952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299367+0000","last_change":"2023-11-13T11:03:20.470055+0000","last_active":"2023-11-13T11:03:22.299367+0000","last_peered":"2023-11-13T11:03:22.299367+0000","last_clean":"2023-11-13T11:03:22.299367+0000","last_became_active":"2023-11-13T11:03:20.469745+0000","last_became_peered":"2023-11-13T11:03:20.469745+0000","last_unstale":"2023-11-13T11:03:22.299367+0000","last_undegraded":"2023-11-13T11:03:22.299367+0000","last_fullsized":"2023-11-13T11:03:22.299367+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:38:54.348922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492118+0000","last_change":"2023-11-13T11:03:19.460723+0000","last_active":"2023-11-13T11:03:20.492118+0000","last_peered":"2023-11-13T11:03:20.492118+0000","last_clean":"2023-11-13T11:03:20.492118+0000","last_became_active":"2023-11-13T11:03:19.460398+0000","last_became_peered":"2023-11-13T11:03:19.460398+0000","last_unstale":"2023-11-13T11:03:20.492118+0000","last_undegraded":"2023-11-13T11:03:20.492118+0000","last_fullsized":"2023-11-13T11:03:20.492118+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:32:40.173268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299033+0000","last_change":"2023-11-13T11:03:20.469995+0000","last_active":"2023-11-13T11:03:22.299033+0000","last_peered":"2023-11-13T11:03:22.299033+0000","last_clean":"2023-11-13T11:03:22.299033+0000","last_became_active":"2023-11-13T11:03:20.469673+0000","last_became_peered":"2023-11-13T11:03:20.469673+0000","last_unstale":"2023-11-13T11:03:22.299033+0000","last_undegraded":"2023-11-13T11:03:22.299033+0000","last_fullsized":"2023-11-13T11:03:22.299033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:16.869476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"68'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492197+0000","last_change":"2023-11-13T11:03:19.462642+0000","last_active":"2023-11-13T11:03:20.492197+0000","last_peered":"2023-11-13T11:03:20.492197+0000","last_clean":"2023-11-13T11:03:20.492197+0000","last_became_active":"2023-11-13T11:03:19.462365+0000","last_became_peered":"2023-11-13T11:03:19.462365+0000","last_unstale":"2023-11-13T11:03:20.492197+0000","last_undegraded":"2023-11-13T11:03:20.492197+0000","last_fullsized":"2023-11-13T11:03:20.492197+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:05:17.285018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.299164+0000","last_change":"2023-11-13T11:03:20.474321+0000","last_active":"2023-11-13T11:03:22.299164+0000","last_peered":"2023-11-13T11:03:22.299164+0000","last_clean":"2023-11-13T11:03:22.299164+0000","last_became_active":"2023-11-13T11:03:20.468744+0000","last_became_peered":"2023-11-13T11:03:20.468744+0000","last_unstale":"2023-11-13T11:03:22.299164+0000","last_undegraded":"2023-11-13T11:03:22.299164+0000","last_fullsized":"2023-11-13T11:03:22.299164+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T15:59:35.919204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059038+0000","last_change":"2023-11-13T11:03:19.469381+0000","last_active":"2023-11-13T11:03:22.059038+0000","last_peered":"2023-11-13T11:03:22.059038+0000","last_clean":"2023-11-13T11:03:22.059038+0000","last_became_active":"2023-11-13T11:03:19.468768+0000","last_became_peered":"2023-11-13T11:03:19.468768+0000","last_unstale":"2023-11-13T11:03:22.059038+0000","last_undegraded":"2023-11-13T11:03:22.059038+0000","last_fullsized":"2023-11-13T11:03:22.059038+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:37:23.154386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.445436+0000","last_change":"2023-11-13T11:03:20.473666+0000","last_active":"2023-11-13T11:03:21.445436+0000","last_peered":"2023-11-13T11:03:21.445436+0000","last_clean":"2023-11-13T11:03:21.445436+0000","last_became_active":"2023-11-13T11:03:20.472677+0000","last_became_peered":"2023-11-13T11:03:20.472677+0000","last_unstale":"2023-11-13T11:03:21.445436+0000","last_undegraded":"2023-11-13T11:03:21.445436+0000","last_fullsized":"2023-11-13T11:03:21.445436+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:27:37.299626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.488193+0000","last_change":"2023-11-13T11:03:19.470541+0000","last_active":"2023-11-13T11:03:20.488193+0000","last_peered":"2023-11-13T11:03:20.488193+0000","last_clean":"2023-11-13T11:03:20.488193+0000","last_became_active":"2023-11-13T11:03:19.470025+0000","last_became_peered":"2023-11-13T11:03:19.470025+0000","last_unstale":"2023-11-13T11:03:20.488193+0000","last_undegraded":"2023-11-13T11:03:20.488193+0000","last_fullsized":"2023-11-13T11:03:20.488193+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:41:17.033629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296924+0000","last_change":"2023-11-13T11:03:20.467427+0000","last_active":"2023-11-13T11:03:22.296924+0000","last_peered":"2023-11-13T11:03:22.296924+0000","last_clean":"2023-11-13T11:03:22.296924+0000","last_became_active":"2023-11-13T11:03:20.467133+0000","last_became_peered":"2023-11-13T11:03:20.467133+0000","last_unstale":"2023-11-13T11:03:22.296924+0000","last_undegraded":"2023-11-13T11:03:22.296924+0000","last_fullsized":"2023-11-13T11:03:22.296924+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:01:16.593902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.442840+0000","last_change":"2023-11-13T11:03:19.460424+0000","last_active":"2023-11-13T11:03:20.442840+0000","last_peered":"2023-11-13T11:03:20.442840+0000","last_clean":"2023-11-13T11:03:20.442840+0000","last_became_active":"2023-11-13T11:03:19.460137+0000","last_became_peered":"2023-11-13T11:03:19.460137+0000","last_unstale":"2023-11-13T11:03:20.442840+0000","last_undegraded":"2023-11-13T11:03:20.442840+0000","last_fullsized":"2023-11-13T11:03:20.442840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:42:27.040835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441389+0000","last_change":"2023-11-13T11:03:20.472596+0000","last_active":"2023-11-13T11:03:21.441389+0000","last_peered":"2023-11-13T11:03:21.441389+0000","last_clean":"2023-11-13T11:03:21.441389+0000","last_became_active":"2023-11-13T11:03:20.472243+0000","last_became_peered":"2023-11-13T11:03:20.472243+0000","last_unstale":"2023-11-13T11:03:21.441389+0000","last_undegraded":"2023-11-13T11:03:21.441389+0000","last_fullsized":"2023-11-13T11:03:21.441389+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:15.570305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.445684+0000","last_change":"2023-11-13T11:03:19.462649+0000","last_active":"2023-11-13T11:03:20.445684+0000","last_peered":"2023-11-13T11:03:20.445684+0000","last_clean":"2023-11-13T11:03:20.445684+0000","last_became_active":"2023-11-13T11:03:19.462308+0000","last_became_peered":"2023-11-13T11:03:19.462308+0000","last_unstale":"2023-11-13T11:03:20.445684+0000","last_undegraded":"2023-11-13T11:03:20.445684+0000","last_fullsized":"2023-11-13T11:03:20.445684+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:21.547257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297182+0000","last_change":"2023-11-13T11:03:20.454510+0000","last_active":"2023-11-13T11:03:22.297182+0000","last_peered":"2023-11-13T11:03:22.297182+0000","last_clean":"2023-11-13T11:03:22.297182+0000","last_became_active":"2023-11-13T11:03:20.454165+0000","last_became_peered":"2023-11-13T11:03:20.454165+0000","last_unstale":"2023-11-13T11:03:22.297182+0000","last_undegraded":"2023-11-13T11:03:22.297182+0000","last_fullsized":"2023-11-13T11:03:22.297182+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:48:04.143689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"68'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.492509+0000","last_change":"2023-11-13T11:03:19.456513+0000","last_active":"2023-11-13T11:03:20.492509+0000","last_peered":"2023-11-13T11:03:20.492509+0000","last_clean":"2023-11-13T11:03:20.492509+0000","last_became_active":"2023-11-13T11:03:19.455795+0000","last_became_peered":"2023-11-13T11:03:19.455795+0000","last_unstale":"2023-11-13T11:03:20.492509+0000","last_undegraded":"2023-11-13T11:03:20.492509+0000","last_fullsized":"2023-11-13T11:03:20.492509+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:38:53.331513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.297194+0000","last_change":"2023-11-13T11:03:20.464089+0000","last_active":"2023-11-13T11:03:22.297194+0000","last_peered":"2023-11-13T11:03:22.297194+0000","last_clean":"2023-11-13T11:03:22.297194+0000","last_became_active":"2023-11-13T11:03:20.463759+0000","last_became_peered":"2023-11-13T11:03:20.463759+0000","last_unstale":"2023-11-13T11:03:22.297194+0000","last_undegraded":"2023-11-13T11:03:22.297194+0000","last_fullsized":"2023-11-13T11:03:22.297194+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:17:06.846928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.442339+0000","last_change":"2023-11-13T11:03:19.460123+0000","last_active":"2023-11-13T11:03:21.442339+0000","last_peered":"2023-11-13T11:03:21.442339+0000","last_clean":"2023-11-13T11:03:21.442339+0000","last_became_active":"2023-11-13T11:03:19.459856+0000","last_became_peered":"2023-11-13T11:03:19.459856+0000","last_unstale":"2023-11-13T11:03:21.442339+0000","last_undegraded":"2023-11-13T11:03:21.442339+0000","last_fullsized":"2023-11-13T11:03:21.442339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:10:15.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298176+0000","last_change":"2023-11-13T11:03:20.469332+0000","last_active":"2023-11-13T11:03:22.298176+0000","last_peered":"2023-11-13T11:03:22.298176+0000","last_clean":"2023-11-13T11:03:22.298176+0000","last_became_active":"2023-11-13T11:03:20.469074+0000","last_became_peered":"2023-11-13T11:03:20.469074+0000","last_unstale":"2023-11-13T11:03:22.298176+0000","last_undegraded":"2023-11-13T11:03:22.298176+0000","last_fullsized":"2023-11-13T11:03:22.298176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:15:47.761631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.445273+0000","last_change":"2023-11-13T11:03:19.468701+0000","last_active":"2023-11-13T11:03:20.445273+0000","last_peered":"2023-11-13T11:03:20.445273+0000","last_clean":"2023-11-13T11:03:20.445273+0000","last_became_active":"2023-11-13T11:03:19.468415+0000","last_became_peered":"2023-11-13T11:03:19.468415+0000","last_unstale":"2023-11-13T11:03:20.445273+0000","last_undegraded":"2023-11-13T11:03:20.445273+0000","last_fullsized":"2023-11-13T11:03:20.445273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T20:11:28.058672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298633+0000","last_change":"2023-11-13T11:03:20.469925+0000","last_active":"2023-11-13T11:03:22.298633+0000","last_peered":"2023-11-13T11:03:22.298633+0000","last_clean":"2023-11-13T11:03:22.298633+0000","last_became_active":"2023-11-13T11:03:20.469129+0000","last_became_peered":"2023-11-13T11:03:20.469129+0000","last_unstale":"2023-11-13T11:03:22.298633+0000","last_undegraded":"2023-11-13T11:03:22.298633+0000","last_fullsized":"2023-11-13T11:03:22.298633+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T14:30:08.514257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.059174+0000","last_change":"2023-11-13T11:03:19.468809+0000","last_active":"2023-11-13T11:03:22.059174+0000","last_peered":"2023-11-13T11:03:22.059174+0000","last_clean":"2023-11-13T11:03:22.059174+0000","last_became_active":"2023-11-13T11:03:19.467750+0000","last_became_peered":"2023-11-13T11:03:19.467750+0000","last_unstale":"2023-11-13T11:03:22.059174+0000","last_undegraded":"2023-11-13T11:03:22.059174+0000","last_fullsized":"2023-11-13T11:03:22.059174+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:45:50.416913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.441062+0000","last_change":"2023-11-13T11:03:20.466160+0000","last_active":"2023-11-13T11:03:21.441062+0000","last_peered":"2023-11-13T11:03:21.441062+0000","last_clean":"2023-11-13T11:03:21.441062+0000","last_became_active":"2023-11-13T11:03:20.465675+0000","last_became_peered":"2023-11-13T11:03:20.465675+0000","last_unstale":"2023-11-13T11:03:21.441062+0000","last_undegraded":"2023-11-13T11:03:21.441062+0000","last_fullsized":"2023-11-13T11:03:21.441062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:27:37.299626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.461189+0000","last_change":"2023-11-13T11:03:19.465809+0000","last_active":"2023-11-13T11:03:20.461189+0000","last_peered":"2023-11-13T11:03:20.461189+0000","last_clean":"2023-11-13T11:03:20.461189+0000","last_became_active":"2023-11-13T11:03:19.465063+0000","last_became_peered":"2023-11-13T11:03:19.465063+0000","last_unstale":"2023-11-13T11:03:20.461189+0000","last_undegraded":"2023-11-13T11:03:20.461189+0000","last_fullsized":"2023-11-13T11:03:20.461189+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T19:00:41.592546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.298363+0000","last_change":"2023-11-13T11:03:20.476368+0000","last_active":"2023-11-13T11:03:22.298363+0000","last_peered":"2023-11-13T11:03:22.298363+0000","last_clean":"2023-11-13T11:03:22.298363+0000","last_became_active":"2023-11-13T11:03:20.476109+0000","last_became_peered":"2023-11-13T11:03:20.476109+0000","last_unstale":"2023-11-13T11:03:22.298363+0000","last_undegraded":"2023-11-13T11:03:22.298363+0000","last_fullsized":"2023-11-13T11:03:22.298363+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:22.564666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.458999+0000","last_change":"2023-11-13T11:03:19.463189+0000","last_active":"2023-11-13T11:03:20.458999+0000","last_peered":"2023-11-13T11:03:20.458999+0000","last_clean":"2023-11-13T11:03:20.458999+0000","last_became_active":"2023-11-13T11:03:19.461814+0000","last_became_peered":"2023-11-13T11:03:19.461814+0000","last_unstale":"2023-11-13T11:03:20.458999+0000","last_undegraded":"2023-11-13T11:03:20.458999+0000","last_fullsized":"2023-11-13T11:03:20.458999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:05.196692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.446764+0000","last_change":"2023-11-13T11:03:20.474978+0000","last_active":"2023-11-13T11:03:21.446764+0000","last_peered":"2023-11-13T11:03:21.446764+0000","last_clean":"2023-11-13T11:03:21.446764+0000","last_became_active":"2023-11-13T11:03:20.474677+0000","last_became_peered":"2023-11-13T11:03:20.474677+0000","last_unstale":"2023-11-13T11:03:21.446764+0000","last_undegraded":"2023-11-13T11:03:21.446764+0000","last_fullsized":"2023-11-13T11:03:21.446764+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T22:43:15.570305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.058922+0000","last_change":"2023-11-13T11:03:19.468080+0000","last_active":"2023-11-13T11:03:22.058922+0000","last_peered":"2023-11-13T11:03:22.058922+0000","last_clean":"2023-11-13T11:03:22.058922+0000","last_became_active":"2023-11-13T11:03:19.467716+0000","last_became_peered":"2023-11-13T11:03:19.467716+0000","last_unstale":"2023-11-13T11:03:22.058922+0000","last_undegraded":"2023-11-13T11:03:22.058922+0000","last_fullsized":"2023-11-13T11:03:22.058922+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:57:39.710883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:22.296906+0000","last_change":"2023-11-13T11:03:20.475905+0000","last_active":"2023-11-13T11:03:22.296906+0000","last_peered":"2023-11-13T11:03:22.296906+0000","last_clean":"2023-11-13T11:03:22.296906+0000","last_became_active":"2023-11-13T11:03:20.474090+0000","last_became_peered":"2023-11-13T11:03:20.474090+0000","last_unstale":"2023-11-13T11:03:22.296906+0000","last_undegraded":"2023-11-13T11:03:22.296906+0000","last_fullsized":"2023-11-13T11:03:22.296906+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T11:57:32.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.446020+0000","last_change":"2023-11-13T11:03:19.459075+0000","last_active":"2023-11-13T11:03:20.446020+0000","last_peered":"2023-11-13T11:03:20.446020+0000","last_clean":"2023-11-13T11:03:20.446020+0000","last_became_active":"2023-11-13T11:03:19.458782+0000","last_became_peered":"2023-11-13T11:03:19.458782+0000","last_unstale":"2023-11-13T11:03:20.446020+0000","last_undegraded":"2023-11-13T11:03:20.446020+0000","last_fullsized":"2023-11-13T11:03:20.446020+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:22:05.196692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-13T11:03:21.836855+0000","last_change":"2023-11-13T11:03:20.465840+0000","last_active":"2023-11-13T11:03:21.836855+0000","last_peered":"2023-11-13T11:03:21.836855+0000","last_clean":"2023-11-13T11:03:21.836855+0000","last_became_active":"2023-11-13T11:03:20.465501+0000","last_became_peered":"2023-11-13T11:03:20.465501+0000","last_unstale":"2023-11-13T11:03:21.836855+0000","last_undegraded":"2023-11-13T11:03:21.836855+0000","last_fullsized":"2023-11-13T11:03:21.836855+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:19.414992+0000","last_clean_scrub_stamp":"2023-11-13T11:03:19.414992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T12:33:22.564666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.446370+0000","last_change":"2023-11-13T11:03:19.464113+0000","last_active":"2023-11-13T11:03:20.446370+0000","last_peered":"2023-11-13T11:03:20.446370+0000","last_clean":"2023-11-13T11:03:20.446370+0000","last_became_active":"2023-11-13T11:03:19.463678+0000","last_became_peered":"2023-11-13T11:03:19.463678+0000","last_unstale":"2023-11-13T11:03:20.446370+0000","last_undegraded":"2023-11-13T11:03:20.446370+0000","last_fullsized":"2023-11-13T11:03:20.446370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T11:03:18.397583+0000","last_clean_scrub_stamp":"2023-11-13T11:03:18.397583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T18:08:31.813846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":208,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-13T11:03:20.440483+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:03:20.440483+0000","last_peered":"2023-11-13T11:03:20.440483+0000","last_clean":"2023-11-13T11:03:20.440483+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:03:20.440483+0000","last_undegraded":"2023-11-13T11:03:20.440483+0000","last_fullsized":"2023-11-13T11:03:20.440483+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44488,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327352,"statfs":{"total":96636764160,"available":96591208448,"internally_reserved":0,"allocated":847872,"data_stored":374056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1060000000000001,"15min":1.1259999999999999},"min":{"1min":0.73899999999999999,"5min":0.63500000000000001,"15min":0.63100000000000001},"max":{"1min":1.706,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.153,"5min":1.2250000000000001,"15min":1.1379999999999999},"min":{"1min":0.82199999999999995,"5min":0.747,"15min":0.68500000000000005},"max":{"1min":1.498,"5min":1.865,"15min":2.2879999999999998},"last":1.1739999999999999}]},{"osd":1,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.252,"15min":1.1799999999999999},"min":{"1min":0.90300000000000002,"5min":0.64000000000000001,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.2889999999999999},"min":{"1min":0.67700000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.649,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.79100000000000004}]},{"osd":2,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.319,"15min":1.276},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.534,"5min":2.153,"15min":2.1680000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.383,"15min":1.3460000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.6100000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.2609999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.429,"15min":1.3779999999999999},"min":{"1min":0.81799999999999995,"5min":0.76300000000000001,"15min":0.63800000000000001},"max":{"1min":1.657,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.458,"15min":1.4379999999999999},"min":{"1min":0.99299999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.762,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.0680000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.456,"15min":1.552},"min":{"1min":0.94899999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.782,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.131},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.512,"15min":1.5740000000000001},"min":{"1min":1.1599999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.9419999999999999,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.101}]},{"osd":5,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.6060000000000001,"15min":1.6399999999999999},"min":{"1min":0.94499999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.8200000000000001,"5min":2.7210000000000001,"15min":3.286},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.625,"5min":1.75,"15min":1.7490000000000001},"min":{"1min":1.4650000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":1.865,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.609}]},{"osd":7,"last update":"Mon Nov 13 11:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.6930000000000001,"15min":1.645},"min":{"1min":1.2849999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0640000000000001,"5min":2.657,"15min":2.657},"last":1.206},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.641,"15min":1.6379999999999999},"min":{"1min":1.1399999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.1349999999999998,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.4410000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509543,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45068,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326772,"statfs":{"total":96636764160,"available":96590614528,"internally_reserved":0,"allocated":1441792,"data_stored":963884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.093,"15min":1.016},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7529999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.2190000000000001,"15min":1.202},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67800000000000005},"max":{"1min":1.639,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.2769999999999999}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.306,"15min":1.2350000000000001},"min":{"1min":0.79100000000000004,"5min":0.72399999999999998,"15min":0.65400000000000003},"max":{"1min":1.73,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.282,"15min":1.2450000000000001},"min":{"1min":0.875,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.95599999999999996}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.389,"15min":1.2649999999999999},"min":{"1min":0.81200000000000006,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":4.4500000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.48},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.3100000000000001,"15min":1.2370000000000001},"min":{"1min":0.78900000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.5760000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.1020000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.387,"15min":1.391},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8300000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.651},{"interface":"front","average":{"1min":1.53,"5min":1.5169999999999999,"15min":1.4870000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.80700000000000005},"max":{"1min":4.6959999999999997,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":1.0700000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3700000000000001,"15min":1.329},"min":{"1min":0.93500000000000005,"5min":0.86599999999999999,"15min":0.55500000000000005},"max":{"1min":1.5800000000000001,"5min":3.484,"15min":3.484},"last":1.016},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.518,"15min":1.53},"min":{"1min":1.093,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.4430000000000001,"5min":3.653,"15min":3.653},"last":1.5549999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.5800000000000001,"15min":1.6040000000000001},"min":{"1min":1.0800000000000001,"5min":0.92000000000000004,"15min":0.78000000000000003},"max":{"1min":4.8760000000000003,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.157},{"interface":"front","average":{"1min":1.367,"5min":1.538,"15min":1.5189999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.61799999999999999},"max":{"1min":1.7609999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.421}]},{"osd":7,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.645,"15min":1.6659999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.1349999999999998,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.833},{"interface":"front","average":{"1min":1.744,"5min":1.7330000000000001,"15min":1.714},"min":{"1min":1.2490000000000001,"5min":1.1200000000000001,"15min":0.69499999999999995},"max":{"1min":4.7919999999999998,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":1.365}]}]},{"osd":4,"up_from":15,"seq":64424509543,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54200,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317640,"statfs":{"total":96636764160,"available":96581263360,"internally_reserved":0,"allocated":831488,"data_stored":372374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.153,"15min":1.089},"min":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":1.986,"15min":1.986},"last":1.012},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.145,"15min":1.085},"min":{"1min":0.69199999999999995,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.087}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2470000000000001,"15min":1.163},"min":{"1min":0.76200000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.774,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.26},"min":{"1min":0.79300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":1.877,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.151}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.298,"15min":1.242},"min":{"1min":0.93300000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.575,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.347,"15min":1.349},"min":{"1min":0.91100000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.72,"5min":1.952,"15min":1.952},"last":1.3959999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.488,"15min":1.421},"min":{"1min":1.1080000000000001,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.786,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.3149999999999999,"15min":1.2969999999999999},"min":{"1min":0.751,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.841,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.274}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.478,"15min":1.4910000000000001},"min":{"1min":0.96399999999999997,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.778,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.476},{"interface":"front","average":{"1min":1.323,"5min":1.371,"15min":1.3939999999999999},"min":{"1min":0.81599999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":2.0529999999999999,"5min":2.089,"15min":2.089},"last":1.2110000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.621,"15min":1.6220000000000001},"min":{"1min":1.0600000000000001,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":1.7769999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.554},{"interface":"front","average":{"1min":1.401,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.92400000000000004,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":1.873,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.1200000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.5369999999999999,"15min":1.5820000000000001},"min":{"1min":1.05,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":2.1480000000000001,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.319},{"interface":"front","average":{"1min":1.454,"5min":1.599,"15min":1.609},"min":{"1min":1.0189999999999999,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":1.931,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.2509999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40316,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331524,"statfs":{"total":96636764160,"available":96595480576,"internally_reserved":0,"allocated":835584,"data_stored":371952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.628,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.282,"5min":1.867,"15min":1.867},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.022,"15min":1.026},"min":{"1min":0.65800000000000003,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.819,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":1.379}]},{"osd":1,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.032,"15min":1.0489999999999999},"min":{"1min":0.625,"5min":0.45600000000000002,"15min":0.438},"max":{"1min":1.8089999999999999,"5min":1.966,"15min":3.71},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.296,"5min":1.1240000000000001,"15min":1.103},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.179,"15min":1.1040000000000001},"min":{"1min":0.72899999999999998,"5min":0.50800000000000001,"15min":0.46600000000000003},"max":{"1min":2.0019999999999998,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.171,"15min":1.2829999999999999},"min":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7809999999999999,"5min":1.952,"15min":3.5710000000000002},"last":1.304}]},{"osd":4,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3300000000000001,"15min":1.4139999999999999},"min":{"1min":0.89200000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.7509999999999999,"5min":2.411,"15min":3.5030000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3879999999999999,"15min":1.355},"min":{"1min":1.1259999999999999,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.831,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.615}]},{"osd":5,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4179999999999999,"15min":1.4750000000000001},"min":{"1min":1.151,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.99,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.454,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0649999999999999,"5min":2.673,"15min":3.3809999999999998},"last":1.258}]},{"osd":6,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.49,"15min":1.49},"min":{"1min":1.1479999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.944,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.546,"5min":1.4870000000000001,"15min":1.5840000000000001},"min":{"1min":1.1060000000000001,"5min":0.85799999999999998,"15min":0.76800000000000002},"max":{"1min":2.093,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.591}]},{"osd":7,"last update":"Mon Nov 13 11:02:31 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5609999999999999,"15min":1.609},"min":{"1min":0.84799999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1339999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.649,"5min":1.6040000000000001,"15min":1.5620000000000001},"min":{"1min":1.1359999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.4089999999999998,"5min":2.75,"15min":2.75},"last":1.4970000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509542,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50052,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321788,"statfs":{"total":96636764160,"available":96585510912,"internally_reserved":0,"allocated":835584,"data_stored":371848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94199999999999995,"15min":0.93100000000000005},"min":{"1min":0.55700000000000005,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.0229999999999999,"15min":0.93400000000000005},"min":{"1min":0.54400000000000004,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":2.9460000000000002,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":0.84699999999999998}]},{"osd":1,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0309999999999999,"15min":0.96299999999999997},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.376},"max":{"1min":3.0720000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.1899999999999999,"15min":1.1419999999999999},"min":{"1min":0.68100000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.3380000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.5980000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.216,"15min":1.2430000000000001},"min":{"1min":0.71599999999999997,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2549999999999999,"15min":1.2749999999999999},"min":{"1min":0.72699999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.4060000000000001,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.417}]},{"osd":4,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4359999999999999,"15min":1.4730000000000001},"min":{"1min":0.89900000000000002,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":2.633,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.393,"15min":1.409},"min":{"1min":0.80000000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":1.3839999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.4690000000000001,"15min":1.482},"min":{"1min":1.034,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.1680000000000001,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.623,"5min":1.591,"15min":1.5680000000000001},"min":{"1min":0.97099999999999997,"5min":0.85999999999999999,"15min":0.80500000000000005},"max":{"1min":2.7509999999999999,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.786}]},{"osd":6,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5189999999999999,"15min":1.55},"min":{"1min":0.89500000000000002,"5min":0.80900000000000005,"15min":0.76500000000000001},"max":{"1min":3.395,"5min":3.395,"15min":4.7569999999999997},"last":1.246},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6120000000000001,"15min":1.599},"min":{"1min":1.2130000000000001,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.5309999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:02:32 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.5700000000000001,"15min":1.607},"min":{"1min":1.0569999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":3.2429999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.694,"5min":1.6479999999999999,"15min":1.6259999999999999},"min":{"1min":1.2090000000000001,"5min":0.91200000000000003,"15min":0.82799999999999996},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":1.6659999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509541,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28416,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343424,"statfs":{"total":96636764160,"available":96607666176,"internally_reserved":0,"allocated":1425408,"data_stored":962194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85699999999999998,"15min":0.94899999999999995},"min":{"1min":0.495,"5min":0.37,"15min":0.35799999999999998},"max":{"1min":1.4910000000000001,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":1.034},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.97399999999999998,"15min":0.95699999999999996},"min":{"1min":0.623,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2689999999999999,"5min":1.869,"15min":1.869},"last":0.97599999999999998}]},{"osd":2,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.0509999999999999,"15min":1},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.40000000000000002},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.873},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0680000000000001,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":1.4670000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.204,"15min":1.175},"min":{"1min":0.68600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.57799999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4019999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.321,"15min":1.3129999999999999},"min":{"1min":0.76200000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.4179999999999999,"15min":1.3859999999999999},"min":{"1min":0.69599999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.0299999999999998,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.3580000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4159999999999999,"15min":1.4139999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.0750000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.387,"15min":1.4079999999999999},"min":{"1min":1.0309999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9410000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.587}]},{"osd":6,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.341,"15min":1.407},"min":{"1min":0.90900000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.649,"5min":1.504,"15min":1.5069999999999999},"min":{"1min":1.0640000000000001,"5min":0.79400000000000004,"15min":0.752},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3879999999999999},"last":1.4450000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:02:34 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.466,"15min":1.494},"min":{"1min":1.228,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.0609999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4399999999999999,"15min":1.4570000000000001},"min":{"1min":0.96899999999999997,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.7569999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44472,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327368,"statfs":{"total":96636764160,"available":96591224832,"internally_reserved":0,"allocated":827392,"data_stored":371792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.56100000000000005,"5min":0.36699999999999999,"15min":0.32100000000000001},"max":{"1min":1.464,"5min":1.74,"15min":1.74},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.589,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.042}]},{"osd":2,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.7230000000000001,"5min":2.105,"15min":2.105},"last":1.4990000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.371},"max":{"1min":1.4179999999999999,"5min":1.968,"15min":2.1669999999999998},"last":0.96999999999999997}]},{"osd":3,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1559999999999999,"15min":1.105},"min":{"1min":0.66700000000000004,"5min":0.54000000000000004,"15min":0.45200000000000001},"max":{"1min":1.571,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.25},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.3009999999999999,"15min":1.3140000000000001},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.8460000000000001,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.4019999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.425,"15min":1.363},"min":{"1min":0.81299999999999994,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.786,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.786},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3680000000000001,"15min":1.4159999999999999},"min":{"1min":0.81999999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5309999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.456}]},{"osd":5,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.492,"15min":1.528},"min":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.611,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.468,"15min":1.482},"min":{"1min":0.91700000000000004,"5min":0.78600000000000003,"15min":0.747},"max":{"1min":1.746,"5min":2.706,"15min":2.706},"last":1.5640000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.597,"15min":1.5329999999999999},"min":{"1min":1.1020000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.669,"5min":4.617,"15min":4.617},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.5860000000000001,"15min":1.573},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.72899999999999998},"max":{"1min":2.0249999999999999,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.6950000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.643,"15min":1.6619999999999999},"min":{"1min":0.85799999999999998,"5min":0.82299999999999995,"15min":0.76700000000000002},"max":{"1min":1.9430000000000001,"5min":2.742,"15min":3.3540000000000001},"last":1.617},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.6699999999999999,"15min":1.6180000000000001},"min":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.77400000000000002},"max":{"1min":1.776,"5min":3.976,"15min":3.976},"last":1.343}]}]},{"osd":7,"up_from":14,"seq":60129542246,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45880,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325960,"statfs":{"total":96636764160,"available":96589783040,"internally_reserved":0,"allocated":827392,"data_stored":371792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.079,"15min":1.1120000000000001},"min":{"1min":0.67200000000000004,"5min":0.63,"15min":0.59899999999999998},"max":{"1min":1.6020000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.05,"5min":1.0740000000000001,"15min":1.0720000000000001},"min":{"1min":0.57599999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4810000000000001,"5min":1.6080000000000001,"15min":2.266},"last":1.1559999999999999}]},{"osd":1,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1519999999999999,"15min":1.0669999999999999},"min":{"1min":0.65900000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.397,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2330000000000001,"15min":1.2290000000000001},"min":{"1min":0.79800000000000004,"5min":0.70099999999999996,"15min":0.64700000000000002},"max":{"1min":1.6080000000000001,"5min":1.893,"15min":2.1899999999999999},"last":1.248}]},{"osd":2,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2809999999999999,"15min":1.298},"min":{"1min":0.76000000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.859,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.655},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.278,"15min":1.2230000000000001},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.736,"5min":1.79,"15min":1.79},"last":1.3360000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.3320000000000001,"15min":1.262},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7210000000000001,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.115},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5049999999999999,"15min":1.4610000000000001},"min":{"1min":0.96299999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0459999999999998,"5min":2.2639999999999998,"15min":2.8159999999999998},"last":1.506}]},{"osd":4,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3440000000000001,"15min":1.3240000000000001},"min":{"1min":0.629,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.6240000000000001,"5min":1.984,"15min":3.0190000000000001},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.405,"15min":1.3759999999999999},"min":{"1min":0.70699999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.756,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.615}]},{"osd":5,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.504,"15min":1.5309999999999999},"min":{"1min":0.88700000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9550000000000001,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.734},{"interface":"front","average":{"1min":1.411,"5min":1.506,"15min":1.48},"min":{"1min":0.73999999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8540000000000001,"5min":2.3079999999999998,"15min":3.3620000000000001},"last":1.6950000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:02:30 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6100000000000001,"15min":1.5720000000000001},"min":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":1.8140000000000001,"5min":2.379,"15min":2.9020000000000001},"last":1.847},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.5620000000000001,"15min":1.512},"min":{"1min":0.76700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.9179999999999999,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.4450000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:03:30.600 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-13T11:03:30.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-13T11:03:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.874+0000 7f3a99cff640 1 Processor -- start 2023-11-13T11:03:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.874+0000 7f3a99cff640 1 -- start start 2023-11-13T11:03:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a99cff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a940b37f0 0x7f3a9406d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a99cff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3a9406b870 0x7f3a9406bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a99cff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a940b3c60 con 0x7f3a940cc740 2023-11-13T11:03:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a99cff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a940b1a70 con 0x7f3a940b37f0 2023-11-13T11:03:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a99cff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a940b1d50 con 0x7f3a9406b870 2023-11-13T11:03:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a8bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a940b37f0 0x7f3a9406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a8bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a940b37f0 0x7f3a9406d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55472/0 (socket says 172.21.15.16:55472) 2023-11-13T11:03:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a8bfff640 1 -- 172.21.15.16:0/148398011 learned_addr learned my addr 172.21.15.16:0/148398011 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a994fe640 1 --2- 172.21.15.16:0/148398011 >> v2:172.21.15.184:3300/0 conn(0x7f3a9406b870 0x7f3a9406bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a8bfff640 1 -- 172.21.15.16:0/148398011 >> v2:172.21.15.184:3300/0 conn(0x7f3a9406b870 msgr2=0x7f3a9406bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a8bfff640 1 --2- 172.21.15.16:0/148398011 >> v2:172.21.15.184:3300/0 conn(0x7f3a9406b870 0x7f3a9406bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a8bfff640 1 -- 172.21.15.16:0/148398011 >> v1:172.21.15.184:6789/0 conn(0x7f3a940cc740 legacy=0x7f3a940ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a8bfff640 1 -- 172.21.15.16:0/148398011 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9406d640 con 0x7f3a940b37f0 2023-11-13T11:03:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.877+0000 7f3a994fe640 1 --2- 172.21.15.16:0/148398011 >> v2:172.21.15.184:3300/0 conn(0x7f3a9406b870 0x7f3a9406bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.879+0000 7f3a8bfff640 1 --2- 172.21.15.16:0/148398011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a940b37f0 0x7f3a9406d100 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3a940cd950 tx=0x7f3a8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bcbd99919506ee server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.879+0000 7f3a8b7fe640 1 -- 172.21.15.16:0/148398011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84031870 con 0x7f3a940b37f0 2023-11-13T11:03:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.879+0000 7f3a8b7fe640 1 -- 172.21.15.16:0/148398011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a840319d0 con 0x7f3a940b37f0 2023-11-13T11:03:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.879+0000 7f3a8b7fe640 1 -- 172.21.15.16:0/148398011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84031d60 con 0x7f3a940b37f0 2023-11-13T11:03:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.880+0000 7f3a99cff640 1 -- 172.21.15.16:0/148398011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a940b37f0 msgr2=0x7f3a9406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.880+0000 7f3a99cff640 1 --2- 172.21.15.16:0/148398011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a940b37f0 0x7f3a9406d100 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3a940cd950 tx=0x7f3a8402b910 comp rx=0 tx=0).stop 2023-11-13T11:03:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.880+0000 7f3a99cff640 1 -- 172.21.15.16:0/148398011 shutdown_connections 2023-11-13T11:03:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.880+0000 7f3a99cff640 1 --2- 172.21.15.16:0/148398011 >> v2:172.21.15.184:3300/0 conn(0x7f3a9406b870 0x7f3a9406bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.880+0000 7f3a99cff640 1 --2- 172.21.15.16:0/148398011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a940b37f0 0x7f3a9406d100 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.880+0000 7f3a99cff640 1 -- 172.21.15.16:0/148398011 >> 172.21.15.16:0/148398011 conn(0x7f3a9406a190 msgr2=0x7f3a9406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.880+0000 7f3a99cff640 1 -- 172.21.15.16:0/148398011 shutdown_connections 2023-11-13T11:03:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.880+0000 7f3a99cff640 1 -- 172.21.15.16:0/148398011 wait complete. 2023-11-13T11:03:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.882+0000 7f3a99cff640 1 Processor -- start 2023-11-13T11:03:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.882+0000 7f3a99cff640 1 -- start start 2023-11-13T11:03:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.883+0000 7f3a99cff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9406b870 0x7f3a940a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.883+0000 7f3a99cff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940b37f0 0x7f3a9415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a99cff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940cc740 0x7f3a94159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a99cff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940af820 con 0x7f3a9406b870 2023-11-13T11:03:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a99cff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a941fa0a0 con 0x7f3a940cc740 2023-11-13T11:03:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a99cff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940afac0 con 0x7f3a940b37f0 2023-11-13T11:03:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a98cfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9406b870 0x7f3a940a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a98cfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9406b870 0x7f3a940a98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55476/0 (socket says 172.21.15.16:55476) 2023-11-13T11:03:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a98cfd640 1 -- 172.21.15.16:0/3919508692 learned_addr learned my addr 172.21.15.16:0/3919508692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a98cfd640 1 -- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940b37f0 msgr2=0x7f3a9415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a98cfd640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940b37f0 0x7f3a9415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a98cfd640 1 -- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940cc740 msgr2=0x7f3a94159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a98cfd640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940cc740 0x7f3a94159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.884+0000 7f3a98cfd640 1 -- 172.21.15.16:0/3919508692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840094d0 con 0x7f3a9406b870 2023-11-13T11:03:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.885+0000 7f3a98cfd640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9406b870 0x7f3a940a98f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000ba50 tx=0x7f3a9000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.885+0000 7f3a897fa640 1 -- 172.21.15.16:0/3919508692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000cc10 con 0x7f3a9406b870 2023-11-13T11:03:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.885+0000 7f3a897fa640 1 -- 172.21.15.16:0/3919508692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9000cd70 con 0x7f3a9406b870 2023-11-13T11:03:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.885+0000 7f3a897fa640 1 -- 172.21.15.16:0/3919508692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90014680 con 0x7f3a9406b870 2023-11-13T11:03:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.885+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a941fa360 con 0x7f3a9406b870 2023-11-13T11:03:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.885+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9415aba0 con 0x7f3a9406b870 2023-11-13T11:03:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.886+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a941fa5a0 con 0x7f3a9406b870 2023-11-13T11:03:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.887+0000 7f3a897fa640 1 -- 172.21.15.16:0/3919508692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a9001d3b0 con 0x7f3a9406b870 2023-11-13T11:03:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.888+0000 7f3a897fa640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a8006ae40 0x7f3a8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.888+0000 7f3a897fa640 1 -- 172.21.15.16:0/3919508692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3a90089700 con 0x7f3a9406b870 2023-11-13T11:03:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.889+0000 7f3a8bfff640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a8006ae40 0x7f3a8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.890+0000 7f3a8bfff640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a8006ae40 0x7f3a8006d300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3a84036000 tx=0x7f3a84030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:30.892+0000 7f3a897fa640 1 -- 172.21.15.16:0/3919508692 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a941fa5a0 con 0x7f3a9406b870 2023-11-13T11:03:31.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.158+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3a941fa5a0 con 0x7f3a9406b870 2023-11-13T11:03:31.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.162+0000 7f3a897fa640 1 -- 172.21.15.16:0/3919508692 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3a900190b0 con 0x7f3a9406b870 2023-11-13T11:03:31.166 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-13T11:03:31.167 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBdAFJl9p66LRAAYRThccLplmNpsr+W647dOA== 2023-11-13T11:03:31.167 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2023-11-13T11:03:31.167 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2023-11-13T11:03:31.167 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-13T11:03:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-11-13T11:03:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a8006ae40 msgr2=0x7f3a8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a8006ae40 0x7f3a8006d300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3a84036000 tx=0x7f3a84030000 comp rx=0 tx=0).stop 2023-11-13T11:03:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9406b870 msgr2=0x7f3a940a98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9406b870 0x7f3a940a98f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000ba50 tx=0x7f3a9000bf10 comp rx=0 tx=0).stop 2023-11-13T11:03:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 shutdown_connections 2023-11-13T11:03:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a8006ae40 0x7f3a8006d300 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a940cc740 0x7f3a94159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a940b37f0 0x7f3a9415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 --2- 172.21.15.16:0/3919508692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a9406b870 0x7f3a940a98f0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 >> 172.21.15.16:0/3919508692 conn(0x7f3a9406a190 msgr2=0x7f3a94163de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 shutdown_connections 2023-11-13T11:03:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.165+0000 7f3a99cff640 1 -- 172.21.15.16:0/3919508692 wait complete. 2023-11-13T11:03:31.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:31.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.395+0000 7f94c8fd2640 1 Processor -- start 2023-11-13T11:03:31.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.396+0000 7f94c8fd2640 1 -- start start 2023-11-13T11:03:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.397+0000 7f94c8fd2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40cb190 0x7f94c415b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:31.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.397+0000 7f94c8fd2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94c40716b0 0x7f94c406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:31.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.397+0000 7f94c8fd2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f94c406a190 0x7f94c406a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:31.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.397+0000 7f94c8fd2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c4158c20 con 0x7f94c40716b0 2023-11-13T11:03:31.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.397+0000 7f94c8fd2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c40b1830 con 0x7f94c406a190 2023-11-13T11:03:31.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.397+0000 7f94c8fd2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c406ab10 con 0x7f94c40cb190 2023-11-13T11:03:31.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94bbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94c40716b0 0x7f94c406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:31.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94bbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94c40716b0 0x7f94c406f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55490/0 (socket says 172.21.15.16:55490) 2023-11-13T11:03:31.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94bbfff640 1 -- 172.21.15.16:0/2024745855 learned_addr learned my addr 172.21.15.16:0/2024745855 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:31.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94c37fe640 1 --2- 172.21.15.16:0/2024745855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40cb190 0x7f94c415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:31.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94c3fff640 1 --2- 172.21.15.16:0/2024745855 >> v2:172.21.15.184:3300/0 conn(0x7f94c406a190 0x7f94c406a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:31.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94bbfff640 1 -- 172.21.15.16:0/2024745855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40cb190 msgr2=0x7f94c415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:31.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94bbfff640 1 --2- 172.21.15.16:0/2024745855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40cb190 0x7f94c415b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94bbfff640 1 -- 172.21.15.16:0/2024745855 >> v2:172.21.15.184:3300/0 conn(0x7f94c406a190 msgr2=0x7f94c406a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:31.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94bbfff640 1 --2- 172.21.15.16:0/2024745855 >> v2:172.21.15.184:3300/0 conn(0x7f94c406a190 0x7f94c406a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.398+0000 7f94bbfff640 1 -- 172.21.15.16:0/2024745855 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c415b780 con 0x7f94c40716b0 2023-11-13T11:03:31.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.399+0000 7f94bbfff640 1 --2- 172.21.15.16:0/2024745855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c40716b0 0x7f94c406f970 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f94c406dc70 tx=0x7f94ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3b351f2b4dddc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:31.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.400+0000 7f94c2ffd640 1 -- 172.21.15.16:0/2024745855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac031870 con 0x7f94c40716b0 2023-11-13T11:03:31.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.400+0000 7f94c2ffd640 1 -- 172.21.15.16:0/2024745855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac0319d0 con 0x7f94c40716b0 2023-11-13T11:03:31.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.400+0000 7f94c2ffd640 1 -- 172.21.15.16:0/2024745855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac031d60 con 0x7f94c40716b0 2023-11-13T11:03:31.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.401+0000 7f94c8fd2640 1 -- 172.21.15.16:0/2024745855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c40716b0 msgr2=0x7f94c406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:31.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.401+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/2024745855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c40716b0 0x7f94c406f970 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f94c406dc70 tx=0x7f94ac02b910 comp rx=0 tx=0).stop 2023-11-13T11:03:31.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.401+0000 7f94c8fd2640 1 -- 172.21.15.16:0/2024745855 shutdown_connections 2023-11-13T11:03:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.401+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/2024745855 >> v2:172.21.15.184:3300/0 conn(0x7f94c406a190 0x7f94c406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.401+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/2024745855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c40716b0 0x7f94c406f970 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.401+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/2024745855 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40cb190 0x7f94c415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.401+0000 7f94c8fd2640 1 -- 172.21.15.16:0/2024745855 >> 172.21.15.16:0/2024745855 conn(0x7f94c40b04a0 msgr2=0x7f94c40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:31.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.401+0000 7f94c8fd2640 1 -- 172.21.15.16:0/2024745855 shutdown_connections 2023-11-13T11:03:31.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.401+0000 7f94c8fd2640 1 -- 172.21.15.16:0/2024745855 wait complete. 2023-11-13T11:03:31.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.403+0000 7f94c8fd2640 1 Processor -- start 2023-11-13T11:03:31.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.403+0000 7f94c8fd2640 1 -- start start 2023-11-13T11:03:31.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c8fd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c406a190 0x7f94c4152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:31.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c8fd2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40716b0 0x7f94c4157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:31.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c8fd2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c40cb190 0x7f94c41ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:31.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c8fd2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4157260 con 0x7f94c406a190 2023-11-13T11:03:31.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c8fd2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4153120 con 0x7f94c40cb190 2023-11-13T11:03:31.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c8fd2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4152ea0 con 0x7f94c40716b0 2023-11-13T11:03:31.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c406a190 0x7f94c4152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:31.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c406a190 0x7f94c4152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55506/0 (socket says 172.21.15.16:55506) 2023-11-13T11:03:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c37fe640 1 -- 172.21.15.16:0/4001872419 learned_addr learned my addr 172.21.15.16:0/4001872419 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c37fe640 1 -- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40716b0 msgr2=0x7f94c4157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94bbfff640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40716b0 0x7f94c4157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c3fff640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c40cb190 0x7f94c41ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:31.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c37fe640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40716b0 0x7f94c4157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c37fe640 1 -- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c40cb190 msgr2=0x7f94c41ee6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c37fe640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c40cb190 0x7f94c41ee6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.404+0000 7f94c37fe640 1 -- 172.21.15.16:0/4001872419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0094d0 con 0x7f94c406a190 2023-11-13T11:03:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.405+0000 7f94c37fe640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c406a190 0x7f94c4152960 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f94b400d620 tx=0x7f94b400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:31.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.405+0000 7f94c0ff9640 1 -- 172.21.15.16:0/4001872419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b400b620 con 0x7f94c406a190 2023-11-13T11:03:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.405+0000 7f94c0ff9640 1 -- 172.21.15.16:0/4001872419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b400b780 con 0x7f94c406a190 2023-11-13T11:03:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.405+0000 7f94c0ff9640 1 -- 172.21.15.16:0/4001872419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b400bb30 con 0x7f94c406a190 2023-11-13T11:03:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.405+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c41568c0 con 0x7f94c406a190 2023-11-13T11:03:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.405+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c4156e80 con 0x7f94c406a190 2023-11-13T11:03:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.406+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c41574d0 con 0x7f94c406a190 2023-11-13T11:03:31.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.407+0000 7f94c0ff9640 1 -- 172.21.15.16:0/4001872419 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94b400bc90 con 0x7f94c406a190 2023-11-13T11:03:31.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.409+0000 7f94c0ff9640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94a406ae40 0x7f94a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:31.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.409+0000 7f94c0ff9640 1 -- 172.21.15.16:0/4001872419 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f94b4089cc0 con 0x7f94c406a190 2023-11-13T11:03:31.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.409+0000 7f94bbfff640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94a406ae40 0x7f94a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:31.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.410+0000 7f94bbfff640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94a406ae40 0x7f94a406d300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94ac036000 tx=0x7f94ac03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:31.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.413+0000 7f94c0ff9640 1 -- 172.21.15.16:0/4001872419 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94b4090080 con 0x7f94c406a190 2023-11-13T11:03:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.647+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94c404e4d0 con 0x7f94c406a190 2023-11-13T11:03:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.648+0000 7f94c0ff9640 1 -- 172.21.15.16:0/4001872419 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f94b4054b90 con 0x7f94c406a190 2023-11-13T11:03:31.653 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:31.653 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1247955601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1247955601},{"type":"v1","addr":"172.21.15.184:6836","nonce":1247955601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3721806669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3721806669},{"type":"v1","addr":"172.21.15.16:6834","nonce":3721806669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2700274237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2700274237},{"type":"v1","addr":"172.21.15.16:6835","nonce":2700274237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:22.988900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/60343961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":60343961},{"type":"v1","addr":"172.21.15.184:6837","nonce":60343961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:03:31.653 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 40 2023-11-13T11:03:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94a406ae40 msgr2=0x7f94a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94a406ae40 0x7f94a406d300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94ac036000 tx=0x7f94ac03c040 comp rx=0 tx=0).stop 2023-11-13T11:03:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c406a190 msgr2=0x7f94c4152960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c406a190 0x7f94c4152960 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f94b400d620 tx=0x7f94b400dae0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 shutdown_connections 2023-11-13T11:03:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f94a406ae40 0x7f94a406d300 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94c40cb190 0x7f94c41ee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94c40716b0 0x7f94c4157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 --2- 172.21.15.16:0/4001872419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c406a190 0x7f94c4152960 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 >> 172.21.15.16:0/4001872419 conn(0x7f94c40b04a0 msgr2=0x7f94c40b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 shutdown_connections 2023-11-13T11:03:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:31.651+0000 7f94c8fd2640 1 -- 172.21.15.16:0/4001872419 wait complete. 2023-11-13T11:03:32.077 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-13T11:03:19.913517+0000', 'modified': '2023-11-13T11:03:22.988900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6885}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6885': {'gid': 6885, 'name': 'b', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6837/60343961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6835', 'nonce': 60343961}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 60343961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-13T11:03:32.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-13T11:03:32.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-13T11:03:32.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.309+0000 7f90d359e640 1 Processor -- start 2023-11-13T11:03:32.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.310+0000 7f90d359e640 1 -- start start 2023-11-13T11:03:32.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.311+0000 7f90d359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f90d406a190 0x7f90d406a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.311+0000 7f90d359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d40b1840 con 0x7f90d406f970 2023-11-13T11:03:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.311+0000 7f90d359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d4071600 con 0x7f90d4161060 2023-11-13T11:03:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.311+0000 7f90d359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d406ab60 con 0x7f90d406a190 2023-11-13T11:03:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.312+0000 7f90cbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f90d406f970 0x7f90d4162900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41084/0 (socket says 172.21.15.16:41084) 2023-11-13T11:03:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.312+0000 7f90cbfff640 1 -- 172.21.15.16:0/401781295 learned_addr learned my addr 172.21.15.16:0/401781295 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.312+0000 7f90d2d9d640 1 --2- 172.21.15.16:0/401781295 >> v2:172.21.15.184:3300/0 conn(0x7f90d406a190 0x7f90d406a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.312+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819984345 0 0) 0x7f90d40b1840 con 0x7f90d406f970 2023-11-13T11:03:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.312+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c0003520 con 0x7f90d406f970 2023-11-13T11:03:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3024068987 0 0) 0x7f90c0003520 con 0x7f90d406f970 2023-11-13T11:03:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 >> v2:172.21.15.184:3300/0 conn(0x7f90d406a190 msgr2=0x7f90d406a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 --2- 172.21.15.16:0/401781295 >> v2:172.21.15.184:3300/0 conn(0x7f90d406a190 0x7f90d406a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 >> v1:172.21.15.184:6789/0 conn(0x7f90d4161060 legacy=0x7f90d41621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d40718e0 con 0x7f90d406f970 2023-11-13T11:03:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f90bc001670 con 0x7f90d406f970 2023-11-13T11:03:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 >> v1:172.21.15.16:6789/0 conn(0x7f90d406f970 legacy=0x7f90d4162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c0003970 0x7f90c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c0005180 0x7f90c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d406ab60 con 0x7f90c00047c0 2023-11-13T11:03:32.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c000c020 con 0x7f90c0003970 2023-11-13T11:03:32.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c000c1e0 con 0x7f90c0005180 2023-11-13T11:03:32.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.313+0000 7f90cbfff640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90d259c640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c0003970 0x7f90c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90d2d9d640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c0005180 0x7f90c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90cbfff640 1 -- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c0005180 msgr2=0x7f90c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90cbfff640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c0005180 0x7f90c000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90cbfff640 1 -- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c0003970 msgr2=0x7f90c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90cbfff640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c0003970 0x7f90c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90cbfff640 1 -- 172.21.15.16:0/401781295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c0003520 con 0x7f90c00047c0 2023-11-13T11:03:32.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90d259c640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90c0003970 0x7f90c0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:32.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90d2d9d640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90c0005180 0x7f90c000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:32.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.314+0000 7f90d2d9d640 1 -- 172.21.15.16:0/401781295 reap_dead start 2023-11-13T11:03:32.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.315+0000 7f90cbfff640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90c0004c40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c240 tx=0x7f90c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9821d66a4a82e1e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.315+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc005ad0 con 0x7f90c00047c0 2023-11-13T11:03:32.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.315+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc005c30 con 0x7f90c00047c0 2023-11-13T11:03:32.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.315+0000 7f90d1d9b640 1 -- 172.21.15.16:0/401781295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc02c9f0 con 0x7f90c00047c0 2023-11-13T11:03:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.316+0000 7f90d359e640 1 -- 172.21.15.16:0/401781295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 msgr2=0x7f90c0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.316+0000 7f90d359e640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90c0004c40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c240 tx=0x7f90c00057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.316+0000 7f90d359e640 1 -- 172.21.15.16:0/401781295 shutdown_connections 2023-11-13T11:03:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.316+0000 7f90d359e640 1 --2- 172.21.15.16:0/401781295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90c0004c40 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.316+0000 7f90d359e640 1 -- 172.21.15.16:0/401781295 >> 172.21.15.16:0/401781295 conn(0x7f90d40b04a0 msgr2=0x7f90d40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.316+0000 7f90d359e640 1 -- 172.21.15.16:0/401781295 shutdown_connections 2023-11-13T11:03:32.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.316+0000 7f90d359e640 1 -- 172.21.15.16:0/401781295 wait complete. 2023-11-13T11:03:32.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.318+0000 7f90d359e640 1 Processor -- start 2023-11-13T11:03:32.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.318+0000 7f90d359e640 1 -- start start 2023-11-13T11:03:32.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.319+0000 7f90d359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90d414cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.319+0000 7f90d359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90d40aa130 0x7f90d40aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.319+0000 7f90d359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90d415b700 0x7f90d415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.319+0000 7f90d359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d415c0c0 con 0x7f90c00047c0 2023-11-13T11:03:32.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.319+0000 7f90d359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d415ed40 con 0x7f90d415b700 2023-11-13T11:03:32.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.319+0000 7f90d359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d415e130 con 0x7f90d40aa130 2023-11-13T11:03:32.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.319+0000 7f90d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90d414cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.319+0000 7f90d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90d414cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55516/0 (socket says 172.21.15.16:55516) 2023-11-13T11:03:32.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.319+0000 7f90d259c640 1 -- 172.21.15.16:0/3254510021 learned_addr learned my addr 172.21.15.16:0/3254510021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:32.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90d259c640 1 -- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90d40aa130 msgr2=0x7f90d40aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90d259c640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90d40aa130 0x7f90d40aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90d259c640 1 -- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90d415b700 msgr2=0x7f90d415bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90d259c640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90d415b700 0x7f90d415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90d259c640 1 -- 172.21.15.16:0/3254510021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d415e410 con 0x7f90c00047c0 2023-11-13T11:03:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90d259c640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90d414cbd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f90c4000f60 tx=0x7f90c400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90caffd640 1 -- 172.21.15.16:0/3254510021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4005480 con 0x7f90c00047c0 2023-11-13T11:03:32.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90caffd640 1 -- 172.21.15.16:0/3254510021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c4003610 con 0x7f90c00047c0 2023-11-13T11:03:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90caffd640 1 -- 172.21.15.16:0/3254510021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c40039c0 con 0x7f90c00047c0 2023-11-13T11:03:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.320+0000 7f90d359e640 1 -- 172.21.15.16:0/3254510021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d415e6f0 con 0x7f90c00047c0 2023-11-13T11:03:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.321+0000 7f90d359e640 1 -- 172.21.15.16:0/3254510021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d414d110 con 0x7f90c00047c0 2023-11-13T11:03:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.323+0000 7f90caffd640 1 -- 172.21.15.16:0/3254510021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f90c4003b20 con 0x7f90c00047c0 2023-11-13T11:03:32.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.324+0000 7f90caffd640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f90b406ae40 0x7f90b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.324+0000 7f90caffd640 1 -- 172.21.15.16:0/3254510021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f90c4089640 con 0x7f90c00047c0 2023-11-13T11:03:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.324+0000 7f90dba2d740 1 Processor -- start 2023-11-13T11:03:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.324+0000 7f90cbfff640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f90b406ae40 0x7f90b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.325+0000 7f90dba2d740 1 -- start start 2023-11-13T11:03:32.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.325+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5118fb0 0x55c4f5119390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.325+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5113cb0 0x55c4f51181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.325+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 0x55c4f515c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.325+0000 7f90dba2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f5077880 con 0x55c4f5118fb0 2023-11-13T11:03:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.325+0000 7f90dba2d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f5086ae0 con 0x55c4f5113cb0 2023-11-13T11:03:32.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.325+0000 7f90dba2d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4f50841d0 con 0x55c4f51186f0 2023-11-13T11:03:32.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90cbfff640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f90b406ae40 0x7f90b406d300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90bc034000 tx=0x7f90bc039040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5118fb0 0x55c4f5119390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5118fb0 0x55c4f5119390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55528/0 (socket says 172.21.15.16:55528) 2023-11-13T11:03:32.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90d259c640 1 -- 172.21.15.16:0/1679877364 learned_addr learned my addr 172.21.15.16:0/1679877364 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:32.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90cbfff640 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5113cb0 0x55c4f51181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90d2d9d640 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 0x55c4f515c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90d259c640 1 -- 172.21.15.16:0/1679877364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 msgr2=0x55c4f515c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90d259c640 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 0x55c4f515c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90d259c640 1 -- 172.21.15.16:0/1679877364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5113cb0 msgr2=0x55c4f51181b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90d259c640 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5113cb0 0x55c4f51181b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90d259c640 1 -- 172.21.15.16:0/1679877364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4f5047860 con 0x55c4f5118fb0 2023-11-13T11:03:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.326+0000 7f90cbfff640 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5113cb0 0x55c4f51181b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.327+0000 7f90d259c640 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5118fb0 0x55c4f5119390 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f90c4087900 tx=0x7f90c4057c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480f6ce639307cff server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.327+0000 7f90b3fff640 1 -- 172.21.15.16:0/1679877364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c405c160 con 0x55c4f5118fb0 2023-11-13T11:03:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.327+0000 7f90b3fff640 1 -- 172.21.15.16:0/1679877364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c40604d0 con 0x55c4f5118fb0 2023-11-13T11:03:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.327+0000 7f90b3fff640 1 -- 172.21.15.16:0/1679877364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4087aa0 con 0x55c4f5118fb0 2023-11-13T11:03:32.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.327+0000 7f90dba2d740 1 -- 172.21.15.16:0/1679877364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5118fb0 msgr2=0x55c4f5119390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.327+0000 7f90dba2d740 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5118fb0 0x55c4f5119390 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f90c4087900 tx=0x7f90c4057c60 comp rx=0 tx=0).stop 2023-11-13T11:03:32.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.328+0000 7f90dba2d740 1 -- 172.21.15.16:0/1679877364 shutdown_connections 2023-11-13T11:03:32.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.328+0000 7f90dba2d740 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 0x55c4f515c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.328+0000 7f90dba2d740 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5113cb0 0x55c4f51181b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.328+0000 7f90dba2d740 1 --2- 172.21.15.16:0/1679877364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5118fb0 0x55c4f5119390 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.328+0000 7f90dba2d740 1 -- 172.21.15.16:0/1679877364 >> 172.21.15.16:0/1679877364 conn(0x55c4f50dabb0 msgr2=0x55c4f515eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.328+0000 7f90dba2d740 1 -- 172.21.15.16:0/1679877364 shutdown_connections 2023-11-13T11:03:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.328+0000 7f90dba2d740 1 -- 172.21.15.16:0/1679877364 wait complete. 2023-11-13T11:03:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.328+0000 7f90dba2d740 1 -- start start 2023-11-13T11:03:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.329+0000 7f90dba2d740 1 Processor -- start 2023-11-13T11:03:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f5113cb0 0x55c4f5140990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f51186f0 0x55c4f51641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f5155f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90dba2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f507fac0 con 0x55c4f51186f0 2023-11-13T11:03:32.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90dba2d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f501c2f0 con 0x55c4f5118fb0 2023-11-13T11:03:32.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90dba2d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4f5039c70 con 0x55c4f5113cb0 2023-11-13T11:03:32.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90dba2d740 10 client.0 _ll_register_callbacks cb 0x55c4f503e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:03:32.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f51186f0 0x55c4f51641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f51186f0 0x55c4f51641c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55534/0 (socket says 172.21.15.16:55534) 2023-11-13T11:03:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90cbfff640 1 -- 172.21.15.16:0/457228450 learned_addr learned my addr 172.21.15.16:0/457228450 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90d2d9d640 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f5155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90cbfff640 1 -- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f5113cb0 msgr2=0x55c4f5140990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90d259c640 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f5113cb0 0x55c4f5140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90cbfff640 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f5113cb0 0x55c4f5140990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90cbfff640 1 -- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 msgr2=0x55c4f5155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90cbfff640 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f5155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90cbfff640 1 -- 172.21.15.16:0/457228450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d414d110 con 0x55c4f51186f0 2023-11-13T11:03:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.330+0000 7f90d2d9d640 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f5155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.331+0000 7f90cbfff640 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f51186f0 0x55c4f51641c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f90bc001930 tx=0x7f90bc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aecbe74ede54a17 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.331+0000 7f90b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:03:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.331+0000 7f90b17fa640 1 -- 172.21.15.16:0/457228450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc048070 con 0x55c4f51186f0 2023-11-13T11:03:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.331+0000 7f90b17fa640 1 -- 172.21.15.16:0/457228450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f90bc04ec10 con 0x55c4f51186f0 2023-11-13T11:03:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.331+0000 7f90dba2d740 1 -- 172.21.15.16:0/457228450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4f5078aa0 con 0x55c4f51186f0 2023-11-13T11:03:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.331+0000 7f90b17fa640 1 -- 172.21.15.16:0/457228450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90bc03c040 con 0x55c4f51186f0 2023-11-13T11:03:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.331+0000 7f90dba2d740 10 client.7032 fetch_fsmap learned FSMap version 40 2023-11-13T11:03:32.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.331+0000 7f90dba2d740 1 -- 172.21.15.16:0/457228450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55c4f507e140 con 0x55c4f51186f0 2023-11-13T11:03:32.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.332+0000 7f90b17fa640 1 -- 172.21.15.16:0/457228450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c4f5078aa0 con 0x55c4f51186f0 2023-11-13T11:03:32.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.332+0000 7f90b17fa640 1 -- 172.21.15.16:0/457228450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f90bc007070 con 0x55c4f51186f0 2023-11-13T11:03:32.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.332+0000 7f90dba2d740 10 client.7032 fetch_fsmap finished waiting for FSMap version 40 2023-11-13T11:03:32.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.332+0000 7f90dba2d740 10 client.7032 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-13T11:03:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.332+0000 7f90dba2d740 20 client.7032 populate_metadata read hostname 'smithi016' 2023-11-13T11:03:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.332+0000 7f90dba2d740 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x55c4f515aa10 0x55c4f5157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.332+0000 7f90dba2d740 4 client.7032 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:03:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.332+0000 7f90dba2d740 1 -- 172.21.15.16:0/457228450 --> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4f50787c0 con 0x55c4f515aa10 2023-11-13T11:03:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.333+0000 7f90d259c640 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x55c4f515aa10 0x55c4f5157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.334+0000 7f90d259c640 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x55c4f515aa10 0x55c4f5157280 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c28a29acef46a966 server_cookie=c26f9e5cd250742b in_seq=0 out_seq=0 2023-11-13T11:03:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.334+0000 7f90b17fa640 10 client.7032 ms_handle_connect on v2:172.21.15.184:6835/60343961 2023-11-13T11:03:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.335+0000 7f90b17fa640 1 -- 172.21.15.16:0/457228450 <== mds.0 v2:172.21.15.184:6835/60343961 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c4f50787c0 con 0x55c4f515aa10 2023-11-13T11:03:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.335+0000 7f90b17fa640 10 client.7032 handle_command_reply: tid=0 2023-11-13T11:03:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.335+0000 7f90dba2d740 1 client.7032 shutdown 2023-11-13T11:03:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.335+0000 7f90dba2d740 1 -- 172.21.15.16:0/457228450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f51186f0 msgr2=0x55c4f51641c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.335+0000 7f90dba2d740 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f51186f0 0x55c4f51641c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f90bc001930 tx=0x7f90bc02e070 comp rx=0 tx=0).stop 2023-11-13T11:03:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90dba2d740 1 -- 172.21.15.16:0/457228450 shutdown_connections 2023-11-13T11:03:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90dba2d740 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f5155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90dba2d740 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f51186f0 0x55c4f51641c0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90dba2d740 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x55c4f515aa10 0x55c4f5157280 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90dba2d740 1 --2- 172.21.15.16:0/457228450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f5113cb0 0x55c4f5140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90dba2d740 1 -- 172.21.15.16:0/457228450 >> 172.21.15.16:0/457228450 conn(0x55c4f50dabb0 msgr2=0x55c4f5112d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90b17fa640 0 client.7032 ms_handle_reset on v2:172.21.15.184:6835/60343961 2023-11-13T11:03:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90dba2d740 1 -- 172.21.15.16:0/457228450 shutdown_connections 2023-11-13T11:03:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90dba2d740 1 -- 172.21.15.16:0/457228450 wait complete. 2023-11-13T11:03:32.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.336+0000 7f90dba2d740 20 client.7032 trim_cache size 0 max 16384 2023-11-13T11:03:32.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.353+0000 7f90dba2d740 1 Processor -- start 2023-11-13T11:03:32.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.353+0000 7f90dba2d740 1 -- start start 2023-11-13T11:03:32.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.354+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.354+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 0x55c4f5140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.354+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f515b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.354+0000 7f90dba2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f5166ce0 con 0x55c4f5113cb0 2023-11-13T11:03:32.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.354+0000 7f90dba2d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4de2f20 con 0x55c4f5118fb0 2023-11-13T11:03:32.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.354+0000 7f90dba2d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4f5081a40 con 0x55c4f51186f0 2023-11-13T11:03:32.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55546/0 (socket says 172.21.15.16:55546) 2023-11-13T11:03:32.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d259c640 1 -- 172.21.15.16:0/3381097981 learned_addr learned my addr 172.21.15.16:0/3381097981 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:32.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d2d9d640 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f515b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90cbfff640 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 0x55c4f5140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d259c640 1 -- 172.21.15.16:0/3381097981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 msgr2=0x55c4f5140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d259c640 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 0x55c4f5140ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d259c640 1 -- 172.21.15.16:0/3381097981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 msgr2=0x55c4f515b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d259c640 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f515b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d259c640 1 -- 172.21.15.16:0/3381097981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4f501bb80 con 0x55c4f5113cb0 2023-11-13T11:03:32.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d2d9d640 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f515b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:32.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.355+0000 7f90d259c640 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f90c406e040 tx=0x7f90c405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116509febb52b435 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90c8ff9640 1 -- 172.21.15.16:0/3381097981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4056570 con 0x55c4f5113cb0 2023-11-13T11:03:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90c8ff9640 1 -- 172.21.15.16:0/3381097981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c4087aa0 con 0x55c4f5113cb0 2023-11-13T11:03:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90c8ff9640 1 -- 172.21.15.16:0/3381097981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4060040 con 0x55c4f5113cb0 2023-11-13T11:03:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90dba2d740 1 -- 172.21.15.16:0/3381097981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 msgr2=0x55c4f51407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90dba2d740 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f90c406e040 tx=0x7f90c405b000 comp rx=0 tx=0).stop 2023-11-13T11:03:32.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90dba2d740 1 -- 172.21.15.16:0/3381097981 shutdown_connections 2023-11-13T11:03:32.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90dba2d740 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f5118fb0 0x55c4f515b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90dba2d740 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f51186f0 0x55c4f5140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90dba2d740 1 --2- 172.21.15.16:0/3381097981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90dba2d740 1 -- 172.21.15.16:0/3381097981 >> 172.21.15.16:0/3381097981 conn(0x55c4f50dabb0 msgr2=0x55c4f510f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90dba2d740 1 -- 172.21.15.16:0/3381097981 shutdown_connections 2023-11-13T11:03:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.356+0000 7f90dba2d740 1 -- 172.21.15.16:0/3381097981 wait complete. 2023-11-13T11:03:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.357+0000 7f90dba2d740 1 -- start start 2023-11-13T11:03:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.357+0000 7f90dba2d740 1 Processor -- start 2023-11-13T11:03:32.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.358+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.358+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f51186f0 0x55c4f5140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.358+0000 7f90dba2d740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f5118fb0 0x55c4f515b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.358+0000 7f90dba2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f51479b0 con 0x55c4f5113cb0 2023-11-13T11:03:32.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.358+0000 7f90dba2d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f513eb30 con 0x55c4f51186f0 2023-11-13T11:03:32.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.358+0000 7f90dba2d740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4f5147760 con 0x55c4f5118fb0 2023-11-13T11:03:32.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.358+0000 7f90dba2d740 10 client.0 _ll_register_callbacks cb 0x7f90c00036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:03:32.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55552/0 (socket says 172.21.15.16:55552) 2023-11-13T11:03:32.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90d259c640 1 -- 172.21.15.16:0/465566247 learned_addr learned my addr 172.21.15.16:0/465566247 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:32.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90d259c640 1 -- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f5118fb0 msgr2=0x55c4f515b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90d259c640 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f5118fb0 0x55c4f515b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90d259c640 1 -- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f51186f0 msgr2=0x55c4f5140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90d259c640 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f51186f0 0x55c4f5140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90d259c640 1 -- 172.21.15.16:0/465566247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4f5047860 con 0x55c4f5113cb0 2023-11-13T11:03:32.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90d259c640 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f90c400def0 tx=0x7f90c4001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933f6b3f04389fcb server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:03:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90b17fa640 1 -- 172.21.15.16:0/465566247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c40548e0 con 0x55c4f5113cb0 2023-11-13T11:03:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90b17fa640 1 -- 172.21.15.16:0/465566247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f90c4054a40 con 0x55c4f5113cb0 2023-11-13T11:03:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.359+0000 7f90dba2d740 1 -- 172.21.15.16:0/465566247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90c40604d0 con 0x55c4f5113cb0 2023-11-13T11:03:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90b17fa640 1 -- 172.21.15.16:0/465566247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90c4060040 con 0x55c4f5113cb0 2023-11-13T11:03:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90dba2d740 10 client.7038 fetch_fsmap learned FSMap version 40 2023-11-13T11:03:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90dba2d740 1 -- 172.21.15.16:0/465566247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55c4f5041170 con 0x55c4f5113cb0 2023-11-13T11:03:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90b17fa640 1 -- 172.21.15.16:0/465566247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c40604d0 con 0x55c4f5113cb0 2023-11-13T11:03:32.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90b17fa640 1 -- 172.21.15.16:0/465566247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f90c40890d0 con 0x55c4f5113cb0 2023-11-13T11:03:32.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90dba2d740 10 client.7038 fetch_fsmap finished waiting for FSMap version 40 2023-11-13T11:03:32.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90dba2d740 10 client.7038 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-13T11:03:32.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90dba2d740 20 client.7038 populate_metadata read hostname 'smithi016' 2023-11-13T11:03:32.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90dba2d740 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x55c4f515b850 0x55c4f515bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90dba2d740 4 client.7038 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-13T11:03:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.360+0000 7f90dba2d740 1 -- 172.21.15.16:0/465566247 --> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c4f4d806c0 con 0x55c4f515b850 2023-11-13T11:03:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.361+0000 7f90cbfff640 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x55c4f515b850 0x55c4f515bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.362+0000 7f90cbfff640 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x55c4f515b850 0x55c4f515bc30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cfb15427c76e4c1 server_cookie=e9a3fb8c73235de8 in_seq=0 out_seq=0 2023-11-13T11:03:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.362+0000 7f90b17fa640 10 client.7038 ms_handle_connect on v2:172.21.15.184:6835/60343961 2023-11-13T11:03:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.362+0000 7f90b17fa640 1 -- 172.21.15.16:0/465566247 <== mds.0 v2:172.21.15.184:6835/60343961 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c4f4d806c0 con 0x55c4f515b850 2023-11-13T11:03:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.362+0000 7f90b17fa640 10 client.7038 handle_command_reply: tid=0 2023-11-13T11:03:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.362+0000 7f90dba2d740 1 client.7038 shutdown 2023-11-13T11:03:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 -- 172.21.15.16:0/465566247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 msgr2=0x55c4f51407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f90c400def0 tx=0x7f90c4001310 comp rx=0 tx=0).stop 2023-11-13T11:03:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 -- 172.21.15.16:0/465566247 shutdown_connections 2023-11-13T11:03:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x55c4f515b850 0x55c4f515bc30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55c4f5118fb0 0x55c4f515b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c4f51186f0 0x55c4f5140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90b17fa640 0 client.7038 ms_handle_reset on v2:172.21.15.184:6835/60343961 2023-11-13T11:03:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 --2- 172.21.15.16:0/465566247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4f5113cb0 0x55c4f51407a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 -- 172.21.15.16:0/465566247 >> 172.21.15.16:0/465566247 conn(0x55c4f50dabb0 msgr2=0x55c4f515eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 -- 172.21.15.16:0/465566247 shutdown_connections 2023-11-13T11:03:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 1 -- 172.21.15.16:0/465566247 wait complete. 2023-11-13T11:03:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.363+0000 7f90dba2d740 20 client.7038 trim_cache size 0 max 16384 2023-11-13T11:03:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:03:32.395 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9", 2023-11-13T11:03:32.395 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-13T11:03:32.395 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6885, 2023-11-13T11:03:32.396 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-13T11:03:32.396 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-13T11:03:32.396 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-13T11:03:32.396 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 11.897564902999999, 2023-11-13T11:03:32.396 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 40, 2023-11-13T11:03:32.397 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 69, 2023-11-13T11:03:32.397 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 69, 2023-11-13T11:03:32.397 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 18.226148589000001 2023-11-13T11:03:32.397 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:03:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 -- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f90b406ae40 msgr2=0x7f90b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f90b406ae40 0x7f90b406d300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90bc034000 tx=0x7f90bc039040 comp rx=0 tx=0).stop 2023-11-13T11:03:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 -- 172.21.15.16:0/3254510021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 msgr2=0x7f90d414cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90d414cbd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f90c4000f60 tx=0x7f90c400d9f0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 -- 172.21.15.16:0/3254510021 shutdown_connections 2023-11-13T11:03:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f90b406ae40 0x7f90b406d300 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90d415b700 0x7f90d415bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90d40aa130 0x7f90d40aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 --2- 172.21.15.16:0/3254510021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90c00047c0 0x7f90d414cbd0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 -- 172.21.15.16:0/3254510021 >> 172.21.15.16:0/3254510021 conn(0x7f90d40b04a0 msgr2=0x7f90d40cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 -- 172.21.15.16:0/3254510021 shutdown_connections 2023-11-13T11:03:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.364+0000 7f90d359e640 1 -- 172.21.15.16:0/3254510021 wait complete. 2023-11-13T11:03:32.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-13T11:03:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.623+0000 7fe020cc3640 1 Processor -- start 2023-11-13T11:03:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.623+0000 7fe020cc3640 1 -- start start 2023-11-13T11:03:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.625+0000 7fe020cc3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c15aa90 0x7fe01c06a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.625+0000 7fe020cc3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe01c06a6d0 0x7fe01c070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.625+0000 7fe020cc3640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c06d680 con 0x7fe01c0708a0 2023-11-13T11:03:32.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.625+0000 7fe020cc3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c06ab50 con 0x7fe01c06a6d0 2023-11-13T11:03:32.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.625+0000 7fe020cc3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c070d30 con 0x7fe01c15aa90 2023-11-13T11:03:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.626+0000 7fe012ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe01c06a6d0 0x7fe01c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.626+0000 7fe012ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe01c06a6d0 0x7fe01c070360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55564/0 (socket says 172.21.15.16:55564) 2023-11-13T11:03:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.626+0000 7fe012ffd640 1 -- 172.21.15.16:0/1084090154 learned_addr learned my addr 172.21.15.16:0/1084090154 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.626+0000 7fe01b7fe640 1 --2- 172.21.15.16:0/1084090154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c15aa90 0x7fe01c06a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.626+0000 7fe012ffd640 1 -- 172.21.15.16:0/1084090154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c15aa90 msgr2=0x7fe01c06a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.626+0000 7fe012ffd640 1 --2- 172.21.15.16:0/1084090154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c15aa90 0x7fe01c06a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.626+0000 7fe012ffd640 1 -- 172.21.15.16:0/1084090154 >> v1:172.21.15.184:6789/0 conn(0x7fe01c0708a0 legacy=0x7fe01c0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.626+0000 7fe012ffd640 1 -- 172.21.15.16:0/1084090154 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c15d630 con 0x7fe01c06a6d0 2023-11-13T11:03:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.627+0000 7fe012ffd640 1 --2- 172.21.15.16:0/1084090154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c06a6d0 0x7fe01c070360 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe004009870 tx=0x7fe00402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2919bef60f945e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.628+0000 7fe01affd640 1 -- 172.21.15.16:0/1084090154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0040317c0 con 0x7fe01c06a6d0 2023-11-13T11:03:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.628+0000 7fe01affd640 1 -- 172.21.15.16:0/1084090154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe004031920 con 0x7fe01c06a6d0 2023-11-13T11:03:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.628+0000 7fe01affd640 1 -- 172.21.15.16:0/1084090154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004031cb0 con 0x7fe01c06a6d0 2023-11-13T11:03:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.628+0000 7fe020cc3640 1 -- 172.21.15.16:0/1084090154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c06a6d0 msgr2=0x7fe01c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.628+0000 7fe020cc3640 1 --2- 172.21.15.16:0/1084090154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c06a6d0 0x7fe01c070360 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe004009870 tx=0x7fe00402b860 comp rx=0 tx=0).stop 2023-11-13T11:03:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.629+0000 7fe020cc3640 1 -- 172.21.15.16:0/1084090154 shutdown_connections 2023-11-13T11:03:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.629+0000 7fe020cc3640 1 --2- 172.21.15.16:0/1084090154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c06a6d0 0x7fe01c070360 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.629+0000 7fe020cc3640 1 --2- 172.21.15.16:0/1084090154 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c15aa90 0x7fe01c06a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.629+0000 7fe020cc3640 1 -- 172.21.15.16:0/1084090154 >> 172.21.15.16:0/1084090154 conn(0x7fe01c0b04a0 msgr2=0x7fe01c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.629+0000 7fe020cc3640 1 -- 172.21.15.16:0/1084090154 shutdown_connections 2023-11-13T11:03:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.629+0000 7fe020cc3640 1 -- 172.21.15.16:0/1084090154 wait complete. 2023-11-13T11:03:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.630+0000 7fe020cc3640 1 Processor -- start 2023-11-13T11:03:32.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.631+0000 7fe020cc3640 1 -- start start 2023-11-13T11:03:32.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.631+0000 7fe020cc3640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c06a6d0 0x7fe01c0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.631+0000 7fe020cc3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0708a0 0x7fe01c0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.631+0000 7fe020cc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c15aa90 0x7fe01c14e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.631+0000 7fe020cc3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c159850 con 0x7fe01c15aa90 2023-11-13T11:03:32.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.631+0000 7fe020cc3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0b2e60 con 0x7fe01c0708a0 2023-11-13T11:03:32.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.631+0000 7fe020cc3640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0b3140 con 0x7fe01c06a6d0 2023-11-13T11:03:32.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.632+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c15aa90 0x7fe01c14e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.632+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c15aa90 0x7fe01c14e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55566/0 (socket says 172.21.15.16:55566) 2023-11-13T11:03:32.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.632+0000 7fe01bfff640 1 -- 172.21.15.16:0/1171624068 learned_addr learned my addr 172.21.15.16:0/1171624068 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:32.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.632+0000 7fe01bfff640 1 -- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c06a6d0 msgr2=0x7fe01c0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.632+0000 7fe01bfff640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c06a6d0 0x7fe01c0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.632+0000 7fe01bfff640 1 -- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0708a0 msgr2=0x7fe01c0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.632+0000 7fe01bfff640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0708a0 0x7fe01c0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.632+0000 7fe01bfff640 1 -- 172.21.15.16:0/1171624068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0040094d0 con 0x7fe01c15aa90 2023-11-13T11:03:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.633+0000 7fe01bfff640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c15aa90 0x7fe01c14e130 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe01400b970 tx=0x7fe01400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.633+0000 7fe018ff9640 1 -- 172.21.15.16:0/1171624068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01400cb40 con 0x7fe01c15aa90 2023-11-13T11:03:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.633+0000 7fe018ff9640 1 -- 172.21.15.16:0/1171624068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01400cca0 con 0x7fe01c15aa90 2023-11-13T11:03:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.633+0000 7fe018ff9640 1 -- 172.21.15.16:0/1171624068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014014680 con 0x7fe01c15aa90 2023-11-13T11:03:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.633+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c0b3420 con 0x7fe01c15aa90 2023-11-13T11:03:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.633+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c0b2b30 con 0x7fe01c15aa90 2023-11-13T11:03:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.634+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c14da70 con 0x7fe01c15aa90 2023-11-13T11:03:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.635+0000 7fe018ff9640 1 -- 172.21.15.16:0/1171624068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe01401d3b0 con 0x7fe01c15aa90 2023-11-13T11:03:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.636+0000 7fe018ff9640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe00006ae70 0x7fe00006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.636+0000 7fe018ff9640 1 -- 172.21.15.16:0/1171624068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe014089780 con 0x7fe01c15aa90 2023-11-13T11:03:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.637+0000 7fe01b7fe640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe00006ae70 0x7fe00006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.638+0000 7fe01b7fe640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe00006ae70 0x7fe00006d330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0b3820 tx=0x7fe00c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.641+0000 7fe018ff9640 1 -- 172.21.15.16:0/1171624068 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe01c14da70 con 0x7fe01c15aa90 2023-11-13T11:03:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.874+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe01c14de50 con 0x7fe01c15aa90 2023-11-13T11:03:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.875+0000 7fe018ff9640 1 -- 172.21.15.16:0/1171624068 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe01c14de50 con 0x7fe01c15aa90 2023-11-13T11:03:32.879 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-13T11:03:32.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe00006ae70 msgr2=0x7fe00006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe00006ae70 0x7fe00006d330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0b3820 tx=0x7fe00c008040 comp rx=0 tx=0).stop 2023-11-13T11:03:32.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c15aa90 msgr2=0x7fe01c14e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:32.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c15aa90 0x7fe01c14e130 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe01400b970 tx=0x7fe01400be30 comp rx=0 tx=0).stop 2023-11-13T11:03:32.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 shutdown_connections 2023-11-13T11:03:32.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe00006ae70 0x7fe00006d330 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c15aa90 0x7fe01c14e130 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01c0708a0 0x7fe01c0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 --2- 172.21.15.16:0/1171624068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe01c06a6d0 0x7fe01c0b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:32.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 >> 172.21.15.16:0/1171624068 conn(0x7fe01c0b04a0 msgr2=0x7fe01c15c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:32.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 shutdown_connections 2023-11-13T11:03:32.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:32.878+0000 7fe020cc3640 1 -- 172.21.15.16:0/1171624068 wait complete. 2023-11-13T11:03:32.900 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-13T11:03:32.901 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-13T11:03:32.901 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-13T11:03:32.901 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-13T11:03:32.902 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-13T11:03:32.902 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-13T11:03:32.902 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-13T11:03:32.903 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-13T11:03:32.903 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-13T11:03:32.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:32.904 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-13T11:03:32.957 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-13T11:03:32.958 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-13T11:03:32.958 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-13T11:03:32.958 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:03:32.958 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-13T11:03:32.959 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:03:32.959 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T11:03:32.959 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:03:32.959 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-13T11:03:32.959 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T11:03:32.960 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:03:32.960 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-13T11:03:32.961 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-13T11:03:32.961 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:03:32.961 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-13T11:03:32.961 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-13T11:03:32.961 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41829sec preferred_lft 41829sec 2023-11-13T11:03:32.962 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-13T11:03:32.962 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:03:32.962 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-13T11:03:32.962 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:03:32.963 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-13T11:03:32.963 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-13T11:03:32.963 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1a:84:8e:80:25:76 brd ff:ff:ff:ff:ff:ff 2023-11-13T11:03:32.963 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-13T11:03:32.964 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:03:32.964 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5426:dff:feb9:4760/64 scope link 2023-11-13T11:03:32.964 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:03:32.964 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-13T11:03:32.964 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1a:84:8e:80:25:76 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:03:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::1884:8eff:fe80:2576/64 scope link 2023-11-13T11:03:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:03:32.965 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-13T11:03:32.965 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether ae:42:49:3f:88:fc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:03:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ac42:49ff:fe3f:88fc/64 scope link 2023-11-13T11:03:32.966 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-13T11:03:32.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:32.967 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-13T11:03:33.055 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-13T11:03:33.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:33.059 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-13T11:03:33.118 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-13T11:03:33.118 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-13T11:03:33.118 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-13T11:03:33.118 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-13T11:03:33.119 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-13T11:03:33.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:33.120 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-13T11:03:33.120 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-13T11:03:33.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-13T11:03:33.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-13T11:03:33.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-13T11:03:33.121 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-13T11:03:33.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:33.321 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-13T11:03:33.377 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-13T11:03:33.378 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-13T11:03:33.379 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-13T11:03:33.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:33.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-13T11:03:33.485 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-13T11:03:33.485 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T11:03:33.486 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-13T11:03:33.548 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-13T11:03:33.549 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-13T11:03:33.549 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-13T11:03:33.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T11:03:33.707 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T11:03:33.765 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T11:03:33.766 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T11:03:33.766 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T11:03:33.766 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T11:03:33.767 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T11:03:33.767 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T11:03:33.767 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T11:03:33.767 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T11:03:33.767 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T11:03:33.768 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T11:03:33.768 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T11:03:33.768 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T11:03:33.768 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T11:03:33.768 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T11:03:33.768 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T11:03:33.769 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T11:03:33.769 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T11:03:33.769 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T11:03:33.769 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T11:03:33.769 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T11:03:33.770 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T11:03:33.770 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T11:03:33.770 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T11:03:33.770 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T11:03:33.771 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T11:03:33.771 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T11:03:33.771 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T11:03:33.771 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T11:03:33.771 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T11:03:33.772 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:03:33.772 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:03:33.772 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:03:33.772 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:03:33.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:33.774 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T11:03:33.830 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-13T11:03:33.830 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-13T11:03:33.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-13T11:03:33.922 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-13T11:03:33.965 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-13T11:03:33.966 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-13T11:03:33.966 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-13T11:03:33.966 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-13T11:03:33.966 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-13T11:03:33.967 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-13T11:03:33.967 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-13T11:03:33.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-13T11:03:33.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-13T11:03:33.967 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-13T11:03:33.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-13T11:03:33.968 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-13T11:03:33.968 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-13T11:03:33.968 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-13T11:03:33.968 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-13T11:03:33.969 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-13T11:03:33.969 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-13T11:03:33.969 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-13T11:03:33.969 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-13T11:03:33.969 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-13T11:03:33.970 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-13T11:03:33.970 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-13T11:03:33.970 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-13T11:03:33.970 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-13T11:03:33.970 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-13T11:03:33.971 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-13T11:03:33.971 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-13T11:03:33.971 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-13T11:03:33.971 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-13T11:03:33.971 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:03:33.972 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:03:33.972 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:03:33.972 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:03:33.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:33.973 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-13T11:03:34.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-13T11:03:34.029+0000 7efe58179480 -1 init, newargv = 0x564ccd452090 newargc=15 2023-11-13T11:03:34.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[117148]: starting ceph client 2023-11-13T11:03:34.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[117148]: starting fuse 2023-11-13T11:03:34.066 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-13T11:03:34.066 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-13T11:03:34.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-13T11:03:34.067 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-13T11:03:34.068 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-13T11:03:34.068 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T11:03:34.069 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-13T11:03:34.069 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:03:34.070 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-13T11:03:34.070 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-13T11:03:34.070 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-13T11:03:34.071 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-13T11:03:34.071 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:03:34.072 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-13T11:03:34.072 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-13T11:03:34.072 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-13T11:03:34.073 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-13T11:03:34.073 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:03:34.073 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-13T11:03:34.074 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-13T11:03:34.074 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-13T11:03:34.075 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-13T11:03:34.075 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-13T11:03:34.075 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-13T11:03:34.076 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-13T11:03:34.076 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-13T11:03:34.076 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:03:34.077 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-13T11:03:34.077 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T11:03:34.286 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.117148.asok 2023-11-13T11:03:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T11:03:34.303 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.117148.asok 2023-11-13T11:03:34.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:34.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.117148.asok status 2023-11-13T11:03:34.557 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:03:34.558 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-13T11:03:34.558 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "35f0debb3cc28955b7132ef4c509cd138a09d56b", 2023-11-13T11:03:34.558 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7288-g35f0debb (35f0debb3cc28955b7132ef4c509cd138a09d56b) reef (dev)", 2023-11-13T11:03:34.558 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-13T11:03:34.559 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-13T11:03:34.559 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-13T11:03:34.559 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "117148", 2023-11-13T11:03:34.559 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-13T11:03:34.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:34.560 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-13T11:03:34.560 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-13T11:03:34.560 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7053, 2023-11-13T11:03:34.560 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-13T11:03:34.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-13T11:03:34.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-13T11:03:34.561 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 7053 2023-11-13T11:03:34.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:34.561 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T11:03:34.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T11:03:34.562 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T11:03:34.562 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2786828148 2023-11-13T11:03:34.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-13T11:03:34.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:34.562 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-13T11:03:34.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-13T11:03:34.563 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-13T11:03:34.563 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2786828148 2023-11-13T11:03:34.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-13T11:03:34.563 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.7053 192.168.0.1:0/2786828148", 2023-11-13T11:03:34.564 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2786828148", 2023-11-13T11:03:34.564 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-13T11:03:34.564 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 40, 2023-11-13T11:03:34.564 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 69, 2023-11-13T11:03:34.564 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-13T11:03:34.565 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-13T11:03:34.565 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-13T11:03:34.565 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:03:34.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:34.574 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-13T11:03:34.574 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-13T11:03:34.629 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-13T11:03:34.630 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-13T11:03:34.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:34.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-13T11:03:34.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_split_size 20 2023-11-13T11:03:34.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.966+0000 7f84eaf25640 1 Processor -- start 2023-11-13T11:03:34.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.966+0000 7f84eaf25640 1 -- start start 2023-11-13T11:03:34.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.968+0000 7f84eaf25640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f84e4070e10 0x7f84e406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:34.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.968+0000 7f84eaf25640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84e40711f0 0x7f84e406bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:34.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.968+0000 7f84eaf25640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e415d8e0 con 0x7f84e4160b80 2023-11-13T11:03:34.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.968+0000 7f84eaf25640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e406b5d0 con 0x7f84e40711f0 2023-11-13T11:03:34.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.968+0000 7f84eaf25640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e41616f0 con 0x7f84e4070e10 2023-11-13T11:03:34.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.968+0000 7f84e3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84e40711f0 0x7f84e406bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:34.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.968+0000 7f84e3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84e40711f0 0x7f84e406bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60262/0 (socket says 172.21.15.16:60262) 2023-11-13T11:03:34.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.968+0000 7f84e3fff640 1 -- 172.21.15.16:0/156664916 learned_addr learned my addr 172.21.15.16:0/156664916 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:34.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.969+0000 7f84e8cc1640 1 --2- 172.21.15.16:0/156664916 >> v2:172.21.15.184:3300/0 conn(0x7f84e4070e10 0x7f84e406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:34.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.969+0000 7f84e3fff640 1 -- 172.21.15.16:0/156664916 >> v2:172.21.15.184:3300/0 conn(0x7f84e4070e10 msgr2=0x7f84e406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:34.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.969+0000 7f84e3fff640 1 --2- 172.21.15.16:0/156664916 >> v2:172.21.15.184:3300/0 conn(0x7f84e4070e10 0x7f84e406b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:34.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.969+0000 7f84e3fff640 1 -- 172.21.15.16:0/156664916 >> v1:172.21.15.184:6789/0 conn(0x7f84e4160b80 legacy=0x7f84e4157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:34.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.969+0000 7f84e3fff640 1 -- 172.21.15.16:0/156664916 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e41619d0 con 0x7f84e40711f0 2023-11-13T11:03:34.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.970+0000 7f84e3fff640 1 --2- 172.21.15.16:0/156664916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 0x7f84e406bf30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f84e4071db0 tx=0x7f84d002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c770149700f295b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:34.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.971+0000 7f84e37fe640 1 -- 172.21.15.16:0/156664916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d00317e0 con 0x7f84e40711f0 2023-11-13T11:03:34.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.971+0000 7f84e37fe640 1 -- 172.21.15.16:0/156664916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d0031940 con 0x7f84e40711f0 2023-11-13T11:03:34.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.971+0000 7f84e37fe640 1 -- 172.21.15.16:0/156664916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0031cd0 con 0x7f84e40711f0 2023-11-13T11:03:34.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.972+0000 7f84eaf25640 1 -- 172.21.15.16:0/156664916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 msgr2=0x7f84e406bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:34.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.972+0000 7f84eaf25640 1 --2- 172.21.15.16:0/156664916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 0x7f84e406bf30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f84e4071db0 tx=0x7f84d002b880 comp rx=0 tx=0).stop 2023-11-13T11:03:34.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.972+0000 7f84eaf25640 1 -- 172.21.15.16:0/156664916 shutdown_connections 2023-11-13T11:03:34.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.972+0000 7f84eaf25640 1 --2- 172.21.15.16:0/156664916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 0x7f84e406bf30 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:34.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.972+0000 7f84eaf25640 1 --2- 172.21.15.16:0/156664916 >> v2:172.21.15.184:3300/0 conn(0x7f84e4070e10 0x7f84e406b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:34.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.972+0000 7f84eaf25640 1 -- 172.21.15.16:0/156664916 >> 172.21.15.16:0/156664916 conn(0x7f84e40b19a0 msgr2=0x7f84e40b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:34.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.972+0000 7f84eaf25640 1 -- 172.21.15.16:0/156664916 shutdown_connections 2023-11-13T11:03:34.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.972+0000 7f84eaf25640 1 -- 172.21.15.16:0/156664916 wait complete. 2023-11-13T11:03:34.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.974+0000 7f84eaf25640 1 Processor -- start 2023-11-13T11:03:34.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.974+0000 7f84eaf25640 1 -- start start 2023-11-13T11:03:34.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.975+0000 7f84eaf25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84e4070e10 0x7f84e4149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:34.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.975+0000 7f84eaf25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 0x7f84e40ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:34.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.975+0000 7f84eaf25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e4160b80 0x7f84e415f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:34.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.975+0000 7f84eaf25640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41582f0 con 0x7f84e40711f0 2023-11-13T11:03:34.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.975+0000 7f84eaf25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4154cc0 con 0x7f84e4160b80 2023-11-13T11:03:34.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.975+0000 7f84eaf25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41549c0 con 0x7f84e4070e10 2023-11-13T11:03:34.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.975+0000 7f84e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 0x7f84e40ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:34.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.975+0000 7f84e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 0x7f84e40ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60278/0 (socket says 172.21.15.16:60278) 2023-11-13T11:03:34.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.975+0000 7f84e3fff640 1 -- 172.21.15.16:0/3382786024 learned_addr learned my addr 172.21.15.16:0/3382786024 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:34.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.976+0000 7f84e8cc1640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84e4070e10 0x7f84e4149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:34.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.976+0000 7f84e94c2640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e4160b80 0x7f84e415f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:34.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.976+0000 7f84e3fff640 1 -- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84e4070e10 msgr2=0x7f84e4149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:34.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.976+0000 7f84e3fff640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84e4070e10 0x7f84e4149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:34.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.976+0000 7f84e3fff640 1 -- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e4160b80 msgr2=0x7f84e415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:34.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.976+0000 7f84e3fff640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e4160b80 0x7f84e415f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:34.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.976+0000 7f84e3fff640 1 -- 172.21.15.16:0/3382786024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d00094d0 con 0x7f84e40711f0 2023-11-13T11:03:34.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.976+0000 7f84e3fff640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 0x7f84e40ad2f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009960 tx=0x7f84d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:34.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.977+0000 7f84e17fa640 1 -- 172.21.15.16:0/3382786024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0032880 con 0x7f84e40711f0 2023-11-13T11:03:34.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.977+0000 7f84e17fa640 1 -- 172.21.15.16:0/3382786024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d00329e0 con 0x7f84e40711f0 2023-11-13T11:03:34.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.977+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e40adbb0 con 0x7f84e40711f0 2023-11-13T11:03:34.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.977+0000 7f84e17fa640 1 -- 172.21.15.16:0/3382786024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0032d90 con 0x7f84e40711f0 2023-11-13T11:03:34.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.977+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e40ae170 con 0x7f84e40711f0 2023-11-13T11:03:34.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.978+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e4072670 con 0x7f84e40711f0 2023-11-13T11:03:34.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.979+0000 7f84e17fa640 1 -- 172.21.15.16:0/3382786024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84d00453f0 con 0x7f84e40711f0 2023-11-13T11:03:34.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.980+0000 7f84e17fa640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f84c806ae40 0x7f84c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:34.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.980+0000 7f84e17fa640 1 -- 172.21.15.16:0/3382786024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f84d00ab890 con 0x7f84e40711f0 2023-11-13T11:03:34.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.981+0000 7f84e8cc1640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f84c806ae40 0x7f84c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:34.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.982+0000 7f84e8cc1640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f84c806ae40 0x7f84c806d300 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f84d8006fd0 tx=0x7f84d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:34.984+0000 7f84e17fa640 1 -- 172.21.15.16:0/3382786024 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84d00767f0 con 0x7f84e40711f0 2023-11-13T11:03:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.178+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7f84e404e5b0 con 0x7f84e40711f0 2023-11-13T11:03:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.185+0000 7f84e17fa640 1 -- 172.21.15.16:0/3382786024 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v15)=0 v15) v1 ==== 117+0+0 (secure 0 0 0) 0x7f84d0079e10 con 0x7f84e40711f0 2023-11-13T11:03:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f84c806ae40 msgr2=0x7f84c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f84c806ae40 0x7f84c806d300 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f84d8006fd0 tx=0x7f84d8008040 comp rx=0 tx=0).stop 2023-11-13T11:03:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 msgr2=0x7f84e40ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 0x7f84e40ad2f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009960 tx=0x7f84d00046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 shutdown_connections 2023-11-13T11:03:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f84c806ae40 0x7f84c806d300 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84e4160b80 0x7f84e415f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40711f0 0x7f84e40ad2f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 --2- 172.21.15.16:0/3382786024 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84e4070e10 0x7f84e4149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 >> 172.21.15.16:0/3382786024 conn(0x7f84e40b19a0 msgr2=0x7f84e40b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.189+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 shutdown_connections 2023-11-13T11:03:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.190+0000 7f84eaf25640 1 -- 172.21.15.16:0/3382786024 wait complete. 2023-11-13T11:03:35.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_merge_size 5 2023-11-13T11:03:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.430+0000 7f80f2c26640 1 Processor -- start 2023-11-13T11:03:35.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.430+0000 7f80f2c26640 1 -- start start 2023-11-13T11:03:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.431+0000 7f80f2c26640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f80ec15cf90 0x7f80ec0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80f2c26640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80ec06c3b0 0x7f80ec0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80f2c26640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec0b39c0 0x7f80ec0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80f2c26640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ec0b3e30 con 0x7f80ec06c3b0 2023-11-13T11:03:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80f2c26640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ec0b2e00 con 0x7f80ec15cf90 2023-11-13T11:03:35.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80f2c26640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec0b21f0 con 0x7f80ec0b39c0 2023-11-13T11:03:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80ebfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80ec06c3b0 0x7f80ec0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80ebfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80ec06c3b0 0x7f80ec0b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60286/0 (socket says 172.21.15.16:60286) 2023-11-13T11:03:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80ebfff640 1 -- 172.21.15.16:0/1792216506 learned_addr learned my addr 172.21.15.16:0/1792216506 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80f09c2640 1 --2- 172.21.15.16:0/1792216506 >> v2:172.21.15.184:3300/0 conn(0x7f80ec15cf90 0x7f80ec0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.432+0000 7f80f11c3640 1 --2- 172.21.15.16:0/1792216506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec0b39c0 0x7f80ec0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.433+0000 7f80ebfff640 1 -- 172.21.15.16:0/1792216506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec0b39c0 msgr2=0x7f80ec0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.433+0000 7f80ebfff640 1 --2- 172.21.15.16:0/1792216506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec0b39c0 0x7f80ec0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.433+0000 7f80ebfff640 1 -- 172.21.15.16:0/1792216506 >> v2:172.21.15.184:3300/0 conn(0x7f80ec15cf90 msgr2=0x7f80ec0b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.433+0000 7f80ebfff640 1 --2- 172.21.15.16:0/1792216506 >> v2:172.21.15.184:3300/0 conn(0x7f80ec15cf90 0x7f80ec0b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.433+0000 7f80ebfff640 1 -- 172.21.15.16:0/1792216506 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec0b24d0 con 0x7f80ec06c3b0 2023-11-13T11:03:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.434+0000 7f80ebfff640 1 --2- 172.21.15.16:0/1792216506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec06c3b0 0x7f80ec0b3480 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f80ec071f80 tx=0x7f80e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d77bbf149de0dc server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.435+0000 7f80eb7fe640 1 -- 172.21.15.16:0/1792216506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0031870 con 0x7f80ec06c3b0 2023-11-13T11:03:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.435+0000 7f80eb7fe640 1 -- 172.21.15.16:0/1792216506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e00319d0 con 0x7f80ec06c3b0 2023-11-13T11:03:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.435+0000 7f80eb7fe640 1 -- 172.21.15.16:0/1792216506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0031d60 con 0x7f80ec06c3b0 2023-11-13T11:03:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.436+0000 7f80f2c26640 1 -- 172.21.15.16:0/1792216506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec06c3b0 msgr2=0x7f80ec0b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.436+0000 7f80f2c26640 1 --2- 172.21.15.16:0/1792216506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec06c3b0 0x7f80ec0b3480 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f80ec071f80 tx=0x7f80e002b910 comp rx=0 tx=0).stop 2023-11-13T11:03:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.436+0000 7f80f2c26640 1 -- 172.21.15.16:0/1792216506 shutdown_connections 2023-11-13T11:03:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.436+0000 7f80f2c26640 1 --2- 172.21.15.16:0/1792216506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec0b39c0 0x7f80ec0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.436+0000 7f80f2c26640 1 --2- 172.21.15.16:0/1792216506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec06c3b0 0x7f80ec0b3480 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.436+0000 7f80f2c26640 1 --2- 172.21.15.16:0/1792216506 >> v2:172.21.15.184:3300/0 conn(0x7f80ec15cf90 0x7f80ec0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.436+0000 7f80f2c26640 1 -- 172.21.15.16:0/1792216506 >> 172.21.15.16:0/1792216506 conn(0x7f80ec06a190 msgr2=0x7f80ec06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.436+0000 7f80f2c26640 1 -- 172.21.15.16:0/1792216506 shutdown_connections 2023-11-13T11:03:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.436+0000 7f80f2c26640 1 -- 172.21.15.16:0/1792216506 wait complete. 2023-11-13T11:03:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.438+0000 7f80f2c26640 1 Processor -- start 2023-11-13T11:03:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.438+0000 7f80f2c26640 1 -- start start 2023-11-13T11:03:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80f2c26640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec06c3b0 0x7f80ec141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80f2c26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec0b39c0 0x7f80ec0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80f2c26640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80ec15cf90 0x7f80ec0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80f2c26640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec142030 con 0x7f80ec0b39c0 2023-11-13T11:03:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80f2c26640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec15e1c0 con 0x7f80ec15cf90 2023-11-13T11:03:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80f2c26640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1443c0 con 0x7f80ec06c3b0 2023-11-13T11:03:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec0b39c0 0x7f80ec0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec0b39c0 0x7f80ec0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60290/0 (socket says 172.21.15.16:60290) 2023-11-13T11:03:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80ebfff640 1 -- 172.21.15.16:0/317243806 learned_addr learned my addr 172.21.15.16:0/317243806 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80f11c3640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80ec15cf90 0x7f80ec0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80f09c2640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec06c3b0 0x7f80ec141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80ebfff640 1 -- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec06c3b0 msgr2=0x7f80ec141af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80ebfff640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec06c3b0 0x7f80ec141af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80ebfff640 1 -- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80ec15cf90 msgr2=0x7f80ec0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80ebfff640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80ec15cf90 0x7f80ec0cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.439+0000 7f80ebfff640 1 -- 172.21.15.16:0/317243806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e00094d0 con 0x7f80ec0b39c0 2023-11-13T11:03:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.440+0000 7f80f11c3640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80ec15cf90 0x7f80ec0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.440+0000 7f80ebfff640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec0b39c0 0x7f80ec0cd900 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000c00 tx=0x7f80e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.440+0000 7f80e97fa640 1 -- 172.21.15.16:0/317243806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e003aca0 con 0x7f80ec0b39c0 2023-11-13T11:03:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.440+0000 7f80e97fa640 1 -- 172.21.15.16:0/317243806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e003ae00 con 0x7f80ec0b39c0 2023-11-13T11:03:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.440+0000 7f80e97fa640 1 -- 172.21.15.16:0/317243806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0044630 con 0x7f80ec0b39c0 2023-11-13T11:03:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.440+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec1446a0 con 0x7f80ec0b39c0 2023-11-13T11:03:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.440+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec144c60 con 0x7f80ec0b39c0 2023-11-13T11:03:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.442+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec15b040 con 0x7f80ec0b39c0 2023-11-13T11:03:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.442+0000 7f80e97fa640 1 -- 172.21.15.16:0/317243806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80e004d450 con 0x7f80ec0b39c0 2023-11-13T11:03:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.443+0000 7f80e97fa640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d806ae40 0x7f80d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.443+0000 7f80e97fa640 1 -- 172.21.15.16:0/317243806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f80e002d040 con 0x7f80ec0b39c0 2023-11-13T11:03:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.444+0000 7f80f09c2640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d806ae40 0x7f80d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.445+0000 7f80f09c2640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d806ae40 0x7f80d806d300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009e10 tx=0x7f80dc00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:35.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.448+0000 7f80e97fa640 1 -- 172.21.15.16:0/317243806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80e0076cf0 con 0x7f80ec0b39c0 2023-11-13T11:03:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.644+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f80ec072520 con 0x7f80ec0b39c0 2023-11-13T11:03:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.651+0000 7f80e97fa640 1 -- 172.21.15.16:0/317243806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v16)=0 v16) v1 ==== 116+0+0 (secure 0 0 0) 0x7f80e00350b0 con 0x7f80ec0b39c0 2023-11-13T11:03:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d806ae40 msgr2=0x7f80d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d806ae40 0x7f80d806d300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009e10 tx=0x7f80dc00b040 comp rx=0 tx=0).stop 2023-11-13T11:03:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec0b39c0 msgr2=0x7f80ec0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec0b39c0 0x7f80ec0cd900 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000c00 tx=0x7f80e00046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 shutdown_connections 2023-11-13T11:03:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f80d806ae40 0x7f80d806d300 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80ec15cf90 0x7f80ec0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec0b39c0 0x7f80ec0cd900 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 --2- 172.21.15.16:0/317243806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80ec06c3b0 0x7f80ec141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 >> 172.21.15.16:0/317243806 conn(0x7f80ec06a190 msgr2=0x7f80ec15b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 shutdown_connections 2023-11-13T11:03:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.654+0000 7f80f2c26640 1 -- 172.21.15.16:0/317243806 wait complete. 2023-11-13T11:03:35.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_split_bits 1 2023-11-13T11:03:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.902+0000 7f9571421640 1 Processor -- start 2023-11-13T11:03:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.903+0000 7f9571421640 1 -- start start 2023-11-13T11:03:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.904+0000 7f9571421640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f956c15cf90 0x7f956c0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.904+0000 7f9571421640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f956c0b3b90 0x7f956c070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.904+0000 7f9571421640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956c0b2e00 con 0x7f956c06c3b0 2023-11-13T11:03:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.905+0000 7f9571421640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956c0b21f0 con 0x7f956c15cf90 2023-11-13T11:03:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.905+0000 7f9571421640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956c0b24d0 con 0x7f956c0b3b90 2023-11-13T11:03:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.905+0000 7f956affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f956c15cf90 0x7f956c0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.905+0000 7f956affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f956c15cf90 0x7f956c0b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60304/0 (socket says 172.21.15.16:60304) 2023-11-13T11:03:35.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.905+0000 7f956affd640 1 -- 172.21.15.16:0/1694964311 learned_addr learned my addr 172.21.15.16:0/1694964311 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:35.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.905+0000 7f956b7fe640 1 --2- 172.21.15.16:0/1694964311 >> v2:172.21.15.184:3300/0 conn(0x7f956c0b3b90 0x7f956c070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.906+0000 7f956affd640 1 -- 172.21.15.16:0/1694964311 >> v2:172.21.15.184:3300/0 conn(0x7f956c0b3b90 msgr2=0x7f956c070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:35.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.906+0000 7f956affd640 1 --2- 172.21.15.16:0/1694964311 >> v2:172.21.15.184:3300/0 conn(0x7f956c0b3b90 0x7f956c070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.906+0000 7f956affd640 1 -- 172.21.15.16:0/1694964311 >> v1:172.21.15.16:6789/0 conn(0x7f956c06c3b0 legacy=0x7f956c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.906+0000 7f956affd640 1 -- 172.21.15.16:0/1694964311 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c0b27b0 con 0x7f956c15cf90 2023-11-13T11:03:35.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.907+0000 7f956affd640 1 --2- 172.21.15.16:0/1694964311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c15cf90 0x7f956c0b1c10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f956c0adae0 tx=0x7f955c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c272a42ce0cd5d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:35.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.907+0000 7f9569ffb640 1 -- 172.21.15.16:0/1694964311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c030860 con 0x7f956c15cf90 2023-11-13T11:03:35.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.907+0000 7f9569ffb640 1 -- 172.21.15.16:0/1694964311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c0309c0 con 0x7f956c15cf90 2023-11-13T11:03:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.907+0000 7f9569ffb640 1 -- 172.21.15.16:0/1694964311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c030d50 con 0x7f956c15cf90 2023-11-13T11:03:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.908+0000 7f9571421640 1 -- 172.21.15.16:0/1694964311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c15cf90 msgr2=0x7f956c0b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.908+0000 7f9571421640 1 --2- 172.21.15.16:0/1694964311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c15cf90 0x7f956c0b1c10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f956c0adae0 tx=0x7f955c02b9a0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.908+0000 7f9571421640 1 -- 172.21.15.16:0/1694964311 shutdown_connections 2023-11-13T11:03:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.908+0000 7f9571421640 1 --2- 172.21.15.16:0/1694964311 >> v2:172.21.15.184:3300/0 conn(0x7f956c0b3b90 0x7f956c070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.908+0000 7f9571421640 1 --2- 172.21.15.16:0/1694964311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c15cf90 0x7f956c0b1c10 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.908+0000 7f9571421640 1 -- 172.21.15.16:0/1694964311 >> 172.21.15.16:0/1694964311 conn(0x7f956c06a190 msgr2=0x7f956c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.908+0000 7f9571421640 1 -- 172.21.15.16:0/1694964311 shutdown_connections 2023-11-13T11:03:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.908+0000 7f9571421640 1 -- 172.21.15.16:0/1694964311 wait complete. 2023-11-13T11:03:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.910+0000 7f9571421640 1 Processor -- start 2023-11-13T11:03:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.910+0000 7f9571421640 1 -- start start 2023-11-13T11:03:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f9571421640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c06c3b0 0x7f956c142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f9571421640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0b3b90 0x7f956c1422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f9571421640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c15cf90 0x7f956c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f9571421640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c15d8c0 con 0x7f956c06c3b0 2023-11-13T11:03:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f9571421640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0cc840 con 0x7f956c15cf90 2023-11-13T11:03:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f9571421640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956c0ccb20 con 0x7f956c0b3b90 2023-11-13T11:03:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f956affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c06c3b0 0x7f956c142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f956affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c06c3b0 0x7f956c142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60316/0 (socket says 172.21.15.16:60316) 2023-11-13T11:03:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f956affd640 1 -- 172.21.15.16:0/2218351142 learned_addr learned my addr 172.21.15.16:0/2218351142 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f956affd640 1 -- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0b3b90 msgr2=0x7f956c1422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f956affd640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0b3b90 0x7f956c1422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f956affd640 1 -- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c15cf90 msgr2=0x7f956c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f956affd640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c15cf90 0x7f956c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.911+0000 7f956affd640 1 -- 172.21.15.16:0/2218351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0094d0 con 0x7f956c06c3b0 2023-11-13T11:03:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.912+0000 7f956affd640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c06c3b0 0x7f956c142c00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f955c02b9e0 tx=0x7f955c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.912+0000 7f95537fe640 1 -- 172.21.15.16:0/2218351142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c039b60 con 0x7f956c06c3b0 2023-11-13T11:03:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.912+0000 7f95537fe640 1 -- 172.21.15.16:0/2218351142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c039cc0 con 0x7f956c06c3b0 2023-11-13T11:03:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.912+0000 7f95537fe640 1 -- 172.21.15.16:0/2218351142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c043650 con 0x7f956c06c3b0 2023-11-13T11:03:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.912+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956c0cce00 con 0x7f956c06c3b0 2023-11-13T11:03:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.912+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956c1427f0 con 0x7f956c06c3b0 2023-11-13T11:03:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.913+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956c0cd320 con 0x7f956c06c3b0 2023-11-13T11:03:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.914+0000 7f95537fe640 1 -- 172.21.15.16:0/2218351142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f955c02f3b0 con 0x7f956c06c3b0 2023-11-13T11:03:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.915+0000 7f95537fe640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f955806ae70 0x7f955806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.915+0000 7f95537fe640 1 -- 172.21.15.16:0/2218351142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f955c0abb30 con 0x7f956c06c3b0 2023-11-13T11:03:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.916+0000 7f956a7fc640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f955806ae70 0x7f955806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.917+0000 7f956a7fc640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f955806ae70 0x7f955806d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9560000f60 tx=0x7f95600079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:35.920+0000 7f95537fe640 1 -- 172.21.15.16:0/2218351142 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f956c0cd320 con 0x7f956c06c3b0 2023-11-13T11:03:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.115+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7f956c0cd320 con 0x7f956c06c3b0 2023-11-13T11:03:36.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.122+0000 7f95537fe640 1 -- 172.21.15.16:0/2218351142 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v17)=0 v17) v1 ==== 116+0+0 (secure 0 0 0) 0x7f956c0cd320 con 0x7f956c06c3b0 2023-11-13T11:03:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f955806ae70 msgr2=0x7f955806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f955806ae70 0x7f955806d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9560000f60 tx=0x7f95600079b0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c06c3b0 msgr2=0x7f956c142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c06c3b0 0x7f956c142c00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f955c02b9e0 tx=0x7f955c004730 comp rx=0 tx=0).stop 2023-11-13T11:03:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 shutdown_connections 2023-11-13T11:03:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f955806ae70 0x7f955806d330 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f956c15cf90 0x7f956c15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f956c0b3b90 0x7f956c1422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 --2- 172.21.15.16:0/2218351142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956c06c3b0 0x7f956c142c00 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.125+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 >> 172.21.15.16:0/2218351142 conn(0x7f956c06a190 msgr2=0x7f956c070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.126+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 shutdown_connections 2023-11-13T11:03:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.126+0000 7f9571421640 1 -- 172.21.15.16:0/2218351142 wait complete. 2023-11-13T11:03:36.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:36.149 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2023-11-13T11:03:36.223 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2023-11-13T11:03:36.223 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T11:03:36.223 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2023-11-13T11:03:36.224 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:03:36.224 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/testdir").st_ino) 2023-11-13T11:03:36.225 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T11:03:36.377 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627776 2023-11-13T11:03:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T11:03:36.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.659+0000 7f3dc759e640 1 Processor -- start 2023-11-13T11:03:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.660+0000 7f3dc759e640 1 -- start start 2023-11-13T11:03:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.661+0000 7f3dc759e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3dc00ae030 0x7f3dc006c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.661+0000 7f3dc759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc00b3a80 con 0x7f3dc0070650 2023-11-13T11:03:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.661+0000 7f3dc759e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc0161880 con 0x7f3dc006fb60 2023-11-13T11:03:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.661+0000 7f3dc759e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc0161b60 con 0x7f3dc00ae030 2023-11-13T11:03:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.662+0000 7f3dc5b3b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3dc0070650 0x7f3dc006c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37970/0 (socket says 172.21.15.16:37970) 2023-11-13T11:03:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.662+0000 7f3dc5b3b640 1 -- 172.21.15.16:0/1750487054 learned_addr learned my addr 172.21.15.16:0/1750487054 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.662+0000 7f3dc4b39640 1 --2- 172.21.15.16:0/1750487054 >> v2:172.21.15.184:3300/0 conn(0x7f3dc00ae030 0x7f3dc006c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.662+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190141195 0 0) 0x7f3dc00b3a80 con 0x7f3dc0070650 2023-11-13T11:03:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.662+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dac003520 con 0x7f3dc0070650 2023-11-13T11:03:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.662+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3950817874 0 0) 0x7f3dac003520 con 0x7f3dc0070650 2023-11-13T11:03:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.662+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 >> v2:172.21.15.184:3300/0 conn(0x7f3dc00ae030 msgr2=0x7f3dc006c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.662+0000 7f3db7fff640 1 --2- 172.21.15.16:0/1750487054 >> v2:172.21.15.184:3300/0 conn(0x7f3dc00ae030 0x7f3dc006c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.662+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 >> v1:172.21.15.184:6789/0 conn(0x7f3dc006fb60 legacy=0x7f3dc006ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc0161e40 con 0x7f3dc0070650 2023-11-13T11:03:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f3db80019f0 con 0x7f3dc0070650 2023-11-13T11:03:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 >> v1:172.21.15.16:6789/0 conn(0x7f3dc0070650 legacy=0x7f3dc006c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3db7fff640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dac0039c0 0x7f3dac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3db7fff640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3db7fff640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dac0051d0 0x7f3dac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0161b60 con 0x7f3dac0047c0 2023-11-13T11:03:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac00c120 con 0x7f3dac0051d0 2023-11-13T11:03:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac00c2e0 con 0x7f3dac0039c0 2023-11-13T11:03:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc4b39640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc5b3b640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dac0051d0 0x7f3dac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc533a640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dac0039c0 0x7f3dac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc4b39640 1 -- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dac0039c0 msgr2=0x7f3dac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc4b39640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dac0039c0 0x7f3dac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc4b39640 1 -- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dac0051d0 msgr2=0x7f3dac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc4b39640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dac0051d0 0x7f3dac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc4b39640 1 -- 172.21.15.16:0/1750487054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac003520 con 0x7f3dac0047c0 2023-11-13T11:03:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc533a640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dac0039c0 0x7f3dac003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc5b3b640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dac0051d0 0x7f3dac00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.663+0000 7f3dc5b3b640 1 -- 172.21.15.16:0/1750487054 reap_dead start 2023-11-13T11:03:36.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.665+0000 7f3dc4b39640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dac004c40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ae4c0 tx=0x7f3dac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee22be929dca31c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:36.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.665+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db002c660 con 0x7f3dac0047c0 2023-11-13T11:03:36.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.665+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db002c7c0 con 0x7f3dac0047c0 2023-11-13T11:03:36.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.665+0000 7f3db7fff640 1 -- 172.21.15.16:0/1750487054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db002cb70 con 0x7f3dac0047c0 2023-11-13T11:03:36.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.665+0000 7f3dc759e640 1 -- 172.21.15.16:0/1750487054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 msgr2=0x7f3dac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:36.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.665+0000 7f3dc759e640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dac004c40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ae4c0 tx=0x7f3dac0057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.666+0000 7f3dc759e640 1 -- 172.21.15.16:0/1750487054 shutdown_connections 2023-11-13T11:03:36.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.666+0000 7f3dc759e640 1 --2- 172.21.15.16:0/1750487054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dac004c40 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.666+0000 7f3dc759e640 1 -- 172.21.15.16:0/1750487054 >> 172.21.15.16:0/1750487054 conn(0x7f3dc006a190 msgr2=0x7f3dc006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:36.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.666+0000 7f3dc759e640 1 -- 172.21.15.16:0/1750487054 shutdown_connections 2023-11-13T11:03:36.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.666+0000 7f3dc759e640 1 -- 172.21.15.16:0/1750487054 wait complete. 2023-11-13T11:03:36.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.668+0000 7f3dc759e640 1 Processor -- start 2023-11-13T11:03:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.668+0000 7f3dc759e640 1 -- start start 2023-11-13T11:03:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dc01627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc759e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc014fe60 0x7f3dc014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dc015c400 0x7f3dc0162ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc01502c0 con 0x7f3dac0047c0 2023-11-13T11:03:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc759e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc015bf70 con 0x7f3dc014fe60 2023-11-13T11:03:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0163d50 con 0x7f3dc015c400 2023-11-13T11:03:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc533a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dc01627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc533a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dc01627c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60324/0 (socket says 172.21.15.16:60324) 2023-11-13T11:03:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc533a640 1 -- 172.21.15.16:0/1760533313 learned_addr learned my addr 172.21.15.16:0/1760533313 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc4b39640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc014fe60 0x7f3dc014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc533a640 1 -- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dc015c400 msgr2=0x7f3dc0162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc533a640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dc015c400 0x7f3dc0162ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc533a640 1 -- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc014fe60 msgr2=0x7f3dc014cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc533a640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc014fe60 0x7f3dc014cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.669+0000 7f3dc533a640 1 -- 172.21.15.16:0/1760533313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc0165970 con 0x7f3dac0047c0 2023-11-13T11:03:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.670+0000 7f3dc533a640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dc01627c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00d4e0 tx=0x7f3dbc00d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.670+0000 7f3db5ffb640 1 -- 172.21.15.16:0/1760533313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc005480 con 0x7f3dac0047c0 2023-11-13T11:03:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.670+0000 7f3db5ffb640 1 -- 172.21.15.16:0/1760533313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc003610 con 0x7f3dac0047c0 2023-11-13T11:03:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.670+0000 7f3db5ffb640 1 -- 172.21.15.16:0/1760533313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc003a00 con 0x7f3dac0047c0 2023-11-13T11:03:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.670+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc0165690 con 0x7f3dac0047c0 2023-11-13T11:03:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.670+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc015c880 con 0x7f3dac0047c0 2023-11-13T11:03:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.671+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc0067090 con 0x7f3dac0047c0 2023-11-13T11:03:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.672+0000 7f3db5ffb640 1 -- 172.21.15.16:0/1760533313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3dbc003b60 con 0x7f3dac0047c0 2023-11-13T11:03:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.673+0000 7f3db5ffb640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3da406ae40 0x7f3da406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.673+0000 7f3db5ffb640 1 -- 172.21.15.16:0/1760533313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3dbc089530 con 0x7f3dac0047c0 2023-11-13T11:03:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.674+0000 7f3dc4b39640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3da406ae40 0x7f3da406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.675+0000 7f3dc4b39640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3da406ae40 0x7f3da406d300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3db0036000 tx=0x7f3db0030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.678+0000 7f3db5ffb640 1 -- 172.21.15.16:0/1760533313 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3dbc054550 con 0x7f3dac0047c0 2023-11-13T11:03:36.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.908+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dc00aeaa0 con 0x7f3dac0047c0 2023-11-13T11:03:36.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.909+0000 7f3db5ffb640 1 -- 172.21.15.16:0/1760533313 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f3dbc057b70 con 0x7f3dac0047c0 2023-11-13T11:03:36.914 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:36.914 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/1247955601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1247955601},{"type":"v1","addr":"172.21.15.184:6836","nonce":1247955601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3721806669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3721806669},{"type":"v1","addr":"172.21.15.16:6834","nonce":3721806669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6879,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2700274237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2700274237},{"type":"v1","addr":"172.21.15.16:6835","nonce":2700274237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:22.988900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6837/60343961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":60343961},{"type":"v1","addr":"172.21.15.184:6837","nonce":60343961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:03:36.914 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 40 2023-11-13T11:03:36.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3da406ae40 msgr2=0x7f3da406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:36.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3da406ae40 0x7f3da406d300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3db0036000 tx=0x7f3db0030000 comp rx=0 tx=0).stop 2023-11-13T11:03:36.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 msgr2=0x7f3dc01627c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:36.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dc01627c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00d4e0 tx=0x7f3dbc00d9a0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 shutdown_connections 2023-11-13T11:03:36.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3da406ae40 0x7f3da406d300 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3dc015c400 0x7f3dc0162ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dc014fe60 0x7f3dc014cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 --2- 172.21.15.16:0/1760533313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac0047c0 0x7f3dc01627c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 >> 172.21.15.16:0/1760533313 conn(0x7f3dc006a190 msgr2=0x7f3dc015a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.912+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 shutdown_connections 2023-11-13T11:03:36.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:36.913+0000 7f3dc759e640 1 -- 172.21.15.16:0/1760533313 wait complete. 2023-11-13T11:03:36.938 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump inode 1099511627776 2023-11-13T11:03:37.178 INFO:teuthology.orchestra.run.smithi184.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2023-11-13T11:03:36.210131+0000","btime":"2023-11-13T11:03:36.210131+0000","mode":16877,"uid":1000,"gid":1301,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2023-11-13T11:03:36.210131+0000","atime":"2023-11-13T11:03:36.210131+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","type":"iauth","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","type":"ixattr","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":7053,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":7053,"want_loner":7053,"mds_caps_wanted":[],"is_subvolume":false} 2023-11-13T11:03:37.179 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2023-11-13T11:03:36.210131+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "iauth", "xlock_by": {} }, "backtrace_version": 2, "btime": "2023-11-13T11:03:36.210131+0000", "change_attr": 0, "client_caps": [ { "client_id": 7053, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2023-11-13T11:03:36.210131+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1301, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 7053, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2023-11-13T11:03:36.210131+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 7053, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ixattr", "xlock_by": {} }, "xattrs": [] } 2023-11-13T11:03:37.180 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:03:37.180 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:37.244 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2023-11-13T11:03:37.245 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-13T11:03:37.245 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2023-11-13T11:03:37.245 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:03:37.246 DEBUG:teuthology.orchestra.run.smithi016:> n = 40 2023-11-13T11:03:37.246 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2023-11-13T11:03:37.247 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2023-11-13T11:03:37.247 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2023-11-13T11:03:37.247 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file" 2023-11-13T11:03:37.248 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:03:37.248 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2023-11-13T11:03:37.248 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2023-11-13T11:03:37.249 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2023-11-13T11:03:37.249 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:03:37.249 DEBUG:teuthology.orchestra.run.smithi016:> try: 2023-11-13T11:03:37.250 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-11-13T11:03:37.250 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-13T11:03:37.251 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2023-11-13T11:03:37.251 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-11-13T11:03:37.251 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2023-11-13T11:03:37.252 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2023-11-13T11:03:37.252 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2023-11-13T11:03:37.252 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2023-11-13T11:03:37.253 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2023-11-13T11:03:37.253 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2023-11-13T11:03:37.254 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2023-11-13T11:03:37.254 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2023-11-13T11:03:37.254 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2023-11-13T11:03:37.255 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2023-11-13T11:03:37.255 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2023-11-13T11:03:37.256 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-11-13T11:03:37.256 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2023-11-13T11:03:37.256 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2023-11-13T11:03:37.257 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2023-11-13T11:03:37.257 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2023-11-13T11:03:37.257 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-13T11:03:37.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-13T11:03:37.737 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:03:37.738 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:37.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.067+0000 7f424f183640 1 Processor -- start 2023-11-13T11:03:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.067+0000 7f424f183640 1 -- start start 2023-11-13T11:03:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.069+0000 7f424f183640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 0x7f4248071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.069+0000 7f424f183640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42480cd0e0 con 0x7f4248155410 2023-11-13T11:03:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.069+0000 7f424f183640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424815b500 con 0x7f4248071980 2023-11-13T11:03:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.069+0000 7f424f183640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4248071e10 con 0x7f42480ccc60 2023-11-13T11:03:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.069+0000 7f424cf1f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4248155410 0x7f42480ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37984/0 (socket says 172.21.15.16:37984) 2023-11-13T11:03:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.069+0000 7f424cf1f640 1 -- 172.21.15.16:0/291167959 learned_addr learned my addr 172.21.15.16:0/291167959 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.069+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343575481 0 0) 0x7f42480cd0e0 con 0x7f4248155410 2023-11-13T11:03:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.069+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4238003520 con 0x7f4248155410 2023-11-13T11:03:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f4247fff640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 0x7f4248071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3733721904 0 0) 0x7f4238003520 con 0x7f4248155410 2023-11-13T11:03:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 msgr2=0x7f4248071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 0x7f4248071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 >> v1:172.21.15.184:6789/0 conn(0x7f4248071980 legacy=0x7f424806fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424815b7e0 con 0x7f4248155410 2023-11-13T11:03:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f4247fff640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 0x7f4248071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:03:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f423c001670 con 0x7f4248155410 2023-11-13T11:03:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 >> v1:172.21.15.16:6789/0 conn(0x7f4248155410 legacy=0x7f42480ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42380039a0 0x7f4238003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 0x7f42380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4238004d00 0x7f4238005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248071e10 con 0x7f4238004d00 2023-11-13T11:03:38.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423800bb10 con 0x7f42380039a0 2023-11-13T11:03:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423800bca0 con 0x7f42480ccc60 2023-11-13T11:03:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.070+0000 7f424d720640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4238004d00 0x7f4238005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.071+0000 7f424cf1f640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42380039a0 0x7f4238003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.071+0000 7f4247fff640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 0x7f42380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.071+0000 7f424d720640 1 -- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 msgr2=0x7f42380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:38.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.071+0000 7f424d720640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 0x7f42380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.071+0000 7f424d720640 1 -- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42380039a0 msgr2=0x7f4238003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:38.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.071+0000 7f424d720640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42380039a0 0x7f4238003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.071+0000 7f424d720640 1 -- 172.21.15.16:0/291167959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4238003520 con 0x7f4238004d00 2023-11-13T11:03:38.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.071+0000 7f4247fff640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42480ccc60 0x7f42380047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.072+0000 7f424d720640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4238004d00 0x7f4238005180 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4240001e50 tx=0x7f42380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633b9e5eb147d0d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.072+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240003b70 con 0x7f4238004d00 2023-11-13T11:03:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.072+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240003cd0 con 0x7f4238004d00 2023-11-13T11:03:38.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.073+0000 7f42477fe640 1 -- 172.21.15.16:0/291167959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240005310 con 0x7f4238004d00 2023-11-13T11:03:38.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.073+0000 7f424f183640 1 -- 172.21.15.16:0/291167959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4238004d00 msgr2=0x7f4238005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.073+0000 7f424f183640 1 --2- 172.21.15.16:0/291167959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4238004d00 0x7f4238005180 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4240001e50 tx=0x7f42380057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.073+0000 7f424d720640 1 -- 172.21.15.16:0/291167959 reap_dead start 2023-11-13T11:03:38.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.073+0000 7f424f183640 1 -- 172.21.15.16:0/291167959 shutdown_connections 2023-11-13T11:03:38.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.073+0000 7f424f183640 1 -- 172.21.15.16:0/291167959 >> 172.21.15.16:0/291167959 conn(0x7f424806a190 msgr2=0x7f424806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:38.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.073+0000 7f424f183640 1 -- 172.21.15.16:0/291167959 shutdown_connections 2023-11-13T11:03:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.073+0000 7f424f183640 1 -- 172.21.15.16:0/291167959 wait complete. 2023-11-13T11:03:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.075+0000 7f424f183640 1 Processor -- start 2023-11-13T11:03:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.075+0000 7f424f183640 1 -- start start 2023-11-13T11:03:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.076+0000 7f424f183640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4238004d00 0x7f42481573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.076+0000 7f424f183640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4248155870 0x7f42480701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.076+0000 7f424f183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424806bb90 0x7f424806c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.076+0000 7f424f183640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248157990 con 0x7f424806bb90 2023-11-13T11:03:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.076+0000 7f424f183640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248157c70 con 0x7f4248155870 2023-11-13T11:03:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.076+0000 7f424f183640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4248157f50 con 0x7f4238004d00 2023-11-13T11:03:38.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.076+0000 7f424d720640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424806bb90 0x7f424806c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f424d720640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424806bb90 0x7f424806c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60340/0 (socket says 172.21.15.16:60340) 2023-11-13T11:03:38.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f424d720640 1 -- 172.21.15.16:0/4068602580 learned_addr learned my addr 172.21.15.16:0/4068602580 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:38.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f424cf1f640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4238004d00 0x7f42481573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f4247fff640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4248155870 0x7f42480701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f424d720640 1 -- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4238004d00 msgr2=0x7f42481573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f424d720640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4238004d00 0x7f42481573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f424d720640 1 -- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4248155870 msgr2=0x7f42480701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f424d720640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4248155870 0x7f42480701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f424d720640 1 -- 172.21.15.16:0/4068602580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4248158230 con 0x7f424806bb90 2023-11-13T11:03:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f4247fff640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4248155870 0x7f42480701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.077+0000 7f424d720640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424806bb90 0x7f424806c010 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4240001f70 tx=0x7f42400014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.078+0000 7f42457fa640 1 -- 172.21.15.16:0/4068602580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424003e070 con 0x7f424806bb90 2023-11-13T11:03:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.078+0000 7f42457fa640 1 -- 172.21.15.16:0/4068602580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240003e40 con 0x7f424806bb90 2023-11-13T11:03:38.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.078+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4248158510 con 0x7f424806bb90 2023-11-13T11:03:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.078+0000 7f42457fa640 1 -- 172.21.15.16:0/4068602580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240006d90 con 0x7f424806bb90 2023-11-13T11:03:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.078+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4248070730 con 0x7f424806bb90 2023-11-13T11:03:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.079+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4248067090 con 0x7f424806bb90 2023-11-13T11:03:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.080+0000 7f42457fa640 1 -- 172.21.15.16:0/4068602580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4240007030 con 0x7f424806bb90 2023-11-13T11:03:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.081+0000 7f42457fa640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f422c06ae40 0x7f422c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.081+0000 7f42457fa640 1 -- 172.21.15.16:0/4068602580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f42400aa980 con 0x7f424806bb90 2023-11-13T11:03:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.082+0000 7f424cf1f640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f422c06ae40 0x7f422c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.083+0000 7f424cf1f640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f422c06ae40 0x7f422c06d300 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f423c0059f0 tx=0x7f423c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.085+0000 7f42457fa640 1 -- 172.21.15.16:0/4068602580 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4240075850 con 0x7f424806bb90 2023-11-13T11:03:38.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.283+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f424815ad90 con 0x7f424806bb90 2023-11-13T11:03:38.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.285+0000 7f42457fa640 1 -- 172.21.15.16:0/4068602580 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16776 (secure 0 0 0) 0x7f4240048020 con 0x7f424806bb90 2023-11-13T11:03:38.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:38.290 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":69,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:21.435542+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:03:17.903258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-13T11:03:18.947896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:38.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.287+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f422c06ae40 msgr2=0x7f422c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.287+0000 7f424f183640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f422c06ae40 0x7f422c06d300 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f423c0059f0 tx=0x7f423c009040 comp rx=0 tx=0).stop 2023-11-13T11:03:38.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.287+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424806bb90 msgr2=0x7f424806c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.287+0000 7f424f183640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424806bb90 0x7f424806c010 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4240001f70 tx=0x7f42400014a0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.288+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 shutdown_connections 2023-11-13T11:03:38.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.288+0000 7f424f183640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f422c06ae40 0x7f422c06d300 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.288+0000 7f424f183640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f424806bb90 0x7f424806c010 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.288+0000 7f424f183640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4248155870 0x7f42480701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.288+0000 7f424f183640 1 --2- 172.21.15.16:0/4068602580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4238004d00 0x7f42481573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.288+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 >> 172.21.15.16:0/4068602580 conn(0x7f424806a190 msgr2=0x7f42480b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.288+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 shutdown_connections 2023-11-13T11:03:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.288+0000 7f424f183640 1 -- 172.21.15.16:0/4068602580 wait complete. 2023-11-13T11:03:38.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T11:03:17.903258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T11:03:18.947896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:03:38.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:03:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.544+0000 7f0557fff640 1 Processor -- start 2023-11-13T11:03:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.545+0000 7f0557fff640 1 -- start start 2023-11-13T11:03:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.547+0000 7f0557fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055815bcc0 0x7f05580cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.547+0000 7f0557fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05580ce090 0x7f05580cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.547+0000 7f0557fff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055806d010 con 0x7f055815b8e0 2023-11-13T11:03:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.547+0000 7f0557fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05580ce540 con 0x7f05580ce090 2023-11-13T11:03:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.547+0000 7f0557fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055806b860 con 0x7f055815bcc0 2023-11-13T11:03:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.547+0000 7f05577fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05580ce090 0x7f05580cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.547+0000 7f05577fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05580ce090 0x7f05580cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60344/0 (socket says 172.21.15.16:60344) 2023-11-13T11:03:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.547+0000 7f05577fe640 1 -- 172.21.15.16:0/438278450 learned_addr learned my addr 172.21.15.16:0/438278450 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.547+0000 7f05567fc640 1 --2- 172.21.15.16:0/438278450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055815bcc0 0x7f05580cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.548+0000 7f05577fe640 1 -- 172.21.15.16:0/438278450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055815bcc0 msgr2=0x7f05580cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.548+0000 7f05577fe640 1 --2- 172.21.15.16:0/438278450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055815bcc0 0x7f05580cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.548+0000 7f05577fe640 1 -- 172.21.15.16:0/438278450 >> v1:172.21.15.184:6789/0 conn(0x7f055815b8e0 legacy=0x7f055806c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.548+0000 7f05577fe640 1 -- 172.21.15.16:0/438278450 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055806bb40 con 0x7f05580ce090 2023-11-13T11:03:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.549+0000 7f05577fe640 1 --2- 172.21.15.16:0/438278450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580ce090 0x7f05580cbae0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0550009e80 tx=0x7f055002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7a4f9a3218258f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.549+0000 7f0555ffb640 1 -- 172.21.15.16:0/438278450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055002ca50 con 0x7f05580ce090 2023-11-13T11:03:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.549+0000 7f0555ffb640 1 -- 172.21.15.16:0/438278450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055002cbb0 con 0x7f05580ce090 2023-11-13T11:03:38.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.549+0000 7f0555ffb640 1 -- 172.21.15.16:0/438278450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05500343f0 con 0x7f05580ce090 2023-11-13T11:03:38.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.550+0000 7f0557fff640 1 -- 172.21.15.16:0/438278450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580ce090 msgr2=0x7f05580cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.550+0000 7f0557fff640 1 --2- 172.21.15.16:0/438278450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580ce090 0x7f05580cbae0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0550009e80 tx=0x7f055002beb0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.550+0000 7f0557fff640 1 -- 172.21.15.16:0/438278450 shutdown_connections 2023-11-13T11:03:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.550+0000 7f0557fff640 1 --2- 172.21.15.16:0/438278450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580ce090 0x7f05580cbae0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.550+0000 7f0557fff640 1 --2- 172.21.15.16:0/438278450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055815bcc0 0x7f05580cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.550+0000 7f0557fff640 1 -- 172.21.15.16:0/438278450 >> 172.21.15.16:0/438278450 conn(0x7f05580b0970 msgr2=0x7f05580b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.550+0000 7f0557fff640 1 -- 172.21.15.16:0/438278450 shutdown_connections 2023-11-13T11:03:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.550+0000 7f0557fff640 1 -- 172.21.15.16:0/438278450 wait complete. 2023-11-13T11:03:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.553+0000 7f0557fff640 1 Processor -- start 2023-11-13T11:03:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.553+0000 7f0557fff640 1 -- start start 2023-11-13T11:03:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.554+0000 7f0557fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05580ce090 0x7f0558150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.554+0000 7f0557fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055815b8e0 0x7f055814ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.554+0000 7f0557fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580af830 0x7f05581517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.554+0000 7f0557fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0558159810 con 0x7f05580af830 2023-11-13T11:03:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.554+0000 7f0557fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0558152230 con 0x7f05580ce090 2023-11-13T11:03:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.554+0000 7f0557fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05581591c0 con 0x7f055815b8e0 2023-11-13T11:03:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f05577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580af830 0x7f05581517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f05577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580af830 0x7f05581517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60346/0 (socket says 172.21.15.16:60346) 2023-11-13T11:03:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f05577fe640 1 -- 172.21.15.16:0/2291098225 learned_addr learned my addr 172.21.15.16:0/2291098225 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f0556ffd640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05580ce090 0x7f0558150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f05577fe640 1 -- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055815b8e0 msgr2=0x7f055814ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f05577fe640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055815b8e0 0x7f055814ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f05577fe640 1 -- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05580ce090 msgr2=0x7f0558150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f05577fe640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05580ce090 0x7f0558150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f05577fe640 1 -- 172.21.15.16:0/2291098225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0550009b30 con 0x7f05580af830 2023-11-13T11:03:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.555+0000 7f0556ffd640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05580ce090 0x7f0558150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.556+0000 7f05577fe640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580af830 0x7f05581517e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0550009fa0 tx=0x7f0550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.556+0000 7f053f7fe640 1 -- 172.21.15.16:0/2291098225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055003e450 con 0x7f05580af830 2023-11-13T11:03:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.556+0000 7f053f7fe640 1 -- 172.21.15.16:0/2291098225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055003e5d0 con 0x7f05580af830 2023-11-13T11:03:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.556+0000 7f053f7fe640 1 -- 172.21.15.16:0/2291098225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055003e980 con 0x7f05580af830 2023-11-13T11:03:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.556+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05581594a0 con 0x7f05580af830 2023-11-13T11:03:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.556+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05580afce0 con 0x7f05580af830 2023-11-13T11:03:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.557+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05581599c0 con 0x7f05580af830 2023-11-13T11:03:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.558+0000 7f053f7fe640 1 -- 172.21.15.16:0/2291098225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f055003eae0 con 0x7f05580af830 2023-11-13T11:03:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.559+0000 7f053f7fe640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f054406ae70 0x7f054406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.559+0000 7f053f7fe640 1 -- 172.21.15.16:0/2291098225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f05500ac270 con 0x7f05580af830 2023-11-13T11:03:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.560+0000 7f0556ffd640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f054406ae70 0x7f054406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.561+0000 7f0556ffd640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f054406ae70 0x7f054406d330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0548003eb0 tx=0x7f0548009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.563+0000 7f053f7fe640 1 -- 172.21.15.16:0/2291098225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05581599c0 con 0x7f05580af830 2023-11-13T11:03:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.756+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05580aff00 con 0x7f05580af830 2023-11-13T11:03:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.757+0000 7f053f7fe640 1 -- 172.21.15.16:0/2291098225 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0550077140 con 0x7f05580af830 2023-11-13T11:03:38.761 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:03:38.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f054406ae70 msgr2=0x7f054406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f054406ae70 0x7f054406d330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0548003eb0 tx=0x7f0548009040 comp rx=0 tx=0).stop 2023-11-13T11:03:38.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580af830 msgr2=0x7f05581517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:38.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580af830 0x7f05581517e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0550009fa0 tx=0x7f0550004a40 comp rx=0 tx=0).stop 2023-11-13T11:03:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 shutdown_connections 2023-11-13T11:03:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f054406ae70 0x7f054406d330 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05580af830 0x7f05581517e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f055815b8e0 0x7f055814ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 --2- 172.21.15.16:0/2291098225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05580ce090 0x7f0558150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.760+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 >> 172.21.15.16:0/2291098225 conn(0x7f05580b0970 msgr2=0x7f05580b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.761+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 shutdown_connections 2023-11-13T11:03:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:38.761+0000 7f0557fff640 1 -- 172.21.15.16:0/2291098225 wait complete. 2023-11-13T11:03:38.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T11:03:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.005+0000 7f06d558a640 1 Processor -- start 2023-11-13T11:03:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.005+0000 7f06d558a640 1 -- start start 2023-11-13T11:03:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.007+0000 7f06d558a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06d015cf90 0x7f06d00b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.007+0000 7f06d558a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d006c3b0 0x7f06d00b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.007+0000 7f06d558a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f06d00b39c0 0x7f06d00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.007+0000 7f06d558a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d00b3e50 con 0x7f06d015cf90 2023-11-13T11:03:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.007+0000 7f06d558a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d00b2e00 con 0x7f06d00b39c0 2023-11-13T11:03:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.007+0000 7f06d558a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d00b21f0 con 0x7f06d006c3b0 2023-11-13T11:03:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.008+0000 7f06ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06d015cf90 0x7f06d00b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.008+0000 7f06ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06d015cf90 0x7f06d00b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60360/0 (socket says 172.21.15.16:60360) 2023-11-13T11:03:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.008+0000 7f06ceffd640 1 -- 172.21.15.16:0/3843605840 learned_addr learned my addr 172.21.15.16:0/3843605840 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.008+0000 7f06ceffd640 1 -- 172.21.15.16:0/3843605840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d006c3b0 msgr2=0x7f06d00b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.008+0000 7f06c7fff640 1 --2- 172.21.15.16:0/3843605840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d006c3b0 0x7f06d00b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.008+0000 7f06ceffd640 1 --2- 172.21.15.16:0/3843605840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d006c3b0 0x7f06d00b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.008+0000 7f06ceffd640 1 -- 172.21.15.16:0/3843605840 >> v2:172.21.15.184:3300/0 conn(0x7f06d00b39c0 msgr2=0x7f06d00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.008+0000 7f06ceffd640 1 --2- 172.21.15.16:0/3843605840 >> v2:172.21.15.184:3300/0 conn(0x7f06d00b39c0 0x7f06d00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.008+0000 7f06ceffd640 1 -- 172.21.15.16:0/3843605840 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d00b24d0 con 0x7f06d015cf90 2023-11-13T11:03:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.009+0000 7f06ceffd640 1 --2- 172.21.15.16:0/3843605840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 0x7f06d00b1c10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f06d006dbb0 tx=0x7f06c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bd53e158a9141b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.010+0000 7f06ce7fc640 1 -- 172.21.15.16:0/3843605840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0031870 con 0x7f06d015cf90 2023-11-13T11:03:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.010+0000 7f06ce7fc640 1 -- 172.21.15.16:0/3843605840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c00319d0 con 0x7f06d015cf90 2023-11-13T11:03:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.010+0000 7f06ce7fc640 1 -- 172.21.15.16:0/3843605840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0031d60 con 0x7f06d015cf90 2023-11-13T11:03:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.010+0000 7f06d558a640 1 -- 172.21.15.16:0/3843605840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 msgr2=0x7f06d00b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.010+0000 7f06d558a640 1 --2- 172.21.15.16:0/3843605840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 0x7f06d00b1c10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f06d006dbb0 tx=0x7f06c002b910 comp rx=0 tx=0).stop 2023-11-13T11:03:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.010+0000 7f06d558a640 1 -- 172.21.15.16:0/3843605840 shutdown_connections 2023-11-13T11:03:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.011+0000 7f06d558a640 1 --2- 172.21.15.16:0/3843605840 >> v2:172.21.15.184:3300/0 conn(0x7f06d00b39c0 0x7f06d00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.011+0000 7f06d558a640 1 --2- 172.21.15.16:0/3843605840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d006c3b0 0x7f06d00b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.011+0000 7f06d558a640 1 --2- 172.21.15.16:0/3843605840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 0x7f06d00b1c10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.011+0000 7f06d558a640 1 -- 172.21.15.16:0/3843605840 >> 172.21.15.16:0/3843605840 conn(0x7f06d006a190 msgr2=0x7f06d006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.011+0000 7f06d558a640 1 -- 172.21.15.16:0/3843605840 shutdown_connections 2023-11-13T11:03:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.011+0000 7f06d558a640 1 -- 172.21.15.16:0/3843605840 wait complete. 2023-11-13T11:03:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.012+0000 7f06d558a640 1 Processor -- start 2023-11-13T11:03:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.013+0000 7f06d558a640 1 -- start start 2023-11-13T11:03:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.013+0000 7f06d558a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d006c3b0 0x7f06d015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06d558a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d00b39c0 0x7f06d013caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06d558a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 0x7f06d013cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06d558a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0151620 con 0x7f06d015cf90 2023-11-13T11:03:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06d558a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d01f3160 con 0x7f06d006c3b0 2023-11-13T11:03:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06d558a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d01404c0 con 0x7f06d00b39c0 2023-11-13T11:03:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 0x7f06d013cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 0x7f06d013cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60374/0 (socket says 172.21.15.16:60374) 2023-11-13T11:03:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06cf7fe640 1 -- 172.21.15.16:0/2333394629 learned_addr learned my addr 172.21.15.16:0/2333394629 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06ceffd640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d006c3b0 0x7f06d015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06cf7fe640 1 -- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d00b39c0 msgr2=0x7f06d013caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06cf7fe640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d00b39c0 0x7f06d013caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06cf7fe640 1 -- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d006c3b0 msgr2=0x7f06d015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06cf7fe640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d006c3b0 0x7f06d015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.014+0000 7f06cf7fe640 1 -- 172.21.15.16:0/2333394629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00094d0 con 0x7f06d015cf90 2023-11-13T11:03:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.015+0000 7f06cf7fe640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 0x7f06d013cfe0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f06c800b970 tx=0x7f06c800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.015+0000 7f06c77fe640 1 -- 172.21.15.16:0/2333394629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c800cb40 con 0x7f06d015cf90 2023-11-13T11:03:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.015+0000 7f06c77fe640 1 -- 172.21.15.16:0/2333394629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c800ccc0 con 0x7f06d015cf90 2023-11-13T11:03:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.015+0000 7f06c77fe640 1 -- 172.21.15.16:0/2333394629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8014680 con 0x7f06d015cf90 2023-11-13T11:03:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.015+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d01407a0 con 0x7f06d015cf90 2023-11-13T11:03:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.015+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d0151b50 con 0x7f06d015cf90 2023-11-13T11:03:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.016+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d0151d70 con 0x7f06d015cf90 2023-11-13T11:03:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.017+0000 7f06c77fe640 1 -- 172.21.15.16:0/2333394629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06c801d3b0 con 0x7f06d015cf90 2023-11-13T11:03:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.018+0000 7f06c77fe640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06b006ae40 0x7f06b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.018+0000 7f06c77fe640 1 -- 172.21.15.16:0/2333394629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f06c80896e0 con 0x7f06d015cf90 2023-11-13T11:03:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.019+0000 7f06ceffd640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06b006ae40 0x7f06b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.020+0000 7f06ceffd640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06b006ae40 0x7f06b006d300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f06c0036000 tx=0x7f06c003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.023+0000 7f06c77fe640 1 -- 172.21.15.16:0/2333394629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06c8054640 con 0x7f06d015cf90 2023-11-13T11:03:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.221+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06d00cf0e0 con 0x7f06d015cf90 2023-11-13T11:03:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.222+0000 7f06c77fe640 1 -- 172.21.15.16:0/2333394629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06c8057c60 con 0x7f06d015cf90 2023-11-13T11:03:39.226 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T11:03:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.224+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06b006ae40 msgr2=0x7f06b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.224+0000 7f06d558a640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06b006ae40 0x7f06b006d300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f06c0036000 tx=0x7f06c003c040 comp rx=0 tx=0).stop 2023-11-13T11:03:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.224+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 msgr2=0x7f06d013cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.224+0000 7f06d558a640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 0x7f06d013cfe0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f06c800b970 tx=0x7f06c800be30 comp rx=0 tx=0).stop 2023-11-13T11:03:39.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.225+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 shutdown_connections 2023-11-13T11:03:39.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.225+0000 7f06d558a640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f06b006ae40 0x7f06b006d300 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.225+0000 7f06d558a640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d015cf90 0x7f06d013cfe0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.225+0000 7f06d558a640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f06d00b39c0 0x7f06d013caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.225+0000 7f06d558a640 1 --2- 172.21.15.16:0/2333394629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06d006c3b0 0x7f06d015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.225+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 >> 172.21.15.16:0/2333394629 conn(0x7f06d006a190 msgr2=0x7f06d00ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:39.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.225+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 shutdown_connections 2023-11-13T11:03:39.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.225+0000 7f06d558a640 1 -- 172.21.15.16:0/2333394629 wait complete. 2023-11-13T11:03:39.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T11:03:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.471+0000 7f07bbfff640 1 Processor -- start 2023-11-13T11:03:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.472+0000 7f07bbfff640 1 -- start start 2023-11-13T11:03:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.474+0000 7f07bbfff640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f07bc15b8e0 0x7f07bc06c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.474+0000 7f07bbfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 0x7f07bc0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.474+0000 7f07bbfff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07bc06b860 con 0x7f07bc15bcc0 2023-11-13T11:03:39.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.474+0000 7f07bbfff640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc06bb40 con 0x7f07bc15b8e0 2023-11-13T11:03:39.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.474+0000 7f07bbfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc06be20 con 0x7f07bc0ce260 2023-11-13T11:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.474+0000 7f07baffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f07bc15b8e0 0x7f07bc06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.475+0000 7f07baffd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f07bc15b8e0 0x7f07bc06c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:51714/0 (socket says 172.21.15.16:51714) 2023-11-13T11:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.475+0000 7f07baffd640 1 -- 172.21.15.16:0/1938356387 learned_addr learned my addr 172.21.15.16:0/1938356387 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.475+0000 7f07bb7fe640 1 --2- 172.21.15.16:0/1938356387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 0x7f07bc0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.475+0000 7f07baffd640 1 -- 172.21.15.16:0/1938356387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 msgr2=0x7f07bc0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.475+0000 7f07baffd640 1 --2- 172.21.15.16:0/1938356387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 0x7f07bc0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.475+0000 7f07baffd640 1 -- 172.21.15.16:0/1938356387 >> v1:172.21.15.184:6789/0 conn(0x7f07bc15bcc0 legacy=0x7f07bc0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.475+0000 7f07baffd640 1 -- 172.21.15.16:0/1938356387 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc06c100 con 0x7f07bc15b8e0 2023-11-13T11:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.477+0000 7f07baffd640 1 --2- 172.21.15.16:0/1938356387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07bc15b8e0 0x7f07bc06c900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009870 tx=0x7f07ac02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05da56b26baf826 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.478+0000 7f07ba7fc640 1 -- 172.21.15.16:0/1938356387 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac030840 con 0x7f07bc15b8e0 2023-11-13T11:03:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.478+0000 7f07ba7fc640 1 -- 172.21.15.16:0/1938356387 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac0309a0 con 0x7f07bc15b8e0 2023-11-13T11:03:39.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.478+0000 7f07ba7fc640 1 -- 172.21.15.16:0/1938356387 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac030d30 con 0x7f07bc15b8e0 2023-11-13T11:03:39.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.479+0000 7f07bbfff640 1 -- 172.21.15.16:0/1938356387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07bc15b8e0 msgr2=0x7f07bc06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.479+0000 7f07bbfff640 1 --2- 172.21.15.16:0/1938356387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07bc15b8e0 0x7f07bc06c900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009870 tx=0x7f07ac02b980 comp rx=0 tx=0).stop 2023-11-13T11:03:39.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.479+0000 7f07bbfff640 1 -- 172.21.15.16:0/1938356387 shutdown_connections 2023-11-13T11:03:39.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.479+0000 7f07bbfff640 1 --2- 172.21.15.16:0/1938356387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 0x7f07bc0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.479+0000 7f07bbfff640 1 --2- 172.21.15.16:0/1938356387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07bc15b8e0 0x7f07bc06c900 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.479+0000 7f07bbfff640 1 -- 172.21.15.16:0/1938356387 >> 172.21.15.16:0/1938356387 conn(0x7f07bc0b0970 msgr2=0x7f07bc0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:39.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.479+0000 7f07bbfff640 1 -- 172.21.15.16:0/1938356387 shutdown_connections 2023-11-13T11:03:39.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.479+0000 7f07bbfff640 1 -- 172.21.15.16:0/1938356387 wait complete. 2023-11-13T11:03:39.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.482+0000 7f07bbfff640 1 Processor -- start 2023-11-13T11:03:39.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.482+0000 7f07bbfff640 1 -- start start 2023-11-13T11:03:39.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.483+0000 7f07bbfff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 0x7f07bc1f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.483+0000 7f07bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc15b8e0 0x7f07bc13dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.483+0000 7f07bbfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07bc156ae0 0x7f07bc156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.483+0000 7f07bbfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc153fe0 con 0x7f07bc15b8e0 2023-11-13T11:03:39.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.483+0000 7f07bbfff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc13d9e0 con 0x7f07bc156ae0 2023-11-13T11:03:39.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.483+0000 7f07bbfff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc153360 con 0x7f07bc0ce260 2023-11-13T11:03:39.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.483+0000 7f07b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc15b8e0 0x7f07bc13dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc15b8e0 0x7f07bc13dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60386/0 (socket says 172.21.15.16:60386) 2023-11-13T11:03:39.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07b27fc640 1 -- 172.21.15.16:0/3756938909 learned_addr learned my addr 172.21.15.16:0/3756938909 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:39.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07baffd640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 0x7f07bc1f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07bb7fe640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07bc156ae0 0x7f07bc156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07b27fc640 1 -- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 msgr2=0x7f07bc1f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07b27fc640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 0x7f07bc1f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07b27fc640 1 -- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07bc156ae0 msgr2=0x7f07bc156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07b27fc640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07bc156ae0 0x7f07bc156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07b27fc640 1 -- 172.21.15.16:0/3756938909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac0094d0 con 0x7f07bc15b8e0 2023-11-13T11:03:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.484+0000 7f07baffd640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 0x7f07bc1f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:03:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.485+0000 7f07b27fc640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc15b8e0 0x7f07bc13dca0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c3c0 tx=0x7f07a400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.485+0000 7f07b3fff640 1 -- 172.21.15.16:0/3756938909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a40052d0 con 0x7f07bc15b8e0 2023-11-13T11:03:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.485+0000 7f07b3fff640 1 -- 172.21.15.16:0/3756938909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a4005450 con 0x7f07bc15b8e0 2023-11-13T11:03:39.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.485+0000 7f07b3fff640 1 -- 172.21.15.16:0/3756938909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4007260 con 0x7f07bc15b8e0 2023-11-13T11:03:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.485+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07bc153640 con 0x7f07bc15b8e0 2023-11-13T11:03:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.485+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc153c00 con 0x7f07bc15b8e0 2023-11-13T11:03:39.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.486+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc153dc0 con 0x7f07bc15b8e0 2023-11-13T11:03:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.487+0000 7f07b3fff640 1 -- 172.21.15.16:0/3756938909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f07a40073c0 con 0x7f07bc15b8e0 2023-11-13T11:03:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.488+0000 7f07b3fff640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f079c06ae70 0x7f079c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.488+0000 7f07b3fff640 1 -- 172.21.15.16:0/3756938909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f07a4088c30 con 0x7f07bc15b8e0 2023-11-13T11:03:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.489+0000 7f07baffd640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f079c06ae70 0x7f079c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.490+0000 7f07baffd640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f079c06ae70 0x7f079c06d330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f07ac036000 tx=0x7f07ac008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:39.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.493+0000 7f07b3fff640 1 -- 172.21.15.16:0/3756938909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07a4053a70 con 0x7f07bc15b8e0 2023-11-13T11:03:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.685+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f07bc154120 con 0x7f07bc15b8e0 2023-11-13T11:03:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.686+0000 7f07b3fff640 1 -- 172.21.15.16:0/3756938909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f07bc154120 con 0x7f07bc15b8e0 2023-11-13T11:03:39.690 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T11:03:39.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f079c06ae70 msgr2=0x7f079c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f079c06ae70 0x7f079c06d330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f07ac036000 tx=0x7f07ac008030 comp rx=0 tx=0).stop 2023-11-13T11:03:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc15b8e0 msgr2=0x7f07bc13dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc15b8e0 0x7f07bc13dca0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c3c0 tx=0x7f07a400c880 comp rx=0 tx=0).stop 2023-11-13T11:03:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 shutdown_connections 2023-11-13T11:03:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f079c06ae70 0x7f079c06d330 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07bc156ae0 0x7f07bc156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc15b8e0 0x7f07bc13dca0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 --2- 172.21.15.16:0/3756938909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07bc0ce260 0x7f07bc1f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.689+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 >> 172.21.15.16:0/3756938909 conn(0x7f07bc0b0970 msgr2=0x7f07bc15d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.690+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 shutdown_connections 2023-11-13T11:03:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.690+0000 7f07bbfff640 1 -- 172.21.15.16:0/3756938909 wait complete. 2023-11-13T11:03:39.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T11:03:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.924+0000 7fb04e7dd640 1 Processor -- start 2023-11-13T11:03:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.924+0000 7fb04e7dd640 1 -- start start 2023-11-13T11:03:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04e7dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb048159230 0x7fb04806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04e7dd640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb0480adc80 0x7fb0480700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04e7dd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 0x7fb04815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04e7dd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0480b18d0 con 0x7fb048159230 2023-11-13T11:03:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04e7dd640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0480caf10 con 0x7fb0480adc80 2023-11-13T11:03:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04e7dd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04806cea0 con 0x7fb04806bfb0 2023-11-13T11:03:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04d7db640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb048159230 0x7fb04806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04d7db640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb048159230 0x7fb04806fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60396/0 (socket says 172.21.15.16:60396) 2023-11-13T11:03:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04d7db640 1 -- 172.21.15.16:0/3983666175 learned_addr learned my addr 172.21.15.16:0/3983666175 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04cfda640 1 --2- 172.21.15.16:0/3983666175 >> v2:172.21.15.184:3300/0 conn(0x7fb0480adc80 0x7fb0480700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04dfdc640 1 --2- 172.21.15.16:0/3983666175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 0x7fb04815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.926+0000 7fb04d7db640 1 -- 172.21.15.16:0/3983666175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 msgr2=0x7fb04815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.927+0000 7fb04d7db640 1 --2- 172.21.15.16:0/3983666175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 0x7fb04815cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.927+0000 7fb04d7db640 1 -- 172.21.15.16:0/3983666175 >> v2:172.21.15.184:3300/0 conn(0x7fb0480adc80 msgr2=0x7fb0480700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:39.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.927+0000 7fb04d7db640 1 --2- 172.21.15.16:0/3983666175 >> v2:172.21.15.184:3300/0 conn(0x7fb0480adc80 0x7fb0480700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.927+0000 7fb04d7db640 1 -- 172.21.15.16:0/3983666175 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04806d9c0 con 0x7fb048159230 2023-11-13T11:03:39.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.927+0000 7fb04dfdc640 1 --2- 172.21.15.16:0/3983666175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 0x7fb04815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.928+0000 7fb04d7db640 1 --2- 172.21.15.16:0/3983666175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb048159230 0x7fb04806fb80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb04806de80 tx=0x7fb04402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3913453bdc007530 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.928+0000 7fb03ffff640 1 -- 172.21.15.16:0/3983666175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044031870 con 0x7fb048159230 2023-11-13T11:03:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.928+0000 7fb03ffff640 1 -- 172.21.15.16:0/3983666175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0440319d0 con 0x7fb048159230 2023-11-13T11:03:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.928+0000 7fb03ffff640 1 -- 172.21.15.16:0/3983666175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044031d60 con 0x7fb048159230 2023-11-13T11:03:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.929+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3983666175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb048159230 msgr2=0x7fb04806fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.929+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3983666175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb048159230 0x7fb04806fb80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb04806de80 tx=0x7fb04402b910 comp rx=0 tx=0).stop 2023-11-13T11:03:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.929+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3983666175 shutdown_connections 2023-11-13T11:03:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.929+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3983666175 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 0x7fb04815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.929+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3983666175 >> v2:172.21.15.184:3300/0 conn(0x7fb0480adc80 0x7fb0480700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.929+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3983666175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb048159230 0x7fb04806fb80 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.929+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3983666175 >> 172.21.15.16:0/3983666175 conn(0x7fb0480b04a0 msgr2=0x7fb0480b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.929+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3983666175 shutdown_connections 2023-11-13T11:03:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.929+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3983666175 wait complete. 2023-11-13T11:03:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.931+0000 7fb04e7dd640 1 Processor -- start 2023-11-13T11:03:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.931+0000 7fb04e7dd640 1 -- start start 2023-11-13T11:03:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.932+0000 7fb04e7dd640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 0x7fb0480acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.932+0000 7fb04e7dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0480adc80 0x7fb048152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.932+0000 7fb04e7dd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb048159230 0x7fb0481526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.932+0000 7fb04e7dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048152f00 con 0x7fb0480adc80 2023-11-13T11:03:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.932+0000 7fb04e7dd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480cbc20 con 0x7fb048159230 2023-11-13T11:03:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.932+0000 7fb04e7dd640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480cb2f0 con 0x7fb04806bfb0 2023-11-13T11:03:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.933+0000 7fb04cfda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0480adc80 0x7fb048152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.933+0000 7fb04cfda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0480adc80 0x7fb048152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60404/0 (socket says 172.21.15.16:60404) 2023-11-13T11:03:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.933+0000 7fb04cfda640 1 -- 172.21.15.16:0/3459739515 learned_addr learned my addr 172.21.15.16:0/3459739515 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.933+0000 7fb04d7db640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 0x7fb0480acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.933+0000 7fb04cfda640 1 -- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 msgr2=0x7fb0480acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.933+0000 7fb04cfda640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 0x7fb0480acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.933+0000 7fb04cfda640 1 -- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb048159230 msgr2=0x7fb0481526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.933+0000 7fb04cfda640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb048159230 0x7fb0481526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.933+0000 7fb04cfda640 1 -- 172.21.15.16:0/3459739515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0440094d0 con 0x7fb0480adc80 2023-11-13T11:03:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.934+0000 7fb04cfda640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0480adc80 0x7fb048152190 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb03800d800 tx=0x7fb03800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.934+0000 7fb03dffb640 1 -- 172.21.15.16:0/3459739515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038018070 con 0x7fb0480adc80 2023-11-13T11:03:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.934+0000 7fb03dffb640 1 -- 172.21.15.16:0/3459739515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb038004020 con 0x7fb0480adc80 2023-11-13T11:03:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.934+0000 7fb03dffb640 1 -- 172.21.15.16:0/3459739515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0380041b0 con 0x7fb0480adc80 2023-11-13T11:03:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.934+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0480cb5d0 con 0x7fb0480adc80 2023-11-13T11:03:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.934+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb048152c10 con 0x7fb0480adc80 2023-11-13T11:03:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.935+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0480cbdb0 con 0x7fb0480adc80 2023-11-13T11:03:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.936+0000 7fb03dffb640 1 -- 172.21.15.16:0/3459739515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb038004310 con 0x7fb0480adc80 2023-11-13T11:03:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.938+0000 7fb03dffb640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb02c06ae40 0x7fb02c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.938+0000 7fb03dffb640 1 -- 172.21.15.16:0/3459739515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb038089620 con 0x7fb0480adc80 2023-11-13T11:03:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.938+0000 7fb04d7db640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb02c06ae40 0x7fb02c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.939+0000 7fb04d7db640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb02c06ae40 0x7fb02c06d300 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb044036000 tx=0x7fb04403c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:39.942+0000 7fb03dffb640 1 -- 172.21.15.16:0/3459739515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb038054580 con 0x7fb0480adc80 2023-11-13T11:03:40.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.151+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb04804e4d0 con 0x7fb0480adc80 2023-11-13T11:03:40.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.153+0000 7fb03dffb640 1 -- 172.21.15.16:0/3459739515 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb038057ba0 con 0x7fb0480adc80 2023-11-13T11:03:40.158 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T11:03:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.155+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb02c06ae40 msgr2=0x7fb02c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.155+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb02c06ae40 0x7fb02c06d300 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb044036000 tx=0x7fb04403c040 comp rx=0 tx=0).stop 2023-11-13T11:03:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0480adc80 msgr2=0x7fb048152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0480adc80 0x7fb048152190 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb03800d800 tx=0x7fb03800dcc0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 shutdown_connections 2023-11-13T11:03:40.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb02c06ae40 0x7fb02c06d300 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb048159230 0x7fb0481526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0480adc80 0x7fb048152190 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 --2- 172.21.15.16:0/3459739515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb04806bfb0 0x7fb0480acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 >> 172.21.15.16:0/3459739515 conn(0x7fb0480b04a0 msgr2=0x7fb04806f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:40.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 shutdown_connections 2023-11-13T11:03:40.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.156+0000 7fb04e7dd640 1 -- 172.21.15.16:0/3459739515 wait complete. 2023-11-13T11:03:40.180 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:03:40.180 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:40.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.457+0000 7fc281dd2640 1 Processor -- start 2023-11-13T11:03:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.457+0000 7fc281dd2640 1 -- start start 2023-11-13T11:03:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.459+0000 7fc281dd2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc27c0cb190 0x7fc27c15b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.459+0000 7fc281dd2640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc27c0716b0 0x7fc27c06f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.459+0000 7fc281dd2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c06a190 0x7fc27c06a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.459+0000 7fc281dd2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27c06ab40 con 0x7fc27c0cb190 2023-11-13T11:03:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.459+0000 7fc281dd2640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27c158c20 con 0x7fc27c0716b0 2023-11-13T11:03:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.459+0000 7fc281dd2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0b1830 con 0x7fc27c06a190 2023-11-13T11:03:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.459+0000 7fc280dd0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc27c0cb190 0x7fc27c15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.459+0000 7fc280dd0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc27c0cb190 0x7fc27c15b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60416/0 (socket says 172.21.15.16:60416) 2023-11-13T11:03:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.459+0000 7fc280dd0640 1 -- 172.21.15.16:0/3215590507 learned_addr learned my addr 172.21.15.16:0/3215590507 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.460+0000 7fc27bfff640 1 --2- 172.21.15.16:0/3215590507 >> v2:172.21.15.184:3300/0 conn(0x7fc27c0716b0 0x7fc27c06f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.460+0000 7fc2815d1640 1 --2- 172.21.15.16:0/3215590507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c06a190 0x7fc27c06a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.460+0000 7fc280dd0640 1 -- 172.21.15.16:0/3215590507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c06a190 msgr2=0x7fc27c06a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.460+0000 7fc280dd0640 1 --2- 172.21.15.16:0/3215590507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c06a190 0x7fc27c06a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.460+0000 7fc280dd0640 1 -- 172.21.15.16:0/3215590507 >> v2:172.21.15.184:3300/0 conn(0x7fc27c0716b0 msgr2=0x7fc27c06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.460+0000 7fc280dd0640 1 --2- 172.21.15.16:0/3215590507 >> v2:172.21.15.184:3300/0 conn(0x7fc27c0716b0 0x7fc27c06f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.460+0000 7fc280dd0640 1 -- 172.21.15.16:0/3215590507 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c15b780 con 0x7fc27c0cb190 2023-11-13T11:03:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.460+0000 7fc2815d1640 1 --2- 172.21.15.16:0/3215590507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c06a190 0x7fc27c06a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.461+0000 7fc280dd0640 1 --2- 172.21.15.16:0/3215590507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 0x7fc27c15b190 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0ae3e0 tx=0x7fc27002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe0dea468ef3d76 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.461+0000 7fc27b7fe640 1 -- 172.21.15.16:0/3215590507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270031880 con 0x7fc27c0cb190 2023-11-13T11:03:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.461+0000 7fc27b7fe640 1 -- 172.21.15.16:0/3215590507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2700319e0 con 0x7fc27c0cb190 2023-11-13T11:03:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.461+0000 7fc27b7fe640 1 -- 172.21.15.16:0/3215590507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270031d70 con 0x7fc27c0cb190 2023-11-13T11:03:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.462+0000 7fc281dd2640 1 -- 172.21.15.16:0/3215590507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 msgr2=0x7fc27c15b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.462+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3215590507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 0x7fc27c15b190 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0ae3e0 tx=0x7fc27002b920 comp rx=0 tx=0).stop 2023-11-13T11:03:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.462+0000 7fc281dd2640 1 -- 172.21.15.16:0/3215590507 shutdown_connections 2023-11-13T11:03:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.462+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3215590507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c06a190 0x7fc27c06a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.462+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3215590507 >> v2:172.21.15.184:3300/0 conn(0x7fc27c0716b0 0x7fc27c06f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.462+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3215590507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 0x7fc27c15b190 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.462+0000 7fc281dd2640 1 -- 172.21.15.16:0/3215590507 >> 172.21.15.16:0/3215590507 conn(0x7fc27c0b04a0 msgr2=0x7fc27c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:40.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.462+0000 7fc281dd2640 1 -- 172.21.15.16:0/3215590507 shutdown_connections 2023-11-13T11:03:40.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.462+0000 7fc281dd2640 1 -- 172.21.15.16:0/3215590507 wait complete. 2023-11-13T11:03:40.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.464+0000 7fc281dd2640 1 Processor -- start 2023-11-13T11:03:40.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.464+0000 7fc281dd2640 1 -- start start 2023-11-13T11:03:40.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.465+0000 7fc281dd2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc27c06a190 0x7fc27c06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.465+0000 7fc281dd2640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c0716b0 0x7fc27c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.465+0000 7fc281dd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 0x7fc27c14d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.465+0000 7fc281dd2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0a9080 con 0x7fc27c0cb190 2023-11-13T11:03:40.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.465+0000 7fc281dd2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0a84d0 con 0x7fc27c06a190 2023-11-13T11:03:40.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.465+0000 7fc281dd2640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0a86e0 con 0x7fc27c0716b0 2023-11-13T11:03:40.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.465+0000 7fc2815d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 0x7fc27c14d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.465+0000 7fc2815d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 0x7fc27c14d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60428/0 (socket says 172.21.15.16:60428) 2023-11-13T11:03:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.465+0000 7fc2815d1640 1 -- 172.21.15.16:0/3865015284 learned_addr learned my addr 172.21.15.16:0/3865015284 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc280dd0640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc27c06a190 0x7fc27c06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc27bfff640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c0716b0 0x7fc27c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc2815d1640 1 -- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c0716b0 msgr2=0x7fc27c0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc2815d1640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c0716b0 0x7fc27c0a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc2815d1640 1 -- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc27c06a190 msgr2=0x7fc27c06cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc2815d1640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc27c06a190 0x7fc27c06cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc2815d1640 1 -- 172.21.15.16:0/3865015284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2700094d0 con 0x7fc27c0cb190 2023-11-13T11:03:40.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc2815d1640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 0x7fc27c14d050 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc27400cef0 tx=0x7fc27400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:40.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc2797fa640 1 -- 172.21.15.16:0/3865015284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400d840 con 0x7fc27c0cb190 2023-11-13T11:03:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc2797fa640 1 -- 172.21.15.16:0/3865015284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27400d9a0 con 0x7fc27c0cb190 2023-11-13T11:03:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc2797fa640 1 -- 172.21.15.16:0/3865015284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400dd50 con 0x7fc27c0cb190 2023-11-13T11:03:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.466+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27c0a89c0 con 0x7fc27c0cb190 2023-11-13T11:03:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.467+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c06d430 con 0x7fc27c0cb190 2023-11-13T11:03:40.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.468+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27c0a91c0 con 0x7fc27c0cb190 2023-11-13T11:03:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.469+0000 7fc2797fa640 1 -- 172.21.15.16:0/3865015284 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc27401e3b0 con 0x7fc27c0cb190 2023-11-13T11:03:40.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.469+0000 7fc2797fa640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc26006ae40 0x7fc26006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.470+0000 7fc2797fa640 1 -- 172.21.15.16:0/3865015284 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc27408a190 con 0x7fc27c0cb190 2023-11-13T11:03:40.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.470+0000 7fc280dd0640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc26006ae40 0x7fc26006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.472+0000 7fc280dd0640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc26006ae40 0x7fc26006d300 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc270036000 tx=0x7fc27003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:40.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.474+0000 7fc2797fa640 1 -- 172.21.15.16:0/3865015284 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc27c0a91c0 con 0x7fc27c0cb190 2023-11-13T11:03:40.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.672+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc27c0a91c0 con 0x7fc27c0cb190 2023-11-13T11:03:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.675+0000 7fc2797fa640 1 -- 172.21.15.16:0/3865015284 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16776 (secure 0 0 0) 0x7fc27c0a91c0 con 0x7fc27c0cb190 2023-11-13T11:03:40.680 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:40.681 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":69,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:21.435542+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:03:17.903258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-13T11:03:18.947896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc26006ae40 msgr2=0x7fc26006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc26006ae40 0x7fc26006d300 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc270036000 tx=0x7fc27003c040 comp rx=0 tx=0).stop 2023-11-13T11:03:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 msgr2=0x7fc27c14d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 0x7fc27c14d050 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc27400cef0 tx=0x7fc27400a710 comp rx=0 tx=0).stop 2023-11-13T11:03:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 shutdown_connections 2023-11-13T11:03:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc26006ae40 0x7fc26006d300 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c0cb190 0x7fc27c14d050 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc27c0716b0 0x7fc27c0a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 --2- 172.21.15.16:0/3865015284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc27c06a190 0x7fc27c06cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 >> 172.21.15.16:0/3865015284 conn(0x7fc27c0b04a0 msgr2=0x7fc27c0b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 shutdown_connections 2023-11-13T11:03:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.678+0000 7fc281dd2640 1 -- 172.21.15.16:0/3865015284 wait complete. 2023-11-13T11:03:40.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T11:03:17.903258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T11:03:18.947896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:03:40.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:03:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.931+0000 7f7b732d7640 1 Processor -- start 2023-11-13T11:03:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.931+0000 7f7b732d7640 1 -- start start 2023-11-13T11:03:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.932+0000 7f7b732d7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0d1850 0x7f7b6c0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.932+0000 7f7b732d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b6c0d2d00 0x7f7b6c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.932+0000 7f7b732d7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c0ce0e0 con 0x7f7b6c15ce70 2023-11-13T11:03:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.932+0000 7f7b732d7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6c0ce3c0 con 0x7f7b6c0d2d00 2023-11-13T11:03:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.932+0000 7f7b732d7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c0ce6a0 con 0x7f7b6c0d1850 2023-11-13T11:03:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.933+0000 7f7b72ad6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b6c0d2d00 0x7f7b6c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.933+0000 7f7b72ad6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b6c0d2d00 0x7f7b6c0c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60440/0 (socket says 172.21.15.16:60440) 2023-11-13T11:03:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.933+0000 7f7b72ad6640 1 -- 172.21.15.16:0/1063621910 learned_addr learned my addr 172.21.15.16:0/1063621910 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.933+0000 7f7b72ad6640 1 -- 172.21.15.16:0/1063621910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0d1850 msgr2=0x7f7b6c0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.933+0000 7f7b71ad4640 1 --2- 172.21.15.16:0/1063621910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0d1850 0x7f7b6c0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.933+0000 7f7b72ad6640 1 --2- 172.21.15.16:0/1063621910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0d1850 0x7f7b6c0d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.933+0000 7f7b72ad6640 1 -- 172.21.15.16:0/1063621910 >> v1:172.21.15.184:6789/0 conn(0x7f7b6c15ce70 legacy=0x7f7b6c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.933+0000 7f7b72ad6640 1 -- 172.21.15.16:0/1063621910 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c0ce980 con 0x7f7b6c0d2d00 2023-11-13T11:03:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.934+0000 7f7b72ad6640 1 --2- 172.21.15.16:0/1063621910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d2d00 0x7f7b6c0c9c30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009e50 tx=0x7f7b6402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12280b5e3d64f918 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.935+0000 7f7b712d3640 1 -- 172.21.15.16:0/1063621910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6402ca70 con 0x7f7b6c0d2d00 2023-11-13T11:03:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.935+0000 7f7b712d3640 1 -- 172.21.15.16:0/1063621910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6402cbd0 con 0x7f7b6c0d2d00 2023-11-13T11:03:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.935+0000 7f7b712d3640 1 -- 172.21.15.16:0/1063621910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b640343f0 con 0x7f7b6c0d2d00 2023-11-13T11:03:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.935+0000 7f7b732d7640 1 -- 172.21.15.16:0/1063621910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d2d00 msgr2=0x7f7b6c0c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.935+0000 7f7b732d7640 1 --2- 172.21.15.16:0/1063621910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d2d00 0x7f7b6c0c9c30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009e50 tx=0x7f7b6402bed0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.935+0000 7f7b732d7640 1 -- 172.21.15.16:0/1063621910 shutdown_connections 2023-11-13T11:03:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.935+0000 7f7b732d7640 1 --2- 172.21.15.16:0/1063621910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d2d00 0x7f7b6c0c9c30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.935+0000 7f7b732d7640 1 --2- 172.21.15.16:0/1063621910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0d1850 0x7f7b6c0d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.935+0000 7f7b732d7640 1 -- 172.21.15.16:0/1063621910 >> 172.21.15.16:0/1063621910 conn(0x7f7b6c08c580 msgr2=0x7f7b6c09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.936+0000 7f7b732d7640 1 -- 172.21.15.16:0/1063621910 shutdown_connections 2023-11-13T11:03:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.936+0000 7f7b732d7640 1 -- 172.21.15.16:0/1063621910 wait complete. 2023-11-13T11:03:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.937+0000 7f7b732d7640 1 Processor -- start 2023-11-13T11:03:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.938+0000 7f7b732d7640 1 -- start start 2023-11-13T11:03:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.938+0000 7f7b732d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d1850 0x7f7b6c1f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.938+0000 7f7b732d7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0d2d00 0x7f7b6c0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.938+0000 7f7b732d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c15ce70 0x7f7b6c0c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.938+0000 7f7b732d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c0c9220 con 0x7f7b6c0d1850 2023-11-13T11:03:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.938+0000 7f7b732d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c15e4c0 con 0x7f7b6c15ce70 2023-11-13T11:03:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.938+0000 7f7b732d7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c0c9430 con 0x7f7b6c0d2d00 2023-11-13T11:03:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b722d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d1850 0x7f7b6c1f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b722d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d1850 0x7f7b6c1f3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60454/0 (socket says 172.21.15.16:60454) 2023-11-13T11:03:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b722d5640 1 -- 172.21.15.16:0/475066391 learned_addr learned my addr 172.21.15.16:0/475066391 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b72ad6640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c15ce70 0x7f7b6c0c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b722d5640 1 -- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0d2d00 msgr2=0x7f7b6c0c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b722d5640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0d2d00 0x7f7b6c0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b722d5640 1 -- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c15ce70 msgr2=0x7f7b6c0c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b722d5640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c15ce70 0x7f7b6c0c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b722d5640 1 -- 172.21.15.16:0/475066391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64009b30 con 0x7f7b6c0d1850 2023-11-13T11:03:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.939+0000 7f7b72ad6640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c15ce70 0x7f7b6c0c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.940+0000 7f7b722d5640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d1850 0x7f7b6c1f3410 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800c430 tx=0x7f7b6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.940+0000 7f7b62ffd640 1 -- 172.21.15.16:0/475066391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b68011790 con 0x7f7b6c0d1850 2023-11-13T11:03:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.940+0000 7f7b62ffd640 1 -- 172.21.15.16:0/475066391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b680118f0 con 0x7f7b6c0d1850 2023-11-13T11:03:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.940+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c15e760 con 0x7f7b6c0d1850 2023-11-13T11:03:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.940+0000 7f7b62ffd640 1 -- 172.21.15.16:0/475066391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b68011ce0 con 0x7f7b6c0d1850 2023-11-13T11:03:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.940+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c0c9710 con 0x7f7b6c0d1850 2023-11-13T11:03:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.941+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c15ec80 con 0x7f7b6c0d1850 2023-11-13T11:03:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.942+0000 7f7b62ffd640 1 -- 172.21.15.16:0/475066391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b6801c460 con 0x7f7b6c0d1850 2023-11-13T11:03:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.943+0000 7f7b62ffd640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b5006aec0 0x7f7b5006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.943+0000 7f7b62ffd640 1 -- 172.21.15.16:0/475066391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7b680895d0 con 0x7f7b6c0d1850 2023-11-13T11:03:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.944+0000 7f7b71ad4640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b5006aec0 0x7f7b5006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.945+0000 7f7b71ad4640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b5006aec0 0x7f7b5006d380 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c160050 tx=0x7f7b5c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:40.947+0000 7f7b62ffd640 1 -- 172.21.15.16:0/475066391 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b6c15ec80 con 0x7f7b6c0d1850 2023-11-13T11:03:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.141+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b6c15ec80 con 0x7f7b6c0d1850 2023-11-13T11:03:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.143+0000 7f7b62ffd640 1 -- 172.21.15.16:0/475066391 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b6c15ec80 con 0x7f7b6c0d1850 2023-11-13T11:03:41.148 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:03:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.145+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b5006aec0 msgr2=0x7f7b5006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.145+0000 7f7b732d7640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b5006aec0 0x7f7b5006d380 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c160050 tx=0x7f7b5c008040 comp rx=0 tx=0).stop 2023-11-13T11:03:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.145+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d1850 msgr2=0x7f7b6c1f3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.145+0000 7f7b732d7640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d1850 0x7f7b6c1f3410 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800c430 tx=0x7f7b6800c8f0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.145+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 shutdown_connections 2023-11-13T11:03:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.146+0000 7f7b732d7640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7b5006aec0 0x7f7b5006d380 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.146+0000 7f7b732d7640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c15ce70 0x7f7b6c0c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.146+0000 7f7b732d7640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0d2d00 0x7f7b6c0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.146+0000 7f7b732d7640 1 --2- 172.21.15.16:0/475066391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c0d1850 0x7f7b6c1f3410 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.146+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 >> 172.21.15.16:0/475066391 conn(0x7f7b6c08c580 msgr2=0x7f7b6c0cafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.146+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 shutdown_connections 2023-11-13T11:03:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.146+0000 7f7b732d7640 1 -- 172.21.15.16:0/475066391 wait complete. 2023-11-13T11:03:41.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T11:03:41.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.392+0000 7f5cb8886640 1 Processor -- start 2023-11-13T11:03:41.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.392+0000 7f5cb8886640 1 -- start start 2023-11-13T11:03:41.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.394+0000 7f5cb8886640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb415b8e0 0x7f5cb406c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.394+0000 7f5cb8886640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5cb415bcc0 0x7f5cb40cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.394+0000 7f5cb8886640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb406b860 con 0x7f5cb40ce090 2023-11-13T11:03:41.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.394+0000 7f5cb8886640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb40ce540 con 0x7f5cb415bcc0 2023-11-13T11:03:41.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.394+0000 7f5cb8886640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb406bb40 con 0x7f5cb415b8e0 2023-11-13T11:03:41.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.395+0000 7f5cb359e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5cb40ce090 0x7f5cb40cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37998/0 (socket says 172.21.15.16:37998) 2023-11-13T11:03:41.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.395+0000 7f5cb359e640 1 -- 172.21.15.16:0/2384696745 learned_addr learned my addr 172.21.15.16:0/2384696745 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.395+0000 7f5cabfff640 1 --2- 172.21.15.16:0/2384696745 >> v2:172.21.15.184:3300/0 conn(0x7f5cb415bcc0 0x7f5cb40cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.395+0000 7f5cb2d9d640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb415b8e0 0x7f5cb406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.395+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014590472 0 0) 0x7f5cb406b860 con 0x7f5cb40ce090 2023-11-13T11:03:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.395+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca0003520 con 0x7f5cb40ce090 2023-11-13T11:03:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179015347 0 0) 0x7f5ca0003520 con 0x7f5cb40ce090 2023-11-13T11:03:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb415b8e0 msgr2=0x7f5cb406c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb415b8e0 0x7f5cb406c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 >> v2:172.21.15.184:3300/0 conn(0x7f5cb415bcc0 msgr2=0x7f5cb40cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 --2- 172.21.15.16:0/2384696745 >> v2:172.21.15.184:3300/0 conn(0x7f5cb415bcc0 0x7f5cb40cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb406be20 con 0x7f5cb40ce090 2023-11-13T11:03:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f5cac0019f0 con 0x7f5cb40ce090 2023-11-13T11:03:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 >> v1:172.21.15.16:6789/0 conn(0x7f5cb40ce090 legacy=0x7f5cb40cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb415b8e0 0x7f5ca0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5ca00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca0004d00 0x7f5ca0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb40ce540 con 0x7f5ca0003f40 2023-11-13T11:03:41.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb406bb40 con 0x7f5ca0004d00 2023-11-13T11:03:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca000b980 con 0x7f5cb415b8e0 2023-11-13T11:03:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.396+0000 7f5cabfff640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5ca00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.397+0000 7f5cabfff640 1 -- 172.21.15.16:0/2384696745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb415b8e0 msgr2=0x7f5ca0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:41.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.397+0000 7f5cabfff640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb415b8e0 0x7f5ca0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.397+0000 7f5cabfff640 1 -- 172.21.15.16:0/2384696745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca0004d00 msgr2=0x7f5ca0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:41.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.397+0000 7f5cabfff640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ca0004d00 0x7f5ca0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.397+0000 7f5cabfff640 1 -- 172.21.15.16:0/2384696745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca0003520 con 0x7f5ca0003f40 2023-11-13T11:03:41.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.398+0000 7f5cabfff640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5ca00047c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009af0 tx=0x7f5ca00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5428af38a56decc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.398+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c02c710 con 0x7f5ca0003f40 2023-11-13T11:03:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.398+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c02c870 con 0x7f5ca0003f40 2023-11-13T11:03:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.398+0000 7f5cb259c640 1 -- 172.21.15.16:0/2384696745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c02cc00 con 0x7f5ca0003f40 2023-11-13T11:03:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.399+0000 7f5cb8886640 1 -- 172.21.15.16:0/2384696745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 msgr2=0x7f5ca00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.399+0000 7f5cb8886640 1 --2- 172.21.15.16:0/2384696745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5ca00047c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009af0 tx=0x7f5ca00057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.399+0000 7f5cb359e640 1 -- 172.21.15.16:0/2384696745 reap_dead start 2023-11-13T11:03:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.399+0000 7f5cb8886640 1 -- 172.21.15.16:0/2384696745 shutdown_connections 2023-11-13T11:03:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.399+0000 7f5cb8886640 1 -- 172.21.15.16:0/2384696745 >> 172.21.15.16:0/2384696745 conn(0x7f5cb40b0970 msgr2=0x7f5cb40b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.399+0000 7f5cb8886640 1 -- 172.21.15.16:0/2384696745 shutdown_connections 2023-11-13T11:03:41.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.399+0000 7f5cb8886640 1 -- 172.21.15.16:0/2384696745 wait complete. 2023-11-13T11:03:41.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.401+0000 7f5cb8886640 1 Processor -- start 2023-11-13T11:03:41.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.401+0000 7f5cb8886640 1 -- start start 2023-11-13T11:03:41.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.402+0000 7f5cb8886640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5cb414aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.402+0000 7f5cb8886640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb414be30 0x7f5cb414c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.402+0000 7f5cb8886640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb41537d0 0x7f5cb4153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.402+0000 7f5cb8886640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4154190 con 0x7f5ca0003f40 2023-11-13T11:03:41.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.402+0000 7f5cb8886640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4156380 con 0x7f5cb414be30 2023-11-13T11:03:41.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.402+0000 7f5cb8886640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4156660 con 0x7f5cb41537d0 2023-11-13T11:03:41.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.402+0000 7f5cb2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5cb414aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.402+0000 7f5cb2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5cb414aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60480/0 (socket says 172.21.15.16:60480) 2023-11-13T11:03:41.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.402+0000 7f5cb2d9d640 1 -- 172.21.15.16:0/1690324694 learned_addr learned my addr 172.21.15.16:0/1690324694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:41.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cb2d9d640 1 -- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb41537d0 msgr2=0x7f5cb4153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cb2d9d640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb41537d0 0x7f5cb4153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cabfff640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb414be30 0x7f5cb414c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cb2d9d640 1 -- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb414be30 msgr2=0x7f5cb414c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cb2d9d640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb414be30 0x7f5cb414c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cb2d9d640 1 -- 172.21.15.16:0/1690324694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb4156940 con 0x7f5ca0003f40 2023-11-13T11:03:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cb2d9d640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5cb414aca0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e7c0 tx=0x7f5ca400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cab7fe640 1 -- 172.21.15.16:0/1690324694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400eed0 con 0x7f5ca0003f40 2023-11-13T11:03:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cab7fe640 1 -- 172.21.15.16:0/1690324694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca400c940 con 0x7f5ca0003f40 2023-11-13T11:03:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cab7fe640 1 -- 172.21.15.16:0/1690324694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400ccd0 con 0x7f5ca0003f40 2023-11-13T11:03:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.403+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb4156c20 con 0x7f5ca0003f40 2023-11-13T11:03:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.404+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb4157420 con 0x7f5ca0003f40 2023-11-13T11:03:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.405+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb404e4d0 con 0x7f5ca0003f40 2023-11-13T11:03:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.406+0000 7f5cab7fe640 1 -- 172.21.15.16:0/1690324694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ca4013470 con 0x7f5ca0003f40 2023-11-13T11:03:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.406+0000 7f5cab7fe640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5c9406ae40 0x7f5c9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.407+0000 7f5cab7fe640 1 -- 172.21.15.16:0/1690324694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5ca40897a0 con 0x7f5ca0003f40 2023-11-13T11:03:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.407+0000 7f5cabfff640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5c9406ae40 0x7f5c9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.408+0000 7f5cabfff640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5c9406ae40 0x7f5c9406d300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c036000 tx=0x7f5c9c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.411+0000 7f5cab7fe640 1 -- 172.21.15.16:0/1690324694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ca4054700 con 0x7f5ca0003f40 2023-11-13T11:03:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.604+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cb40b0d80 con 0x7f5ca0003f40 2023-11-13T11:03:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.605+0000 7f5cab7fe640 1 -- 172.21.15.16:0/1690324694 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ca4057d20 con 0x7f5ca0003f40 2023-11-13T11:03:41.611 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T11:03:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.607+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5c9406ae40 msgr2=0x7f5c9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.607+0000 7f5cb8886640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5c9406ae40 0x7f5c9406d300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c036000 tx=0x7f5c9c030000 comp rx=0 tx=0).stop 2023-11-13T11:03:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.607+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 msgr2=0x7f5cb414aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.607+0000 7f5cb8886640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5cb414aca0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e7c0 tx=0x7f5ca400ec80 comp rx=0 tx=0).stop 2023-11-13T11:03:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.608+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 shutdown_connections 2023-11-13T11:03:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.608+0000 7f5cb8886640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5c9406ae40 0x7f5c9406d300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.608+0000 7f5cb8886640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5cb41537d0 0x7f5cb4153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.608+0000 7f5cb8886640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb414be30 0x7f5cb414c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.608+0000 7f5cb8886640 1 --2- 172.21.15.16:0/1690324694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0003f40 0x7f5cb414aca0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.608+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 >> 172.21.15.16:0/1690324694 conn(0x7f5cb40b0970 msgr2=0x7f5cb40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.608+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 shutdown_connections 2023-11-13T11:03:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.608+0000 7f5cb8886640 1 -- 172.21.15.16:0/1690324694 wait complete. 2023-11-13T11:03:41.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T11:03:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.863+0000 7f3164a85640 1 Processor -- start 2023-11-13T11:03:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.863+0000 7f3164a85640 1 -- start start 2023-11-13T11:03:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.865+0000 7f3164a85640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f316015b8e0 0x7f316006c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.865+0000 7f3164a85640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316006b860 con 0x7f31600ce260 2023-11-13T11:03:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.865+0000 7f3164a85640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316006bb40 con 0x7f316015bcc0 2023-11-13T11:03:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.865+0000 7f3164a85640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316006be20 con 0x7f316015b8e0 2023-11-13T11:03:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.866+0000 7f315ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f31600ce260 0x7f31600cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38004/0 (socket says 172.21.15.16:38004) 2023-11-13T11:03:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.866+0000 7f315ffff640 1 -- 172.21.15.16:0/1428555161 learned_addr learned my addr 172.21.15.16:0/1428555161 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:41.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.866+0000 7f315f7fe640 1 --2- 172.21.15.16:0/1428555161 >> v2:172.21.15.184:3300/0 conn(0x7f316015b8e0 0x7f316006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.866+0000 7f315effd640 1 -- 172.21.15.16:0/1428555161 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320595510 0 0) 0x7f316006b860 con 0x7f31600ce260 2023-11-13T11:03:41.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.866+0000 7f315effd640 1 -- 172.21.15.16:0/1428555161 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3140003520 con 0x7f31600ce260 2023-11-13T11:03:41.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.867+0000 7f315f7fe640 1 -- 172.21.15.16:0/1428555161 >> v1:172.21.15.184:6789/0 conn(0x7f316015bcc0 legacy=0x7f31600cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.867+0000 7f315effd640 1 -- 172.21.15.16:0/1428555161 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1349202339 0 0) 0x7f3140003520 con 0x7f31600ce260 2023-11-13T11:03:41.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.867+0000 7f315f7fe640 1 -- 172.21.15.16:0/1428555161 >> v1:172.21.15.16:6789/0 conn(0x7f31600ce260 legacy=0x7f31600cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.867+0000 7f315f7fe640 1 -- 172.21.15.16:0/1428555161 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316006c100 con 0x7f316015b8e0 2023-11-13T11:03:41.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.868+0000 7f315f7fe640 1 --2- 172.21.15.16:0/1428555161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316015b8e0 0x7f316006c900 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f31500097a0 tx=0x7f315002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0510a6c729f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:41.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.869+0000 7f315effd640 1 -- 172.21.15.16:0/1428555161 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315002f050 con 0x7f316015b8e0 2023-11-13T11:03:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.869+0000 7f315effd640 1 -- 172.21.15.16:0/1428555161 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3150031500 con 0x7f316015b8e0 2023-11-13T11:03:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.869+0000 7f315effd640 1 -- 172.21.15.16:0/1428555161 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31580019f0 con 0x7f316015b8e0 2023-11-13T11:03:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.869+0000 7f3164a85640 1 -- 172.21.15.16:0/1428555161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316015b8e0 msgr2=0x7f316006c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.869+0000 7f3164a85640 1 --2- 172.21.15.16:0/1428555161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316015b8e0 0x7f316006c900 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f31500097a0 tx=0x7f315002b820 comp rx=0 tx=0).stop 2023-11-13T11:03:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.869+0000 7f3164a85640 1 -- 172.21.15.16:0/1428555161 shutdown_connections 2023-11-13T11:03:41.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.869+0000 7f3164a85640 1 --2- 172.21.15.16:0/1428555161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316015b8e0 0x7f316006c900 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.869+0000 7f3164a85640 1 -- 172.21.15.16:0/1428555161 >> 172.21.15.16:0/1428555161 conn(0x7f31600b0970 msgr2=0x7f31600b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:41.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.870+0000 7f3164a85640 1 -- 172.21.15.16:0/1428555161 shutdown_connections 2023-11-13T11:03:41.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.870+0000 7f3164a85640 1 -- 172.21.15.16:0/1428555161 wait complete. 2023-11-13T11:03:41.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.871+0000 7f3164a85640 1 Processor -- start 2023-11-13T11:03:41.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.872+0000 7f3164a85640 1 -- start start 2023-11-13T11:03:41.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3164a85640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31600ce260 0x7f3160146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3164a85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316015b8e0 0x7f31601555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3164a85640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3160153050 0x7f31600ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3164a85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3160146650 con 0x7f316015b8e0 2023-11-13T11:03:41.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3164a85640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3160154550 con 0x7f31600ce260 2023-11-13T11:03:41.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3164a85640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3160154830 con 0x7f3160153050 2023-11-13T11:03:41.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3156ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316015b8e0 0x7f31601555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3156ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316015b8e0 0x7f31601555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60484/0 (socket says 172.21.15.16:60484) 2023-11-13T11:03:41.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3156ffd640 1 -- 172.21.15.16:0/2029752675 learned_addr learned my addr 172.21.15.16:0/2029752675 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:41.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f315f7fe640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31600ce260 0x7f3160146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3156ffd640 1 -- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3160153050 msgr2=0x7f31600ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f315ffff640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3160153050 0x7f31600ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3156ffd640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3160153050 0x7f31600ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3156ffd640 1 -- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31600ce260 msgr2=0x7f3160146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:41.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3156ffd640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31600ce260 0x7f3160146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:41.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f3156ffd640 1 -- 172.21.15.16:0/2029752675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3150009400 con 0x7f316015b8e0 2023-11-13T11:03:41.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.873+0000 7f315f7fe640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31600ce260 0x7f3160146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:41.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.874+0000 7f3156ffd640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316015b8e0 0x7f31601555f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f314800c400 tx=0x7f314800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:41.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.874+0000 7f315cff9640 1 -- 172.21.15.16:0/2029752675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31480052d0 con 0x7f316015b8e0 2023-11-13T11:03:41.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.874+0000 7f315cff9640 1 -- 172.21.15.16:0/2029752675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3148005430 con 0x7f316015b8e0 2023-11-13T11:03:41.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.874+0000 7f315cff9640 1 -- 172.21.15.16:0/2029752675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148003830 con 0x7f316015b8e0 2023-11-13T11:03:41.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.874+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3160154b10 con 0x7f316015b8e0 2023-11-13T11:03:41.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.874+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31601550d0 con 0x7f316015b8e0 2023-11-13T11:03:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.876+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316004e4d0 con 0x7f316015b8e0 2023-11-13T11:03:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.876+0000 7f315cff9640 1 -- 172.21.15.16:0/2029752675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3148003990 con 0x7f316015b8e0 2023-11-13T11:03:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.877+0000 7f315cff9640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f314406ae40 0x7f314406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.877+0000 7f315cff9640 1 -- 172.21.15.16:0/2029752675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3148088df0 con 0x7f316015b8e0 2023-11-13T11:03:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.878+0000 7f315f7fe640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f314406ae40 0x7f314406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.879+0000 7f315f7fe640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f314406ae40 0x7f314406d300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3150035000 tx=0x7f3150008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:41.882+0000 7f315cff9640 1 -- 172.21.15.16:0/2029752675 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3148053c40 con 0x7f316015b8e0 2023-11-13T11:03:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.078+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f31600b18c0 con 0x7f316015b8e0 2023-11-13T11:03:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.078+0000 7f315cff9640 1 -- 172.21.15.16:0/2029752675 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3148074e60 con 0x7f316015b8e0 2023-11-13T11:03:42.084 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T11:03:42.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f314406ae40 msgr2=0x7f314406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f314406ae40 0x7f314406d300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3150035000 tx=0x7f3150008030 comp rx=0 tx=0).stop 2023-11-13T11:03:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316015b8e0 msgr2=0x7f31601555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316015b8e0 0x7f31601555f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f314800c400 tx=0x7f314800c8c0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 shutdown_connections 2023-11-13T11:03:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f314406ae40 0x7f314406d300 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3160153050 0x7f31600ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316015b8e0 0x7f31601555f0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 --2- 172.21.15.16:0/2029752675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31600ce260 0x7f3160146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 >> 172.21.15.16:0/2029752675 conn(0x7f31600b0970 msgr2=0x7f316015a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:42.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.081+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 shutdown_connections 2023-11-13T11:03:42.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.082+0000 7f3164a85640 1 -- 172.21.15.16:0/2029752675 wait complete. 2023-11-13T11:03:42.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-13T11:03:42.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.337+0000 7f642e8df640 1 Processor -- start 2023-11-13T11:03:42.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.338+0000 7f642e8df640 1 -- start start 2023-11-13T11:03:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.339+0000 7f642e8df640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f64280d2ae0 0x7f64280d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.339+0000 7f642e8df640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64280b3cf0 con 0x7f642815cfc0 2023-11-13T11:03:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.339+0000 7f642e8df640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64280d3480 con 0x7f64280b29d0 2023-11-13T11:03:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.339+0000 7f642e8df640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64280b3550 con 0x7f64280d2ae0 2023-11-13T11:03:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.339+0000 7f642d8dd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f642815cfc0 0x7f64280cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38016/0 (socket says 172.21.15.16:38016) 2023-11-13T11:03:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.339+0000 7f642d8dd640 1 -- 172.21.15.16:0/396916434 learned_addr learned my addr 172.21.15.16:0/396916434 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:42.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642d0dc640 1 --2- 172.21.15.16:0/396916434 >> v2:172.21.15.184:3300/0 conn(0x7f64280d2ae0 0x7f64280d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:42.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52395799 0 0) 0x7f64280b3cf0 con 0x7f642815cfc0 2023-11-13T11:03:42.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6414003520 con 0x7f642815cfc0 2023-11-13T11:03:42.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1165593616 0 0) 0x7f6414003520 con 0x7f642815cfc0 2023-11-13T11:03:42.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 >> v2:172.21.15.184:3300/0 conn(0x7f64280d2ae0 msgr2=0x7f64280d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:42.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 --2- 172.21.15.16:0/396916434 >> v2:172.21.15.184:3300/0 conn(0x7f64280d2ae0 0x7f64280d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 >> v1:172.21.15.184:6789/0 conn(0x7f64280b29d0 legacy=0x7f64280b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:42.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280cde80 con 0x7f642815cfc0 2023-11-13T11:03:42.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f64240019f0 con 0x7f642815cfc0 2023-11-13T11:03:42.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 >> v1:172.21.15.16:6789/0 conn(0x7f642815cfc0 legacy=0x7f64280cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:42.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64140039c0 0x7f6414003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:42.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140047c0 0x7f6414004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:42.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414005250 0x7f641400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:42.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64280b3550 con 0x7f64140039c0 2023-11-13T11:03:42.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641400c1a0 con 0x7f64140047c0 2023-11-13T11:03:42.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641400c360 con 0x7f6414005250 2023-11-13T11:03:42.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.340+0000 7f642d8dd640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64140039c0 0x7f6414003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642d0dc640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140047c0 0x7f6414004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642d8dd640 1 -- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414005250 msgr2=0x7f641400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642e0de640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414005250 0x7f641400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642d8dd640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414005250 0x7f641400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642d8dd640 1 -- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140047c0 msgr2=0x7f6414004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642d8dd640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140047c0 0x7f6414004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642d8dd640 1 -- 172.21.15.16:0/396916434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6414003520 con 0x7f64140039c0 2023-11-13T11:03:42.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642e0de640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6414005250 0x7f641400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:42.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642d0dc640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140047c0 0x7f6414004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:42.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.341+0000 7f642e0de640 1 -- 172.21.15.16:0/396916434 reap_dead start 2023-11-13T11:03:42.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.342+0000 7f642d8dd640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64140039c0 0x7f6414003dc0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f642400a810 tx=0x7f64140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a428a170be5df198 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.342+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424034630 con 0x7f64140039c0 2023-11-13T11:03:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.342+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642402f070 con 0x7f64140039c0 2023-11-13T11:03:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.342+0000 7f642c8db640 1 -- 172.21.15.16:0/396916434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424034a10 con 0x7f64140039c0 2023-11-13T11:03:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.343+0000 7f642e8df640 1 -- 172.21.15.16:0/396916434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64140039c0 msgr2=0x7f6414003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.343+0000 7f642e8df640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64140039c0 0x7f6414003dc0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f642400a810 tx=0x7f64140057f0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.343+0000 7f642e8df640 1 -- 172.21.15.16:0/396916434 shutdown_connections 2023-11-13T11:03:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.343+0000 7f642e8df640 1 --2- 172.21.15.16:0/396916434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64140039c0 0x7f6414003dc0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.343+0000 7f642e8df640 1 -- 172.21.15.16:0/396916434 >> 172.21.15.16:0/396916434 conn(0x7f642808c570 msgr2=0x7f642809be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.343+0000 7f642e8df640 1 -- 172.21.15.16:0/396916434 shutdown_connections 2023-11-13T11:03:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.343+0000 7f642e8df640 1 -- 172.21.15.16:0/396916434 wait complete. 2023-11-13T11:03:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.345+0000 7f642e8df640 1 Processor -- start 2023-11-13T11:03:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.345+0000 7f642e8df640 1 -- start start 2023-11-13T11:03:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.346+0000 7f642e8df640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140039c0 0x7f64280b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.346+0000 7f642e8df640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64280b7c00 0x7f64280b8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.346+0000 7f642e8df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64280b5930 0x7f64280b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:42.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.346+0000 7f642e8df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64280c7f90 con 0x7f64280b5930 2023-11-13T11:03:42.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.346+0000 7f642e8df640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64280b85c0 con 0x7f64140039c0 2023-11-13T11:03:42.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.346+0000 7f642e8df640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6428158d20 con 0x7f64280b7c00 2023-11-13T11:03:42.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.346+0000 7f642e0de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64280b5930 0x7f64280b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:42.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.346+0000 7f642e0de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64280b5930 0x7f64280b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60508/0 (socket says 172.21.15.16:60508) 2023-11-13T11:03:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.346+0000 7f642e0de640 1 -- 172.21.15.16:0/3190866493 learned_addr learned my addr 172.21.15.16:0/3190866493 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f642d8dd640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140039c0 0x7f64280b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f642d0dc640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64280b7c00 0x7f64280b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f642e0de640 1 -- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64280b7c00 msgr2=0x7f64280b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f642e0de640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64280b7c00 0x7f64280b8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f642e0de640 1 -- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140039c0 msgr2=0x7f64280b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f642e0de640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140039c0 0x7f64280b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f642e0de640 1 -- 172.21.15.16:0/3190866493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280c70c0 con 0x7f64280b5930 2023-11-13T11:03:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f642d8dd640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140039c0 0x7f64280b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f642e0de640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64280b5930 0x7f64280b5db0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f64280cd540 tx=0x7f642000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f641e7fc640 1 -- 172.21.15.16:0/3190866493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420005440 con 0x7f64280b5930 2023-11-13T11:03:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.347+0000 7f641e7fc640 1 -- 172.21.15.16:0/3190866493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6420007020 con 0x7f64280b5930 2023-11-13T11:03:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.348+0000 7f641e7fc640 1 -- 172.21.15.16:0/3190866493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64200073d0 con 0x7f64280b5930 2023-11-13T11:03:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.348+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64280c7310 con 0x7f64280b5930 2023-11-13T11:03:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.348+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64280c78d0 con 0x7f64280b5930 2023-11-13T11:03:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.349+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642804e4d0 con 0x7f64280b5930 2023-11-13T11:03:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.350+0000 7f641e7fc640 1 -- 172.21.15.16:0/3190866493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6420007530 con 0x7f64280b5930 2023-11-13T11:03:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.351+0000 7f641e7fc640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640c06ae40 0x7f640c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.351+0000 7f641e7fc640 1 -- 172.21.15.16:0/3190866493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6420088b60 con 0x7f64280b5930 2023-11-13T11:03:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.351+0000 7f642d8dd640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640c06ae40 0x7f640c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.352+0000 7f642d8dd640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640c06ae40 0x7f640c06d300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6424034000 tx=0x7f642402f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.355+0000 7f641e7fc640 1 -- 172.21.15.16:0/3190866493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6420053a30 con 0x7f64280b5930 2023-11-13T11:03:42.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.560+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f64280b8b00 con 0x7f64280b5930 2023-11-13T11:03:42.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.561+0000 7f641e7fc640 1 -- 172.21.15.16:0/3190866493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6420007cf0 con 0x7f64280b5930 2023-11-13T11:03:42.567 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-13T11:03:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640c06ae40 msgr2=0x7f640c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:42.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640c06ae40 0x7f640c06d300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6424034000 tx=0x7f642402f000 comp rx=0 tx=0).stop 2023-11-13T11:03:42.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64280b5930 msgr2=0x7f64280b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:42.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64280b5930 0x7f64280b5db0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f64280cd540 tx=0x7f642000d890 comp rx=0 tx=0).stop 2023-11-13T11:03:42.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 shutdown_connections 2023-11-13T11:03:42.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f640c06ae40 0x7f640c06d300 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64280b5930 0x7f64280b5db0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f64280b7c00 0x7f64280b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 --2- 172.21.15.16:0/3190866493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64140039c0 0x7f64280b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 >> 172.21.15.16:0/3190866493 conn(0x7f642808c570 msgr2=0x7f64280d1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 shutdown_connections 2023-11-13T11:03:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:42.564+0000 7f642e8df640 1 -- 172.21.15.16:0/3190866493 wait complete. 2023-11-13T11:03:42.590 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-13T11:03:42.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:42.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-13T11:03:42.683 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[117148]: fuse finished with error 0 and tester_r 0 2023-11-13T11:03:42.692 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:03:48.794 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:03:48.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:48.795 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-13T11:03:48.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2023-11-13T11:03:49.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.093+0000 7f7ced92e640 1 Processor -- start 2023-11-13T11:03:49.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.093+0000 7f7ced92e640 1 -- start start 2023-11-13T11:03:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.094+0000 7f7ced92e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ce8158fa0 0x7f7ce80cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7ced92e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce80b1880 0x7f7ce8071300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7ced92e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce80b1d10 con 0x7f7ce806bf50 2023-11-13T11:03:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7ced92e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce80b1ff0 con 0x7f7ce8158fa0 2023-11-13T11:03:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7ced92e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80b22d0 con 0x7f7ce80b1880 2023-11-13T11:03:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7cec92c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ce8158fa0 0x7f7ce80cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7cec92c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ce8158fa0 0x7f7ce80cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36584/0 (socket says 172.21.15.16:36584) 2023-11-13T11:03:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7cec92c640 1 -- 172.21.15.16:0/4261554086 learned_addr learned my addr 172.21.15.16:0/4261554086 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7ced12d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7ce806bf50 0x7f7ce806c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38406/0 (socket says 172.21.15.16:38406) 2023-11-13T11:03:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/4261554086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce80b1880 0x7f7ce8071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7cec92c640 1 -- 172.21.15.16:0/4261554086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce80b1880 msgr2=0x7f7ce8071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7cec92c640 1 --2- 172.21.15.16:0/4261554086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce80b1880 0x7f7ce8071300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7cec92c640 1 -- 172.21.15.16:0/4261554086 >> v1:172.21.15.16:6789/0 conn(0x7f7ce806bf50 legacy=0x7f7ce806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7cec92c640 1 -- 172.21.15.16:0/4261554086 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce80b25b0 con 0x7f7ce8158fa0 2023-11-13T11:03:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.095+0000 7f7ce77fe640 1 -- 172.21.15.16:0/4261554086 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603607465 0 0) 0x7f7ce80b1d10 con 0x7f7ce806bf50 2023-11-13T11:03:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.097+0000 7f7cec92c640 1 --2- 172.21.15.16:0/4261554086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 0x7f7ce80cc1a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00bb10 tx=0x7f7cdc02e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1861342e49cd3b9a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.097+0000 7f7ce77fe640 1 -- 172.21.15.16:0/4261554086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc036440 con 0x7f7ce8158fa0 2023-11-13T11:03:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.097+0000 7f7ce77fe640 1 -- 172.21.15.16:0/4261554086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc0365a0 con 0x7f7ce8158fa0 2023-11-13T11:03:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.097+0000 7f7ce77fe640 1 -- 172.21.15.16:0/4261554086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc036930 con 0x7f7ce8158fa0 2023-11-13T11:03:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.097+0000 7f7ced92e640 1 -- 172.21.15.16:0/4261554086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 msgr2=0x7f7ce80cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.097+0000 7f7ced92e640 1 --2- 172.21.15.16:0/4261554086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 0x7f7ce80cc1a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00bb10 tx=0x7f7cdc02e150 comp rx=0 tx=0).stop 2023-11-13T11:03:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.098+0000 7f7ced92e640 1 -- 172.21.15.16:0/4261554086 shutdown_connections 2023-11-13T11:03:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.098+0000 7f7ced92e640 1 --2- 172.21.15.16:0/4261554086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce80b1880 0x7f7ce8071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.098+0000 7f7ced92e640 1 --2- 172.21.15.16:0/4261554086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 0x7f7ce80cc1a0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.098+0000 7f7ced92e640 1 -- 172.21.15.16:0/4261554086 >> 172.21.15.16:0/4261554086 conn(0x7f7ce80b0970 msgr2=0x7f7ce80b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.098+0000 7f7ced92e640 1 -- 172.21.15.16:0/4261554086 shutdown_connections 2023-11-13T11:03:49.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.098+0000 7f7ced92e640 1 -- 172.21.15.16:0/4261554086 wait complete. 2023-11-13T11:03:49.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.099+0000 7f7ced92e640 1 Processor -- start 2023-11-13T11:03:49.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.100+0000 7f7ced92e640 1 -- start start 2023-11-13T11:03:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ced92e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce806bf50 0x7f7ce8149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ced92e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 0x7f7ce80cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ced92e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce80ccdc0 0x7f7ce815cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ced92e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce815c0a0 con 0x7f7ce8158fa0 2023-11-13T11:03:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ced92e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80ad1d0 con 0x7f7ce80ccdc0 2023-11-13T11:03:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ced92e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce815b770 con 0x7f7ce806bf50 2023-11-13T11:03:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ce7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 0x7f7ce80cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ce7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 0x7f7ce80cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36588/0 (socket says 172.21.15.16:36588) 2023-11-13T11:03:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ce7fff640 1 -- 172.21.15.16:0/2488391877 learned_addr learned my addr 172.21.15.16:0/2488391877 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7cec92c640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce806bf50 0x7f7ce8149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ce7fff640 1 -- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce806bf50 msgr2=0x7f7ce8149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce806bf50 0x7f7ce8149140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ce7fff640 1 -- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce80ccdc0 msgr2=0x7f7ce815cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce80ccdc0 0x7f7ce815cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.101+0000 7f7ce7fff640 1 -- 172.21.15.16:0/2488391877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc00b770 con 0x7f7ce8158fa0 2023-11-13T11:03:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.102+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 0x7f7ce80cc880 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400d800 tx=0x7f7cd400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.102+0000 7f7ce57fa640 1 -- 172.21.15.16:0/2488391877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd400b6c0 con 0x7f7ce8158fa0 2023-11-13T11:03:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.102+0000 7f7ce57fa640 1 -- 172.21.15.16:0/2488391877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd400b820 con 0x7f7ce8158fa0 2023-11-13T11:03:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.102+0000 7f7ced92e640 1 -- 172.21.15.16:0/2488391877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce815ba50 con 0x7f7ce8158fa0 2023-11-13T11:03:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.102+0000 7f7ce57fa640 1 -- 172.21.15.16:0/2488391877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd400bbb0 con 0x7f7ce8158fa0 2023-11-13T11:03:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.102+0000 7f7ced92e640 1 -- 172.21.15.16:0/2488391877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce815b480 con 0x7f7ce8158fa0 2023-11-13T11:03:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.105+0000 7f7ce57fa640 1 -- 172.21.15.16:0/2488391877 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7cd400bd10 con 0x7f7ce8158fa0 2023-11-13T11:03:49.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.106+0000 7f7ce57fa640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ccc06ae70 0x7f7ccc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.106+0000 7f7ce57fa640 1 -- 172.21.15.16:0/2488391877 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7cd4089d70 con 0x7f7ce8158fa0 2023-11-13T11:03:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.106+0000 7f7cf1800740 1 Processor -- start 2023-11-13T11:03:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.106+0000 7f7cf1800740 1 -- start start 2023-11-13T11:03:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.106+0000 7f7cec92c640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ccc06ae70 0x7f7ccc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.107+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef35210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.107+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef62650 0x556e7ef65000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.107+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef35750 0x556e7ef65540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.107+0000 7f7cf1800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7eea7e90 con 0x556e7ef62270 2023-11-13T11:03:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.107+0000 7f7cf1800740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7eea5580 con 0x556e7ef35750 2023-11-13T11:03:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.107+0000 7f7cf1800740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e7eea0e70 con 0x556e7ef62650 2023-11-13T11:03:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef35210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef35210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36594/0 (socket says 172.21.15.16:36594) 2023-11-13T11:03:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 -- 172.21.15.16:0/3642824463 learned_addr learned my addr 172.21.15.16:0/3642824463 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ccc06ae70 0x7f7ccc06d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc038000 tx=0x7f7cdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/3642824463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef62650 0x556e7ef65000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 -- 172.21.15.16:0/3642824463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef62650 msgr2=0x556e7ef65000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 --2- 172.21.15.16:0/3642824463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef62650 0x556e7ef65000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 -- 172.21.15.16:0/3642824463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef35750 msgr2=0x556e7ef65540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7ced12d640 1 --2- 172.21.15.16:0/3642824463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef35750 0x556e7ef65540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 --2- 172.21.15.16:0/3642824463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef35750 0x556e7ef65540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 -- 172.21.15.16:0/3642824463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e7ee6b020 con 0x556e7ef62270 2023-11-13T11:03:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.108+0000 7f7cec92c640 1 --2- 172.21.15.16:0/3642824463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef35210 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00bc30 tx=0x7f7cdc033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d15c56e6c7fcd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.109+0000 7f7cd27fc640 1 -- 172.21.15.16:0/3642824463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc046040 con 0x556e7ef62270 2023-11-13T11:03:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.109+0000 7f7cd27fc640 1 -- 172.21.15.16:0/3642824463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc041740 con 0x556e7ef62270 2023-11-13T11:03:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.109+0000 7f7cd27fc640 1 -- 172.21.15.16:0/3642824463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc041ab0 con 0x556e7ef62270 2023-11-13T11:03:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.110+0000 7f7cf1800740 1 -- 172.21.15.16:0/3642824463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 msgr2=0x556e7ef35210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.110+0000 7f7cf1800740 1 --2- 172.21.15.16:0/3642824463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef35210 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00bc30 tx=0x7f7cdc033000 comp rx=0 tx=0).stop 2023-11-13T11:03:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.110+0000 7f7cf1800740 1 -- 172.21.15.16:0/3642824463 shutdown_connections 2023-11-13T11:03:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.110+0000 7f7cf1800740 1 --2- 172.21.15.16:0/3642824463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef35750 0x556e7ef65540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.110+0000 7f7cf1800740 1 --2- 172.21.15.16:0/3642824463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef62650 0x556e7ef65000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.110+0000 7f7cf1800740 1 --2- 172.21.15.16:0/3642824463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef35210 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.110+0000 7f7cf1800740 1 -- 172.21.15.16:0/3642824463 >> 172.21.15.16:0/3642824463 conn(0x556e7ef37480 msgr2=0x556e7ef80000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.110+0000 7f7cf1800740 1 -- 172.21.15.16:0/3642824463 shutdown_connections 2023-11-13T11:03:49.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.110+0000 7f7cf1800740 1 -- 172.21.15.16:0/3642824463 wait complete. 2023-11-13T11:03:49.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.111+0000 7f7cf1800740 1 -- start start 2023-11-13T11:03:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.111+0000 7f7cf1800740 1 Processor -- start 2023-11-13T11:03:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef65ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62650 0x556e7ef6b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef5fce0 0x556e7ef6b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cf1800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7ee3d480 con 0x556e7ef62270 2023-11-13T11:03:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cf1800740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7ee5adc0 con 0x556e7ef62650 2023-11-13T11:03:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cf1800740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e7eb99f90 con 0x556e7ef5fce0 2023-11-13T11:03:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cf1800740 10 client.0 _ll_register_callbacks cb 0x556e7ebb2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:03:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cec92c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef65ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cec92c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef65ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36606/0 (socket says 172.21.15.16:36606) 2023-11-13T11:03:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cec92c640 1 -- 172.21.15.16:0/4290242908 learned_addr learned my addr 172.21.15.16:0/4290242908 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cec92c640 1 -- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef5fce0 msgr2=0x556e7ef6b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cec92c640 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef5fce0 0x556e7ef6b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cec92c640 1 -- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62650 msgr2=0x556e7ef6b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cec92c640 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62650 0x556e7ef6b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.112+0000 7f7cec92c640 1 -- 172.21.15.16:0/4290242908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce80b25b0 con 0x556e7ef62270 2023-11-13T11:03:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.113+0000 7f7cec92c640 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef65ff0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00b2c0 tx=0x7f7cdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee200bfdbb69f62b server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.113+0000 7f7cc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:03:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.113+0000 7f7cc77fe640 1 -- 172.21.15.16:0/4290242908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc041cc0 con 0x556e7ef62270 2023-11-13T11:03:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.113+0000 7f7cc77fe640 1 -- 172.21.15.16:0/4290242908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7cdc041e20 con 0x556e7ef62270 2023-11-13T11:03:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.113+0000 7f7cf1800740 1 -- 172.21.15.16:0/4290242908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e7ee99e50 con 0x556e7ef62270 2023-11-13T11:03:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.113+0000 7f7cc77fe640 1 -- 172.21.15.16:0/4290242908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cdc003610 con 0x556e7ef62270 2023-11-13T11:03:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.113+0000 7f7cf1800740 10 client.7152 fetch_fsmap learned FSMap version 40 2023-11-13T11:03:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.113+0000 7f7cf1800740 1 -- 172.21.15.16:0/4290242908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x556e7ee9f4f0 con 0x556e7ef62270 2023-11-13T11:03:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.114+0000 7f7cc77fe640 1 -- 172.21.15.16:0/4290242908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556e7ee99e50 con 0x556e7ef62270 2023-11-13T11:03:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.114+0000 7f7cc77fe640 1 -- 172.21.15.16:0/4290242908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7cdc03d030 con 0x556e7ef62270 2023-11-13T11:03:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.114+0000 7f7cf1800740 10 client.7152 fetch_fsmap finished waiting for FSMap version 40 2023-11-13T11:03:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.114+0000 7f7cf1800740 10 client.7152 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-11-13T11:03:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.114+0000 7f7cf1800740 20 client.7152 populate_metadata read hostname 'smithi016' 2023-11-13T11:03:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.114+0000 7f7cf1800740 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x556e7ef69e00 0x556e7ef6a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.114+0000 7f7cf1800740 4 client.7152 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:03:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.114+0000 7f7cf1800740 1 -- 172.21.15.16:0/4290242908 --> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e7ee99b70 con 0x556e7ef69e00 2023-11-13T11:03:49.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.114+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x556e7ef69e00 0x556e7ef6a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.115+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x556e7ef69e00 0x556e7ef6a230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8780752b6719779c server_cookie=3ceb401efdd127bf in_seq=0 out_seq=0 2023-11-13T11:03:49.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.115+0000 7f7cc77fe640 10 client.7152 ms_handle_connect on v2:172.21.15.184:6835/60343961 2023-11-13T11:03:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.117+0000 7f7cc77fe640 1 -- 172.21.15.16:0/4290242908 <== mds.0 v2:172.21.15.184:6835/60343961 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e7ee99b70 con 0x556e7ef69e00 2023-11-13T11:03:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.117+0000 7f7cc77fe640 10 client.7152 handle_command_reply: tid=0 2023-11-13T11:03:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.117+0000 7f7cf1800740 1 client.7152 shutdown 2023-11-13T11:03:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 -- 172.21.15.16:0/4290242908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 msgr2=0x556e7ef65ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef65ff0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00b2c0 tx=0x7f7cdc004730 comp rx=0 tx=0).stop 2023-11-13T11:03:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 -- 172.21.15.16:0/4290242908 shutdown_connections 2023-11-13T11:03:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x556e7ef69e00 0x556e7ef6a230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef5fce0 0x556e7ef6b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62650 0x556e7ef6b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 --2- 172.21.15.16:0/4290242908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef65ff0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cc77fe640 0 client.7152 ms_handle_reset on v2:172.21.15.184:6835/60343961 2023-11-13T11:03:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 -- 172.21.15.16:0/4290242908 >> 172.21.15.16:0/4290242908 conn(0x556e7ef35750 msgr2=0x556e7ef34970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 -- 172.21.15.16:0/4290242908 shutdown_connections 2023-11-13T11:03:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 1 -- 172.21.15.16:0/4290242908 wait complete. 2023-11-13T11:03:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.118+0000 7f7cf1800740 20 client.7152 trim_cache size 0 max 16384 2023-11-13T11:03:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.133+0000 7f7cf1800740 1 Processor -- start 2023-11-13T11:03:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.133+0000 7f7cf1800740 1 -- start start 2023-11-13T11:03:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.133+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef5fce0 0x556e7ef34c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef3c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62650 0x556e7ef825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7cf1800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7ef82b10 con 0x556e7ef62270 2023-11-13T11:03:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7cf1800740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7ef69570 con 0x556e7ef62650 2023-11-13T11:03:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7cf1800740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e7efa2ca0 con 0x556e7ef5fce0 2023-11-13T11:03:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7ce7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef3c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7ce7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef3c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36618/0 (socket says 172.21.15.16:36618) 2023-11-13T11:03:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7ce7fff640 1 -- 172.21.15.16:0/3708773907 learned_addr learned my addr 172.21.15.16:0/3708773907 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7ced12d640 1 --2- 172.21.15.16:0/3708773907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62650 0x556e7ef825d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7ce7fff640 1 -- 172.21.15.16:0/3708773907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef5fce0 msgr2=0x556e7ef34c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/3708773907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef5fce0 0x556e7ef34c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7ce7fff640 1 -- 172.21.15.16:0/3708773907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62650 msgr2=0x556e7ef825d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/3708773907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62650 0x556e7ef825d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.134+0000 7f7ce7fff640 1 -- 172.21.15.16:0/3708773907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e7ee3cd10 con 0x556e7ef62270 2023-11-13T11:03:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.135+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/3708773907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef3c440 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4058890 tx=0x7f7cd405ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40eaedf87baffadf server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.135+0000 7f7cd2ffd640 1 -- 172.21.15.16:0/3708773907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40881d0 con 0x556e7ef62270 2023-11-13T11:03:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.135+0000 7f7cd2ffd640 1 -- 172.21.15.16:0/3708773907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd4056ca0 con 0x556e7ef62270 2023-11-13T11:03:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.135+0000 7f7cd2ffd640 1 -- 172.21.15.16:0/3708773907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4070070 con 0x556e7ef62270 2023-11-13T11:03:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.135+0000 7f7cf1800740 1 -- 172.21.15.16:0/3708773907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 msgr2=0x556e7ef3c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.135+0000 7f7cf1800740 1 --2- 172.21.15.16:0/3708773907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef3c440 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4058890 tx=0x7f7cd405ca50 comp rx=0 tx=0).stop 2023-11-13T11:03:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.135+0000 7f7cf1800740 1 -- 172.21.15.16:0/3708773907 shutdown_connections 2023-11-13T11:03:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.135+0000 7f7cf1800740 1 --2- 172.21.15.16:0/3708773907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62650 0x556e7ef825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.135+0000 7f7cf1800740 1 --2- 172.21.15.16:0/3708773907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef62270 0x556e7ef3c440 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.136+0000 7f7cf1800740 1 --2- 172.21.15.16:0/3708773907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef5fce0 0x556e7ef34c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.136+0000 7f7cf1800740 1 -- 172.21.15.16:0/3708773907 >> 172.21.15.16:0/3708773907 conn(0x556e7ef35750 msgr2=0x556e7ef39290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.136+0000 7f7cf1800740 1 -- 172.21.15.16:0/3708773907 shutdown_connections 2023-11-13T11:03:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.136+0000 7f7cf1800740 1 -- 172.21.15.16:0/3708773907 wait complete. 2023-11-13T11:03:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.136+0000 7f7cf1800740 1 -- start start 2023-11-13T11:03:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.137+0000 7f7cf1800740 1 Processor -- start 2023-11-13T11:03:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef5fce0 0x556e7ef81150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62270 0x556e7ef81690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cf1800740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef62650 0x556e7ef39f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cf1800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7ef7f6e0 con 0x556e7ef5fce0 2023-11-13T11:03:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cf1800740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7ef7fb40 con 0x556e7ef62270 2023-11-13T11:03:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cf1800740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e7ef83730 con 0x556e7ef62650 2023-11-13T11:03:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cf1800740 10 client.0 _ll_register_callbacks cb 0x7f7cdc039070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:03:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cec92c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef5fce0 0x556e7ef81150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cec92c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef5fce0 0x556e7ef81150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36622/0 (socket says 172.21.15.16:36622) 2023-11-13T11:03:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cec92c640 1 -- 172.21.15.16:0/1413108544 learned_addr learned my addr 172.21.15.16:0/1413108544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cec92c640 1 -- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef62650 msgr2=0x556e7ef39f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cec92c640 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef62650 0x556e7ef39f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cec92c640 1 -- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62270 msgr2=0x556e7ef81690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cec92c640 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62270 0x556e7ef81690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.138+0000 7f7cec92c640 1 -- 172.21.15.16:0/1413108544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce815b480 con 0x556e7ef5fce0 2023-11-13T11:03:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.139+0000 7f7cec92c640 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef5fce0 0x556e7ef81150 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc043010 tx=0x7f7cdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e379b37508067a4a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.139+0000 7f7cc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:03:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.139+0000 7f7cc77fe640 1 -- 172.21.15.16:0/1413108544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc041cd0 con 0x556e7ef5fce0 2023-11-13T11:03:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.139+0000 7f7cc77fe640 1 -- 172.21.15.16:0/1413108544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7cdc055420 con 0x556e7ef5fce0 2023-11-13T11:03:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.139+0000 7f7cf1800740 1 -- 172.21.15.16:0/1413108544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cdc041740 con 0x556e7ef5fce0 2023-11-13T11:03:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.139+0000 7f7cc77fe640 1 -- 172.21.15.16:0/1413108544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cdc041e30 con 0x556e7ef5fce0 2023-11-13T11:03:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.139+0000 7f7cf1800740 10 client.7158 fetch_fsmap learned FSMap version 40 2023-11-13T11:03:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.139+0000 7f7cf1800740 1 -- 172.21.15.16:0/1413108544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x556e7ee62300 con 0x556e7ef5fce0 2023-11-13T11:03:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.139+0000 7f7cc77fe640 1 -- 172.21.15.16:0/1413108544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc041740 con 0x556e7ef5fce0 2023-11-13T11:03:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.140+0000 7f7cc77fe640 1 -- 172.21.15.16:0/1413108544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7cdc03d040 con 0x556e7ef5fce0 2023-11-13T11:03:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.140+0000 7f7cf1800740 10 client.7158 fetch_fsmap finished waiting for FSMap version 40 2023-11-13T11:03:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.140+0000 7f7cf1800740 10 client.7158 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-11-13T11:03:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.140+0000 7f7cf1800740 20 client.7158 populate_metadata read hostname 'smithi016' 2023-11-13T11:03:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.140+0000 7f7cf1800740 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x556e7ef34a50 0x556e7ef3a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.140+0000 7f7cf1800740 4 client.7158 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-11-13T11:03:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.140+0000 7f7cf1800740 1 -- 172.21.15.16:0/1413108544 --> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e7eba17e0 con 0x556e7ef34a50 2023-11-13T11:03:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.140+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x556e7ef34a50 0x556e7ef3a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.141+0000 7f7ce7fff640 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x556e7ef34a50 0x556e7ef3a4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edcb0ae8a349e6c2 server_cookie=3766b5d012baba40 in_seq=0 out_seq=0 2023-11-13T11:03:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.141+0000 7f7cc77fe640 10 client.7158 ms_handle_connect on v2:172.21.15.184:6835/60343961 2023-11-13T11:03:49.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.173+0000 7f7cc77fe640 1 -- 172.21.15.16:0/1413108544 <== mds.0 v2:172.21.15.184:6835/60343961 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e7eba17e0 con 0x556e7ef34a50 2023-11-13T11:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.173+0000 7f7cc77fe640 10 client.7158 handle_command_reply: tid=0 2023-11-13T11:03:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.173+0000 7f7cf1800740 1 client.7158 shutdown 2023-11-13T11:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 -- 172.21.15.16:0/1413108544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef5fce0 msgr2=0x556e7ef81150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef5fce0 0x556e7ef81150 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc043010 tx=0x7f7cdc0046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 -- 172.21.15.16:0/1413108544 shutdown_connections 2023-11-13T11:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:6835/60343961,v1:172.21.15.184:6837/60343961] conn(0x556e7ef34a50 0x556e7ef3a4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556e7ef62650 0x556e7ef39f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e7ef62270 0x556e7ef81690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 --2- 172.21.15.16:0/1413108544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7ef5fce0 0x556e7ef81150 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cc77fe640 0 client.7158 ms_handle_reset on v2:172.21.15.184:6835/60343961 2023-11-13T11:03:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 -- 172.21.15.16:0/1413108544 >> 172.21.15.16:0/1413108544 conn(0x556e7ef35750 msgr2=0x556e7ef65de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 -- 172.21.15.16:0/1413108544 shutdown_connections 2023-11-13T11:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 1 -- 172.21.15.16:0/1413108544 wait complete. 2023-11-13T11:03:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.174+0000 7f7cf1800740 20 client.7158 trim_cache size 0 max 16384 2023-11-13T11:03:49.183 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:03:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "", 2023-11-13T11:03:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0 2023-11-13T11:03:49.184 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 -- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ccc06ae70 msgr2=0x7f7ccc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ccc06ae70 0x7f7ccc06d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc038000 tx=0x7f7cdc008040 comp rx=0 tx=0).stop 2023-11-13T11:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 -- 172.21.15.16:0/2488391877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 msgr2=0x7f7ce80cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 0x7f7ce80cc880 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400d800 tx=0x7f7cd400dcc0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 -- 172.21.15.16:0/2488391877 shutdown_connections 2023-11-13T11:03:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f7ccc06ae70 0x7f7ccc06d330 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ce80ccdc0 0x7f7ce815cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8158fa0 0x7f7ce80cc880 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 --2- 172.21.15.16:0/2488391877 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ce806bf50 0x7f7ce8149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.175+0000 7f7ced92e640 1 -- 172.21.15.16:0/2488391877 >> 172.21.15.16:0/2488391877 conn(0x7f7ce80b0970 msgr2=0x7f7ce80b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.176+0000 7f7ced92e640 1 -- 172.21.15.16:0/2488391877 shutdown_connections 2023-11-13T11:03:49.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.176+0000 7f7ced92e640 1 -- 172.21.15.16:0/2488391877 wait complete. 2023-11-13T11:03:49.199 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-13T11:03:49.200 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:03:49.200 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-13T11:03:49.200 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:03:49.201 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-13T11:03:49.201 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:03:49.202 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-13T11:03:49.202 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:03:49.215 INFO:tasks.ceph.mds.a:Stopped 2023-11-13T11:03:49.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2023-11-13T11:03:49.218 INFO:tasks.ceph.mds.c:Stopped 2023-11-13T11:03:49.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2023-11-13T11:03:49.222 INFO:tasks.ceph.mds.d:Stopped 2023-11-13T11:03:49.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2023-11-13T11:03:49.223 INFO:tasks.ceph.mds.b:Stopped 2023-11-13T11:03:49.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2023-11-13T11:03:49.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.458+0000 7ff28759e640 1 Processor -- start 2023-11-13T11:03:49.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.459+0000 7ff28759e640 1 -- start start 2023-11-13T11:03:49.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.460+0000 7ff28759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff28815aa90 0x7ff28806a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.460+0000 7ff28759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 0x7ff288070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.460+0000 7ff28759e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28806d680 con 0x7ff2880708a0 2023-11-13T11:03:49.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.460+0000 7ff28759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28806ab50 con 0x7ff28815aa90 2023-11-13T11:03:49.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.460+0000 7ff28759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff288070d30 con 0x7ff28806a6d0 2023-11-13T11:03:49.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.461+0000 7ff28659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff28815aa90 0x7ff28806a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.461+0000 7ff28659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff28815aa90 0x7ff28806a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36632/0 (socket says 172.21.15.16:36632) 2023-11-13T11:03:49.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.461+0000 7ff28659c640 1 -- 172.21.15.16:0/2867620848 learned_addr learned my addr 172.21.15.16:0/2867620848 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.461+0000 7ff285d9b640 1 --2- 172.21.15.16:0/2867620848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 0x7ff288070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.461+0000 7ff28659c640 1 -- 172.21.15.16:0/2867620848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 msgr2=0x7ff288070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.461+0000 7ff28659c640 1 --2- 172.21.15.16:0/2867620848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 0x7ff288070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.461+0000 7ff28659c640 1 -- 172.21.15.16:0/2867620848 >> v1:172.21.15.184:6789/0 conn(0x7ff2880708a0 legacy=0x7ff2880b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.461+0000 7ff28659c640 1 -- 172.21.15.16:0/2867620848 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28815d630 con 0x7ff28815aa90 2023-11-13T11:03:49.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.463+0000 7ff28659c640 1 --2- 172.21.15.16:0/2867620848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28815aa90 0x7ff28806a190 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff27802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1296292e36a7a68 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.463+0000 7ff28559a640 1 -- 172.21.15.16:0/2867620848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278031870 con 0x7ff28815aa90 2023-11-13T11:03:49.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.463+0000 7ff28559a640 1 -- 172.21.15.16:0/2867620848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2780319d0 con 0x7ff28815aa90 2023-11-13T11:03:49.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.463+0000 7ff28559a640 1 -- 172.21.15.16:0/2867620848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278031da0 con 0x7ff28815aa90 2023-11-13T11:03:49.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.464+0000 7ff28759e640 1 -- 172.21.15.16:0/2867620848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28815aa90 msgr2=0x7ff28806a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.464+0000 7ff28759e640 1 --2- 172.21.15.16:0/2867620848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28815aa90 0x7ff28806a190 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff27802b910 comp rx=0 tx=0).stop 2023-11-13T11:03:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.465+0000 7ff28759e640 1 -- 172.21.15.16:0/2867620848 shutdown_connections 2023-11-13T11:03:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.465+0000 7ff28759e640 1 --2- 172.21.15.16:0/2867620848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 0x7ff288070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.465+0000 7ff28759e640 1 --2- 172.21.15.16:0/2867620848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28815aa90 0x7ff28806a190 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.465+0000 7ff28759e640 1 -- 172.21.15.16:0/2867620848 >> 172.21.15.16:0/2867620848 conn(0x7ff2880b04a0 msgr2=0x7ff2880b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.465+0000 7ff28759e640 1 -- 172.21.15.16:0/2867620848 shutdown_connections 2023-11-13T11:03:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.465+0000 7ff28759e640 1 -- 172.21.15.16:0/2867620848 wait complete. 2023-11-13T11:03:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.466+0000 7fbcfdf27640 1 Processor -- start 2023-11-13T11:03:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.467+0000 7ff28759e640 1 Processor -- start 2023-11-13T11:03:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.467+0000 7fbcfdf27640 1 -- start start 2023-11-13T11:03:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.467+0000 7f6f494e7640 1 Processor -- start 2023-11-13T11:03:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.467+0000 7ff28759e640 1 -- start start 2023-11-13T11:03:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.467+0000 7f6f494e7640 1 -- start start 2023-11-13T11:03:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.468+0000 7ff28759e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 0x7ff288156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.468+0000 7ff28759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880708a0 0x7ff288157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.468+0000 7ff28759e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28815aa90 0x7ff288166c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.468+0000 7ff28759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff288157120 con 0x7ff2880708a0 2023-11-13T11:03:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.468+0000 7ff28759e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881584f0 con 0x7ff28815aa90 2023-11-13T11:03:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.468+0000 7ff28759e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881587d0 con 0x7ff28806a6d0 2023-11-13T11:03:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7fbcfdf27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcf8161060 0x7fbcf81621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7fbcfdf27640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbcf806f970 0x7fbcf8162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7f6f494e7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6f4406fb60 0x7f6f4406ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7f6f494e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f440ae030 0x7f6f44070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7f6f494e7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f440709c0 con 0x7f6f44164d60 2023-11-13T11:03:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7f6f494e7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4406d070 con 0x7f6f4406fb60 2023-11-13T11:03:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7f6f494e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440b3a80 con 0x7f6f440ae030 2023-11-13T11:03:49.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7fbcfdf27640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf8162700 con 0x7fbcf806a190 2023-11-13T11:03:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7fbcfdf27640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf80b1840 con 0x7fbcf8161060 2023-11-13T11:03:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7fbcfdf27640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf8071600 con 0x7fbcf806f970 2023-11-13T11:03:49.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7f6f437fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6f44164d60 0x7f6f441651f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38420/0 (socket says 172.21.15.16:38420) 2023-11-13T11:03:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7f6f437fe640 1 -- 172.21.15.16:0/428085116 learned_addr learned my addr 172.21.15.16:0/428085116 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28815aa90 0x7ff288166c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7ff285d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880708a0 0x7ff288157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7ff285d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880708a0 0x7ff288157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36648/0 (socket says 172.21.15.16:36648) 2023-11-13T11:03:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28815aa90 0x7ff288166c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:42302/0 (socket says 172.21.15.16:42302) 2023-11-13T11:03:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7ff285d9b640 1 -- 172.21.15.16:0/3643311785 learned_addr learned my addr 172.21.15.16:0/3643311785 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.469+0000 7ff28659c640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 0x7ff288156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7fbcfcf25640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcf8161060 0x7fbcf81621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7fbcfcf25640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcf8161060 0x7fbcf81621c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36652/0 (socket says 172.21.15.16:36652) 2023-11-13T11:03:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7fbcfcf25640 1 -- 172.21.15.16:0/1476442690 learned_addr learned my addr 172.21.15.16:0/1476442690 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7f6f427fc640 1 --2- 172.21.15.16:0/428085116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f440ae030 0x7f6f44070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7f6f42ffd640 1 --2- 172.21.15.16:0/428085116 >> v2:172.21.15.184:3300/0 conn(0x7f6f4406fb60 0x7f6f4406ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7fbcf7fff640 1 --2- 172.21.15.16:0/1476442690 >> v2:172.21.15.184:3300/0 conn(0x7fbcf806f970 0x7fbcf8162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7ff285d9b640 1 -- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 msgr2=0x7ff288156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7ff285d9b640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 0x7ff288156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7ff285d9b640 1 -- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28815aa90 msgr2=0x7ff288166c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7ff285d9b640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28815aa90 0x7ff288166c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.470+0000 7ff285d9b640 1 -- 172.21.15.16:0/3643311785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2780094d0 con 0x7ff2880708a0 2023-11-13T11:03:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7f6f41ffb640 1 -- 172.21.15.16:0/428085116 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687721490 0 0) 0x7f6f440709c0 con 0x7f6f44164d60 2023-11-13T11:03:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7f6f42ffd640 1 -- 172.21.15.16:0/428085116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f440ae030 msgr2=0x7f6f44070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7ff285d9b640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880708a0 0x7ff288157bc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00d5d0 tx=0x7ff27c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7fbcfcf25640 1 -- 172.21.15.16:0/1476442690 >> v2:172.21.15.184:3300/0 conn(0x7fbcf806f970 msgr2=0x7fbcf8162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7fbcfcf25640 1 --2- 172.21.15.16:0/1476442690 >> v2:172.21.15.184:3300/0 conn(0x7fbcf806f970 0x7fbcf8162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7fbcfcf25640 1 -- 172.21.15.16:0/1476442690 >> v1:172.21.15.184:6789/0 conn(0x7fbcf806a190 legacy=0x7fbcf80b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7f6f42ffd640 1 --2- 172.21.15.16:0/428085116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f440ae030 0x7f6f44070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7f6f42ffd640 1 -- 172.21.15.16:0/428085116 >> v1:172.21.15.16:6789/0 conn(0x7f6f44164d60 legacy=0x7f6f441651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7f6f42ffd640 1 -- 172.21.15.16:0/428085116 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f44161880 con 0x7f6f4406fb60 2023-11-13T11:03:49.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7fbcfcf25640 1 -- 172.21.15.16:0/1476442690 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf80718e0 con 0x7fbcf8161060 2023-11-13T11:03:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7ff276ffd640 1 -- 172.21.15.16:0/3643311785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c0125f0 con 0x7ff2880708a0 2023-11-13T11:03:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7ff276ffd640 1 -- 172.21.15.16:0/3643311785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c012770 con 0x7ff2880708a0 2023-11-13T11:03:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.471+0000 7ff276ffd640 1 -- 172.21.15.16:0/3643311785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c012b20 con 0x7ff2880708a0 2023-11-13T11:03:49.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.472+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff288158ab0 con 0x7ff2880708a0 2023-11-13T11:03:49.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.472+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff288159070 con 0x7ff2880708a0 2023-11-13T11:03:49.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.472+0000 7fbcfcf25640 1 --2- 172.21.15.16:0/1476442690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 0x7fbcf81621c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80ae3e0 tx=0x7fbcec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8feb42897d0ff76a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.472+0000 7f6f42ffd640 1 --2- 172.21.15.16:0/428085116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f4406fb60 0x7f6f4406ff40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6f340097a0 tx=0x7f6f3402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c3e08bdc890105 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.473+0000 7f6f41ffb640 1 -- 172.21.15.16:0/428085116 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3402d040 con 0x7f6f4406fb60 2023-11-13T11:03:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.473+0000 7f6f41ffb640 1 -- 172.21.15.16:0/428085116 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f34031810 con 0x7f6f4406fb60 2023-11-13T11:03:49.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.473+0000 7f6f41ffb640 1 -- 172.21.15.16:0/428085116 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34031ba0 con 0x7f6f4406fb60 2023-11-13T11:03:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.473+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28804e4d0 con 0x7ff2880708a0 2023-11-13T11:03:49.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f6f494e7640 1 -- 172.21.15.16:0/428085116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f4406fb60 msgr2=0x7f6f4406ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f6f494e7640 1 --2- 172.21.15.16:0/428085116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f4406fb60 0x7f6f4406ff40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6f340097a0 tx=0x7f6f3402b8b0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f6f494e7640 1 -- 172.21.15.16:0/428085116 shutdown_connections 2023-11-13T11:03:49.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f6f494e7640 1 --2- 172.21.15.16:0/428085116 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f440ae030 0x7f6f44070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f6f494e7640 1 --2- 172.21.15.16:0/428085116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f4406fb60 0x7f6f4406ff40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f18266af640 1 Processor -- start 2023-11-13T11:03:49.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f6f494e7640 1 -- 172.21.15.16:0/428085116 >> 172.21.15.16:0/428085116 conn(0x7f6f4406a190 msgr2=0x7f6f4406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f18266af640 1 -- start start 2023-11-13T11:03:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7fbcf77fe640 1 -- 172.21.15.16:0/1476442690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec031870 con 0x7fbcf8161060 2023-11-13T11:03:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f6f494e7640 1 -- 172.21.15.16:0/428085116 shutdown_connections 2023-11-13T11:03:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7f6f494e7640 1 -- 172.21.15.16:0/428085116 wait complete. 2023-11-13T11:03:49.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7ff276ffd640 1 -- 172.21.15.16:0/3643311785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff27c012c80 con 0x7ff2880708a0 2023-11-13T11:03:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7fbcf77fe640 1 -- 172.21.15.16:0/1476442690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec0319d0 con 0x7fbcf8161060 2023-11-13T11:03:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.474+0000 7fbcf77fe640 1 -- 172.21.15.16:0/1476442690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec031d60 con 0x7fbcf8161060 2023-11-13T11:03:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.475+0000 7fbcfdf27640 1 -- 172.21.15.16:0/1476442690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 msgr2=0x7fbcf81621c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.475+0000 7fbcfdf27640 1 --2- 172.21.15.16:0/1476442690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 0x7fbcf81621c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80ae3e0 tx=0x7fbcec02b910 comp rx=0 tx=0).stop 2023-11-13T11:03:49.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.475+0000 7ff276ffd640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.475+0000 7ff276ffd640 1 -- 172.21.15.16:0/3643311785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff27c089b30 con 0x7ff2880708a0 2023-11-13T11:03:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.475+0000 7fbcfdf27640 1 -- 172.21.15.16:0/1476442690 shutdown_connections 2023-11-13T11:03:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.475+0000 7fbcfdf27640 1 --2- 172.21.15.16:0/1476442690 >> v2:172.21.15.184:3300/0 conn(0x7fbcf806f970 0x7fbcf8162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.475+0000 7fbcfdf27640 1 --2- 172.21.15.16:0/1476442690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 0x7fbcf81621c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.475+0000 7fbcfdf27640 1 -- 172.21.15.16:0/1476442690 >> 172.21.15.16:0/1476442690 conn(0x7fbcf80b04a0 msgr2=0x7fbcf80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7fbcfdf27640 1 -- 172.21.15.16:0/1476442690 shutdown_connections 2023-11-13T11:03:49.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7f18266af640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f182015d0f0 0x7f18200d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7fbcfdf27640 1 -- 172.21.15.16:0/1476442690 wait complete. 2023-11-13T11:03:49.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7f18266af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18200cfe20 0x7f18200d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7f18266af640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18200d3c70 con 0x7f18200d0ac0 2023-11-13T11:03:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7f18266af640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18200c69d0 con 0x7f18200cfe20 2023-11-13T11:03:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7f18266af640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18200c6cb0 con 0x7f182015d0f0 2023-11-13T11:03:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7ff28659c640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7f181f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18200cfe20 0x7f18200d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7f181f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18200cfe20 0x7f18200d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36658/0 (socket says 172.21.15.16:36658) 2023-11-13T11:03:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7f1824c4c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f18200d0ac0 0x7f18200b3950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38432/0 (socket says 172.21.15.16:38432) 2023-11-13T11:03:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.476+0000 7f181f7fe640 1 -- 172.21.15.16:0/564152207 learned_addr learned my addr 172.21.15.16:0/564152207 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.477+0000 7f181ffff640 1 --2- 172.21.15.16:0/564152207 >> v2:172.21.15.184:3300/0 conn(0x7f182015d0f0 0x7f18200d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.477+0000 7f6f494e7640 1 Processor -- start 2023-11-13T11:03:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.477+0000 7f181f7fe640 1 -- 172.21.15.16:0/564152207 >> v2:172.21.15.184:3300/0 conn(0x7f182015d0f0 msgr2=0x7f18200d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.477+0000 7f181f7fe640 1 --2- 172.21.15.16:0/564152207 >> v2:172.21.15.184:3300/0 conn(0x7f182015d0f0 0x7f18200d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.477+0000 7f181f7fe640 1 -- 172.21.15.16:0/564152207 >> v1:172.21.15.16:6789/0 conn(0x7f18200d0ac0 legacy=0x7f18200b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.477+0000 7f181f7fe640 1 -- 172.21.15.16:0/564152207 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18200c6f90 con 0x7f18200cfe20 2023-11-13T11:03:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.477+0000 7f6f494e7640 1 -- start start 2023-11-13T11:03:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.478+0000 7ff28659c640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff278036000 tx=0x7ff27803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.478+0000 7f181f7fe640 1 --2- 172.21.15.16:0/564152207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18200cfe20 0x7f18200d1800 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f182008d4c0 tx=0x7f181402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4f6ed6c8b06cab server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.478+0000 7f6f494e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f4406fb60 0x7f6f441f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.478+0000 7f6f494e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f440ae030 0x7f6f44159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.478+0000 7f6f494e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f44164d60 0x7f6f4415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.478+0000 7f6f494e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441f7b20 con 0x7f6f44164d60 2023-11-13T11:03:49.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.478+0000 7f6f494e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440cb490 con 0x7f6f440ae030 2023-11-13T11:03:49.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.478+0000 7f6f494e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440cb770 con 0x7f6f4406fb60 2023-11-13T11:03:49.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7fbcfdf27640 1 Processor -- start 2023-11-13T11:03:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f44164d60 0x7f6f4415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f44164d60 0x7f6f4415c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36660/0 (socket says 172.21.15.16:36660) 2023-11-13T11:03:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f437fe640 1 -- 172.21.15.16:0/2532332023 learned_addr learned my addr 172.21.15.16:0/2532332023 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7fbcfdf27640 1 -- start start 2023-11-13T11:03:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f42ffd640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f4406fb60 0x7f6f441f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f427fc640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f440ae030 0x7f6f44159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f437fe640 1 -- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f4406fb60 msgr2=0x7f6f441f75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f437fe640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f4406fb60 0x7f6f441f75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f437fe640 1 -- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f440ae030 msgr2=0x7f6f44159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f437fe640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f440ae030 0x7f6f44159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.479+0000 7f6f437fe640 1 -- 172.21.15.16:0/2532332023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f34009400 con 0x7f6f44164d60 2023-11-13T11:03:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.480+0000 7f6f437fe640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f44164d60 0x7f6f4415c460 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00c780 tx=0x7f6f3c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfdf27640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf806a190 0x7fbcf814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfdf27640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf806f970 0x7fbcf815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfdf27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 0x7fbcf80acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfdf27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf815d9c0 con 0x7fbcf8161060 2023-11-13T11:03:49.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfdf27640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf814cf60 con 0x7fbcf806a190 2023-11-13T11:03:49.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfdf27640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf815cd40 con 0x7fbcf806f970 2023-11-13T11:03:49.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfd726640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 0x7fbcf80acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfd726640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 0x7fbcf80acc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36662/0 (socket says 172.21.15.16:36662) 2023-11-13T11:03:49.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfd726640 1 -- 172.21.15.16:0/457705216 learned_addr learned my addr 172.21.15.16:0/457705216 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcfcf25640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf806a190 0x7fbcf814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.481+0000 7fbcf7fff640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf806f970 0x7fbcf815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.482+0000 7fbcfd726640 1 -- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf806f970 msgr2=0x7fbcf815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.482+0000 7fbcfd726640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf806f970 0x7fbcf815dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.482+0000 7fbcfd726640 1 -- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf806a190 msgr2=0x7fbcf814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.482+0000 7fbcfd726640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf806a190 0x7fbcf814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.482+0000 7fbcfd726640 1 -- 172.21.15.16:0/457705216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0094d0 con 0x7fbcf8161060 2023-11-13T11:03:49.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.482+0000 7fbcfcf25640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf806a190 0x7fbcf814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:03:49.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.482+0000 7fbcfd726640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 0x7fbcf80acc10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b970 tx=0x7fbcf000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7f6f2b7fe640 1 -- 172.21.15.16:0/2532332023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00ce90 con 0x7f6f44164d60 2023-11-13T11:03:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7f181effd640 1 -- 172.21.15.16:0/564152207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181402f850 con 0x7f18200cfe20 2023-11-13T11:03:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7f181effd640 1 -- 172.21.15.16:0/564152207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181402f9b0 con 0x7f18200cfe20 2023-11-13T11:03:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7f181effd640 1 -- 172.21.15.16:0/564152207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181402fd40 con 0x7f18200cfe20 2023-11-13T11:03:49.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7fbcf57fa640 1 -- 172.21.15.16:0/457705216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf000cb40 con 0x7fbcf8161060 2023-11-13T11:03:49.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7fbcf57fa640 1 -- 172.21.15.16:0/457705216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf000cca0 con 0x7fbcf8161060 2023-11-13T11:03:49.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7fbcf57fa640 1 -- 172.21.15.16:0/457705216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0014680 con 0x7fbcf8161060 2023-11-13T11:03:49.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f440cba50 con 0x7f6f44164d60 2023-11-13T11:03:49.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7f6f2b7fe640 1 -- 172.21.15.16:0/2532332023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c00a640 con 0x7f6f44164d60 2023-11-13T11:03:49.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7f6f2b7fe640 1 -- 172.21.15.16:0/2532332023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00a9d0 con 0x7f6f44164d60 2023-11-13T11:03:49.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f440cc010 con 0x7f6f44164d60 2023-11-13T11:03:49.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.483+0000 7ff276ffd640 1 -- 172.21.15.16:0/3643311785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff27c054b50 con 0x7ff2880708a0 2023-11-13T11:03:49.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.484+0000 7f18266af640 1 -- 172.21.15.16:0/564152207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18200cfe20 msgr2=0x7f18200d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.484+0000 7f18266af640 1 --2- 172.21.15.16:0/564152207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18200cfe20 0x7f18200d1800 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f182008d4c0 tx=0x7f181402ba30 comp rx=0 tx=0).stop 2023-11-13T11:03:49.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.484+0000 7f18266af640 1 -- 172.21.15.16:0/564152207 shutdown_connections 2023-11-13T11:03:49.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.484+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf815d020 con 0x7fbcf8161060 2023-11-13T11:03:49.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.484+0000 7f18266af640 1 --2- 172.21.15.16:0/564152207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18200cfe20 0x7f18200d1800 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.484+0000 7f18266af640 1 --2- 172.21.15.16:0/564152207 >> v2:172.21.15.184:3300/0 conn(0x7f182015d0f0 0x7f18200d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.484+0000 7f18266af640 1 -- 172.21.15.16:0/564152207 >> 172.21.15.16:0/564152207 conn(0x7f18200d2ce0 msgr2=0x7f18200d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:49.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.485+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf815d5e0 con 0x7fbcf8161060 2023-11-13T11:03:49.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.485+0000 7f18266af640 1 -- 172.21.15.16:0/564152207 shutdown_connections 2023-11-13T11:03:49.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.485+0000 7f18266af640 1 -- 172.21.15.16:0/564152207 wait complete. 2023-11-13T11:03:49.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.486+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f440cc230 con 0x7f6f44164d60 2023-11-13T11:03:49.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.486+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf815db00 con 0x7fbcf8161060 2023-11-13T11:03:49.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.486+0000 7f6f2b7fe640 1 -- 172.21.15.16:0/2532332023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f3c00ab30 con 0x7f6f44164d60 2023-11-13T11:03:49.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.488+0000 7f6f2b7fe640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6f3006ae40 0x7f6f3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.488+0000 7f6f2b7fe640 1 -- 172.21.15.16:0/2532332023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6f3c088f60 con 0x7f6f44164d60 2023-11-13T11:03:49.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.488+0000 7fbcf57fa640 1 -- 172.21.15.16:0/457705216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcf001d3b0 con 0x7fbcf8161060 2023-11-13T11:03:49.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.488+0000 7f6f42ffd640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6f3006ae40 0x7f6f3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.488+0000 7f18266af640 1 Processor -- start 2023-11-13T11:03:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.489+0000 7f18266af640 1 -- start start 2023-11-13T11:03:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.489+0000 7fbcf57fa640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcdc06aec0 0x7fbcdc06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.489+0000 7fbcf57fa640 1 -- 172.21.15.16:0/457705216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbcf0089550 con 0x7fbcf8161060 2023-11-13T11:03:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f18266af640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18200cfe20 0x7f18200b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f6f42ffd640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6f3006ae40 0x7f6f3006d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6f34036000 tx=0x7f6f3403c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f18266af640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18200d0ac0 0x7f18200b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f18266af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f182015d0f0 0x7f1820159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f18266af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18200b8330 con 0x7f182015d0f0 2023-11-13T11:03:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f18266af640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18200b7db0 con 0x7f18200d0ac0 2023-11-13T11:03:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f18266af640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18200b8050 con 0x7f18200cfe20 2023-11-13T11:03:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7fbcfcf25640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcdc06aec0 0x7fbcdc06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f1824c4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f182015d0f0 0x7f1820159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f1824c4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f182015d0f0 0x7f1820159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36666/0 (socket says 172.21.15.16:36666) 2023-11-13T11:03:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f1824c4c640 1 -- 172.21.15.16:0/2918961722 learned_addr learned my addr 172.21.15.16:0/2918961722 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f181ffff640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18200cfe20 0x7f18200b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.490+0000 7f181f7fe640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18200d0ac0 0x7f18200b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.491+0000 7f1824c4c640 1 -- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18200cfe20 msgr2=0x7f18200b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.491+0000 7f1824c4c640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18200cfe20 0x7f18200b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.491+0000 7f1824c4c640 1 -- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18200d0ac0 msgr2=0x7f18200b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.491+0000 7f1824c4c640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18200d0ac0 0x7f18200b7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.491+0000 7f1824c4c640 1 -- 172.21.15.16:0/2918961722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18140094d0 con 0x7f182015d0f0 2023-11-13T11:03:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.491+0000 7f181f7fe640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18200d0ac0 0x7f18200b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:03:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.491+0000 7f1824c4c640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f182015d0f0 0x7f1820159e60 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f181800c780 tx=0x7f181800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.492+0000 7fbcfcf25640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcdc06aec0 0x7fbcdc06d380 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbcec036000 tx=0x7fbcec03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.493+0000 7f6f2b7fe640 1 -- 172.21.15.16:0/2532332023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f3c053ec0 con 0x7f6f44164d60 2023-11-13T11:03:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.497+0000 7f181cff9640 1 -- 172.21.15.16:0/2918961722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800ce90 con 0x7f182015d0f0 2023-11-13T11:03:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.497+0000 7f181cff9640 1 -- 172.21.15.16:0/2918961722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181800a640 con 0x7f182015d0f0 2023-11-13T11:03:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.497+0000 7f181cff9640 1 -- 172.21.15.16:0/2918961722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800a9f0 con 0x7f182015d0f0 2023-11-13T11:03:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.497+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1820159380 con 0x7f182015d0f0 2023-11-13T11:03:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.497+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1820147cc0 con 0x7f182015d0f0 2023-11-13T11:03:49.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.497+0000 7fbcf57fa640 1 -- 172.21.15.16:0/457705216 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbcf815db00 con 0x7fbcf8161060 2023-11-13T11:03:49.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.499+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18201591c0 con 0x7f182015d0f0 2023-11-13T11:03:49.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.501+0000 7f181cff9640 1 -- 172.21.15.16:0/2918961722 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f181800ab50 con 0x7f182015d0f0 2023-11-13T11:03:49.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.502+0000 7f181cff9640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f180c06aec0 0x7f180c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:49.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.502+0000 7f181cff9640 1 -- 172.21.15.16:0/2918961722 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1818088c50 con 0x7f182015d0f0 2023-11-13T11:03:49.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.503+0000 7f181ffff640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f180c06aec0 0x7f180c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:49.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.504+0000 7f181ffff640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f180c06aec0 0x7f180c06d380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1810009b20 tx=0x7f1810009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:49.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.511+0000 7f181cff9640 1 -- 172.21.15.16:0/2918961722 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1818053ce0 con 0x7f182015d0f0 2023-11-13T11:03:49.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.754+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7ff28806c9e0 con 0x7ff2880708a0 2023-11-13T11:03:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.764+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7fbcf815db00 con 0x7fbcf8161060 2023-11-13T11:03:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.798+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f6f44067090 con 0x7f6f44164d60 2023-11-13T11:03:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:49.823+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f18200d3c70 con 0x7f182015d0f0 2023-11-13T11:03:49.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:03:49.888 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:03:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.000+0000 7ff276ffd640 1 -- 172.21.15.16:0/3643311785 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6876 v41) v1 ==== 95+0+0 (secure 0 0 0) 0x7ff27c058170 con 0x7ff2880708a0 2023-11-13T11:03:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:failed mds gid 6876 2023-11-13T11:03:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 msgr2=0x7ff26806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff278036000 tx=0x7ff27803c040 comp rx=0 tx=0).stop 2023-11-13T11:03:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880708a0 msgr2=0x7ff288157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880708a0 0x7ff288157bc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00d5d0 tx=0x7ff27c00da90 comp rx=0 tx=0).stop 2023-11-13T11:03:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 shutdown_connections 2023-11-13T11:03:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff26806ae40 0x7ff26806d300 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff28815aa90 0x7ff288166c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2880708a0 0x7ff288157bc0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 --2- 172.21.15.16:0/3643311785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff28806a6d0 0x7ff288156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.004+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 >> 172.21.15.16:0/3643311785 conn(0x7ff2880b04a0 msgr2=0x7ff28815be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.005+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 shutdown_connections 2023-11-13T11:03:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.005+0000 7ff28759e640 1 -- 172.21.15.16:0/3643311785 wait complete. 2023-11-13T11:03:50.023 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-13T11:03:50.018+0000 7fe1528cd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-13T11:03:50.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.024+0000 7fbcf57fa640 1 -- 172.21.15.16:0/457705216 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5195 v42) v1 ==== 95+0+0 (secure 0 0 0) 0x7fbcf815db00 con 0x7fbcf8161060 2023-11-13T11:03:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.024+0000 7f6f2b7fe640 1 -- 172.21.15.16:0/2532332023 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6879 v42) v1 ==== 95+0+0 (secure 0 0 0) 0x7f6f3c018020 con 0x7f6f44164d60 2023-11-13T11:03:50.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.024+0000 7f181cff9640 1 -- 172.21.15.16:0/2918961722 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6885 v42) v1 ==== 95+0+0 (secure 0 0 0) 0x7f18200d3c70 con 0x7f182015d0f0 2023-11-13T11:03:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:failed mds gid 5195 2023-11-13T11:03:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:failed mds gid 6885 2023-11-13T11:03:50.031 INFO:teuthology.orchestra.run.smithi016.stderr:failed mds gid 6879 2023-11-13T11:03:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.028+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcdc06aec0 msgr2=0x7fbcdc06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.028+0000 7fbcfdf27640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcdc06aec0 0x7fbcdc06d380 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbcec036000 tx=0x7fbcec03c040 comp rx=0 tx=0).stop 2023-11-13T11:03:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.028+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 msgr2=0x7fbcf80acc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.028+0000 7fbcfdf27640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 0x7fbcf80acc10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b970 tx=0x7fbcf000be30 comp rx=0 tx=0).stop 2023-11-13T11:03:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6f3006ae40 msgr2=0x7f6f3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6f3006ae40 0x7f6f3006d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6f34036000 tx=0x7f6f3403c040 comp rx=0 tx=0).stop 2023-11-13T11:03:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f44164d60 msgr2=0x7f6f4415c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f44164d60 0x7f6f4415c460 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00c780 tx=0x7f6f3c00cc40 comp rx=0 tx=0).stop 2023-11-13T11:03:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f180c06aec0 msgr2=0x7f180c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f180c06aec0 0x7f180c06d380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1810009b20 tx=0x7f1810009190 comp rx=0 tx=0).stop 2023-11-13T11:03:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f182015d0f0 msgr2=0x7f1820159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f182015d0f0 0x7f1820159e60 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f181800c780 tx=0x7f181800cc40 comp rx=0 tx=0).stop 2023-11-13T11:03:50.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 shutdown_connections 2023-11-13T11:03:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7fbcfdf27640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbcdc06aec0 0x7fbcdc06d380 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7fbcfdf27640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcf8161060 0x7fbcf80acc10 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7fbcfdf27640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcf806f970 0x7fbcf815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7fbcfdf27640 1 --2- 172.21.15.16:0/457705216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcf806a190 0x7fbcf814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 >> 172.21.15.16:0/457705216 conn(0x7fbcf80b04a0 msgr2=0x7fbcf81651e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 shutdown_connections 2023-11-13T11:03:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6f3006ae40 0x7f6f3006d300 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f44164d60 0x7f6f4415c460 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f440ae030 0x7f6f44159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 --2- 172.21.15.16:0/2532332023 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f4406fb60 0x7f6f441f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 >> 172.21.15.16:0/2532332023 conn(0x7f6f4406a190 msgr2=0x7f6f4415a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 shutdown_connections 2023-11-13T11:03:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f6f494e7640 1 -- 172.21.15.16:0/2532332023 wait complete. 2023-11-13T11:03:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 shutdown_connections 2023-11-13T11:03:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f180c06aec0 0x7f180c06d380 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f182015d0f0 0x7f1820159e60 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18200d0ac0 0x7f18200b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 --2- 172.21.15.16:0/2918961722 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18200cfe20 0x7f18200b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.029+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 >> 172.21.15.16:0/2918961722 conn(0x7f18200d2ce0 msgr2=0x7f182006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.030+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 shutdown_connections 2023-11-13T11:03:50.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.030+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 shutdown_connections 2023-11-13T11:03:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.030+0000 7f18266af640 1 -- 172.21.15.16:0/2918961722 wait complete. 2023-11-13T11:03:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.030+0000 7fbcfdf27640 1 -- 172.21.15.16:0/457705216 wait complete. 2023-11-13T11:03:50.048 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-13T11:03:50.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-13T11:03:50.059 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-13T11:03:50.059 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-13T11:03:50.062 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-13T11:03:50.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-13T11:03:50.065 INFO:tasks.ceph.mds.a:Started 2023-11-13T11:03:50.065 DEBUG:teuthology.parallel:result is None 2023-11-13T11:03:50.066 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-13T11:03:50.067 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-13T11:03:50.068 INFO:tasks.ceph.mds.d:Started 2023-11-13T11:03:50.069 DEBUG:teuthology.parallel:result is None 2023-11-13T11:03:50.076 INFO:tasks.ceph.mds.b:Started 2023-11-13T11:03:50.077 DEBUG:teuthology.parallel:result is None 2023-11-13T11:03:50.112 INFO:tasks.ceph.mds.c:Started 2023-11-13T11:03:50.112 DEBUG:teuthology.parallel:result is None 2023-11-13T11:03:50.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:50.223 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-13T11:03:50.234 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-11-13T11:03:50.234 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-11-13T11:03:50.264 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-13T11:03:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.291+0000 7fb9c23e9640 1 Processor -- start 2023-11-13T11:03:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.291+0000 7fb9c23e9640 1 -- start start 2023-11-13T11:03:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.293+0000 7fb9c23e9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb9bc0d0f00 0x7fb9bc0b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.293+0000 7fb9c23e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d1320 0x7fb9bc0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.293+0000 7fb9c23e9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc0d0590 con 0x7fb9bc15cea0 2023-11-13T11:03:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.293+0000 7fb9c23e9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9bc0b2950 con 0x7fb9bc0d0f00 2023-11-13T11:03:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.293+0000 7fb9c23e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0ce5c0 con 0x7fb9bc0d1320 2023-11-13T11:03:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.293+0000 7fb9c13e7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9bc15cea0 0x7fb9bc0d07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38444/0 (socket says 172.21.15.16:38444) 2023-11-13T11:03:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.293+0000 7fb9c13e7640 1 -- 172.21.15.16:0/3657760819 learned_addr learned my addr 172.21.15.16:0/3657760819 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.294+0000 7fb9c1be8640 1 --2- 172.21.15.16:0/3657760819 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d1320 0x7fb9bc0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.294+0000 7fb9c0be6640 1 --2- 172.21.15.16:0/3657760819 >> v2:172.21.15.184:3300/0 conn(0x7fb9bc0d0f00 0x7fb9bc0b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.294+0000 7fb9b3fff640 1 -- 172.21.15.16:0/3657760819 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783433788 0 0) 0x7fb9bc0d0590 con 0x7fb9bc15cea0 2023-11-13T11:03:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.294+0000 7fb9b3fff640 1 -- 172.21.15.16:0/3657760819 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9a8003520 con 0x7fb9bc15cea0 2023-11-13T11:03:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.294+0000 7fb9c0be6640 1 -- 172.21.15.16:0/3657760819 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d1320 msgr2=0x7fb9bc0b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.294+0000 7fb9c0be6640 1 --2- 172.21.15.16:0/3657760819 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d1320 0x7fb9bc0b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.294+0000 7fb9c0be6640 1 -- 172.21.15.16:0/3657760819 >> v1:172.21.15.16:6789/0 conn(0x7fb9bc15cea0 legacy=0x7fb9bc0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.294+0000 7fb9c0be6640 1 -- 172.21.15.16:0/3657760819 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc0ce8a0 con 0x7fb9bc0d0f00 2023-11-13T11:03:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.295+0000 7fb9c0be6640 1 --2- 172.21.15.16:0/3657760819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9bc0d0f00 0x7fb9bc0b2de0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097a0 tx=0x7fb9ac02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fed1574c48333921 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9b3fff640 1 -- 172.21.15.16:0/3657760819 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac02d040 con 0x7fb9bc0d0f00 2023-11-13T11:03:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9b3fff640 1 -- 172.21.15.16:0/3657760819 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac031810 con 0x7fb9bc0d0f00 2023-11-13T11:03:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9b3fff640 1 -- 172.21.15.16:0/3657760819 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac031ba0 con 0x7fb9bc0d0f00 2023-11-13T11:03:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9c23e9640 1 -- 172.21.15.16:0/3657760819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9bc0d0f00 msgr2=0x7fb9bc0b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9c23e9640 1 --2- 172.21.15.16:0/3657760819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9bc0d0f00 0x7fb9bc0b2de0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097a0 tx=0x7fb9ac02b8b0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9c23e9640 1 -- 172.21.15.16:0/3657760819 shutdown_connections 2023-11-13T11:03:50.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9c23e9640 1 --2- 172.21.15.16:0/3657760819 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d1320 0x7fb9bc0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9c23e9640 1 --2- 172.21.15.16:0/3657760819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9bc0d0f00 0x7fb9bc0b2de0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9c23e9640 1 -- 172.21.15.16:0/3657760819 >> 172.21.15.16:0/3657760819 conn(0x7fb9bc0d2c20 msgr2=0x7fb9bc0d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9c23e9640 1 -- 172.21.15.16:0/3657760819 shutdown_connections 2023-11-13T11:03:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.296+0000 7fb9c23e9640 1 -- 172.21.15.16:0/3657760819 wait complete. 2023-11-13T11:03:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.298+0000 7fb9c23e9640 1 Processor -- start 2023-11-13T11:03:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.298+0000 7fb9c23e9640 1 -- start start 2023-11-13T11:03:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.299+0000 7fb9c23e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d0f00 0x7fb9bc1510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.299+0000 7fb9c23e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc0d1320 0x7fb9bc0b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.299+0000 7fb9c23e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9bc15cea0 0x7fb9bc1f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.299+0000 7fb9c23e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0b5e70 con 0x7fb9bc0d1320 2023-11-13T11:03:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.299+0000 7fb9c23e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc1515e0 con 0x7fb9bc15cea0 2023-11-13T11:03:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.299+0000 7fb9c23e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0b67d0 con 0x7fb9bc0d0f00 2023-11-13T11:03:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.299+0000 7fb9c0be6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc0d1320 0x7fb9bc0b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c0be6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc0d1320 0x7fb9bc0b6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36702/0 (socket says 172.21.15.16:36702) 2023-11-13T11:03:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c0be6640 1 -- 172.21.15.16:0/2846341746 learned_addr learned my addr 172.21.15.16:0/2846341746 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c1be8640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9bc15cea0 0x7fb9bc1f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c13e7640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d0f00 0x7fb9bc1510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c0be6640 1 -- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d0f00 msgr2=0x7fb9bc1510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c0be6640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d0f00 0x7fb9bc1510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c0be6640 1 -- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9bc15cea0 msgr2=0x7fb9bc1f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c0be6640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9bc15cea0 0x7fb9bc1f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c0be6640 1 -- 172.21.15.16:0/2846341746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac009400 con 0x7fb9bc0d1320 2023-11-13T11:03:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.300+0000 7fb9c0be6640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc0d1320 0x7fb9bc0b6290 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0098c0 tx=0x7fb9ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.301+0000 7fb9b1ffb640 1 -- 172.21.15.16:0/2846341746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac03ad60 con 0x7fb9bc0d1320 2023-11-13T11:03:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.301+0000 7fb9b1ffb640 1 -- 172.21.15.16:0/2846341746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac044400 con 0x7fb9bc0d1320 2023-11-13T11:03:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.301+0000 7fb9b1ffb640 1 -- 172.21.15.16:0/2846341746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac0445b0 con 0x7fb9bc0d1320 2023-11-13T11:03:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.301+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc0b57d0 con 0x7fb9bc0d1320 2023-11-13T11:03:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.301+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc1f6a10 con 0x7fb9bc0d1320 2023-11-13T11:03:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.302+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc0b5fb0 con 0x7fb9bc0d1320 2023-11-13T11:03:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.303+0000 7fb9b1ffb640 1 -- 172.21.15.16:0/2846341746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9ac04d440 con 0x7fb9bc0d1320 2023-11-13T11:03:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.304+0000 7fb9b1ffb640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb9a006ae40 0x7fb9a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.304+0000 7fb9b1ffb640 1 -- 172.21.15.16:0/2846341746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fb9ac0ab9a0 con 0x7fb9bc0d1320 2023-11-13T11:03:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.304+0000 7fb9c13e7640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb9a006ae40 0x7fb9a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.305+0000 7fb9c13e7640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb9a006ae40 0x7fb9a006d300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80011e0 tx=0x7fb9b8007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.308+0000 7fb9b1ffb640 1 -- 172.21.15.16:0/2846341746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9ac0768c0 con 0x7fb9bc0d1320 2023-11-13T11:03:50.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.541+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9bc08c2d0 con 0x7fb9bc0d1320 2023-11-13T11:03:50.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.542+0000 7fb9b1ffb640 1 -- 172.21.15.16:0/2846341746 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+1743 (secure 0 0 0) 0x7fb9ac0356e0 con 0x7fb9bc0d1320 2023-11-13T11:03:50.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:50.548 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:50.019672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:03:50.548 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 42 2023-11-13T11:03:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb9a006ae40 msgr2=0x7fb9a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb9a006ae40 0x7fb9a006d300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80011e0 tx=0x7fb9b8007d30 comp rx=0 tx=0).stop 2023-11-13T11:03:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc0d1320 msgr2=0x7fb9bc0b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc0d1320 0x7fb9bc0b6290 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0098c0 tx=0x7fb9ac0046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 shutdown_connections 2023-11-13T11:03:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fb9a006ae40 0x7fb9a006d300 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9bc15cea0 0x7fb9bc1f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc0d1320 0x7fb9bc0b6290 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 --2- 172.21.15.16:0/2846341746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9bc0d0f00 0x7fb9bc1510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.545+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 >> 172.21.15.16:0/2846341746 conn(0x7fb9bc0d2c20 msgr2=0x7fb9bc06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.546+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 shutdown_connections 2023-11-13T11:03:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:50.546+0000 7fb9c23e9640 1 -- 172.21.15.16:0/2846341746 wait complete. 2023-11-13T11:03:50.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-13T11:03:19.913517+0000', 'modified': '2023-11-13T11:03:50.019672+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-13T11:03:50.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2023-11-13T11:03:51.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:52.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.197+0000 7f21a9135640 1 Processor -- start 2023-11-13T11:03:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.198+0000 7f21a9135640 1 -- start start 2023-11-13T11:03:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.199+0000 7f21a9135640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a415cc90 0x7f21a406a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.199+0000 7f21a9135640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21a4070cf0 0x7f21a406a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.199+0000 7f21a9135640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a406fc50 con 0x7f21a4071320 2023-11-13T11:03:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.199+0000 7f21a9135640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a406ff30 con 0x7f21a4070cf0 2023-11-13T11:03:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.199+0000 7f21a9135640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4070210 con 0x7f21a415cc90 2023-11-13T11:03:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.199+0000 7f21a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21a4070cf0 0x7f21a406a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.199+0000 7f21a8934640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f21a4071320 0x7f21a406b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38460/0 (socket says 172.21.15.16:38460) 2023-11-13T11:03:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.199+0000 7f21a8934640 1 -- 172.21.15.16:0/39790689 learned_addr learned my addr 172.21.15.16:0/39790689 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.200+0000 7f21a3fff640 1 --2- 172.21.15.16:0/39790689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a415cc90 0x7f21a406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.200+0000 7f21a37fe640 1 -- 172.21.15.16:0/39790689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a415cc90 msgr2=0x7f21a406a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.200+0000 7f21a37fe640 1 --2- 172.21.15.16:0/39790689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a415cc90 0x7f21a406a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.200+0000 7f21a37fe640 1 -- 172.21.15.16:0/39790689 >> v1:172.21.15.16:6789/0 conn(0x7f21a4071320 legacy=0x7f21a406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.200+0000 7f21a37fe640 1 -- 172.21.15.16:0/39790689 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a40704f0 con 0x7f21a4070cf0 2023-11-13T11:03:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.200+0000 7f21a3fff640 1 --2- 172.21.15.16:0/39790689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a415cc90 0x7f21a406a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.201+0000 7f21a37fe640 1 --2- 172.21.15.16:0/39790689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4070cf0 0x7f21a406a6d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f21980097a0 tx=0x7f219802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedbf1c6b84f897d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.201+0000 7f21a2ffd640 1 -- 172.21.15.16:0/39790689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219802d040 con 0x7f21a4070cf0 2023-11-13T11:03:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.201+0000 7f21a2ffd640 1 -- 172.21.15.16:0/39790689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2198031810 con 0x7f21a4070cf0 2023-11-13T11:03:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.201+0000 7f21a2ffd640 1 -- 172.21.15.16:0/39790689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198031ba0 con 0x7f21a4070cf0 2023-11-13T11:03:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.202+0000 7f21a9135640 1 -- 172.21.15.16:0/39790689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4070cf0 msgr2=0x7f21a406a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.202+0000 7f21a9135640 1 --2- 172.21.15.16:0/39790689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4070cf0 0x7f21a406a6d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f21980097a0 tx=0x7f219802b8b0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.202+0000 7f21a9135640 1 -- 172.21.15.16:0/39790689 shutdown_connections 2023-11-13T11:03:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.202+0000 7f21a9135640 1 --2- 172.21.15.16:0/39790689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4070cf0 0x7f21a406a6d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.202+0000 7f21a9135640 1 --2- 172.21.15.16:0/39790689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a415cc90 0x7f21a406a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.202+0000 7f21a9135640 1 -- 172.21.15.16:0/39790689 >> 172.21.15.16:0/39790689 conn(0x7f21a40b04a0 msgr2=0x7f21a40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.202+0000 7f21a9135640 1 -- 172.21.15.16:0/39790689 shutdown_connections 2023-11-13T11:03:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.202+0000 7f21a9135640 1 -- 172.21.15.16:0/39790689 wait complete. 2023-11-13T11:03:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.204+0000 7f21a9135640 1 Processor -- start 2023-11-13T11:03:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.204+0000 7f21a9135640 1 -- start start 2023-11-13T11:03:52.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.205+0000 7f21a9135640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a4070cf0 0x7f21a41f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:52.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.205+0000 7f21a9135640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a4071320 0x7f21a414b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:52.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.205+0000 7f21a9135640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415cc90 0x7f21a415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:52.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.205+0000 7f21a9135640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a415d1f0 con 0x7f21a415cc90 2023-11-13T11:03:52.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.205+0000 7f21a9135640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a41f67d0 con 0x7f21a4070cf0 2023-11-13T11:03:52.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.205+0000 7f21a9135640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a41f6a70 con 0x7f21a4071320 2023-11-13T11:03:52.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.205+0000 7f21a8934640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415cc90 0x7f21a415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:52.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.205+0000 7f21a8934640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415cc90 0x7f21a415d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36718/0 (socket says 172.21.15.16:36718) 2023-11-13T11:03:52.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.205+0000 7f21a8934640 1 -- 172.21.15.16:0/4017934862 learned_addr learned my addr 172.21.15.16:0/4017934862 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:52.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a37fe640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a4071320 0x7f21a414b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:52.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a3fff640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a4070cf0 0x7f21a41f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:52.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a8934640 1 -- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a4071320 msgr2=0x7f21a414b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:52.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a8934640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a4071320 0x7f21a414b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a8934640 1 -- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a4070cf0 msgr2=0x7f21a41f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:52.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a8934640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a4070cf0 0x7f21a41f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a8934640 1 -- 172.21.15.16:0/4017934862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2198009400 con 0x7f21a415cc90 2023-11-13T11:03:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a37fe640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a4071320 0x7f21a414b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a3fff640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a4070cf0 0x7f21a41f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a8934640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415cc90 0x7f21a415d9b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f219c00cbc0 tx=0x7f219c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a0ff9640 1 -- 172.21.15.16:0/4017934862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c00ad40 con 0x7f21a415cc90 2023-11-13T11:03:52.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a0ff9640 1 -- 172.21.15.16:0/4017934862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c012400 con 0x7f21a415cc90 2023-11-13T11:03:52.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a0ff9640 1 -- 172.21.15.16:0/4017934862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c00aea0 con 0x7f21a415cc90 2023-11-13T11:03:52.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a41f6d50 con 0x7f21a415cc90 2023-11-13T11:03:52.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.206+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a415d490 con 0x7f21a415cc90 2023-11-13T11:03:52.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.208+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a415d680 con 0x7f21a415cc90 2023-11-13T11:03:52.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.208+0000 7f21a0ff9640 1 -- 172.21.15.16:0/4017934862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f219c007070 con 0x7f21a415cc90 2023-11-13T11:03:52.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.209+0000 7f21a0ff9640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f219006ae40 0x7f219006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:52.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.210+0000 7f21a0ff9640 1 -- 172.21.15.16:0/4017934862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f219c089610 con 0x7f21a415cc90 2023-11-13T11:03:52.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.210+0000 7f21a3fff640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f219006ae40 0x7f219006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:52.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.211+0000 7f21a3fff640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f219006ae40 0x7f219006d300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2194006fd0 tx=0x7f2194008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:52.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.214+0000 7f21a0ff9640 1 -- 172.21.15.16:0/4017934862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f219c0545d0 con 0x7f21a415cc90 2023-11-13T11:03:52.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.448+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21a404e4d0 con 0x7f21a415cc90 2023-11-13T11:03:52.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.449+0000 7f21a0ff9640 1 -- 172.21.15.16:0/4017934862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f219c057bf0 con 0x7f21a415cc90 2023-11-13T11:03:52.455 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:52.455 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/960469256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":960469256},{"type":"v1","addr":"172.21.15.16:6833","nonce":960469256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1265200946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1265200946},{"type":"v1","addr":"172.21.15.184:6837","nonce":1265200946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3768737652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3768737652},{"type":"v1","addr":"172.21.15.16:6835","nonce":3768737652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:50.999121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"b","rank":0,"incarnation":43,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.184:6836/2632887594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2632887594},{"type":"v1","addr":"172.21.15.184:6836","nonce":2632887594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:03:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2023-11-13T11:03:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f219006ae40 msgr2=0x7f219006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:52.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f219006ae40 0x7f219006d300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2194006fd0 tx=0x7f2194008040 comp rx=0 tx=0).stop 2023-11-13T11:03:52.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415cc90 msgr2=0x7f21a415d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:52.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415cc90 0x7f21a415d9b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f219c00cbc0 tx=0x7f219c00a430 comp rx=0 tx=0).stop 2023-11-13T11:03:52.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 shutdown_connections 2023-11-13T11:03:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f219006ae40 0x7f219006d300 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415cc90 0x7f21a415d9b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a4071320 0x7f21a414b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 --2- 172.21.15.16:0/4017934862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a4070cf0 0x7f21a41f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.452+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 >> 172.21.15.16:0/4017934862 conn(0x7f21a40b04a0 msgr2=0x7f21a40ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.453+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 shutdown_connections 2023-11-13T11:03:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:52.453+0000 7f21a9135640 1 -- 172.21.15.16:0/4017934862 wait complete. 2023-11-13T11:03:52.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-13T11:03:19.913517+0000', 'modified': '2023-11-13T11:03:50.999121+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7200}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7200': {'gid': 7200, 'name': 'b', 'rank': 0, 'incarnation': 43, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.184:6836/2632887594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 2632887594}, {'type': 'v1', 'addr': '172.21.15.184:6836', 'nonce': 2632887594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-13T11:03:52.880 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7200:up:reconnect 2023-11-13T11:03:53.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.114+0000 7fe95c9bf640 1 Processor -- start 2023-11-13T11:03:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.114+0000 7fe95c9bf640 1 -- start start 2023-11-13T11:03:54.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.115+0000 7fe95c9bf640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe9580d2ae0 0x7fe9580d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:54.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.115+0000 7fe95c9bf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9580b29d0 0x7fe9580b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:54.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.115+0000 7fe95c9bf640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9580b3cf0 con 0x7fe95815cfc0 2023-11-13T11:03:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.115+0000 7fe95c9bf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9580d3480 con 0x7fe9580b29d0 2023-11-13T11:03:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.115+0000 7fe95c9bf640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9580cde80 con 0x7fe9580d2ae0 2023-11-13T11:03:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.116+0000 7fe95759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9580b29d0 0x7fe9580b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.116+0000 7fe95759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9580b29d0 0x7fe9580b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52086/0 (socket says 172.21.15.16:52086) 2023-11-13T11:03:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.116+0000 7fe95759e640 1 -- 172.21.15.16:0/4274312606 learned_addr learned my addr 172.21.15.16:0/4274312606 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.116+0000 7fe95659c640 1 --2- 172.21.15.16:0/4274312606 >> v2:172.21.15.184:3300/0 conn(0x7fe9580d2ae0 0x7fe9580d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.116+0000 7fe95759e640 1 -- 172.21.15.16:0/4274312606 >> v2:172.21.15.184:3300/0 conn(0x7fe9580d2ae0 msgr2=0x7fe9580d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.116+0000 7fe95759e640 1 --2- 172.21.15.16:0/4274312606 >> v2:172.21.15.184:3300/0 conn(0x7fe9580d2ae0 0x7fe9580d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.116+0000 7fe95759e640 1 -- 172.21.15.16:0/4274312606 >> v1:172.21.15.184:6789/0 conn(0x7fe95815cfc0 legacy=0x7fe9580cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.116+0000 7fe95759e640 1 -- 172.21.15.16:0/4274312606 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580b3730 con 0x7fe9580b29d0 2023-11-13T11:03:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.118+0000 7fe95759e640 1 --2- 172.21.15.16:0/4274312606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580b29d0 0x7fe9580b2e40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe95808ce30 tx=0x7fe95002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa893426a170171 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:54.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.118+0000 7fe955d9b640 1 -- 172.21.15.16:0/4274312606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9500317e0 con 0x7fe9580b29d0 2023-11-13T11:03:54.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.118+0000 7fe955d9b640 1 -- 172.21.15.16:0/4274312606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe950031940 con 0x7fe9580b29d0 2023-11-13T11:03:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.118+0000 7fe955d9b640 1 -- 172.21.15.16:0/4274312606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950031cd0 con 0x7fe9580b29d0 2023-11-13T11:03:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.119+0000 7fe95c9bf640 1 -- 172.21.15.16:0/4274312606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580b29d0 msgr2=0x7fe9580b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.119+0000 7fe95c9bf640 1 --2- 172.21.15.16:0/4274312606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580b29d0 0x7fe9580b2e40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe95808ce30 tx=0x7fe95002b880 comp rx=0 tx=0).stop 2023-11-13T11:03:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.119+0000 7fe95c9bf640 1 -- 172.21.15.16:0/4274312606 shutdown_connections 2023-11-13T11:03:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.119+0000 7fe95c9bf640 1 --2- 172.21.15.16:0/4274312606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580b29d0 0x7fe9580b2e40 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.119+0000 7fe95c9bf640 1 --2- 172.21.15.16:0/4274312606 >> v2:172.21.15.184:3300/0 conn(0x7fe9580d2ae0 0x7fe9580d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.119+0000 7fe95c9bf640 1 -- 172.21.15.16:0/4274312606 >> 172.21.15.16:0/4274312606 conn(0x7fe95808c570 msgr2=0x7fe95809be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.119+0000 7fe95c9bf640 1 -- 172.21.15.16:0/4274312606 shutdown_connections 2023-11-13T11:03:54.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.119+0000 7fe95c9bf640 1 -- 172.21.15.16:0/4274312606 wait complete. 2023-11-13T11:03:54.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.121+0000 7fe95c9bf640 1 Processor -- start 2023-11-13T11:03:54.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.121+0000 7fe95c9bf640 1 -- start start 2023-11-13T11:03:54.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.122+0000 7fe95c9bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9580b29d0 0x7fe9581f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:54.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.122+0000 7fe95c9bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580d2ae0 0x7fe95815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:54.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.122+0000 7fe95c9bf640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe95815cfc0 0x7fe9581f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:54.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.122+0000 7fe95c9bf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9581f3c90 con 0x7fe9580d2ae0 2023-11-13T11:03:54.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.122+0000 7fe95c9bf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95815c5d0 con 0x7fe9580b29d0 2023-11-13T11:03:54.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.122+0000 7fe95c9bf640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9581f3190 con 0x7fe95815cfc0 2023-11-13T11:03:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.122+0000 7fe95659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580d2ae0 0x7fe95815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.122+0000 7fe95659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580d2ae0 0x7fe95815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52096/0 (socket says 172.21.15.16:52096) 2023-11-13T11:03:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.122+0000 7fe95659c640 1 -- 172.21.15.16:0/3310180410 learned_addr learned my addr 172.21.15.16:0/3310180410 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe956d9d640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9580b29d0 0x7fe9581f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe95659c640 1 -- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe95815cfc0 msgr2=0x7fe9581f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe95659c640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe95815cfc0 0x7fe9581f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe95659c640 1 -- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9580b29d0 msgr2=0x7fe9581f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe95659c640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9580b29d0 0x7fe9581f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe95659c640 1 -- 172.21.15.16:0/3310180410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9500094d0 con 0x7fe9580d2ae0 2023-11-13T11:03:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe956d9d640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9580b29d0 0x7fe9581f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe95659c640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580d2ae0 0x7fe95815c090 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe94000d5f0 tx=0x7fe94000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe94f7fe640 1 -- 172.21.15.16:0/3310180410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9400125f0 con 0x7fe9580d2ae0 2023-11-13T11:03:54.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe94f7fe640 1 -- 172.21.15.16:0/3310180410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe940012750 con 0x7fe9580d2ae0 2023-11-13T11:03:54.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe94f7fe640 1 -- 172.21.15.16:0/3310180410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940012b00 con 0x7fe9580d2ae0 2023-11-13T11:03:54.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.123+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9581f3470 con 0x7fe9580d2ae0 2023-11-13T11:03:54.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.124+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9580cc0f0 con 0x7fe9580d2ae0 2023-11-13T11:03:54.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.125+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9580cc280 con 0x7fe9580d2ae0 2023-11-13T11:03:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.126+0000 7fe94f7fe640 1 -- 172.21.15.16:0/3310180410 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe940012c60 con 0x7fe9580d2ae0 2023-11-13T11:03:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.127+0000 7fe94f7fe640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe93806ae40 0x7fe93806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.127+0000 7fe94f7fe640 1 -- 172.21.15.16:0/3310180410 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fe940089d20 con 0x7fe9580d2ae0 2023-11-13T11:03:54.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.127+0000 7fe956d9d640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe93806ae40 0x7fe93806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:54.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.128+0000 7fe956d9d640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe93806ae40 0x7fe93806d300 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe948004520 tx=0x7fe948009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:54.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.131+0000 7fe94f7fe640 1 -- 172.21.15.16:0/3310180410 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe940054b90 con 0x7fe9580d2ae0 2023-11-13T11:03:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.366+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe95804e4d0 con 0x7fe9580d2ae0 2023-11-13T11:03:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.367+0000 7fe94f7fe640 1 -- 172.21.15.16:0/3310180410 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fe9400581b0 con 0x7fe9580d2ae0 2023-11-13T11:03:54.375 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:54.375 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/960469256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":960469256},{"type":"v1","addr":"172.21.15.16:6833","nonce":960469256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1265200946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1265200946},{"type":"v1","addr":"172.21.15.184:6837","nonce":1265200946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3768737652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3768737652},{"type":"v1","addr":"172.21.15.16:6835","nonce":3768737652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:53.003626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"b","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6836/2632887594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2632887594},{"type":"v1","addr":"172.21.15.184:6836","nonce":2632887594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:03:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2023-11-13T11:03:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.370+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe93806ae40 msgr2=0x7fe93806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.370+0000 7fe95c9bf640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe93806ae40 0x7fe93806d300 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe948004520 tx=0x7fe948009040 comp rx=0 tx=0).stop 2023-11-13T11:03:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.370+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580d2ae0 msgr2=0x7fe95815c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.370+0000 7fe95c9bf640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580d2ae0 0x7fe95815c090 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe94000d5f0 tx=0x7fe94000dab0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.371+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 shutdown_connections 2023-11-13T11:03:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.371+0000 7fe95c9bf640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe93806ae40 0x7fe93806d300 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.371+0000 7fe95c9bf640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe95815cfc0 0x7fe9581f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.371+0000 7fe95c9bf640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9580d2ae0 0x7fe95815c090 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.371+0000 7fe95c9bf640 1 --2- 172.21.15.16:0/3310180410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9580b29d0 0x7fe9581f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.371+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 >> 172.21.15.16:0/3310180410 conn(0x7fe95808c570 msgr2=0x7fe9580c86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.371+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 shutdown_connections 2023-11-13T11:03:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:54.371+0000 7fe95c9bf640 1 -- 172.21.15.16:0/3310180410 wait complete. 2023-11-13T11:03:54.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-13T11:03:19.913517+0000', 'modified': '2023-11-13T11:03:53.003626+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7200}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7200': {'gid': 7200, 'name': 'b', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.184:6836/2632887594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 2632887594}, {'type': 'v1', 'addr': '172.21.15.184:6836', 'nonce': 2632887594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-13T11:03:54.801 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-13T11:03:54.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-13T11:03:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.032+0000 7f32b64bb640 1 Processor -- start 2023-11-13T11:03:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.032+0000 7f32b64bb640 1 -- start start 2023-11-13T11:03:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.034+0000 7f32b64bb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b015b190 0x7f32b006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.034+0000 7f32b64bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32b006a190 0x7f32b006a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.034+0000 7f32b64bb640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b0158c20 con 0x7f32b00cb190 2023-11-13T11:03:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.034+0000 7f32b64bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b00b1830 con 0x7f32b006a190 2023-11-13T11:03:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.034+0000 7f32b64bb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b006ab10 con 0x7f32b015b190 2023-11-13T11:03:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.034+0000 7f32b5cba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32b006a190 0x7f32b006a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.034+0000 7f32b5cba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32b006a190 0x7f32b006a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52102/0 (socket says 172.21.15.16:52102) 2023-11-13T11:03:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.034+0000 7f32b5cba640 1 -- 172.21.15.16:0/623295834 learned_addr learned my addr 172.21.15.16:0/623295834 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.035+0000 7f32b4cb8640 1 --2- 172.21.15.16:0/623295834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b015b190 0x7f32b006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.035+0000 7f32b5cba640 1 -- 172.21.15.16:0/623295834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b015b190 msgr2=0x7f32b006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.035+0000 7f32b5cba640 1 --2- 172.21.15.16:0/623295834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b015b190 0x7f32b006f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.035+0000 7f32b5cba640 1 -- 172.21.15.16:0/623295834 >> v1:172.21.15.184:6789/0 conn(0x7f32b00cb190 legacy=0x7f32b00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.035+0000 7f32b5cba640 1 -- 172.21.15.16:0/623295834 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b015b780 con 0x7f32b006a190 2023-11-13T11:03:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.037+0000 7f32b5cba640 1 --2- 172.21.15.16:0/623295834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b006a190 0x7f32b006a5d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009e50 tx=0x7f32a802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d88a299f4c8b26 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.037+0000 7f32a3fff640 1 -- 172.21.15.16:0/623295834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a802c9e0 con 0x7f32b006a190 2023-11-13T11:03:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.037+0000 7f32a3fff640 1 -- 172.21.15.16:0/623295834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a802cb40 con 0x7f32b006a190 2023-11-13T11:03:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.037+0000 7f32a3fff640 1 -- 172.21.15.16:0/623295834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80343f0 con 0x7f32b006a190 2023-11-13T11:03:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.038+0000 7f32b64bb640 1 -- 172.21.15.16:0/623295834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b006a190 msgr2=0x7f32b006a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.038+0000 7f32b64bb640 1 --2- 172.21.15.16:0/623295834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b006a190 0x7f32b006a5d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009e50 tx=0x7f32a802be40 comp rx=0 tx=0).stop 2023-11-13T11:03:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.038+0000 7f32b64bb640 1 -- 172.21.15.16:0/623295834 shutdown_connections 2023-11-13T11:03:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.038+0000 7f32b64bb640 1 --2- 172.21.15.16:0/623295834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b006a190 0x7f32b006a5d0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.038+0000 7f32b64bb640 1 --2- 172.21.15.16:0/623295834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b015b190 0x7f32b006f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.038+0000 7f32b64bb640 1 -- 172.21.15.16:0/623295834 >> 172.21.15.16:0/623295834 conn(0x7f32b00b04a0 msgr2=0x7f32b00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:55.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.038+0000 7f32b64bb640 1 -- 172.21.15.16:0/623295834 shutdown_connections 2023-11-13T11:03:55.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.038+0000 7f32b64bb640 1 -- 172.21.15.16:0/623295834 wait complete. 2023-11-13T11:03:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.040+0000 7f32b64bb640 1 Processor -- start 2023-11-13T11:03:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.040+0000 7f32b64bb640 1 -- start start 2023-11-13T11:03:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.041+0000 7f32b64bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b006a190 0x7f32b01f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.041+0000 7f32b64bb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b00cb190 0x7f32b015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.041+0000 7f32b64bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015b190 0x7f32b01f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.041+0000 7f32b64bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b015c620 con 0x7f32b015b190 2023-11-13T11:03:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.041+0000 7f32b64bb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b014db70 con 0x7f32b006a190 2023-11-13T11:03:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.041+0000 7f32b64bb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b015c260 con 0x7f32b00cb190 2023-11-13T11:03:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.041+0000 7f32b5cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015b190 0x7f32b01f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.041+0000 7f32b5cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015b190 0x7f32b01f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52104/0 (socket says 172.21.15.16:52104) 2023-11-13T11:03:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.041+0000 7f32b5cba640 1 -- 172.21.15.16:0/4211604777 learned_addr learned my addr 172.21.15.16:0/4211604777 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b4cb8640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b00cb190 0x7f32b015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b54b9640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b006a190 0x7f32b01f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b5cba640 1 -- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b00cb190 msgr2=0x7f32b015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b5cba640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b00cb190 0x7f32b015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b5cba640 1 -- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b006a190 msgr2=0x7f32b01f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b5cba640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b006a190 0x7f32b01f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b5cba640 1 -- 172.21.15.16:0/4211604777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a8009b30 con 0x7f32b015b190 2023-11-13T11:03:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b4cb8640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b00cb190 0x7f32b015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b54b9640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b006a190 0x7f32b01f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32b5cba640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015b190 0x7f32b01f5b70 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f32a8036040 tx=0x7f32a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.042+0000 7f32a1ffb640 1 -- 172.21.15.16:0/4211604777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8009510 con 0x7f32b015b190 2023-11-13T11:03:55.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.043+0000 7f32a1ffb640 1 -- 172.21.15.16:0/4211604777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a803e450 con 0x7f32b015b190 2023-11-13T11:03:55.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.043+0000 7f32a1ffb640 1 -- 172.21.15.16:0/4211604777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a803e7e0 con 0x7f32b015b190 2023-11-13T11:03:55.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.043+0000 7f32b64bb640 1 -- 172.21.15.16:0/4211604777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b01f63a0 con 0x7f32b015b190 2023-11-13T11:03:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.043+0000 7f32b64bb640 1 -- 172.21.15.16:0/4211604777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b01f6960 con 0x7f32b015b190 2023-11-13T11:03:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.045+0000 7f32a1ffb640 1 -- 172.21.15.16:0/4211604777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32a803e940 con 0x7f32b015b190 2023-11-13T11:03:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.046+0000 7f32a1ffb640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f329406ae40 0x7f329406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.046+0000 7f32a1ffb640 1 -- 172.21.15.16:0/4211604777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f32a80ac5a0 con 0x7f32b015b190 2023-11-13T11:03:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.047+0000 7f32b54b9640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f329406ae40 0x7f329406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.047+0000 7f32ba38a740 1 Processor -- start 2023-11-13T11:03:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.047+0000 7f32ba38a740 1 -- start start 2023-11-13T11:03:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.048+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d8660390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.048+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d865f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.048+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.048+0000 7f32ba38a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d85be880 con 0x5653d865ffb0 2023-11-13T11:03:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.048+0000 7f32ba38a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d85cdae0 con 0x5653d865f6f0 2023-11-13T11:03:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.048+0000 7f32ba38a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653d85cb1d0 con 0x5653d865acb0 2023-11-13T11:03:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.048+0000 7f32b54b9640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f329406ae40 0x7f329406d300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f32ac001730 tx=0x7f32ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.048+0000 7f32b54b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d8660390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b54b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d8660390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52116/0 (socket says 172.21.15.16:52116) 2023-11-13T11:03:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b54b9640 1 -- 172.21.15.16:0/349787616 learned_addr learned my addr 172.21.15.16:0/349787616 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b4cb8640 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d865f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b5cba640 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b54b9640 1 -- 172.21.15.16:0/349787616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 msgr2=0x5653d865f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b54b9640 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d865f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b54b9640 1 -- 172.21.15.16:0/349787616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 msgr2=0x5653d86a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b54b9640 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b54b9640 1 -- 172.21.15.16:0/349787616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d858e860 con 0x5653d865ffb0 2023-11-13T11:03:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b5cba640 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.049+0000 7f32b54b9640 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d8660390 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00daf0 tx=0x7f32ac010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10448d96c84584fd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.050+0000 7f3292ffd640 1 -- 172.21.15.16:0/349787616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac014db0 con 0x5653d865ffb0 2023-11-13T11:03:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.050+0000 7f3292ffd640 1 -- 172.21.15.16:0/349787616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac007030 con 0x5653d865ffb0 2023-11-13T11:03:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.050+0000 7f3292ffd640 1 -- 172.21.15.16:0/349787616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac0073a0 con 0x5653d865ffb0 2023-11-13T11:03:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.050+0000 7f32ba38a740 1 -- 172.21.15.16:0/349787616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 msgr2=0x5653d8660390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.050+0000 7f32ba38a740 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d8660390 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00daf0 tx=0x7f32ac010000 comp rx=0 tx=0).stop 2023-11-13T11:03:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.051+0000 7f32ba38a740 1 -- 172.21.15.16:0/349787616 shutdown_connections 2023-11-13T11:03:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.051+0000 7f32ba38a740 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.051+0000 7f32ba38a740 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d865f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.051+0000 7f32ba38a740 1 --2- 172.21.15.16:0/349787616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d8660390 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.051+0000 7f32ba38a740 1 -- 172.21.15.16:0/349787616 >> 172.21.15.16:0/349787616 conn(0x5653d8621bb0 msgr2=0x5653d86a5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.051+0000 7f32ba38a740 1 -- 172.21.15.16:0/349787616 shutdown_connections 2023-11-13T11:03:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.051+0000 7f32ba38a740 1 -- 172.21.15.16:0/349787616 wait complete. 2023-11-13T11:03:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.051+0000 7f32ba38a740 1 -- start start 2023-11-13T11:03:55.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.052+0000 7f32ba38a740 1 Processor -- start 2023-11-13T11:03:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d8685040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32ba38a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d85c6ac0 con 0x5653d865ffb0 2023-11-13T11:03:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32ba38a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d85632f0 con 0x5653d865f6f0 2023-11-13T11:03:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32ba38a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653d8580c70 con 0x5653d865acb0 2023-11-13T11:03:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32ba38a740 10 client.0 _ll_register_callbacks cb 0x5653d85853b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:03:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32b5cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32b5cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52124/0 (socket says 172.21.15.16:52124) 2023-11-13T11:03:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32b5cba640 1 -- 172.21.15.16:0/1667095682 learned_addr learned my addr 172.21.15.16:0/1667095682 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32b54b9640 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d8685040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.053+0000 7f32b4cb8640 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.054+0000 7f32b5cba640 1 -- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 msgr2=0x5653d8685040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.054+0000 7f32b5cba640 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d8685040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.054+0000 7f32b5cba640 1 -- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 msgr2=0x5653d86ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.054+0000 7f32b5cba640 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.054+0000 7f32b5cba640 1 -- 172.21.15.16:0/1667095682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac00d7d0 con 0x5653d865ffb0 2023-11-13T11:03:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.054+0000 7f32b4cb8640 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.054+0000 7f32b5cba640 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a89a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f32a802c450 tx=0x7f32a807b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec4ea46a8ce6ebd server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.054+0000 7f328bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:03:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.056+0000 7f328bfff640 1 -- 172.21.15.16:0/1667095682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a807f0d0 con 0x5653d865ffb0 2023-11-13T11:03:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.056+0000 7f328bfff640 1 -- 172.21.15.16:0/1667095682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f32a807f2b0 con 0x5653d865ffb0 2023-11-13T11:03:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.056+0000 7f32ba38a740 1 -- 172.21.15.16:0/1667095682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653d85bfaa0 con 0x5653d865ffb0 2023-11-13T11:03:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.056+0000 7f328bfff640 1 -- 172.21.15.16:0/1667095682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32a8083460 con 0x5653d865ffb0 2023-11-13T11:03:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.056+0000 7f32ba38a740 10 client.7245 fetch_fsmap learned FSMap version 46 2023-11-13T11:03:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.056+0000 7f32ba38a740 1 -- 172.21.15.16:0/1667095682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x5653d85c5140 con 0x5653d865ffb0 2023-11-13T11:03:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.056+0000 7f328bfff640 1 -- 172.21.15.16:0/1667095682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5653d85bfaa0 con 0x5653d865ffb0 2023-11-13T11:03:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.056+0000 7f328bfff640 1 -- 172.21.15.16:0/1667095682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32a804f020 con 0x5653d865ffb0 2023-11-13T11:03:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.057+0000 7f32ba38a740 10 client.7245 fetch_fsmap finished waiting for FSMap version 46 2023-11-13T11:03:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.057+0000 7f32ba38a740 10 client.7245 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-13T11:03:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.057+0000 7f32ba38a740 20 client.7245 populate_metadata read hostname 'smithi016' 2023-11-13T11:03:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.057+0000 7f32ba38a740 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] conn(0x5653d8683d70 0x5653d8684150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.057+0000 7f32ba38a740 4 client.7245 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-13T11:03:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.057+0000 7f32ba38a740 1 -- 172.21.15.16:0/1667095682 --> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653d85bf7c0 con 0x5653d8683d70 2023-11-13T11:03:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.057+0000 7f32b54b9640 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] conn(0x5653d8683d70 0x5653d8684150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.058+0000 7f32b54b9640 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] conn(0x5653d8683d70 0x5653d8684150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd5118ffd9788b1 server_cookie=f5c056e5f6149eb3 in_seq=0 out_seq=0 2023-11-13T11:03:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.058+0000 7f328bfff640 10 client.7245 ms_handle_connect on v2:172.21.15.184:6834/2632887594 2023-11-13T11:03:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.060+0000 7f328bfff640 1 -- 172.21.15.16:0/1667095682 <== mds.0 v2:172.21.15.184:6834/2632887594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653d85bf7c0 con 0x5653d8683d70 2023-11-13T11:03:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.060+0000 7f328bfff640 10 client.7245 handle_command_reply: tid=0 2023-11-13T11:03:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.060+0000 7f32ba38a740 1 client.7245 shutdown 2023-11-13T11:03:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 -- 172.21.15.16:0/1667095682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 msgr2=0x5653d86a89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a89a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f32a802c450 tx=0x7f32a807b860 comp rx=0 tx=0).stop 2023-11-13T11:03:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 -- 172.21.15.16:0/1667095682 shutdown_connections 2023-11-13T11:03:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] conn(0x5653d8683d70 0x5653d8684150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a89a0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 --2- 172.21.15.16:0/1667095682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d8685040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 -- 172.21.15.16:0/1667095682 >> 172.21.15.16:0/1667095682 conn(0x5653d8621bb0 msgr2=0x5653d86a5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f328bfff640 0 client.7245 ms_handle_reset on v2:172.21.15.184:6834/2632887594 2023-11-13T11:03:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 -- 172.21.15.16:0/1667095682 shutdown_connections 2023-11-13T11:03:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 1 -- 172.21.15.16:0/1667095682 wait complete. 2023-11-13T11:03:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.061+0000 7f32ba38a740 20 client.7245 trim_cache size 0 max 16384 2023-11-13T11:03:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.078+0000 7f32ba38a740 1 Processor -- start 2023-11-13T11:03:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.079+0000 7f32ba38a740 1 -- start start 2023-11-13T11:03:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.079+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d865ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.079+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.079+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.079+0000 7f32ba38a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d8329f20 con 0x5653d865ffb0 2023-11-13T11:03:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.079+0000 7f32ba38a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d8660f50 con 0x5653d865f6f0 2023-11-13T11:03:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.079+0000 7f32ba38a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653d8661880 con 0x5653d865acb0 2023-11-13T11:03:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.080+0000 7f32b5cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.080+0000 7f32b5cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52134/0 (socket says 172.21.15.16:52134) 2023-11-13T11:03:55.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.080+0000 7f32b5cba640 1 -- 172.21.15.16:0/749652492 learned_addr learned my addr 172.21.15.16:0/749652492 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.080+0000 7f32b5cba640 1 -- 172.21.15.16:0/749652492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 msgr2=0x5653d865ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.080+0000 7f32b5cba640 1 --2- 172.21.15.16:0/749652492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d865ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.080+0000 7f32b5cba640 1 -- 172.21.15.16:0/749652492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 msgr2=0x5653d86a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.080+0000 7f32b5cba640 1 --2- 172.21.15.16:0/749652492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.080+0000 7f32b5cba640 1 -- 172.21.15.16:0/749652492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d8562b80 con 0x5653d865ffb0 2023-11-13T11:03:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.080+0000 7f32b5cba640 1 --2- 172.21.15.16:0/749652492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a5fe0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009e50 tx=0x7f32a8043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1a3eda4dc3cf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32937fe640 1 -- 172.21.15.16:0/749652492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a807f0d0 con 0x5653d865ffb0 2023-11-13T11:03:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32937fe640 1 -- 172.21.15.16:0/749652492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a807f2b0 con 0x5653d865ffb0 2023-11-13T11:03:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32937fe640 1 -- 172.21.15.16:0/749652492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8083460 con 0x5653d865ffb0 2023-11-13T11:03:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32ba38a740 1 -- 172.21.15.16:0/749652492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 msgr2=0x5653d86a5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32ba38a740 1 --2- 172.21.15.16:0/749652492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a5fe0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009e50 tx=0x7f32a8043000 comp rx=0 tx=0).stop 2023-11-13T11:03:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32ba38a740 1 -- 172.21.15.16:0/749652492 shutdown_connections 2023-11-13T11:03:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32ba38a740 1 --2- 172.21.15.16:0/749652492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d86a5fe0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32ba38a740 1 --2- 172.21.15.16:0/749652492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865f6f0 0x5653d86a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32ba38a740 1 --2- 172.21.15.16:0/749652492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865acb0 0x5653d865ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32ba38a740 1 -- 172.21.15.16:0/749652492 >> 172.21.15.16:0/749652492 conn(0x5653d8621bb0 msgr2=0x5653d8660390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32ba38a740 1 -- 172.21.15.16:0/749652492 shutdown_connections 2023-11-13T11:03:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.081+0000 7f32ba38a740 1 -- 172.21.15.16:0/749652492 wait complete. 2023-11-13T11:03:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.082+0000 7f32ba38a740 1 -- start start 2023-11-13T11:03:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.082+0000 7f32ba38a740 1 Processor -- start 2023-11-13T11:03:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.083+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865acb0 0x5653d86a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.083+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865f6f0 0x5653d86a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.083+0000 7f32ba38a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d870f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.083+0000 7f32ba38a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d8710850 con 0x5653d865ffb0 2023-11-13T11:03:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.083+0000 7f32ba38a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d8683320 con 0x5653d865acb0 2023-11-13T11:03:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.083+0000 7f32ba38a740 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653d86805e0 con 0x5653d865f6f0 2023-11-13T11:03:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.083+0000 7f32ba38a740 10 client.0 _ll_register_callbacks cb 0x7f32a80774a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-13T11:03:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b5cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d870f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b5cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d870f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52136/0 (socket says 172.21.15.16:52136) 2023-11-13T11:03:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b5cba640 1 -- 172.21.15.16:0/2139675929 learned_addr learned my addr 172.21.15.16:0/2139675929 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b5cba640 1 -- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865f6f0 msgr2=0x5653d86a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b4cb8640 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865f6f0 0x5653d86a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b54b9640 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865acb0 0x5653d86a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b5cba640 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865f6f0 0x5653d86a5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b5cba640 1 -- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865acb0 msgr2=0x5653d86a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b5cba640 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865acb0 0x5653d86a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b5cba640 1 -- 172.21.15.16:0/2139675929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac00d7d0 con 0x5653d865ffb0 2023-11-13T11:03:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b54b9640 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865acb0 0x5653d86a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f32b5cba640 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d870f530 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f32a807b0f0 tx=0x7f32a80aa7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e7f1cb99c86817 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.084+0000 7f328bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-13T11:03:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f328bfff640 1 -- 172.21.15.16:0/2139675929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a803edf0 con 0x5653d865ffb0 2023-11-13T11:03:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f328bfff640 1 -- 172.21.15.16:0/2139675929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f32a8083460 con 0x5653d865ffb0 2023-11-13T11:03:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f32ba38a740 1 -- 172.21.15.16:0/2139675929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32ac007030 con 0x5653d865ffb0 2023-11-13T11:03:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f328bfff640 1 -- 172.21.15.16:0/2139675929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32a807f0d0 con 0x5653d865ffb0 2023-11-13T11:03:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f32ba38a740 10 client.7251 fetch_fsmap learned FSMap version 46 2023-11-13T11:03:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f32ba38a740 1 -- 172.21.15.16:0/2139675929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x5653d8588170 con 0x5653d865ffb0 2023-11-13T11:03:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f328bfff640 1 -- 172.21.15.16:0/2139675929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac007030 con 0x5653d865ffb0 2023-11-13T11:03:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f328bfff640 1 -- 172.21.15.16:0/2139675929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32a804f040 con 0x5653d865ffb0 2023-11-13T11:03:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f32ba38a740 10 client.7251 fetch_fsmap finished waiting for FSMap version 46 2023-11-13T11:03:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f32ba38a740 10 client.7251 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-13T11:03:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f32ba38a740 20 client.7251 populate_metadata read hostname 'smithi016' 2023-11-13T11:03:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f32ba38a740 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] conn(0x5653d87103a0 0x5653d870fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f32ba38a740 4 client.7251 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-13T11:03:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.085+0000 7f32ba38a740 1 -- 172.21.15.16:0/2139675929 --> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5653d82c76c0 con 0x5653d87103a0 2023-11-13T11:03:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.086+0000 7f32b54b9640 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] conn(0x5653d87103a0 0x5653d870fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.087+0000 7f32b54b9640 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] conn(0x5653d87103a0 0x5653d870fa70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3e7f542f3bdb3c server_cookie=f6794da151cde424 in_seq=0 out_seq=0 2023-11-13T11:03:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.087+0000 7f328bfff640 10 client.7251 ms_handle_connect on v2:172.21.15.184:6834/2632887594 2023-11-13T11:03:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f328bfff640 1 -- 172.21.15.16:0/2139675929 <== mds.0 v2:172.21.15.184:6834/2632887594 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5653d82c76c0 con 0x5653d87103a0 2023-11-13T11:03:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f328bfff640 10 client.7251 handle_command_reply: tid=0 2023-11-13T11:03:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 client.7251 shutdown 2023-11-13T11:03:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 -- 172.21.15.16:0/2139675929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 msgr2=0x5653d870f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d870f530 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f32a807b0f0 tx=0x7f32a80aa7f0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 -- 172.21.15.16:0/2139675929 shutdown_connections 2023-11-13T11:03:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:6834/2632887594,v1:172.21.15.184:6836/2632887594] conn(0x5653d87103a0 0x5653d870fa70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5653d865ffb0 0x5653d870f530 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5653d865f6f0 0x5653d86a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 --2- 172.21.15.16:0/2139675929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653d865acb0 0x5653d86a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f328bfff640 0 client.7251 ms_handle_reset on v2:172.21.15.184:6834/2632887594 2023-11-13T11:03:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 -- 172.21.15.16:0/2139675929 >> 172.21.15.16:0/2139675929 conn(0x5653d8621bb0 msgr2=0x5653d86abe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 -- 172.21.15.16:0/2139675929 shutdown_connections 2023-11-13T11:03:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 1 -- 172.21.15.16:0/2139675929 wait complete. 2023-11-13T11:03:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.088+0000 7f32ba38a740 20 client.7251 trim_cache size 0 max 16384 2023-11-13T11:03:55.105 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:03:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9", 2023-11-13T11:03:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-13T11:03:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7200, 2023-11-13T11:03:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-13T11:03:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-13T11:03:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-13T11:03:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 4.1146099730000003, 2023-11-13T11:03:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 46, 2023-11-13T11:03:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 73, 2023-11-13T11:03:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 73, 2023-11-13T11:03:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 4.8544858980000001 2023-11-13T11:03:55.107 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:03:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 -- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f329406ae40 msgr2=0x7f329406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f329406ae40 0x7f329406d300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f32ac001730 tx=0x7f32ac009040 comp rx=0 tx=0).stop 2023-11-13T11:03:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 -- 172.21.15.16:0/4211604777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015b190 msgr2=0x7f32b01f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015b190 0x7f32b01f5b70 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f32a8036040 tx=0x7f32a8004a40 comp rx=0 tx=0).stop 2023-11-13T11:03:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 -- 172.21.15.16:0/4211604777 shutdown_connections 2023-11-13T11:03:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f329406ae40 0x7f329406d300 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015b190 0x7f32b01f5b70 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32b00cb190 0x7f32b015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 --2- 172.21.15.16:0/4211604777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b006a190 0x7f32b01f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 -- 172.21.15.16:0/4211604777 >> 172.21.15.16:0/4211604777 conn(0x7f32b00b04a0 msgr2=0x7f32b00b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 -- 172.21.15.16:0/4211604777 shutdown_connections 2023-11-13T11:03:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.089+0000 7f32b64bb640 1 -- 172.21.15.16:0/4211604777 wait complete. 2023-11-13T11:03:55.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_split_size 100 2023-11-13T11:03:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.329+0000 7fd709578640 1 Processor -- start 2023-11-13T11:03:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.329+0000 7fd709578640 1 -- start start 2023-11-13T11:03:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.331+0000 7fd709578640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7040ce550 0x7fd7040ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.331+0000 7fd709578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 0x7fd7040b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.331+0000 7fd709578640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7040ceef0 con 0x7fd704144570 2023-11-13T11:03:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.331+0000 7fd709578640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd704144950 con 0x7fd7040ce550 2023-11-13T11:03:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.331+0000 7fd709578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040ce300 con 0x7fd7040b19d0 2023-11-13T11:03:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.331+0000 7fd7027fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7040ce550 0x7fd7040ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.331+0000 7fd7027fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7040ce550 0x7fd7040ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52138/0 (socket says 172.21.15.16:52138) 2023-11-13T11:03:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.332+0000 7fd7027fc640 1 -- 172.21.15.16:0/618406936 learned_addr learned my addr 172.21.15.16:0/618406936 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.332+0000 7fd7037fe640 1 --2- 172.21.15.16:0/618406936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 0x7fd7040b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.332+0000 7fd7027fc640 1 -- 172.21.15.16:0/618406936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 msgr2=0x7fd7040b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.332+0000 7fd7027fc640 1 --2- 172.21.15.16:0/618406936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 0x7fd7040b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.332+0000 7fd7027fc640 1 -- 172.21.15.16:0/618406936 >> v1:172.21.15.184:6789/0 conn(0x7fd704144570 legacy=0x7fd70415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.332+0000 7fd7027fc640 1 -- 172.21.15.16:0/618406936 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7040cd260 con 0x7fd7040ce550 2023-11-13T11:03:55.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.334+0000 7fd7027fc640 1 --2- 172.21.15.16:0/618406936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 0x7fd7040ce9b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009870 tx=0x7fd6f802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf9e6bf57520663 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.335+0000 7fd701ffb640 1 -- 172.21.15.16:0/618406936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8030840 con 0x7fd7040ce550 2023-11-13T11:03:55.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.335+0000 7fd701ffb640 1 -- 172.21.15.16:0/618406936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f80309a0 con 0x7fd7040ce550 2023-11-13T11:03:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.335+0000 7fd701ffb640 1 -- 172.21.15.16:0/618406936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8030d30 con 0x7fd7040ce550 2023-11-13T11:03:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.335+0000 7fd709578640 1 -- 172.21.15.16:0/618406936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 msgr2=0x7fd7040ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.335+0000 7fd709578640 1 --2- 172.21.15.16:0/618406936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 0x7fd7040ce9b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009870 tx=0x7fd6f802b980 comp rx=0 tx=0).stop 2023-11-13T11:03:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.335+0000 7fd709578640 1 -- 172.21.15.16:0/618406936 shutdown_connections 2023-11-13T11:03:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.335+0000 7fd709578640 1 --2- 172.21.15.16:0/618406936 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 0x7fd7040b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.335+0000 7fd709578640 1 --2- 172.21.15.16:0/618406936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 0x7fd7040ce9b0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.335+0000 7fd709578640 1 -- 172.21.15.16:0/618406936 >> 172.21.15.16:0/618406936 conn(0x7fd70406a190 msgr2=0x7fd70406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.336+0000 7fd709578640 1 -- 172.21.15.16:0/618406936 shutdown_connections 2023-11-13T11:03:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.336+0000 7fd709578640 1 -- 172.21.15.16:0/618406936 wait complete. 2023-11-13T11:03:55.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.337+0000 7fd709578640 1 Processor -- start 2023-11-13T11:03:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.338+0000 7fd709578640 1 -- start start 2023-11-13T11:03:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.338+0000 7fd709578640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 0x7fd70413a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd709578640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 0x7fd70414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd709578640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704144570 0x7fd70414c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd709578640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70414bd00 con 0x7fd7040ce550 2023-11-13T11:03:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd709578640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70413a9a0 con 0x7fd704144570 2023-11-13T11:03:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd709578640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70414b0f0 con 0x7fd7040b19d0 2023-11-13T11:03:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7027fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 0x7fd70414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7027fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 0x7fd70414a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52146/0 (socket says 172.21.15.16:52146) 2023-11-13T11:03:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7027fc640 1 -- 172.21.15.16:0/3916386684 learned_addr learned my addr 172.21.15.16:0/3916386684 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd702ffd640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 0x7fd70413a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7037fe640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704144570 0x7fd70414c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7027fc640 1 -- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 msgr2=0x7fd70413a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7027fc640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 0x7fd70413a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7027fc640 1 -- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704144570 msgr2=0x7fd70414c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7027fc640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704144570 0x7fd70414c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7027fc640 1 -- 172.21.15.16:0/3916386684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f80094d0 con 0x7fd7040ce550 2023-11-13T11:03:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.339+0000 7fd7037fe640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704144570 0x7fd70414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.340+0000 7fd7027fc640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 0x7fd70414a630 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd6f802b9c0 tx=0x7fd6f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.340+0000 7fd6eb7fe640 1 -- 172.21.15.16:0/3916386684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8039c90 con 0x7fd7040ce550 2023-11-13T11:03:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.340+0000 7fd6eb7fe640 1 -- 172.21.15.16:0/3916386684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f8039df0 con 0x7fd7040ce550 2023-11-13T11:03:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.340+0000 7fd6eb7fe640 1 -- 172.21.15.16:0/3916386684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f80446c0 con 0x7fd7040ce550 2023-11-13T11:03:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.340+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70414b3d0 con 0x7fd7040ce550 2023-11-13T11:03:55.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.340+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70414b990 con 0x7fd7040ce550 2023-11-13T11:03:55.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.342+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70414bb20 con 0x7fd7040ce550 2023-11-13T11:03:55.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.343+0000 7fd6eb7fe640 1 -- 172.21.15.16:0/3916386684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6f802f390 con 0x7fd7040ce550 2023-11-13T11:03:55.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.343+0000 7fd6eb7fe640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd6f006ae40 0x7fd6f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.344+0000 7fd6eb7fe640 1 -- 172.21.15.16:0/3916386684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd6f803e070 con 0x7fd7040ce550 2023-11-13T11:03:55.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.344+0000 7fd702ffd640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd6f006ae40 0x7fd6f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.345+0000 7fd702ffd640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd6f006ae40 0x7fd6f006d300 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4000f60 tx=0x7fd6f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.348+0000 7fd6eb7fe640 1 -- 172.21.15.16:0/3916386684 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6f8076f60 con 0x7fd7040ce550 2023-11-13T11:03:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.546+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7fd7040b3840 con 0x7fd7040ce550 2023-11-13T11:03:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.551+0000 7fd6eb7fe640 1 -- 172.21.15.16:0/3916386684 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v18)=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd6f807a580 con 0x7fd7040ce550 2023-11-13T11:03:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.553+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd6f006ae40 msgr2=0x7fd6f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.553+0000 7fd709578640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd6f006ae40 0x7fd6f006d300 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4000f60 tx=0x7fd6f4009040 comp rx=0 tx=0).stop 2023-11-13T11:03:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 msgr2=0x7fd70414a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 0x7fd70414a630 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd6f802b9c0 tx=0x7fd6f8004730 comp rx=0 tx=0).stop 2023-11-13T11:03:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 shutdown_connections 2023-11-13T11:03:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd6f006ae40 0x7fd6f006d300 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704144570 0x7fd70414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7040ce550 0x7fd70414a630 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 --2- 172.21.15.16:0/3916386684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b19d0 0x7fd70413a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 >> 172.21.15.16:0/3916386684 conn(0x7fd70406a190 msgr2=0x7fd7040b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 shutdown_connections 2023-11-13T11:03:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.554+0000 7fd709578640 1 -- 172.21.15.16:0/3916386684 wait complete. 2023-11-13T11:03:55.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_merge_size 30 2023-11-13T11:03:55.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.800+0000 7f58f981a640 1 Processor -- start 2023-11-13T11:03:55.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.800+0000 7f58f981a640 1 -- start start 2023-11-13T11:03:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.801+0000 7f58f981a640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58f40ce1d0 0x7f58f40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.801+0000 7f58f981a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f415e390 0x7f58f40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.801+0000 7f58f981a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f415d8d0 con 0x7f58f40ce5b0 2023-11-13T11:03:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.801+0000 7f58f981a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f415af00 con 0x7f58f415e390 2023-11-13T11:03:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.801+0000 7f58f981a640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f415db90 con 0x7f58f40ce1d0 2023-11-13T11:03:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f415e390 0x7f58f40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f415e390 0x7f58f40cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52152/0 (socket says 172.21.15.16:52152) 2023-11-13T11:03:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f37fe640 1 -- 172.21.15.16:0/1320639637 learned_addr learned my addr 172.21.15.16:0/1320639637 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f27fc640 1 --1- 172.21.15.16:0/1320639637 >> v1:172.21.15.16:6789/0 conn(0x7f58f40ce5b0 0x7f58f415a130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57034/0 (socket says 172.21.15.16:57034) 2023-11-13T11:03:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f37fe640 1 -- 172.21.15.16:0/1320639637 >> v2:172.21.15.184:3300/0 conn(0x7f58f40ce1d0 msgr2=0x7f58f40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:55.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f2ffd640 1 --2- 172.21.15.16:0/1320639637 >> v2:172.21.15.184:3300/0 conn(0x7f58f40ce1d0 0x7f58f40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f37fe640 1 --2- 172.21.15.16:0/1320639637 >> v2:172.21.15.184:3300/0 conn(0x7f58f40ce1d0 0x7f58f40ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f37fe640 1 -- 172.21.15.16:0/1320639637 >> v1:172.21.15.16:6789/0 conn(0x7f58f40ce5b0 legacy=0x7f58f415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f37fe640 1 -- 172.21.15.16:0/1320639637 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f415de70 con 0x7f58f415e390 2023-11-13T11:03:55.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f1ffb640 1 -- 172.21.15.16:0/1320639637 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43667677 0 0) 0x7f58f415d8d0 con 0x7f58f40ce5b0 2023-11-13T11:03:55.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.802+0000 7f58f2ffd640 1 --2- 172.21.15.16:0/1320639637 >> v2:172.21.15.184:3300/0 conn(0x7f58f40ce1d0 0x7f58f40ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.803+0000 7f58f37fe640 1 --2- 172.21.15.16:0/1320639637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f415e390 0x7f58f40cef10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f58f4072970 tx=0x7f58ec02b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc39a9116a31a1d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.804+0000 7f58f1ffb640 1 -- 172.21.15.16:0/1320639637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec030880 con 0x7f58f415e390 2023-11-13T11:03:55.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.804+0000 7f58f1ffb640 1 -- 172.21.15.16:0/1320639637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec0309e0 con 0x7f58f415e390 2023-11-13T11:03:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.804+0000 7f58f1ffb640 1 -- 172.21.15.16:0/1320639637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec030d70 con 0x7f58f415e390 2023-11-13T11:03:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.805+0000 7f58f981a640 1 -- 172.21.15.16:0/1320639637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f415e390 msgr2=0x7f58f40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.805+0000 7f58f981a640 1 --2- 172.21.15.16:0/1320639637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f415e390 0x7f58f40cef10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f58f4072970 tx=0x7f58ec02b9c0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.805+0000 7f58f981a640 1 -- 172.21.15.16:0/1320639637 shutdown_connections 2023-11-13T11:03:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.805+0000 7f58f981a640 1 --2- 172.21.15.16:0/1320639637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f415e390 0x7f58f40cef10 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.805+0000 7f58f981a640 1 --2- 172.21.15.16:0/1320639637 >> v2:172.21.15.184:3300/0 conn(0x7f58f40ce1d0 0x7f58f40ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.805+0000 7f58f981a640 1 -- 172.21.15.16:0/1320639637 >> 172.21.15.16:0/1320639637 conn(0x7f58f406a190 msgr2=0x7f58f406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.805+0000 7f58f981a640 1 -- 172.21.15.16:0/1320639637 shutdown_connections 2023-11-13T11:03:55.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.805+0000 7f58f981a640 1 -- 172.21.15.16:0/1320639637 wait complete. 2023-11-13T11:03:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.807+0000 7f58f981a640 1 Processor -- start 2023-11-13T11:03:55.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.807+0000 7f58f981a640 1 -- start start 2023-11-13T11:03:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f981a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f40ce1d0 0x7f58f41409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f981a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58f415e390 0x7f58f406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f981a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f406d260 0x7f58f4158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f981a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f414d2c0 con 0x7f58f40ce1d0 2023-11-13T11:03:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f981a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4142760 con 0x7f58f415e390 2023-11-13T11:03:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f981a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f414c410 con 0x7f58f406d260 2023-11-13T11:03:55.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f40ce1d0 0x7f58f41409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f40ce1d0 0x7f58f41409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52166/0 (socket says 172.21.15.16:52166) 2023-11-13T11:03:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f2ffd640 1 -- 172.21.15.16:0/278484545 learned_addr learned my addr 172.21.15.16:0/278484545 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f2ffd640 1 -- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f406d260 msgr2=0x7f58f4158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.808+0000 7f58f2ffd640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f406d260 0x7f58f4158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.809+0000 7f58f2ffd640 1 -- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58f415e390 msgr2=0x7f58f406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.809+0000 7f58f2ffd640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58f415e390 0x7f58f406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.809+0000 7f58f2ffd640 1 -- 172.21.15.16:0/278484545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0094d0 con 0x7f58f40ce1d0 2023-11-13T11:03:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.809+0000 7f58f2ffd640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f40ce1d0 0x7f58f41409a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f58e400cf60 tx=0x7f58e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.809+0000 7f58ebfff640 1 -- 172.21.15.16:0/278484545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e400d840 con 0x7f58f40ce1d0 2023-11-13T11:03:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.809+0000 7f58ebfff640 1 -- 172.21.15.16:0/278484545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e400d9a0 con 0x7f58f40ce1d0 2023-11-13T11:03:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.809+0000 7f58ebfff640 1 -- 172.21.15.16:0/278484545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e400dd30 con 0x7f58f40ce1d0 2023-11-13T11:03:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.809+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f414c6b0 con 0x7f58f40ce1d0 2023-11-13T11:03:55.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.809+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f414cc70 con 0x7f58f40ce1d0 2023-11-13T11:03:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.810+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f4072520 con 0x7f58f40ce1d0 2023-11-13T11:03:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.811+0000 7f58ebfff640 1 -- 172.21.15.16:0/278484545 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58e4015440 con 0x7f58f40ce1d0 2023-11-13T11:03:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.812+0000 7f58ebfff640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f58d406ae40 0x7f58d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.812+0000 7f58ebfff640 1 -- 172.21.15.16:0/278484545 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f58e408a110 con 0x7f58f40ce1d0 2023-11-13T11:03:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.813+0000 7f58f27fc640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f58d406ae40 0x7f58d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:55.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.814+0000 7f58f27fc640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f58d406ae40 0x7f58d406d300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f58dc007fd0 tx=0x7f58dc007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:55.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:55.816+0000 7f58ebfff640 1 -- 172.21.15.16:0/278484545 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58e40550d0 con 0x7f58f40ce1d0 2023-11-13T11:03:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.013+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f58f404e5b0 con 0x7f58f40ce1d0 2023-11-13T11:03:56.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.020+0000 7f58ebfff640 1 -- 172.21.15.16:0/278484545 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v19)=0 v19) v1 ==== 117+0+0 (secure 0 0 0) 0x7f58e40586f0 con 0x7f58f40ce1d0 2023-11-13T11:03:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f58d406ae40 msgr2=0x7f58d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f58d406ae40 0x7f58d406d300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f58dc007fd0 tx=0x7f58dc007920 comp rx=0 tx=0).stop 2023-11-13T11:03:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f40ce1d0 msgr2=0x7f58f41409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f40ce1d0 0x7f58f41409a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f58e400cf60 tx=0x7f58e400cf90 comp rx=0 tx=0).stop 2023-11-13T11:03:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 shutdown_connections 2023-11-13T11:03:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f58d406ae40 0x7f58d406d300 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58f406d260 0x7f58f4158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58f415e390 0x7f58f406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 --2- 172.21.15.16:0/278484545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f40ce1d0 0x7f58f41409a0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 >> 172.21.15.16:0/278484545 conn(0x7f58f406a190 msgr2=0x7f58f414ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 shutdown_connections 2023-11-13T11:03:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.023+0000 7f58f981a640 1 -- 172.21.15.16:0/278484545 wait complete. 2023-11-13T11:03:56.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:56.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.269+0000 7fbab6154640 1 Processor -- start 2023-11-13T11:03:56.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.269+0000 7fbab6154640 1 -- start start 2023-11-13T11:03:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.271+0000 7fbab6154640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbab006f970 0x7fbab0162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.271+0000 7fbab6154640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbab006a190 0x7fbab006a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.271+0000 7fbab6154640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab00b1840 con 0x7fbab0161060 2023-11-13T11:03:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.271+0000 7fbab6154640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab006ab60 con 0x7fbab006a190 2023-11-13T11:03:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.271+0000 7fbab6154640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab0071600 con 0x7fbab006f970 2023-11-13T11:03:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.271+0000 7fbab5152640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbab0161060 0x7fbab01621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57040/0 (socket says 172.21.15.16:57040) 2023-11-13T11:03:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.271+0000 7fbab5152640 1 -- 172.21.15.16:0/2363994011 learned_addr learned my addr 172.21.15.16:0/2363994011 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.271+0000 7fbab5953640 1 --2- 172.21.15.16:0/2363994011 >> v2:172.21.15.16:3300/0 conn(0x7fbab006a190 0x7fbab006a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.271+0000 7fbab4951640 1 --2- 172.21.15.16:0/2363994011 >> v2:172.21.15.184:3300/0 conn(0x7fbab006f970 0x7fbab0162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:56.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.272+0000 7fbab5953640 1 -- 172.21.15.16:0/2363994011 >> v2:172.21.15.184:3300/0 conn(0x7fbab006f970 msgr2=0x7fbab0162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:56.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.272+0000 7fbab5953640 1 --2- 172.21.15.16:0/2363994011 >> v2:172.21.15.184:3300/0 conn(0x7fbab006f970 0x7fbab0162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.272+0000 7fbaa7fff640 1 -- 172.21.15.16:0/2363994011 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305013778 0 0) 0x7fbab00b1840 con 0x7fbab0161060 2023-11-13T11:03:56.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.272+0000 7fbab5953640 1 -- 172.21.15.16:0/2363994011 >> v1:172.21.15.16:6789/0 conn(0x7fbab0161060 legacy=0x7fbab01621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.272+0000 7fbab5953640 1 -- 172.21.15.16:0/2363994011 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab00718e0 con 0x7fbab006a190 2023-11-13T11:03:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.273+0000 7fbab5953640 1 --2- 172.21.15.16:0/2363994011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006a190 0x7fbab006a620 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbab00ae3e0 tx=0x7fbaa802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4281711576809ab server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:56.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.273+0000 7fbaa7fff640 1 -- 172.21.15.16:0/2363994011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8031830 con 0x7fbab006a190 2023-11-13T11:03:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.273+0000 7fbaa7fff640 1 -- 172.21.15.16:0/2363994011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa8031990 con 0x7fbab006a190 2023-11-13T11:03:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.273+0000 7fbaa7fff640 1 -- 172.21.15.16:0/2363994011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8031d20 con 0x7fbab006a190 2023-11-13T11:03:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.274+0000 7fbab6154640 1 -- 172.21.15.16:0/2363994011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006a190 msgr2=0x7fbab006a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.274+0000 7fbab6154640 1 --2- 172.21.15.16:0/2363994011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006a190 0x7fbab006a620 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbab00ae3e0 tx=0x7fbaa802b8d0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.274+0000 7fbab6154640 1 -- 172.21.15.16:0/2363994011 shutdown_connections 2023-11-13T11:03:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.274+0000 7fbab6154640 1 --2- 172.21.15.16:0/2363994011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006a190 0x7fbab006a620 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.274+0000 7fbab6154640 1 --2- 172.21.15.16:0/2363994011 >> v2:172.21.15.184:3300/0 conn(0x7fbab006f970 0x7fbab0162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.274+0000 7fbab6154640 1 -- 172.21.15.16:0/2363994011 >> 172.21.15.16:0/2363994011 conn(0x7fbab00b04a0 msgr2=0x7fbab00b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.274+0000 7fbab6154640 1 -- 172.21.15.16:0/2363994011 shutdown_connections 2023-11-13T11:03:56.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.274+0000 7fbab6154640 1 -- 172.21.15.16:0/2363994011 wait complete. 2023-11-13T11:03:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.276+0000 7fbab6154640 1 Processor -- start 2023-11-13T11:03:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.276+0000 7fbab6154640 1 -- start start 2023-11-13T11:03:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab6154640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbab006a190 0x7fbab0151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab6154640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006f970 0x7fbab00af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab6154640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbab0161060 0x7fbab00af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab6154640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab015f090 con 0x7fbab006f970 2023-11-13T11:03:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab6154640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab015e6f0 con 0x7fbab006a190 2023-11-13T11:03:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab6154640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab015e9d0 con 0x7fbab0161060 2023-11-13T11:03:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab4951640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006f970 0x7fbab00af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab4951640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006f970 0x7fbab00af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52190/0 (socket says 172.21.15.16:52190) 2023-11-13T11:03:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab4951640 1 -- 172.21.15.16:0/2323508500 learned_addr learned my addr 172.21.15.16:0/2323508500 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.277+0000 7fbab5152640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbab006a190 0x7fbab0151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.278+0000 7fbab5953640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbab0161060 0x7fbab00af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.278+0000 7fbab4951640 1 -- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbab0161060 msgr2=0x7fbab00af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.278+0000 7fbab4951640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbab0161060 0x7fbab00af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.278+0000 7fbab4951640 1 -- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbab006a190 msgr2=0x7fbab0151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.278+0000 7fbab4951640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbab006a190 0x7fbab0151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.278+0000 7fbab4951640 1 -- 172.21.15.16:0/2323508500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa8009400 con 0x7fbab006f970 2023-11-13T11:03:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.278+0000 7fbab4951640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006f970 0x7fbab00af150 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000d5f0 tx=0x7fbaa000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.279+0000 7fbaa5ffb640 1 -- 172.21.15.16:0/2323508500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00125f0 con 0x7fbab006f970 2023-11-13T11:03:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.279+0000 7fbaa5ffb640 1 -- 172.21.15.16:0/2323508500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa0012750 con 0x7fbab006f970 2023-11-13T11:03:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.279+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab015ecb0 con 0x7fbab006f970 2023-11-13T11:03:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.279+0000 7fbaa5ffb640 1 -- 172.21.15.16:0/2323508500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0012b00 con 0x7fbab006f970 2023-11-13T11:03:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.279+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab015a150 con 0x7fbab006f970 2023-11-13T11:03:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.280+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab004e4d0 con 0x7fbab006f970 2023-11-13T11:03:56.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.281+0000 7fbaa5ffb640 1 -- 172.21.15.16:0/2323508500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbaa0012c60 con 0x7fbab006f970 2023-11-13T11:03:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.283+0000 7fbaa5ffb640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba9406ae40 0x7fba9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.283+0000 7fbaa5ffb640 1 -- 172.21.15.16:0/2323508500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fbaa0089d20 con 0x7fbab006f970 2023-11-13T11:03:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.283+0000 7fbab5152640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba9406ae40 0x7fba9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.285+0000 7fbab5152640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba9406ae40 0x7fba9406d300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0012e0 tx=0x7fbaac007ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.288+0000 7fbaa5ffb640 1 -- 172.21.15.16:0/2323508500 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbaa0054b90 con 0x7fbab006f970 2023-11-13T11:03:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.526+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbab00b33e0 con 0x7fbab006f970 2023-11-13T11:03:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.527+0000 7fbaa5ffb640 1 -- 172.21.15.16:0/2323508500 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fbaa00581b0 con 0x7fbab006f970 2023-11-13T11:03:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:56.533 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/960469256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":960469256},{"type":"v1","addr":"172.21.15.16:6833","nonce":960469256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1265200946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1265200946},{"type":"v1","addr":"172.21.15.184:6837","nonce":1265200946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3768737652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3768737652},{"type":"v1","addr":"172.21.15.16:6835","nonce":3768737652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:53.003626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"b","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6836/2632887594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2632887594},{"type":"v1","addr":"172.21.15.184:6836","nonce":2632887594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:03:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2023-11-13T11:03:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba9406ae40 msgr2=0x7fba9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba9406ae40 0x7fba9406d300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0012e0 tx=0x7fbaac007ed0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006f970 msgr2=0x7fbab00af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:56.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006f970 0x7fbab00af150 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000d5f0 tx=0x7fbaa000dab0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 shutdown_connections 2023-11-13T11:03:56.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fba9406ae40 0x7fba9406d300 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbab0161060 0x7fbab00af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006f970 0x7fbab00af150 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 --2- 172.21.15.16:0/2323508500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbab006a190 0x7fbab0151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 >> 172.21.15.16:0/2323508500 conn(0x7fbab00b04a0 msgr2=0x7fbab0162fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 shutdown_connections 2023-11-13T11:03:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:56.530+0000 7fbab6154640 1 -- 172.21.15.16:0/2323508500 wait complete. 2023-11-13T11:03:56.556 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump inode 1099511627776 2023-11-13T11:03:56.796 INFO:teuthology.orchestra.run.smithi184.stdout:{"path":"/testdir","ino":1099511627776,"rdev":0,"ctime":"2023-11-13T11:03:37.705109+0000","btime":"2023-11-13T11:03:36.210131+0000","mode":16877,"uid":1000,"gid":1301,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2023-11-13T11:03:37.705109+0000","atime":"2023-11-13T11:03:36.210131+0000","time_warp_seq":0,"change_attr":31,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2023-11-13T11:03:37.705109+0000","num_files":40,"num_subdirs":0,"change_attr":31},"rstat":{"version":0,"rbytes":70,"rfiles":40,"rsubdirs":1,"rsnaps":0,"rctime":"2023-11-13T11:03:37.709300+0000"},"accounted_rstat":{"version":0,"rbytes":70,"rfiles":40,"rsubdirs":1,"rsnaps":0,"rctime":"2023-11-13T11:03:37.709300+0000"},"version":166,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[{"frag":"*","children":1},{"frag":"1*","children":1}]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{},"nref":0,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"is_subvolume":false} 2023-11-13T11:03:56.797 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 70, "rctime": "2023-11-13T11:03:37.709300+0000", "rfiles": 40, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2023-11-13T11:03:36.210131+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2023-11-13T11:03:36.210131+0000", "change_attr": 31, "client_caps": [], "client_ranges": [], "ctime": "2023-11-13T11:03:37.705109+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [ { "children": 1, "frag": "*" }, { "children": 1, "frag": "1*" } ] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 31, "mtime": "2023-11-13T11:03:37.705109+0000", "num_files": 40, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1301, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2023-11-13T11:03:37.705109+0000", "nestlock": {}, "nlink": 1, "nref": 0, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/testdir", "pins": {}, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 70, "rctime": "2023-11-13T11:03:37.709300+0000", "rfiles": 40, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 166, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2023-11-13T11:03:56.798 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-13T11:03:56.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:03:56.931 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-13T11:03:57.066 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-13T11:03:57.067 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:03:57.067 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:57.131 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:03:57.132 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:03:57.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:57.132 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-13T11:03:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-13T11:03:57.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:03:57.188 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:03:57.189 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:57.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:03:57.251 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:03:57.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:57.252 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-13T11:03:57.307 INFO:teuthology.orchestra.run.smithi184.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-13T11:03:57.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:03:57.309 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:03:57.309 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:57.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:03:57.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:03:57.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:57.376 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-13T11:03:57.435 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-13T11:03:57.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:03:57.437 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:03:57.437 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:03:57.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:03:57.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:03:57.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:03:57.499 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-13T11:03:57.550 INFO:teuthology.orchestra.run.smithi184.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-13T11:03:57.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:03:57.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:03:57.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.778+0000 7f37d58f8640 1 Processor -- start 2023-11-13T11:03:57.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.779+0000 7f37d58f8640 1 -- start start 2023-11-13T11:03:57.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.780+0000 7f37d58f8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37d015cfc0 0x7f37d00cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:57.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.780+0000 7f37d58f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d00d2ae0 0x7f37d00d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:57.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.780+0000 7f37d58f8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d00b3cf0 con 0x7f37d00b29d0 2023-11-13T11:03:57.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.780+0000 7f37d58f8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d00d3480 con 0x7f37d015cfc0 2023-11-13T11:03:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.780+0000 7f37d58f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d00b3730 con 0x7f37d00d2ae0 2023-11-13T11:03:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.780+0000 7f37d48f6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37d015cfc0 0x7f37d00cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.781+0000 7f37d48f6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37d015cfc0 0x7f37d00cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52198/0 (socket says 172.21.15.16:52198) 2023-11-13T11:03:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.781+0000 7f37d48f6640 1 -- 172.21.15.16:0/619895228 learned_addr learned my addr 172.21.15.16:0/619895228 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.781+0000 7f37cffff640 1 --2- 172.21.15.16:0/619895228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d00d2ae0 0x7f37d00d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.781+0000 7f37d48f6640 1 -- 172.21.15.16:0/619895228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d00d2ae0 msgr2=0x7f37d00d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.781+0000 7f37d48f6640 1 --2- 172.21.15.16:0/619895228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d00d2ae0 0x7f37d00d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.781+0000 7f37d48f6640 1 -- 172.21.15.16:0/619895228 >> v1:172.21.15.184:6789/0 conn(0x7f37d00b29d0 legacy=0x7f37d00b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:57.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.781+0000 7f37d48f6640 1 -- 172.21.15.16:0/619895228 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d00b3a10 con 0x7f37d015cfc0 2023-11-13T11:03:57.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.782+0000 7f37d48f6640 1 --2- 172.21.15.16:0/619895228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d015cfc0 0x7f37d00cd770 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009870 tx=0x7f37c402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f60fe1c5412bda3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.782+0000 7f37cf7fe640 1 -- 172.21.15.16:0/619895228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c40317c0 con 0x7f37d015cfc0 2023-11-13T11:03:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37cf7fe640 1 -- 172.21.15.16:0/619895228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c4031920 con 0x7f37d015cfc0 2023-11-13T11:03:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37cf7fe640 1 -- 172.21.15.16:0/619895228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4031cb0 con 0x7f37d015cfc0 2023-11-13T11:03:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37d58f8640 1 -- 172.21.15.16:0/619895228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d015cfc0 msgr2=0x7f37d00cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37d58f8640 1 --2- 172.21.15.16:0/619895228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d015cfc0 0x7f37d00cd770 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009870 tx=0x7f37c402b860 comp rx=0 tx=0).stop 2023-11-13T11:03:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37d58f8640 1 -- 172.21.15.16:0/619895228 shutdown_connections 2023-11-13T11:03:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37d58f8640 1 --2- 172.21.15.16:0/619895228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d00d2ae0 0x7f37d00d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37d58f8640 1 --2- 172.21.15.16:0/619895228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d015cfc0 0x7f37d00cd770 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37d58f8640 1 -- 172.21.15.16:0/619895228 >> 172.21.15.16:0/619895228 conn(0x7f37d008c570 msgr2=0x7f37d009be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37d58f8640 1 -- 172.21.15.16:0/619895228 shutdown_connections 2023-11-13T11:03:57.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.783+0000 7f37d58f8640 1 -- 172.21.15.16:0/619895228 wait complete. 2023-11-13T11:03:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.785+0000 7f37d58f8640 1 Processor -- start 2023-11-13T11:03:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.785+0000 7f37d58f8640 1 -- start start 2023-11-13T11:03:57.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.786+0000 7f37d58f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d00b29d0 0x7f37d00cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:57.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.786+0000 7f37d58f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d00d2ae0 0x7f37d00c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:57.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.786+0000 7f37d58f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d015cfc0 0x7f37d0150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:57.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.786+0000 7f37d58f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d00ca740 con 0x7f37d00b29d0 2023-11-13T11:03:57.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.786+0000 7f37d58f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d00cb5a0 con 0x7f37d00d2ae0 2023-11-13T11:03:57.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.786+0000 7f37d58f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d00ca990 con 0x7f37d015cfc0 2023-11-13T11:03:57.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d48f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d00b29d0 0x7f37d00cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:57.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d48f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d00b29d0 0x7f37d00cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52210/0 (socket says 172.21.15.16:52210) 2023-11-13T11:03:57.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d48f6640 1 -- 172.21.15.16:0/2080681748 learned_addr learned my addr 172.21.15.16:0/2080681748 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:57.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37cffff640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d00d2ae0 0x7f37d00c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:57.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d50f7640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d015cfc0 0x7f37d0150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:57.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d48f6640 1 -- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d015cfc0 msgr2=0x7f37d0150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:57.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d48f6640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d015cfc0 0x7f37d0150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:57.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d48f6640 1 -- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d00d2ae0 msgr2=0x7f37d00c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:57.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d48f6640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d00d2ae0 0x7f37d00c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:57.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d48f6640 1 -- 172.21.15.16:0/2080681748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40094d0 con 0x7f37d00b29d0 2023-11-13T11:03:57.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37cffff640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d00d2ae0 0x7f37d00c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:57.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.787+0000 7f37d50f7640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d015cfc0 0x7f37d0150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:57.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.788+0000 7f37d48f6640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d00b29d0 0x7f37d00cba30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f37c4000c00 tx=0x7f37c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:57.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.788+0000 7f37cd7fa640 1 -- 172.21.15.16:0/2080681748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4032860 con 0x7f37d00b29d0 2023-11-13T11:03:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.788+0000 7f37cd7fa640 1 -- 172.21.15.16:0/2080681748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c40329e0 con 0x7f37d00b29d0 2023-11-13T11:03:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.788+0000 7f37cd7fa640 1 -- 172.21.15.16:0/2080681748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4032d90 con 0x7f37d00b29d0 2023-11-13T11:03:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.788+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d00cac70 con 0x7f37d00b29d0 2023-11-13T11:03:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.788+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d00cb230 con 0x7f37d00b29d0 2023-11-13T11:03:57.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.789+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d00d0b40 con 0x7f37d00b29d0 2023-11-13T11:03:57.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.790+0000 7f37cd7fa640 1 -- 172.21.15.16:0/2080681748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37c404e450 con 0x7f37d00b29d0 2023-11-13T11:03:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.791+0000 7f37cd7fa640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37b406ae40 0x7f37b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.791+0000 7f37cd7fa640 1 -- 172.21.15.16:0/2080681748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f37c40ab9b0 con 0x7f37d00b29d0 2023-11-13T11:03:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.792+0000 7f37cffff640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37b406ae40 0x7f37b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:57.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.793+0000 7f37cffff640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37b406ae40 0x7f37b406d300 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f37bc006fd0 tx=0x7f37bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:57.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:57.796+0000 7f37cd7fa640 1 -- 172.21.15.16:0/2080681748 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37d00d0b40 con 0x7f37d00b29d0 2023-11-13T11:03:58.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.028+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37d00d0b40 con 0x7f37d00b29d0 2023-11-13T11:03:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.029+0000 7f37cd7fa640 1 -- 172.21.15.16:0/2080681748 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f37d00d0b40 con 0x7f37d00b29d0 2023-11-13T11:03:58.034 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:58.035 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/960469256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":960469256},{"type":"v1","addr":"172.21.15.16:6833","nonce":960469256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1265200946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1265200946},{"type":"v1","addr":"172.21.15.184:6837","nonce":1265200946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3768737652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3768737652},{"type":"v1","addr":"172.21.15.16:6835","nonce":3768737652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:53.003626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"b","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6836/2632887594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2632887594},{"type":"v1","addr":"172.21.15.184:6836","nonce":2632887594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:03:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2023-11-13T11:03:58.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37b406ae40 msgr2=0x7f37b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37b406ae40 0x7f37b406d300 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f37bc006fd0 tx=0x7f37bc008040 comp rx=0 tx=0).stop 2023-11-13T11:03:58.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d00b29d0 msgr2=0x7f37d00cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d00b29d0 0x7f37d00cba30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f37c4000c00 tx=0x7f37c40046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 shutdown_connections 2023-11-13T11:03:58.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f37b406ae40 0x7f37b406d300 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37d015cfc0 0x7f37d0150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37d00d2ae0 0x7f37d00c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 --2- 172.21.15.16:0/2080681748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37d00b29d0 0x7f37d00cba30 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 >> 172.21.15.16:0/2080681748 conn(0x7f37d008c570 msgr2=0x7f37d006a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 shutdown_connections 2023-11-13T11:03:58.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.032+0000 7f37d58f8640 1 -- 172.21.15.16:0/2080681748 wait complete. 2023-11-13T11:03:58.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:03:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.281+0000 7f9e6f5bb640 1 Processor -- start 2023-11-13T11:03:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.281+0000 7f9e6f5bb640 1 -- start start 2023-11-13T11:03:58.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.283+0000 7f9e6f5bb640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9e680cecc0 0x7f9e6806cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.283+0000 7f9e6f5bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e6806bb90 0x7f9e68070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.283+0000 7f9e6f5bb640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e680b2900 con 0x7f9e6815cf20 2023-11-13T11:03:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.283+0000 7f9e6f5bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6815d300 con 0x7f9e6806bb90 2023-11-13T11:03:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.283+0000 7f9e6f5bb640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e680b1fd0 con 0x7f9e680cecc0 2023-11-13T11:03:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.283+0000 7f9e6db58640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e6806bb90 0x7f9e68070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.283+0000 7f9e6db58640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e6806bb90 0x7f9e68070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52224/0 (socket says 172.21.15.16:52224) 2023-11-13T11:03:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.283+0000 7f9e6db58640 1 -- 172.21.15.16:0/2562478268 learned_addr learned my addr 172.21.15.16:0/2562478268 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:58.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.284+0000 7f9e6cb56640 1 --2- 172.21.15.16:0/2562478268 >> v2:172.21.15.184:3300/0 conn(0x7f9e680cecc0 0x7f9e6806cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.284+0000 7f9e6db58640 1 -- 172.21.15.16:0/2562478268 >> v2:172.21.15.184:3300/0 conn(0x7f9e680cecc0 msgr2=0x7f9e6806cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.284+0000 7f9e6db58640 1 --2- 172.21.15.16:0/2562478268 >> v2:172.21.15.184:3300/0 conn(0x7f9e680cecc0 0x7f9e6806cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.284+0000 7f9e6db58640 1 -- 172.21.15.16:0/2562478268 >> v1:172.21.15.184:6789/0 conn(0x7f9e6815cf20 legacy=0x7f9e680ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.284+0000 7f9e6db58640 1 -- 172.21.15.16:0/2562478268 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680b22b0 con 0x7f9e6806bb90 2023-11-13T11:03:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.285+0000 7f9e6db58640 1 --2- 172.21.15.16:0/2562478268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6806bb90 0x7f9e68070330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9e680cadd0 tx=0x7f9e6002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64314bcbb00aba57 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.285+0000 7f9e5ffff640 1 -- 172.21.15.16:0/2562478268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6002ca80 con 0x7f9e6806bb90 2023-11-13T11:03:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.285+0000 7f9e5ffff640 1 -- 172.21.15.16:0/2562478268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6002cbe0 con 0x7f9e6806bb90 2023-11-13T11:03:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.286+0000 7f9e5ffff640 1 -- 172.21.15.16:0/2562478268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e600343f0 con 0x7f9e6806bb90 2023-11-13T11:03:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.286+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/2562478268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6806bb90 msgr2=0x7f9e68070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.286+0000 7f9e6f5bb640 1 --2- 172.21.15.16:0/2562478268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6806bb90 0x7f9e68070330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9e680cadd0 tx=0x7f9e6002bee0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.286+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/2562478268 shutdown_connections 2023-11-13T11:03:58.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.286+0000 7f9e6f5bb640 1 --2- 172.21.15.16:0/2562478268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6806bb90 0x7f9e68070330 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.286+0000 7f9e6f5bb640 1 --2- 172.21.15.16:0/2562478268 >> v2:172.21.15.184:3300/0 conn(0x7f9e680cecc0 0x7f9e6806cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.286+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/2562478268 >> 172.21.15.16:0/2562478268 conn(0x7f9e6806a190 msgr2=0x7f9e6806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:58.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.286+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/2562478268 shutdown_connections 2023-11-13T11:03:58.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.286+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/2562478268 wait complete. 2023-11-13T11:03:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.288+0000 7f9e6f5bb640 1 Processor -- start 2023-11-13T11:03:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.288+0000 7f9e6f5bb640 1 -- start start 2023-11-13T11:03:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.289+0000 7f9e6f5bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e6806bb90 0x7f9e68160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.289+0000 7f9e6f5bb640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e680cecc0 0x7f9e68154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.289+0000 7f9e6f5bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6815cf20 0x7f9e68142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.289+0000 7f9e6f5bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e681418e0 con 0x7f9e6815cf20 2023-11-13T11:03:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.289+0000 7f9e6f5bb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68140cd0 con 0x7f9e6806bb90 2023-11-13T11:03:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.289+0000 7f9e6f5bb640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68140fb0 con 0x7f9e680cecc0 2023-11-13T11:03:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6db58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6815cf20 0x7f9e68142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6db58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6815cf20 0x7f9e68142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52226/0 (socket says 172.21.15.16:52226) 2023-11-13T11:03:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6db58640 1 -- 172.21.15.16:0/4231298447 learned_addr learned my addr 172.21.15.16:0/4231298447 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6d357640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e6806bb90 0x7f9e68160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6db58640 1 -- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e680cecc0 msgr2=0x7f9e68154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6db58640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e680cecc0 0x7f9e68154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6db58640 1 -- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e6806bb90 msgr2=0x7f9e68160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6db58640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e6806bb90 0x7f9e68160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6db58640 1 -- 172.21.15.16:0/4231298447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e60009b30 con 0x7f9e6815cf20 2023-11-13T11:03:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.290+0000 7f9e6d357640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e6806bb90 0x7f9e68160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.291+0000 7f9e6db58640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6815cf20 0x7f9e68142e50 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9e60036040 tx=0x7f9e60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.291+0000 7f9e5dffb640 1 -- 172.21.15.16:0/4231298447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60009510 con 0x7f9e6815cf20 2023-11-13T11:03:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.291+0000 7f9e5dffb640 1 -- 172.21.15.16:0/4231298447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6003e450 con 0x7f9e6815cf20 2023-11-13T11:03:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.291+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e68141290 con 0x7f9e6815cf20 2023-11-13T11:03:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.291+0000 7f9e5dffb640 1 -- 172.21.15.16:0/4231298447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6003e800 con 0x7f9e6815cf20 2023-11-13T11:03:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.291+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e68160550 con 0x7f9e6815cf20 2023-11-13T11:03:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.292+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e68141a90 con 0x7f9e6815cf20 2023-11-13T11:03:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.294+0000 7f9e5dffb640 1 -- 172.21.15.16:0/4231298447 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e6002cd50 con 0x7f9e6815cf20 2023-11-13T11:03:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.294+0000 7f9e5dffb640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e4c06ae70 0x7f9e4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.295+0000 7f9e5dffb640 1 -- 172.21.15.16:0/4231298447 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f9e600ac280 con 0x7f9e6815cf20 2023-11-13T11:03:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.295+0000 7f9e6d357640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e4c06ae70 0x7f9e4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.297+0000 7f9e6d357640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e4c06ae70 0x7f9e4c06d330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400a830 tx=0x7f9e64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.299+0000 7f9e5dffb640 1 -- 172.21.15.16:0/4231298447 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e68141a90 con 0x7f9e6815cf20 2023-11-13T11:03:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.493+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e680ae790 con 0x7f9e6815cf20 2023-11-13T11:03:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.495+0000 7f9e5dffb640 1 -- 172.21.15.16:0/4231298447 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16903 (secure 0 0 0) 0x7f9e60077180 con 0x7f9e6815cf20 2023-11-13T11:03:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:03:58.501 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:50.019687+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:03:17.903258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-13T11:03:18.947896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6837/60343961":"2023-11-14T11:03:50.019659+0000","172.21.15.184:6835/60343961":"2023-11-14T11:03:50.019659+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:03:58.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e4c06ae70 msgr2=0x7f9e4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e4c06ae70 0x7f9e4c06d330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400a830 tx=0x7f9e64008040 comp rx=0 tx=0).stop 2023-11-13T11:03:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6815cf20 msgr2=0x7f9e68142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6815cf20 0x7f9e68142e50 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9e60036040 tx=0x7f9e60004a40 comp rx=0 tx=0).stop 2023-11-13T11:03:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 shutdown_connections 2023-11-13T11:03:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f9e4c06ae70 0x7f9e4c06d330 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6815cf20 0x7f9e68142e50 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e680cecc0 0x7f9e68154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 --2- 172.21.15.16:0/4231298447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e6806bb90 0x7f9e68160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 >> 172.21.15.16:0/4231298447 conn(0x7f9e6806a190 msgr2=0x7f9e6806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 shutdown_connections 2023-11-13T11:03:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.498+0000 7f9e6f5bb640 1 -- 172.21.15.16:0/4231298447 wait complete. 2023-11-13T11:03:58.524 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-13T10:55:14.038047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-13T11:03:17.903258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-13T11:03:18.947896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-13T11:03:58.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-13T11:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.760+0000 7fd9bffff640 1 Processor -- start 2023-11-13T11:03:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.760+0000 7fd9bffff640 1 -- start start 2023-11-13T11:03:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.762+0000 7fd9bffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9c00d1850 0x7fd9c00d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.762+0000 7fd9bffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c00d2d00 0x7fd9c00c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.762+0000 7fd9bffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c00ce0e0 con 0x7fd9c015ce70 2023-11-13T11:03:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.762+0000 7fd9bffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c00ce3c0 con 0x7fd9c00d1850 2023-11-13T11:03:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.762+0000 7fd9bffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00ce6a0 con 0x7fd9c00d2d00 2023-11-13T11:03:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9beffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9c015ce70 0x7fd9c00cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57056/0 (socket says 172.21.15.16:57056) 2023-11-13T11:03:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9beffd640 1 -- 172.21.15.16:0/2247014789 learned_addr learned my addr 172.21.15.16:0/2247014789 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:03:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9b7fff640 1 --2- 172.21.15.16:0/2247014789 >> v2:172.21.15.16:3300/0 conn(0x7fd9c00d1850 0x7fd9c00d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9be7fc640 1 -- 172.21.15.16:0/2247014789 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187537585 0 0) 0x7fd9c00ce0e0 con 0x7fd9c015ce70 2023-11-13T11:03:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9bf7fe640 1 --2- 172.21.15.16:0/2247014789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c00d2d00 0x7fd9c00c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9be7fc640 1 -- 172.21.15.16:0/2247014789 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9ac003520 con 0x7fd9c015ce70 2023-11-13T11:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9b7fff640 1 -- 172.21.15.16:0/2247014789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c00d2d00 msgr2=0x7fd9c00c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9b7fff640 1 --2- 172.21.15.16:0/2247014789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c00d2d00 0x7fd9c00c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9b7fff640 1 -- 172.21.15.16:0/2247014789 >> v1:172.21.15.16:6789/0 conn(0x7fd9c015ce70 legacy=0x7fd9c00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.763+0000 7fd9b7fff640 1 -- 172.21.15.16:0/2247014789 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00ce980 con 0x7fd9c00d1850 2023-11-13T11:03:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.764+0000 7fd9b7fff640 1 --2- 172.21.15.16:0/2247014789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d1850 0x7fd9c00d27c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80097a0 tx=0x7fd9a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582e0a610ca294a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.765+0000 7fd9be7fc640 1 -- 172.21.15.16:0/2247014789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8031830 con 0x7fd9c00d1850 2023-11-13T11:03:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.765+0000 7fd9be7fc640 1 -- 172.21.15.16:0/2247014789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a8031990 con 0x7fd9c00d1850 2023-11-13T11:03:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.765+0000 7fd9be7fc640 1 -- 172.21.15.16:0/2247014789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8031d20 con 0x7fd9c00d1850 2023-11-13T11:03:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.765+0000 7fd9bffff640 1 -- 172.21.15.16:0/2247014789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d1850 msgr2=0x7fd9c00d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.765+0000 7fd9bffff640 1 --2- 172.21.15.16:0/2247014789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d1850 0x7fd9c00d27c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80097a0 tx=0x7fd9a802b8d0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.766+0000 7fd9bffff640 1 -- 172.21.15.16:0/2247014789 shutdown_connections 2023-11-13T11:03:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.766+0000 7fd9bffff640 1 --2- 172.21.15.16:0/2247014789 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c00d2d00 0x7fd9c00c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.766+0000 7fd9bffff640 1 --2- 172.21.15.16:0/2247014789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d1850 0x7fd9c00d27c0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.766+0000 7fd9bffff640 1 -- 172.21.15.16:0/2247014789 >> 172.21.15.16:0/2247014789 conn(0x7fd9c008c580 msgr2=0x7fd9c009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.766+0000 7fd9bffff640 1 -- 172.21.15.16:0/2247014789 shutdown_connections 2023-11-13T11:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.766+0000 7fd9bffff640 1 -- 172.21.15.16:0/2247014789 wait complete. 2023-11-13T11:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.767+0000 7fd9bffff640 1 Processor -- start 2023-11-13T11:03:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.768+0000 7fd9bffff640 1 -- start start 2023-11-13T11:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.768+0000 7fd9bffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9c00d1850 0x7fd9c015a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d2d00 0x7fd9c01f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9bffff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c015ce70 0x7fd9c009fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9bffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c015a540 con 0x7fd9c00d2d00 2023-11-13T11:03:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9bffff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00b6ae0 con 0x7fd9c00d1850 2023-11-13T11:03:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9bffff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c009d170 con 0x7fd9c015ce70 2023-11-13T11:03:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d2d00 0x7fd9c01f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d2d00 0x7fd9c01f38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52246/0 (socket says 172.21.15.16:52246) 2023-11-13T11:03:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9b7fff640 1 -- 172.21.15.16:0/708910872 learned_addr learned my addr 172.21.15.16:0/708910872 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9bf7fe640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c015ce70 0x7fd9c009fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9beffd640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9c00d1850 0x7fd9c015a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9b7fff640 1 -- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c015ce70 msgr2=0x7fd9c009fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.769+0000 7fd9b7fff640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c015ce70 0x7fd9c009fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.770+0000 7fd9b7fff640 1 -- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9c00d1850 msgr2=0x7fd9c015a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.770+0000 7fd9b7fff640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9c00d1850 0x7fd9c015a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.770+0000 7fd9b7fff640 1 -- 172.21.15.16:0/708910872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a8009400 con 0x7fd9c00d2d00 2023-11-13T11:03:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.770+0000 7fd9b7fff640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d2d00 0x7fd9c01f38b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80098c0 tx=0x7fd9a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.770+0000 7fd9b77fe640 1 -- 172.21.15.16:0/708910872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a803a630 con 0x7fd9c00d2d00 2023-11-13T11:03:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.770+0000 7fd9b77fe640 1 -- 172.21.15.16:0/708910872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a803a7b0 con 0x7fd9c00d2d00 2023-11-13T11:03:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.770+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c009d410 con 0x7fd9c00d2d00 2023-11-13T11:03:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.770+0000 7fd9b77fe640 1 -- 172.21.15.16:0/708910872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a803ab60 con 0x7fd9c00d2d00 2023-11-13T11:03:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.770+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c009d9d0 con 0x7fd9c00d2d00 2023-11-13T11:03:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.772+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c00675b0 con 0x7fd9c00d2d00 2023-11-13T11:03:58.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.773+0000 7fd9b77fe640 1 -- 172.21.15.16:0/708910872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd9a803acc0 con 0x7fd9c00d2d00 2023-11-13T11:03:58.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.773+0000 7fd9b77fe640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd9a006ae40 0x7fd9a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:58.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.774+0000 7fd9b77fe640 1 -- 172.21.15.16:0/708910872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd9a80ab3f0 con 0x7fd9c00d2d00 2023-11-13T11:03:58.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.774+0000 7fd9beffd640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd9a006ae40 0x7fd9a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:58.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.775+0000 7fd9beffd640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd9a006ae40 0x7fd9a006d300 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00011e0 tx=0x7fd9b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:58.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.778+0000 7fd9b77fe640 1 -- 172.21.15.16:0/708910872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd9a8076300 con 0x7fd9c00d2d00 2023-11-13T11:03:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.966+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9c004e4d0 con 0x7fd9c00d2d00 2023-11-13T11:03:58.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.968+0000 7fd9b77fe640 1 -- 172.21.15.16:0/708910872 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9a8079920 con 0x7fd9c00d2d00 2023-11-13T11:03:58.973 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-13T11:03:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd9a006ae40 msgr2=0x7fd9a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd9a006ae40 0x7fd9a006d300 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00011e0 tx=0x7fd9b0009040 comp rx=0 tx=0).stop 2023-11-13T11:03:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d2d00 msgr2=0x7fd9c01f38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d2d00 0x7fd9c01f38b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80098c0 tx=0x7fd9a80046c0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 shutdown_connections 2023-11-13T11:03:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fd9a006ae40 0x7fd9a006d300 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9c015ce70 0x7fd9c009fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c00d2d00 0x7fd9c01f38b0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 --2- 172.21.15.16:0/708910872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9c00d1850 0x7fd9c015a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 >> 172.21.15.16:0/708910872 conn(0x7fd9c008c580 msgr2=0x7fd9c00cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:58.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 shutdown_connections 2023-11-13T11:03:58.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:58.970+0000 7fd9bffff640 1 -- 172.21.15.16:0/708910872 wait complete. 2023-11-13T11:03:58.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-13T11:03:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.210+0000 7f8a356ca640 1 Processor -- start 2023-11-13T11:03:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.210+0000 7f8a356ca640 1 -- start start 2023-11-13T11:03:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a356ca640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a300af650 0x7f8a300b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a356ca640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a3014a9d0 0x7f8a300ad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a356ca640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 0x7f8a300c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a356ca640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a300afa30 con 0x7f8a3014a9d0 2023-11-13T11:03:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a356ca640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a300c4390 con 0x7f8a300af650 2023-11-13T11:03:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a356ca640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300b0340 con 0x7f8a300c39a0 2023-11-13T11:03:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a2f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a3014a9d0 0x7f8a300ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a2f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a3014a9d0 0x7f8a300ad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52248/0 (socket says 172.21.15.16:52248) 2023-11-13T11:03:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a2f7fe640 1 -- 172.21.15.16:0/2236299602 learned_addr learned my addr 172.21.15.16:0/2236299602 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.212+0000 7f8a2ffff640 1 --2- 172.21.15.16:0/2236299602 >> v2:172.21.15.184:3300/0 conn(0x7f8a300af650 0x7f8a300b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.213+0000 7f8a34ec9640 1 --2- 172.21.15.16:0/2236299602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 0x7f8a300c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.213+0000 7f8a2f7fe640 1 -- 172.21.15.16:0/2236299602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 msgr2=0x7f8a300c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.213+0000 7f8a2f7fe640 1 --2- 172.21.15.16:0/2236299602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 0x7f8a300c3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.213+0000 7f8a2f7fe640 1 -- 172.21.15.16:0/2236299602 >> v2:172.21.15.184:3300/0 conn(0x7f8a300af650 msgr2=0x7f8a300b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.213+0000 7f8a2f7fe640 1 --2- 172.21.15.16:0/2236299602 >> v2:172.21.15.184:3300/0 conn(0x7f8a300af650 0x7f8a300b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.213+0000 7f8a2f7fe640 1 -- 172.21.15.16:0/2236299602 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a300b1160 con 0x7f8a3014a9d0 2023-11-13T11:03:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.213+0000 7f8a2ffff640 1 --2- 172.21.15.16:0/2236299602 >> v2:172.21.15.184:3300/0 conn(0x7f8a300af650 0x7f8a300b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.214+0000 7f8a2f7fe640 1 --2- 172.21.15.16:0/2236299602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a3014a9d0 0x7f8a300ad800 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8a3006cba0 tx=0x7f8a2402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd2fd940cc10c1e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.214+0000 7f8a2effd640 1 -- 172.21.15.16:0/2236299602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a240317c0 con 0x7f8a3014a9d0 2023-11-13T11:03:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.214+0000 7f8a2effd640 1 -- 172.21.15.16:0/2236299602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a24031920 con 0x7f8a3014a9d0 2023-11-13T11:03:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.214+0000 7f8a2effd640 1 -- 172.21.15.16:0/2236299602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a24031cb0 con 0x7f8a3014a9d0 2023-11-13T11:03:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.215+0000 7f8a356ca640 1 -- 172.21.15.16:0/2236299602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a3014a9d0 msgr2=0x7f8a300ad800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.215+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2236299602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a3014a9d0 0x7f8a300ad800 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8a3006cba0 tx=0x7f8a2402b860 comp rx=0 tx=0).stop 2023-11-13T11:03:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.215+0000 7f8a356ca640 1 -- 172.21.15.16:0/2236299602 shutdown_connections 2023-11-13T11:03:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.215+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2236299602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 0x7f8a300c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.215+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2236299602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a3014a9d0 0x7f8a300ad800 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.215+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2236299602 >> v2:172.21.15.184:3300/0 conn(0x7f8a300af650 0x7f8a300b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.215+0000 7f8a356ca640 1 -- 172.21.15.16:0/2236299602 >> 172.21.15.16:0/2236299602 conn(0x7f8a300b2520 msgr2=0x7f8a300b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.215+0000 7f8a356ca640 1 -- 172.21.15.16:0/2236299602 shutdown_connections 2023-11-13T11:03:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.215+0000 7f8a356ca640 1 -- 172.21.15.16:0/2236299602 wait complete. 2023-11-13T11:03:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.217+0000 7f8a356ca640 1 Processor -- start 2023-11-13T11:03:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.217+0000 7f8a356ca640 1 -- start start 2023-11-13T11:03:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.218+0000 7f8a356ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a300af650 0x7f8a300cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.218+0000 7f8a356ca640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 0x7f8a300c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.218+0000 7f8a356ca640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3014a9d0 0x7f8a30149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.218+0000 7f8a356ca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300c4a10 con 0x7f8a300af650 2023-11-13T11:03:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.218+0000 7f8a356ca640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300c5440 con 0x7f8a3014a9d0 2023-11-13T11:03:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.218+0000 7f8a356ca640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300caa00 con 0x7f8a300c39a0 2023-11-13T11:03:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a300af650 0x7f8a300cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a300af650 0x7f8a300cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52250/0 (socket says 172.21.15.16:52250) 2023-11-13T11:03:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2ffff640 1 -- 172.21.15.16:0/2205505673 learned_addr learned my addr 172.21.15.16:0/2205505673 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2f7fe640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 0x7f8a300c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a34ec9640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3014a9d0 0x7f8a30149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2ffff640 1 -- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 msgr2=0x7f8a300c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2ffff640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 0x7f8a300c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2ffff640 1 -- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3014a9d0 msgr2=0x7f8a30149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2ffff640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3014a9d0 0x7f8a30149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2ffff640 1 -- 172.21.15.16:0/2205505673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240094d0 con 0x7f8a300af650 2023-11-13T11:03:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a34ec9640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3014a9d0 0x7f8a30149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:03:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.219+0000 7f8a2ffff640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a300af650 0x7f8a300cb000 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000cf60 tx=0x7f8a2000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.220+0000 7f8a2cff9640 1 -- 172.21.15.16:0/2205505673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2000d840 con 0x7f8a300af650 2023-11-13T11:03:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.220+0000 7f8a2cff9640 1 -- 172.21.15.16:0/2205505673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2000d9a0 con 0x7f8a300af650 2023-11-13T11:03:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.220+0000 7f8a2cff9640 1 -- 172.21.15.16:0/2205505673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2000dd30 con 0x7f8a300af650 2023-11-13T11:03:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.220+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a300ca720 con 0x7f8a300af650 2023-11-13T11:03:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.220+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a300c50d0 con 0x7f8a300af650 2023-11-13T11:03:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.221+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a30066490 con 0x7f8a300af650 2023-11-13T11:03:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.222+0000 7f8a2cff9640 1 -- 172.21.15.16:0/2205505673 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a20015440 con 0x7f8a300af650 2023-11-13T11:03:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.223+0000 7f8a2cff9640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a1c06ae40 0x7f8a1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.223+0000 7f8a2cff9640 1 -- 172.21.15.16:0/2205505673 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f8a2008a150 con 0x7f8a300af650 2023-11-13T11:03:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.224+0000 7f8a2f7fe640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a1c06ae40 0x7f8a1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.225+0000 7f8a2f7fe640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a1c06ae40 0x7f8a1c06d300 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8a24036000 tx=0x7f8a2403d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.227+0000 7f8a2cff9640 1 -- 172.21.15.16:0/2205505673 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a20055100 con 0x7f8a300af650 2023-11-13T11:03:59.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.423+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a300b5070 con 0x7f8a300af650 2023-11-13T11:03:59.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.423+0000 7f8a2cff9640 1 -- 172.21.15.16:0/2205505673 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a20058720 con 0x7f8a300af650 2023-11-13T11:03:59.429 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-13T11:03:59.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a1c06ae40 msgr2=0x7f8a1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a1c06ae40 0x7f8a1c06d300 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8a24036000 tx=0x7f8a2403d040 comp rx=0 tx=0).stop 2023-11-13T11:03:59.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a300af650 msgr2=0x7f8a300cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a300af650 0x7f8a300cb000 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000cf60 tx=0x7f8a2000cf90 comp rx=0 tx=0).stop 2023-11-13T11:03:59.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 shutdown_connections 2023-11-13T11:03:59.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8a1c06ae40 0x7f8a1c06d300 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a3014a9d0 0x7f8a30149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a300c39a0 0x7f8a300c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 --2- 172.21.15.16:0/2205505673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a300af650 0x7f8a300cb000 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 >> 172.21.15.16:0/2205505673 conn(0x7f8a300b2520 msgr2=0x7f8a300cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:59.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 shutdown_connections 2023-11-13T11:03:59.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.426+0000 7f8a356ca640 1 -- 172.21.15.16:0/2205505673 wait complete. 2023-11-13T11:03:59.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-13T11:03:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.655+0000 7f4fe3fff640 1 Processor -- start 2023-11-13T11:03:59.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.656+0000 7f4fe3fff640 1 -- start start 2023-11-13T11:03:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.657+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fe415ce70 0x7f4fe40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.657+0000 7f4fe3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fe40d1850 0x7f4fe40c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.657+0000 7f4fe3fff640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe40ce0e0 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe3fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40ce3c0 con 0x7f4fe40d1850 2023-11-13T11:03:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe3fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40ce6a0 con 0x7f4fe415ce70 2023-11-13T11:03:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fe40d1850 0x7f4fe40c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fe40d1850 0x7f4fe40c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52264/0 (socket says 172.21.15.16:52264) 2023-11-13T11:03:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe37fe640 1 -- 172.21.15.16:0/3251502109 learned_addr learned my addr 172.21.15.16:0/3251502109 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:59.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe2ffd640 1 --2- 172.21.15.16:0/3251502109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fe415ce70 0x7f4fe40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe37fe640 1 -- 172.21.15.16:0/3251502109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fe415ce70 msgr2=0x7f4fe40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:59.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe37fe640 1 --2- 172.21.15.16:0/3251502109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fe415ce70 0x7f4fe40cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe37fe640 1 -- 172.21.15.16:0/3251502109 >> v1:172.21.15.184:6789/0 conn(0x7f4fe40cdbb0 legacy=0x7f4fe40d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:03:59.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.658+0000 7f4fe37fe640 1 -- 172.21.15.16:0/3251502109 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40ce980 con 0x7f4fe40d1850 2023-11-13T11:03:59.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.659+0000 7f4fe37fe640 1 --2- 172.21.15.16:0/3251502109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40d1850 0x7f4fe40c9c30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0098a0 tx=0x7f4fdc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cc412c00660641 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:59.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.660+0000 7f4fe1ffb640 1 -- 172.21.15.16:0/3251502109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc031640 con 0x7f4fe40d1850 2023-11-13T11:03:59.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.660+0000 7f4fe1ffb640 1 -- 172.21.15.16:0/3251502109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0317a0 con 0x7f4fe40d1850 2023-11-13T11:03:59.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.660+0000 7f4fe1ffb640 1 -- 172.21.15.16:0/3251502109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc031b30 con 0x7f4fe40d1850 2023-11-13T11:03:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.660+0000 7f4fe3fff640 1 -- 172.21.15.16:0/3251502109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40d1850 msgr2=0x7f4fe40c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.660+0000 7f4fe3fff640 1 --2- 172.21.15.16:0/3251502109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40d1850 0x7f4fe40c9c30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0098a0 tx=0x7f4fdc02b8f0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.660+0000 7f4fe3fff640 1 -- 172.21.15.16:0/3251502109 shutdown_connections 2023-11-13T11:03:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.660+0000 7f4fe3fff640 1 --2- 172.21.15.16:0/3251502109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40d1850 0x7f4fe40c9c30 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.660+0000 7f4fe3fff640 1 --2- 172.21.15.16:0/3251502109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fe415ce70 0x7f4fe40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.660+0000 7f4fe3fff640 1 -- 172.21.15.16:0/3251502109 >> 172.21.15.16:0/3251502109 conn(0x7f4fe408c580 msgr2=0x7f4fe409bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.661+0000 7f4fe3fff640 1 -- 172.21.15.16:0/3251502109 shutdown_connections 2023-11-13T11:03:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.661+0000 7f4fe3fff640 1 -- 172.21.15.16:0/3251502109 wait complete. 2023-11-13T11:03:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.662+0000 7f4fe3fff640 1 Processor -- start 2023-11-13T11:03:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.662+0000 7f4fe3fff640 1 -- start start 2023-11-13T11:03:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.663+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40cdbb0 0x7f4fe40c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.663+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fe40d1850 0x7f4fe40c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.663+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fe415ce70 0x7f4fe40c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.663+0000 7f4fe3fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe415e630 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.663+0000 7f4fe3fff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c93f0 con 0x7f4fe40d1850 2023-11-13T11:03:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.663+0000 7f4fe3fff640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c8b50 con 0x7f4fe415ce70 2023-11-13T11:03:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.663+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40cdbb0 0x7f4fe40c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40cdbb0 0x7f4fe40c5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52276/0 (socket says 172.21.15.16:52276) 2023-11-13T11:03:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fe2ffd640 1 -- 172.21.15.16:0/2117928794 learned_addr learned my addr 172.21.15.16:0/2117928794 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:03:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fe2ffd640 1 -- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fe415ce70 msgr2=0x7f4fe40c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fe2ffd640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fe415ce70 0x7f4fe40c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fe2ffd640 1 -- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fe40d1850 msgr2=0x7f4fe40c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fe2ffd640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fe40d1850 0x7f4fe40c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fe2ffd640 1 -- 172.21.15.16:0/2117928794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0094d0 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fe2ffd640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40cdbb0 0x7f4fe40c5a40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c7f0 tx=0x7f4fd400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/2117928794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4004220 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/2117928794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd4004380 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe40c8e30 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.664+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/2117928794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4007840 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.665+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe40c5f80 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.665+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe40675b0 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.667+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/2117928794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fd40079a0 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.667+0000 7f4fdb7fe640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4fc806ae70 0x7f4fc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:03:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.668+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/2117928794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f4fd4089460 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.668+0000 7f4fe27fc640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4fc806ae70 0x7f4fc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:03:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.669+0000 7f4fe27fc640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4fc806ae70 0x7f4fc806d330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4fe41566f0 tx=0x7f4fcc007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:03:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.672+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/2117928794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4fd4054360 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.855+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fe415e770 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.856+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/2117928794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fe415e770 con 0x7f4fe40cdbb0 2023-11-13T11:03:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-13T11:03:59.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4fc806ae70 msgr2=0x7f4fc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4fc806ae70 0x7f4fc806d330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4fe41566f0 tx=0x7f4fcc007d30 comp rx=0 tx=0).stop 2023-11-13T11:03:59.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40cdbb0 msgr2=0x7f4fe40c5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:03:59.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40cdbb0 0x7f4fe40c5a40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c7f0 tx=0x7f4fd400ccb0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 shutdown_connections 2023-11-13T11:03:59.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4fc806ae70 0x7f4fc806d330 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4fe415ce70 0x7f4fe40c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fe40d1850 0x7f4fe40c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 --2- 172.21.15.16:0/2117928794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fe40cdbb0 0x7f4fe40c5a40 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:03:59.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 >> 172.21.15.16:0/2117928794 conn(0x7f4fe408c580 msgr2=0x7f4fe40ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:03:59.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 shutdown_connections 2023-11-13T11:03:59.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:03:59.859+0000 7f4fe3fff640 1 -- 172.21.15.16:0/2117928794 wait complete. 2023-11-13T11:03:59.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:04:00.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.098+0000 7fa176295640 1 Processor -- start 2023-11-13T11:04:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.098+0000 7fa176295640 1 -- start start 2023-11-13T11:04:00.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.100+0000 7fa176295640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa17015b190 0x7fa17006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.100+0000 7fa176295640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa17006a190 0x7fa17006a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.100+0000 7fa176295640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa170158c20 con 0x7fa1700cb190 2023-11-13T11:04:00.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.100+0000 7fa176295640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17006ab40 con 0x7fa17006a190 2023-11-13T11:04:00.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.100+0000 7fa176295640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1700b1830 con 0x7fa17015b190 2023-11-13T11:04:00.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.100+0000 7fa175a94640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa17006a190 0x7fa17006a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.101+0000 7fa175293640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa1700cb190 0x7fa1700716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57060/0 (socket says 172.21.15.16:57060) 2023-11-13T11:04:00.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.101+0000 7fa175293640 1 -- 172.21.15.16:0/955679030 learned_addr learned my addr 172.21.15.16:0/955679030 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:00.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.101+0000 7fa175a94640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa17006a190 0x7fa17006a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52278/0 (socket says 172.21.15.16:52278) 2023-11-13T11:04:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.101+0000 7fa174a92640 1 --2- 172.21.15.16:0/955679030 >> v2:172.21.15.184:3300/0 conn(0x7fa17015b190 0x7fa17006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.101+0000 7fa175a94640 1 -- 172.21.15.16:0/955679030 >> v2:172.21.15.184:3300/0 conn(0x7fa17015b190 msgr2=0x7fa17006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:00.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.101+0000 7fa175a94640 1 --2- 172.21.15.16:0/955679030 >> v2:172.21.15.184:3300/0 conn(0x7fa17015b190 0x7fa17006f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.101+0000 7fa175a94640 1 -- 172.21.15.16:0/955679030 >> v1:172.21.15.16:6789/0 conn(0x7fa1700cb190 legacy=0x7fa1700716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.101+0000 7fa167fff640 1 -- 172.21.15.16:0/955679030 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287312684 0 0) 0x7fa170158c20 con 0x7fa1700cb190 2023-11-13T11:04:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.101+0000 7fa175a94640 1 -- 172.21.15.16:0/955679030 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17015b780 con 0x7fa17006a190 2023-11-13T11:04:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.102+0000 7fa175a94640 1 --2- 172.21.15.16:0/955679030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17006a190 0x7fa17006a600 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa1700ae3e0 tx=0x7fa16802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe1be23fb913b03 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa167fff640 1 -- 172.21.15.16:0/955679030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16802cbe0 con 0x7fa17006a190 2023-11-13T11:04:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa167fff640 1 -- 172.21.15.16:0/955679030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16802cd40 con 0x7fa17006a190 2023-11-13T11:04:00.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa167fff640 1 -- 172.21.15.16:0/955679030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168034660 con 0x7fa17006a190 2023-11-13T11:04:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa176295640 1 -- 172.21.15.16:0/955679030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17006a190 msgr2=0x7fa17006a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa176295640 1 --2- 172.21.15.16:0/955679030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17006a190 0x7fa17006a600 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa1700ae3e0 tx=0x7fa16802be30 comp rx=0 tx=0).stop 2023-11-13T11:04:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa176295640 1 -- 172.21.15.16:0/955679030 shutdown_connections 2023-11-13T11:04:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa176295640 1 --2- 172.21.15.16:0/955679030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17006a190 0x7fa17006a600 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa176295640 1 --2- 172.21.15.16:0/955679030 >> v2:172.21.15.184:3300/0 conn(0x7fa17015b190 0x7fa17006f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa176295640 1 -- 172.21.15.16:0/955679030 >> 172.21.15.16:0/955679030 conn(0x7fa1700b04a0 msgr2=0x7fa1700b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:00.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa176295640 1 -- 172.21.15.16:0/955679030 shutdown_connections 2023-11-13T11:04:00.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.103+0000 7fa176295640 1 -- 172.21.15.16:0/955679030 wait complete. 2023-11-13T11:04:00.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.105+0000 7fa176295640 1 Processor -- start 2023-11-13T11:04:00.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.105+0000 7fa176295640 1 -- start start 2023-11-13T11:04:00.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.106+0000 7fa176295640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa17006a190 0x7fa1701501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.106+0000 7fa176295640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1700cb190 0x7fa170155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.106+0000 7fa176295640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17015b190 0x7fa1700c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.106+0000 7fa176295640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1701548c0 con 0x7fa17015b190 2023-11-13T11:04:00.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.106+0000 7fa176295640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa170154ba0 con 0x7fa1700cb190 2023-11-13T11:04:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.106+0000 7fa176295640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa170154e80 con 0x7fa17006a190 2023-11-13T11:04:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.106+0000 7fa175a94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17015b190 0x7fa1700c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa175a94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17015b190 0x7fa1700c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52292/0 (socket says 172.21.15.16:52292) 2023-11-13T11:04:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa175a94640 1 -- 172.21.15.16:0/2242202706 learned_addr learned my addr 172.21.15.16:0/2242202706 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa174a92640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1700cb190 0x7fa170155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa175293640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa17006a190 0x7fa1701501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa175a94640 1 -- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa17006a190 msgr2=0x7fa1701501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa175a94640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa17006a190 0x7fa1701501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa175a94640 1 -- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1700cb190 msgr2=0x7fa170155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa175a94640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1700cb190 0x7fa170155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa175a94640 1 -- 172.21.15.16:0/2242202706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168009a70 con 0x7fa17015b190 2023-11-13T11:04:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa174a92640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1700cb190 0x7fa170155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.107+0000 7fa175a94640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17015b190 0x7fa1700c89d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa168009e80 tx=0x7fa168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.108+0000 7fa165ffb640 1 -- 172.21.15.16:0/2242202706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168040650 con 0x7fa17015b190 2023-11-13T11:04:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.108+0000 7fa165ffb640 1 -- 172.21.15.16:0/2242202706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1680407d0 con 0x7fa17015b190 2023-11-13T11:04:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.108+0000 7fa165ffb640 1 -- 172.21.15.16:0/2242202706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168040b80 con 0x7fa17015b190 2023-11-13T11:04:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.108+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa170155160 con 0x7fa17015b190 2023-11-13T11:04:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.108+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa170150700 con 0x7fa17015b190 2023-11-13T11:04:00.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.109+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa170155680 con 0x7fa17015b190 2023-11-13T11:04:00.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.110+0000 7fa165ffb640 1 -- 172.21.15.16:0/2242202706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa168040ce0 con 0x7fa17015b190 2023-11-13T11:04:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.111+0000 7fa165ffb640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa15406ae40 0x7fa15406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.111+0000 7fa165ffb640 1 -- 172.21.15.16:0/2242202706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fa1680ac650 con 0x7fa17015b190 2023-11-13T11:04:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.112+0000 7fa175293640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa15406ae40 0x7fa15406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.113+0000 7fa175293640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa15406ae40 0x7fa15406d300 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007fd0 tx=0x7fa16c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:00.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.115+0000 7fa165ffb640 1 -- 172.21.15.16:0/2242202706 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa170155680 con 0x7fa17015b190 2023-11-13T11:04:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.352+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa170155680 con 0x7fa17015b190 2023-11-13T11:04:00.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.353+0000 7fa165ffb640 1 -- 172.21.15.16:0/2242202706 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fa170155680 con 0x7fa17015b190 2023-11-13T11:04:00.359 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:00.359 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/960469256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":960469256},{"type":"v1","addr":"172.21.15.16:6833","nonce":960469256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1265200946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1265200946},{"type":"v1","addr":"172.21.15.184:6837","nonce":1265200946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3768737652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3768737652},{"type":"v1","addr":"172.21.15.16:6835","nonce":3768737652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:53.003626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"b","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6836/2632887594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2632887594},{"type":"v1","addr":"172.21.15.184:6836","nonce":2632887594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:04:00.360 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2023-11-13T11:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa15406ae40 msgr2=0x7fa15406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa15406ae40 0x7fa15406d300 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007fd0 tx=0x7fa16c009040 comp rx=0 tx=0).stop 2023-11-13T11:04:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17015b190 msgr2=0x7fa1700c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17015b190 0x7fa1700c89d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa168009e80 tx=0x7fa168004a40 comp rx=0 tx=0).stop 2023-11-13T11:04:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 shutdown_connections 2023-11-13T11:04:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fa15406ae40 0x7fa15406d300 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17015b190 0x7fa1700c89d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1700cb190 0x7fa170155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 --2- 172.21.15.16:0/2242202706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa17006a190 0x7fa1701501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 >> 172.21.15.16:0/2242202706 conn(0x7fa1700b04a0 msgr2=0x7fa170157f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 shutdown_connections 2023-11-13T11:04:00.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.356+0000 7fa176295640 1 -- 172.21.15.16:0/2242202706 wait complete. 2023-11-13T11:04:00.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:04:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.607+0000 7f91bcb75640 1 Processor -- start 2023-11-13T11:04:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.607+0000 7f91bcb75640 1 -- start start 2023-11-13T11:04:00.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.609+0000 7f91bcb75640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91b80cb190 0x7f91b815b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.609+0000 7f91bcb75640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f91b80716b0 0x7f91b806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.609+0000 7f91bcb75640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b8158c20 con 0x7f91b806a190 2023-11-13T11:04:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.609+0000 7f91bcb75640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80b1830 con 0x7f91b80716b0 2023-11-13T11:04:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.609+0000 7f91bcb75640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815b780 con 0x7f91b80cb190 2023-11-13T11:04:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.609+0000 7f91b7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f91b806a190 0x7f91b80b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57070/0 (socket says 172.21.15.16:57070) 2023-11-13T11:04:00.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.609+0000 7f91b7fff640 1 -- 172.21.15.16:0/981189404 learned_addr learned my addr 172.21.15.16:0/981189404 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b77fe640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91b80cb190 0x7f91b815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91affff640 1 --2- 172.21.15.16:0/981189404 >> v2:172.21.15.184:3300/0 conn(0x7f91b80716b0 0x7f91b806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983380756 0 0) 0x7f91b8158c20 con 0x7f91b806a190 2023-11-13T11:04:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91a4003520 con 0x7f91b806a190 2023-11-13T11:04:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3134142893 0 0) 0x7f91a4003520 con 0x7f91b806a190 2023-11-13T11:04:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91b80cb190 msgr2=0x7f91b815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91b80cb190 0x7f91b815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 >> v2:172.21.15.184:3300/0 conn(0x7f91b80716b0 msgr2=0x7f91b806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 --2- 172.21.15.16:0/981189404 >> v2:172.21.15.184:3300/0 conn(0x7f91b80716b0 0x7f91b806f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b815ba60 con 0x7f91b806a190 2023-11-13T11:04:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f91b00019f0 con 0x7f91b806a190 2023-11-13T11:04:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 >> v1:172.21.15.16:6789/0 conn(0x7f91b806a190 legacy=0x7f91b80b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91b80cb190 0x7f91a4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a4003f40 0x7f91a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91a4004d00 0x7f91a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80b1830 con 0x7f91a4003f40 2023-11-13T11:04:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815b780 con 0x7f91a4004d00 2023-11-13T11:04:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.610+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a400b980 con 0x7f91b80cb190 2023-11-13T11:04:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.611+0000 7f91affff640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a4003f40 0x7f91a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.611+0000 7f91b77fe640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91b80cb190 0x7f91a4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.611+0000 7f91b7fff640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91a4004d00 0x7f91a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.611+0000 7f91affff640 1 -- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91b80cb190 msgr2=0x7f91a4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.611+0000 7f91affff640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91b80cb190 0x7f91a4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.611+0000 7f91affff640 1 -- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91a4004d00 msgr2=0x7f91a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.611+0000 7f91affff640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91a4004d00 0x7f91a4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.611+0000 7f91affff640 1 -- 172.21.15.16:0/981189404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a4003520 con 0x7f91a4003f40 2023-11-13T11:04:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.612+0000 7f91affff640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a4003f40 0x7f91a40047c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009af0 tx=0x7f91a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761ffd91840281f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a002c710 con 0x7f91a4003f40 2023-11-13T11:04:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a002c870 con 0x7f91a4003f40 2023-11-13T11:04:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91b6ffd640 1 -- 172.21.15.16:0/981189404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a002cc00 con 0x7f91a4003f40 2023-11-13T11:04:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91bcb75640 1 -- 172.21.15.16:0/981189404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a4003f40 msgr2=0x7f91a40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91bcb75640 1 --2- 172.21.15.16:0/981189404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a4003f40 0x7f91a40047c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009af0 tx=0x7f91a40057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91b7fff640 1 -- 172.21.15.16:0/981189404 reap_dead start 2023-11-13T11:04:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91bcb75640 1 -- 172.21.15.16:0/981189404 shutdown_connections 2023-11-13T11:04:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91bcb75640 1 -- 172.21.15.16:0/981189404 >> 172.21.15.16:0/981189404 conn(0x7f91b80b04a0 msgr2=0x7f91b80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91bcb75640 1 -- 172.21.15.16:0/981189404 shutdown_connections 2023-11-13T11:04:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.613+0000 7f91bcb75640 1 -- 172.21.15.16:0/981189404 wait complete. 2023-11-13T11:04:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.615+0000 7f91bcb75640 1 Processor -- start 2023-11-13T11:04:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.615+0000 7f91bcb75640 1 -- start start 2023-11-13T11:04:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.616+0000 7f91bcb75640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91a4003f40 0x7f91b81f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.616+0000 7f91bcb75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91b81f5dd0 0x7f91b8158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.616+0000 7f91bcb75640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91b8159330 0x7f91b8147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91bcb75640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815a740 con 0x7f91b81f5dd0 2023-11-13T11:04:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91bcb75640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b81f6250 con 0x7f91b8159330 2023-11-13T11:04:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91bcb75640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8159bc0 con 0x7f91a4003f40 2023-11-13T11:04:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91b81f5dd0 0x7f91b8158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91b81f5dd0 0x7f91b8158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52304/0 (socket says 172.21.15.16:52304) 2023-11-13T11:04:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91affff640 1 -- 172.21.15.16:0/2122073132 learned_addr learned my addr 172.21.15.16:0/2122073132 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91b77fe640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91a4003f40 0x7f91b81f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91affff640 1 -- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91a4003f40 msgr2=0x7f91b81f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91affff640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91a4003f40 0x7f91b81f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91b7fff640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91b8159330 0x7f91b8147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91affff640 1 -- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91b8159330 msgr2=0x7f91b8147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91affff640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91b8159330 0x7f91b8147980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91affff640 1 -- 172.21.15.16:0/2122073132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b8159ea0 con 0x7f91b81f5dd0 2023-11-13T11:04:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.617+0000 7f91b77fe640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91a4003f40 0x7f91b81f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.618+0000 7f91affff640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91b81f5dd0 0x7f91b8158df0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f91a0037040 tx=0x7f91a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.618+0000 7f91b4ff9640 1 -- 172.21.15.16:0/2122073132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0003810 con 0x7f91b81f5dd0 2023-11-13T11:04:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.618+0000 7f91b4ff9640 1 -- 172.21.15.16:0/2122073132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a002de90 con 0x7f91b81f5dd0 2023-11-13T11:04:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.618+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b815a180 con 0x7f91b81f5dd0 2023-11-13T11:04:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.618+0000 7f91b4ff9640 1 -- 172.21.15.16:0/2122073132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0003b70 con 0x7f91b81f5dd0 2023-11-13T11:04:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.618+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b81f2080 con 0x7f91b81f5dd0 2023-11-13T11:04:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.619+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b8148680 con 0x7f91b81f5dd0 2023-11-13T11:04:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.620+0000 7f91b4ff9640 1 -- 172.21.15.16:0/2122073132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f91a0003cd0 con 0x7f91b81f5dd0 2023-11-13T11:04:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.621+0000 7f91b4ff9640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f919806ae40 0x7f919806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.621+0000 7f91b4ff9640 1 -- 172.21.15.16:0/2122073132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f91a00abaf0 con 0x7f91b81f5dd0 2023-11-13T11:04:00.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.622+0000 7f91b77fe640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f919806ae40 0x7f919806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.623+0000 7f91b77fe640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f919806ae40 0x7f919806d300 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006ce0 tx=0x7f91a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.626+0000 7f91b4ff9640 1 -- 172.21.15.16:0/2122073132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91b8148680 con 0x7f91b81f5dd0 2023-11-13T11:04:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.819+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91b8148680 con 0x7f91b81f5dd0 2023-11-13T11:04:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.823+0000 7f91b4ff9640 1 -- 172.21.15.16:0/2122073132 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16903 (secure 0 0 0) 0x7f91b8148680 con 0x7f91b81f5dd0 2023-11-13T11:04:00.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:00.829 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:50.019687+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:03:17.903258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-13T11:03:18.947896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6837/60343961":"2023-11-14T11:03:50.019659+0000","172.21.15.184:6835/60343961":"2023-11-14T11:03:50.019659+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:04:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f919806ae40 msgr2=0x7f919806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f919806ae40 0x7f919806d300 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006ce0 tx=0x7f91a8008040 comp rx=0 tx=0).stop 2023-11-13T11:04:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91b81f5dd0 msgr2=0x7f91b8158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91b81f5dd0 0x7f91b8158df0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f91a0037040 tx=0x7f91a0004730 comp rx=0 tx=0).stop 2023-11-13T11:04:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 shutdown_connections 2023-11-13T11:04:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f919806ae40 0x7f919806d300 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91b8159330 0x7f91b8147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91b81f5dd0 0x7f91b8158df0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 --2- 172.21.15.16:0/2122073132 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91a4003f40 0x7f91b81f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 >> 172.21.15.16:0/2122073132 conn(0x7f91b80b04a0 msgr2=0x7f91b80ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 shutdown_connections 2023-11-13T11:04:00.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:00.826+0000 7f91bcb75640 1 -- 172.21.15.16:0/2122073132 wait complete. 2023-11-13T11:04:00.850 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-13T11:04:00.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:04:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.068+0000 7f1d80e3f640 1 Processor -- start 2023-11-13T11:04:01.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.068+0000 7f1d80e3f640 1 -- start start 2023-11-13T11:04:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.070+0000 7f1d80e3f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 0x7f1d7c06cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.070+0000 7f1d80e3f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d7c06bb90 0x7f1d7c0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.070+0000 7f1d80e3f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c0b2900 con 0x7f1d7c0b2d90 2023-11-13T11:04:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.070+0000 7f1d80e3f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c15d300 con 0x7f1d7c06bb90 2023-11-13T11:04:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.070+0000 7f1d80e3f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0b1fd0 con 0x7f1d7c15cf20 2023-11-13T11:04:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.070+0000 7f1d79d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d7c06bb90 0x7f1d7c0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.070+0000 7f1d79d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d7c06bb90 0x7f1d7c0ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52312/0 (socket says 172.21.15.16:52312) 2023-11-13T11:04:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.070+0000 7f1d79d9b640 1 -- 172.21.15.16:0/1118175421 learned_addr learned my addr 172.21.15.16:0/1118175421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.071+0000 7f1d7a59c640 1 --2- 172.21.15.16:0/1118175421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 0x7f1d7c06cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.071+0000 7f1d7959a640 1 -- 172.21.15.16:0/1118175421 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360371009 0 0) 0x7f1d7c0b2900 con 0x7f1d7c0b2d90 2023-11-13T11:04:01.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.071+0000 7f1d79d9b640 1 -- 172.21.15.16:0/1118175421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 msgr2=0x7f1d7c06cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:01.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.071+0000 7f1d79d9b640 1 --2- 172.21.15.16:0/1118175421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 0x7f1d7c06cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.071+0000 7f1d79d9b640 1 -- 172.21.15.16:0/1118175421 >> v1:172.21.15.16:6789/0 conn(0x7f1d7c0b2d90 legacy=0x7f1d7c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.071+0000 7f1d79d9b640 1 -- 172.21.15.16:0/1118175421 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0b22b0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.072+0000 7f1d79d9b640 1 --2- 172.21.15.16:0/1118175421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 0x7f1d7c0ce5b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009870 tx=0x7f1d7002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5b39c996f276b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.073+0000 7f1d7959a640 1 -- 172.21.15.16:0/1118175421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70030840 con 0x7f1d7c06bb90 2023-11-13T11:04:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.073+0000 7f1d7959a640 1 -- 172.21.15.16:0/1118175421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d700309a0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.073+0000 7f1d7959a640 1 -- 172.21.15.16:0/1118175421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70030d30 con 0x7f1d7c06bb90 2023-11-13T11:04:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.073+0000 7f1d80e3f640 1 -- 172.21.15.16:0/1118175421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 msgr2=0x7f1d7c0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.073+0000 7f1d80e3f640 1 --2- 172.21.15.16:0/1118175421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 0x7f1d7c0ce5b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009870 tx=0x7f1d7002b980 comp rx=0 tx=0).stop 2023-11-13T11:04:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.074+0000 7f1d80e3f640 1 -- 172.21.15.16:0/1118175421 shutdown_connections 2023-11-13T11:04:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.074+0000 7f1d80e3f640 1 --2- 172.21.15.16:0/1118175421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 0x7f1d7c0ce5b0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.074+0000 7f1d80e3f640 1 --2- 172.21.15.16:0/1118175421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 0x7f1d7c06cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.074+0000 7f1d80e3f640 1 -- 172.21.15.16:0/1118175421 >> 172.21.15.16:0/1118175421 conn(0x7f1d7c06a190 msgr2=0x7f1d7c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.074+0000 7f1d80e3f640 1 -- 172.21.15.16:0/1118175421 shutdown_connections 2023-11-13T11:04:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.074+0000 7f1d80e3f640 1 -- 172.21.15.16:0/1118175421 wait complete. 2023-11-13T11:04:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.076+0000 7f1d80e3f640 1 Processor -- start 2023-11-13T11:04:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.076+0000 7f1d80e3f640 1 -- start start 2023-11-13T11:04:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.077+0000 7f1d80e3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 0x7f1d7c153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.077+0000 7f1d80e3f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d7c0b2d90 0x7f1d7c156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.077+0000 7f1d80e3f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 0x7f1d7c1433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.077+0000 7f1d80e3f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c15eaf0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.077+0000 7f1d80e3f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c156bf0 con 0x7f1d7c0b2d90 2023-11-13T11:04:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.077+0000 7f1d80e3f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c156ed0 con 0x7f1d7c15cf20 2023-11-13T11:04:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.077+0000 7f1d7a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 0x7f1d7c153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.077+0000 7f1d7a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 0x7f1d7c153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52316/0 (socket says 172.21.15.16:52316) 2023-11-13T11:04:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.077+0000 7f1d7a59c640 1 -- 172.21.15.16:0/4270176665 learned_addr learned my addr 172.21.15.16:0/4270176665 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.078+0000 7f1d7ad9d640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 0x7f1d7c1433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.078+0000 7f1d79d9b640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d7c0b2d90 0x7f1d7c156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.078+0000 7f1d7a59c640 1 -- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 msgr2=0x7f1d7c1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.078+0000 7f1d7a59c640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 0x7f1d7c1433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.078+0000 7f1d7a59c640 1 -- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d7c0b2d90 msgr2=0x7f1d7c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.078+0000 7f1d7a59c640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d7c0b2d90 0x7f1d7c156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.078+0000 7f1d7a59c640 1 -- 172.21.15.16:0/4270176665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d700094d0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.078+0000 7f1d79d9b640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d7c0b2d90 0x7f1d7c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:01.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.078+0000 7f1d7a59c640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 0x7f1d7c153fb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d5f0 tx=0x7f1d6c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:01.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.079+0000 7f1d6affd640 1 -- 172.21.15.16:0/4270176665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0125f0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.079+0000 7f1d6affd640 1 -- 172.21.15.16:0/4270176665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c012750 con 0x7f1d7c06bb90 2023-11-13T11:04:01.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.079+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c15e4a0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.079+0000 7f1d6affd640 1 -- 172.21.15.16:0/4270176665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c012b00 con 0x7f1d7c06bb90 2023-11-13T11:04:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.079+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c1567c0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.080+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c1569b0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.081+0000 7f1d6affd640 1 -- 172.21.15.16:0/4270176665 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d6c012c60 con 0x7f1d7c06bb90 2023-11-13T11:04:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.082+0000 7f1d6affd640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5c06ae90 0x7f1d5c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.083+0000 7f1d6affd640 1 -- 172.21.15.16:0/4270176665 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f1d6c089ce0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.083+0000 7f1d79d9b640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5c06ae90 0x7f1d5c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.084+0000 7f1d79d9b640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5c06ae90 0x7f1d5c06d350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1d70036000 tx=0x7f1d7003b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:01.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.088+0000 7f1d6affd640 1 -- 172.21.15.16:0/4270176665 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d7c1569b0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.318+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d7c1569b0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.320+0000 7f1d6affd640 1 -- 172.21.15.16:0/4270176665 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f1d7c1569b0 con 0x7f1d7c06bb90 2023-11-13T11:04:01.325 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:01.325 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/960469256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":960469256},{"type":"v1","addr":"172.21.15.16:6833","nonce":960469256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1265200946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1265200946},{"type":"v1","addr":"172.21.15.184:6837","nonce":1265200946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3768737652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3768737652},{"type":"v1","addr":"172.21.15.16:6835","nonce":3768737652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:53.003626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"b","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6836/2632887594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2632887594},{"type":"v1","addr":"172.21.15.184:6836","nonce":2632887594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:04:01.326 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2023-11-13T11:04:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.322+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5c06ae90 msgr2=0x7f1d5c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.322+0000 7f1d80e3f640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5c06ae90 0x7f1d5c06d350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1d70036000 tx=0x7f1d7003b040 comp rx=0 tx=0).stop 2023-11-13T11:04:01.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.322+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 msgr2=0x7f1d7c153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.322+0000 7f1d80e3f640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 0x7f1d7c153fb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d5f0 tx=0x7f1d6c00dab0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.323+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 shutdown_connections 2023-11-13T11:04:01.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.323+0000 7f1d80e3f640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1d5c06ae90 0x7f1d5c06d350 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.323+0000 7f1d80e3f640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1d7c15cf20 0x7f1d7c1433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.323+0000 7f1d80e3f640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d7c0b2d90 0x7f1d7c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.323+0000 7f1d80e3f640 1 --2- 172.21.15.16:0/4270176665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d7c06bb90 0x7f1d7c153fb0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.323+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 >> 172.21.15.16:0/4270176665 conn(0x7f1d7c06a190 msgr2=0x7f1d7c0b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:01.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.323+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 shutdown_connections 2023-11-13T11:04:01.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.323+0000 7f1d80e3f640 1 -- 172.21.15.16:0/4270176665 wait complete. 2023-11-13T11:04:01.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:04:01.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.561+0000 7f5d1e4ad640 1 Processor -- start 2023-11-13T11:04:01.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.561+0000 7f5d1e4ad640 1 -- start start 2023-11-13T11:04:01.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.563+0000 7f5d1e4ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d18161060 0x7f5d181621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.563+0000 7f5d1e4ad640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d18162700 con 0x7f5d1806f970 2023-11-13T11:04:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.563+0000 7f5d1e4ad640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d180b1840 con 0x7f5d1806a190 2023-11-13T11:04:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.563+0000 7f5d1e4ad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d18071600 con 0x7f5d18161060 2023-11-13T11:04:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.563+0000 7f5d1d4ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d18161060 0x7f5d181621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.564+0000 7f5d1d4ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d18161060 0x7f5d181621c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52322/0 (socket says 172.21.15.16:52322) 2023-11-13T11:04:01.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.564+0000 7f5d1d4ab640 1 -- 172.21.15.16:0/1653110931 learned_addr learned my addr 172.21.15.16:0/1653110931 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:01.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.564+0000 7f5d1d4ab640 1 -- 172.21.15.16:0/1653110931 >> v1:172.21.15.184:6789/0 conn(0x7f5d1806a190 legacy=0x7f5d180b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:01.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.564+0000 7f5d1d4ab640 1 -- 172.21.15.16:0/1653110931 >> v1:172.21.15.16:6789/0 conn(0x7f5d1806f970 legacy=0x7f5d18162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.564+0000 7f5d1d4ab640 1 -- 172.21.15.16:0/1653110931 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d180718e0 con 0x7f5d18161060 2023-11-13T11:04:01.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.564+0000 7f5d0ffff640 1 -- 172.21.15.16:0/1653110931 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179084545 0 0) 0x7f5d18162700 con 0x7f5d1806f970 2023-11-13T11:04:01.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.565+0000 7f5d1d4ab640 1 --2- 172.21.15.16:0/1653110931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d18161060 0x7f5d181621c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009870 tx=0x7f5d1402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6e0e663f320eb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:01.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.566+0000 7f5d0ffff640 1 -- 172.21.15.16:0/1653110931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14030850 con 0x7f5d18161060 2023-11-13T11:04:01.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.566+0000 7f5d0ffff640 1 -- 172.21.15.16:0/1653110931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d140309b0 con 0x7f5d18161060 2023-11-13T11:04:01.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.566+0000 7f5d0ffff640 1 -- 172.21.15.16:0/1653110931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14030d40 con 0x7f5d18161060 2023-11-13T11:04:01.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.566+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/1653110931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d18161060 msgr2=0x7f5d181621c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.566+0000 7f5d1e4ad640 1 --2- 172.21.15.16:0/1653110931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d18161060 0x7f5d181621c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009870 tx=0x7f5d1402b990 comp rx=0 tx=0).stop 2023-11-13T11:04:01.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.567+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/1653110931 shutdown_connections 2023-11-13T11:04:01.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.567+0000 7f5d1e4ad640 1 --2- 172.21.15.16:0/1653110931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d18161060 0x7f5d181621c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.567+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/1653110931 >> 172.21.15.16:0/1653110931 conn(0x7f5d180b04a0 msgr2=0x7f5d180b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:01.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.567+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/1653110931 shutdown_connections 2023-11-13T11:04:01.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.567+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/1653110931 wait complete. 2023-11-13T11:04:01.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.568+0000 7f5d1e4ad640 1 Processor -- start 2023-11-13T11:04:01.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.569+0000 7f5d1e4ad640 1 -- start start 2023-11-13T11:04:01.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1e4ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1806a190 0x7f5d18147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1e4ad640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d1806f970 0x7f5d18144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1e4ad640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d18161060 0x7f5d180af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1e4ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d18147950 con 0x7f5d1806a190 2023-11-13T11:04:01.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1e4ad640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d18144660 con 0x7f5d1806f970 2023-11-13T11:04:01.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1e4ad640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1815d120 con 0x7f5d18161060 2023-11-13T11:04:01.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1d4ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1806a190 0x7f5d18147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1d4ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1806a190 0x7f5d18147410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52324/0 (socket says 172.21.15.16:52324) 2023-11-13T11:04:01.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1d4ab640 1 -- 172.21.15.16:0/2809566836 learned_addr learned my addr 172.21.15.16:0/2809566836 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:01.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1ccaa640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d1806f970 0x7f5d18144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1d4ab640 1 -- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d18161060 msgr2=0x7f5d180af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1d4ab640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d18161060 0x7f5d180af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1d4ab640 1 -- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d1806f970 msgr2=0x7f5d18144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1d4ab640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d1806f970 0x7f5d18144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.570+0000 7f5d1d4ab640 1 -- 172.21.15.16:0/2809566836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d140094d0 con 0x7f5d1806a190 2023-11-13T11:04:01.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.571+0000 7f5d1d4ab640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1806a190 0x7f5d18147410 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009870 tx=0x7f5d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:01.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.571+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2809566836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1403e070 con 0x7f5d1806a190 2023-11-13T11:04:01.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.571+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2809566836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d14004030 con 0x7f5d1806a190 2023-11-13T11:04:01.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.571+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1815eb90 con 0x7f5d1806a190 2023-11-13T11:04:01.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.571+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2809566836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d140043c0 con 0x7f5d1806a190 2023-11-13T11:04:01.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.571+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1815f150 con 0x7f5d1806a190 2023-11-13T11:04:01.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.572+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1815e9f0 con 0x7f5d1806a190 2023-11-13T11:04:01.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.573+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2809566836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d14042070 con 0x7f5d1806a190 2023-11-13T11:04:01.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.575+0000 7f5d0dffb640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5cfc06ae40 0x7f5cfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:01.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.575+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2809566836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5d140ab820 con 0x7f5d1806a190 2023-11-13T11:04:01.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.575+0000 7f5d1ccaa640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5cfc06ae40 0x7f5cfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:01.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.577+0000 7f5d1ccaa640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5cfc06ae40 0x7f5cfc06d300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5d18160830 tx=0x7f5d08009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:01.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.579+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2809566836 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d1815e9f0 con 0x7f5d1806a190 2023-11-13T11:04:01.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.808+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d1815e9f0 con 0x7f5d1806a190 2023-11-13T11:04:01.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.810+0000 7f5d0dffb640 1 -- 172.21.15.16:0/2809566836 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f5d1815e9f0 con 0x7f5d1806a190 2023-11-13T11:04:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:01.816 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/960469256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":960469256},{"type":"v1","addr":"172.21.15.16:6833","nonce":960469256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1265200946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1265200946},{"type":"v1","addr":"172.21.15.184:6837","nonce":1265200946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3768737652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3768737652},{"type":"v1","addr":"172.21.15.16:6835","nonce":3768737652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-13T11:03:19.913517+0000","modified":"2023-11-13T11:03:53.003626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"b","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.184:6836/2632887594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":2632887594},{"type":"v1","addr":"172.21.15.184:6836","nonce":2632887594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-13T11:04:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2023-11-13T11:04:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5cfc06ae40 msgr2=0x7f5cfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5cfc06ae40 0x7f5cfc06d300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5d18160830 tx=0x7f5d08009040 comp rx=0 tx=0).stop 2023-11-13T11:04:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1806a190 msgr2=0x7f5d18147410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1806a190 0x7f5d18147410 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009870 tx=0x7f5d140046c0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 shutdown_connections 2023-11-13T11:04:01.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f5cfc06ae40 0x7f5cfc06d300 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5d18161060 0x7f5d180af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d1806f970 0x7f5d18144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 --2- 172.21.15.16:0/2809566836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d1806a190 0x7f5d18147410 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.812+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 >> 172.21.15.16:0/2809566836 conn(0x7f5d180b04a0 msgr2=0x7f5d181630a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.813+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 shutdown_connections 2023-11-13T11:04:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:01.813+0000 7f5d1e4ad640 1 -- 172.21.15.16:0/2809566836 wait complete. 2023-11-13T11:04:01.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:04:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.061+0000 7feb5b59e640 1 Processor -- start 2023-11-13T11:04:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.061+0000 7feb5b59e640 1 -- start start 2023-11-13T11:04:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.062+0000 7feb5b59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb5c0b30e0 0x7feb5c0d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.062+0000 7feb5b59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5c0b3cf0 con 0x7feb5c0d2ae0 2023-11-13T11:04:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.062+0000 7feb5b59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5c0d3480 con 0x7feb5c15cfc0 2023-11-13T11:04:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.062+0000 7feb5b59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c0b3550 con 0x7feb5c0b30e0 2023-11-13T11:04:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.063+0000 7feb5ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb5c0b30e0 0x7feb5c0d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.063+0000 7feb5ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb5c0b30e0 0x7feb5c0d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52334/0 (socket says 172.21.15.16:52334) 2023-11-13T11:04:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.063+0000 7feb5ad9d640 1 -- 172.21.15.16:0/3751191894 learned_addr learned my addr 172.21.15.16:0/3751191894 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.063+0000 7feb53fff640 1 --1- 172.21.15.16:0/3751191894 >> v1:172.21.15.16:6789/0 conn(0x7feb5c0d2ae0 0x7feb5c0b29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57090/0 (socket says 172.21.15.16:57090) 2023-11-13T11:04:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.063+0000 7feb5ad9d640 1 -- 172.21.15.16:0/3751191894 >> v1:172.21.15.184:6789/0 conn(0x7feb5c15cfc0 legacy=0x7feb5c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.063+0000 7feb5ad9d640 1 -- 172.21.15.16:0/3751191894 >> v1:172.21.15.16:6789/0 conn(0x7feb5c0d2ae0 legacy=0x7feb5c0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.063+0000 7feb5ad9d640 1 -- 172.21.15.16:0/3751191894 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c0cde80 con 0x7feb5c0b30e0 2023-11-13T11:04:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.064+0000 7feb5ad9d640 1 --2- 172.21.15.16:0/3751191894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c0b30e0 0x7feb5c0d2f40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7feb54009e50 tx=0x7feb5402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0544777f8fa38c server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.065+0000 7feb59d9b640 1 -- 172.21.15.16:0/3751191894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5402cb10 con 0x7feb5c0b30e0 2023-11-13T11:04:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.065+0000 7feb59d9b640 1 -- 172.21.15.16:0/3751191894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5402cc70 con 0x7feb5c0b30e0 2023-11-13T11:04:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.065+0000 7feb59d9b640 1 -- 172.21.15.16:0/3751191894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb540343f0 con 0x7feb5c0b30e0 2023-11-13T11:04:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.065+0000 7feb5b59e640 1 -- 172.21.15.16:0/3751191894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c0b30e0 msgr2=0x7feb5c0d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.065+0000 7feb5b59e640 1 --2- 172.21.15.16:0/3751191894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c0b30e0 0x7feb5c0d2f40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7feb54009e50 tx=0x7feb5402bf70 comp rx=0 tx=0).stop 2023-11-13T11:04:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.066+0000 7feb5b59e640 1 -- 172.21.15.16:0/3751191894 shutdown_connections 2023-11-13T11:04:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.066+0000 7feb5b59e640 1 --2- 172.21.15.16:0/3751191894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c0b30e0 0x7feb5c0d2f40 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.066+0000 7feb5b59e640 1 -- 172.21.15.16:0/3751191894 >> 172.21.15.16:0/3751191894 conn(0x7feb5c08c570 msgr2=0x7feb5c09be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.066+0000 7feb5b59e640 1 -- 172.21.15.16:0/3751191894 shutdown_connections 2023-11-13T11:04:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.066+0000 7feb5b59e640 1 -- 172.21.15.16:0/3751191894 wait complete. 2023-11-13T11:04:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.067+0000 7feb5b59e640 1 Processor -- start 2023-11-13T11:04:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.068+0000 7feb5b59e640 1 -- start start 2023-11-13T11:04:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.068+0000 7feb5b59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb5c0b30e0 0x7feb5c15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5b59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb5c0d2ae0 0x7feb5c0c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c15cfc0 0x7feb5c0c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c15d430 con 0x7feb5c15cfc0 2023-11-13T11:04:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5b59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c15d710 con 0x7feb5c0d2ae0 2023-11-13T11:04:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5b59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c15d9f0 con 0x7feb5c0b30e0 2023-11-13T11:04:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c15cfc0 0x7feb5c0c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c15cfc0 0x7feb5c0c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52342/0 (socket says 172.21.15.16:52342) 2023-11-13T11:04:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5ad9d640 1 -- 172.21.15.16:0/2989360974 learned_addr learned my addr 172.21.15.16:0/2989360974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5ad9d640 1 -- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb5c0b30e0 msgr2=0x7feb5c15b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5ad9d640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb5c0b30e0 0x7feb5c15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5ad9d640 1 -- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb5c0d2ae0 msgr2=0x7feb5c0c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5ad9d640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb5c0d2ae0 0x7feb5c0c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.069+0000 7feb5ad9d640 1 -- 172.21.15.16:0/2989360974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb54009b30 con 0x7feb5c15cfc0 2023-11-13T11:04:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.070+0000 7feb5ad9d640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c15cfc0 0x7feb5c0c8c10 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7feb54009e50 tx=0x7feb54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.070+0000 7feb52ffd640 1 -- 172.21.15.16:0/2989360974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54009510 con 0x7feb5c15cfc0 2023-11-13T11:04:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.070+0000 7feb52ffd640 1 -- 172.21.15.16:0/2989360974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5403e450 con 0x7feb5c15cfc0 2023-11-13T11:04:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.070+0000 7feb52ffd640 1 -- 172.21.15.16:0/2989360974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5403e800 con 0x7feb5c15cfc0 2023-11-13T11:04:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.070+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c15dcd0 con 0x7feb5c15cfc0 2023-11-13T11:04:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.070+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c15e290 con 0x7feb5c15cfc0 2023-11-13T11:04:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.071+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5c04e4d0 con 0x7feb5c15cfc0 2023-11-13T11:04:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.072+0000 7feb52ffd640 1 -- 172.21.15.16:0/2989360974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb5403e960 con 0x7feb5c15cfc0 2023-11-13T11:04:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.073+0000 7feb52ffd640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb3c06ae40 0x7feb3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.073+0000 7feb52ffd640 1 -- 172.21.15.16:0/2989360974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7feb540ac670 con 0x7feb5c15cfc0 2023-11-13T11:04:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.074+0000 7feb5a59c640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb3c06ae40 0x7feb3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.075+0000 7feb5a59c640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb3c06ae40 0x7feb3c06d300 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feb4c000f60 tx=0x7feb4c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.078+0000 7feb52ffd640 1 -- 172.21.15.16:0/2989360974 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb54077630 con 0x7feb5c15cfc0 2023-11-13T11:04:02.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.275+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb5c0b8b00 con 0x7feb5c15cfc0 2023-11-13T11:04:02.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.277+0000 7feb52ffd640 1 -- 172.21.15.16:0/2989360974 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16903 (secure 0 0 0) 0x7feb5407ac50 con 0x7feb5c15cfc0 2023-11-13T11:04:02.282 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:02.282 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:03:50.019687+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-13T11:03:17.903258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-13T11:03:18.947896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6837/60343961":"2023-11-14T11:03:50.019659+0000","172.21.15.184:6835/60343961":"2023-11-14T11:03:50.019659+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:04:02.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb3c06ae40 msgr2=0x7feb3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb3c06ae40 0x7feb3c06d300 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feb4c000f60 tx=0x7feb4c0079b0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c15cfc0 msgr2=0x7feb5c0c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c15cfc0 0x7feb5c0c8c10 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7feb54009e50 tx=0x7feb54004a40 comp rx=0 tx=0).stop 2023-11-13T11:04:02.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 shutdown_connections 2023-11-13T11:04:02.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb3c06ae40 0x7feb3c06d300 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb5c15cfc0 0x7feb5c0c8c10 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb5c0d2ae0 0x7feb5c0c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 --2- 172.21.15.16:0/2989360974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb5c0b30e0 0x7feb5c15b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.279+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 >> 172.21.15.16:0/2989360974 conn(0x7feb5c08c570 msgr2=0x7feb5c0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:02.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.280+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 shutdown_connections 2023-11-13T11:04:02.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.280+0000 7feb5b59e640 1 -- 172.21.15.16:0/2989360974 wait complete. 2023-11-13T11:04:02.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-13T11:04:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.510+0000 7ff744fa9640 1 Processor -- start 2023-11-13T11:04:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.510+0000 7ff744fa9640 1 -- start start 2023-11-13T11:04:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.512+0000 7ff744fa9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7400ccc60 0x7ff740071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.512+0000 7ff744fa9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff740071980 0x7ff74006fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.512+0000 7ff744fa9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7400cd110 con 0x7ff740155410 2023-11-13T11:04:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.512+0000 7ff744fa9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff740071e10 con 0x7ff7400ccc60 2023-11-13T11:04:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.512+0000 7ff744fa9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff74015b500 con 0x7ff740071980 2023-11-13T11:04:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.512+0000 7ff73e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff740155410 0x7ff7400ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57106/0 (socket says 172.21.15.16:57106) 2023-11-13T11:04:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.512+0000 7ff73e59c640 1 -- 172.21.15.16:0/861114049 learned_addr learned my addr 172.21.15.16:0/861114049 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:02.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.512+0000 7ff73dd9b640 1 --2- 172.21.15.16:0/861114049 >> v2:172.21.15.16:3300/0 conn(0x7ff7400ccc60 0x7ff740071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.513+0000 7ff73ed9d640 1 --2- 172.21.15.16:0/861114049 >> v2:172.21.15.184:3300/0 conn(0x7ff740071980 0x7ff74006fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.513+0000 7ff73d59a640 1 -- 172.21.15.16:0/861114049 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686145540 0 0) 0x7ff7400cd110 con 0x7ff740155410 2023-11-13T11:04:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.513+0000 7ff73d59a640 1 -- 172.21.15.16:0/861114049 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff728003520 con 0x7ff740155410 2023-11-13T11:04:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.513+0000 7ff73dd9b640 1 -- 172.21.15.16:0/861114049 >> v2:172.21.15.184:3300/0 conn(0x7ff740071980 msgr2=0x7ff74006fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.513+0000 7ff73dd9b640 1 --2- 172.21.15.16:0/861114049 >> v2:172.21.15.184:3300/0 conn(0x7ff740071980 0x7ff74006fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.513+0000 7ff73dd9b640 1 -- 172.21.15.16:0/861114049 >> v1:172.21.15.16:6789/0 conn(0x7ff740155410 legacy=0x7ff7400ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.513+0000 7ff73dd9b640 1 -- 172.21.15.16:0/861114049 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74015b7e0 con 0x7ff7400ccc60 2023-11-13T11:04:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.514+0000 7ff73dd9b640 1 --2- 172.21.15.16:0/861114049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7400ccc60 0x7ff740071440 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff74015e700 tx=0x7ff73402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99060b85d3a74f3e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.514+0000 7ff73d59a640 1 -- 172.21.15.16:0/861114049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734031830 con 0x7ff7400ccc60 2023-11-13T11:04:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.514+0000 7ff73d59a640 1 -- 172.21.15.16:0/861114049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff734031990 con 0x7ff7400ccc60 2023-11-13T11:04:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.514+0000 7ff73d59a640 1 -- 172.21.15.16:0/861114049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734031d20 con 0x7ff7400ccc60 2023-11-13T11:04:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.515+0000 7ff744fa9640 1 -- 172.21.15.16:0/861114049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7400ccc60 msgr2=0x7ff740071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.515+0000 7ff744fa9640 1 --2- 172.21.15.16:0/861114049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7400ccc60 0x7ff740071440 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff74015e700 tx=0x7ff73402b8d0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.515+0000 7ff744fa9640 1 -- 172.21.15.16:0/861114049 shutdown_connections 2023-11-13T11:04:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.515+0000 7ff744fa9640 1 --2- 172.21.15.16:0/861114049 >> v2:172.21.15.184:3300/0 conn(0x7ff740071980 0x7ff74006fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.515+0000 7ff744fa9640 1 --2- 172.21.15.16:0/861114049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7400ccc60 0x7ff740071440 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.515+0000 7ff744fa9640 1 -- 172.21.15.16:0/861114049 >> 172.21.15.16:0/861114049 conn(0x7ff74006a190 msgr2=0x7ff74006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.515+0000 7ff744fa9640 1 -- 172.21.15.16:0/861114049 shutdown_connections 2023-11-13T11:04:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.515+0000 7ff744fa9640 1 -- 172.21.15.16:0/861114049 wait complete. 2023-11-13T11:04:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.517+0000 7ff744fa9640 1 Processor -- start 2023-11-13T11:04:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.517+0000 7ff744fa9640 1 -- start start 2023-11-13T11:04:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.518+0000 7ff744fa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff740071980 0x7ff740155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.518+0000 7ff744fa9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7400ccc60 0x7ff740149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.518+0000 7ff744fa9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff740155410 0x7ff74015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.518+0000 7ff744fa9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74014f220 con 0x7ff740071980 2023-11-13T11:04:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.518+0000 7ff744fa9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff740158080 con 0x7ff740155410 2023-11-13T11:04:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.518+0000 7ff744fa9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7401495d0 con 0x7ff7400ccc60 2023-11-13T11:04:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.518+0000 7ff73e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff740071980 0x7ff740155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.518+0000 7ff73e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff740071980 0x7ff740155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52368/0 (socket says 172.21.15.16:52368) 2023-11-13T11:04:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.518+0000 7ff73e59c640 1 -- 172.21.15.16:0/4275701715 learned_addr learned my addr 172.21.15.16:0/4275701715 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff73e59c640 1 -- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7400ccc60 msgr2=0x7ff740149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff73e59c640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7400ccc60 0x7ff740149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff73e59c640 1 -- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff740155410 msgr2=0x7ff74015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff73e59c640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff740155410 0x7ff74015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff73e59c640 1 -- 172.21.15.16:0/4275701715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff734009400 con 0x7ff740071980 2023-11-13T11:04:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff73e59c640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff740071980 0x7ff740155870 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff73000c500 tx=0x7ff73000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff72effd640 1 -- 172.21.15.16:0/4275701715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730005ce0 con 0x7ff740071980 2023-11-13T11:04:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff72effd640 1 -- 172.21.15.16:0/4275701715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff730005e40 con 0x7ff740071980 2023-11-13T11:04:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff72effd640 1 -- 172.21.15.16:0/4275701715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730007330 con 0x7ff740071980 2023-11-13T11:04:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff740158320 con 0x7ff740071980 2023-11-13T11:04:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.519+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74014ef10 con 0x7ff740071980 2023-11-13T11:04:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.520+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff740067090 con 0x7ff740071980 2023-11-13T11:04:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.521+0000 7ff72effd640 1 -- 172.21.15.16:0/4275701715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff730007550 con 0x7ff740071980 2023-11-13T11:04:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.522+0000 7ff72effd640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff72006ae40 0x7ff72006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.522+0000 7ff72effd640 1 -- 172.21.15.16:0/4275701715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff730088e60 con 0x7ff740071980 2023-11-13T11:04:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.523+0000 7ff73dd9b640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff72006ae40 0x7ff72006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.524+0000 7ff73dd9b640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff72006ae40 0x7ff72006d300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff734036000 tx=0x7ff734030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.526+0000 7ff72effd640 1 -- 172.21.15.16:0/4275701715 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff730053e20 con 0x7ff740071980 2023-11-13T11:04:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.750+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff74015f670 con 0x7ff740071980 2023-11-13T11:04:02.756 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-13T11:04:02.752+0000 7fe1550d2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-13T11:04:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.755+0000 7ff72effd640 1 -- 172.21.15.16:0/4275701715 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff730017020 con 0x7ff740071980 2023-11-13T11:04:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-13T11:04:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.757+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff72006ae40 msgr2=0x7ff72006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.757+0000 7ff744fa9640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff72006ae40 0x7ff72006d300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff734036000 tx=0x7ff734030000 comp rx=0 tx=0).stop 2023-11-13T11:04:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.757+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff740071980 msgr2=0x7ff740155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.757+0000 7ff744fa9640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff740071980 0x7ff740155870 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff73000c500 tx=0x7ff73000c9c0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.758+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 shutdown_connections 2023-11-13T11:04:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.758+0000 7ff744fa9640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff72006ae40 0x7ff72006d300 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.758+0000 7ff744fa9640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff740155410 0x7ff74015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.758+0000 7ff744fa9640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7400ccc60 0x7ff740149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.758+0000 7ff744fa9640 1 --2- 172.21.15.16:0/4275701715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff740071980 0x7ff740155870 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.758+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 >> 172.21.15.16:0/4275701715 conn(0x7ff74006a190 msgr2=0x7ff740153760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.758+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 shutdown_connections 2023-11-13T11:04:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.759+0000 7ff744fa9640 1 -- 172.21.15.16:0/4275701715 wait complete. 2023-11-13T11:04:02.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-13T11:04:02.851 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-11-13T11:04:03.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.999+0000 7f8561207640 1 Processor -- start 2023-11-13T11:04:03.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:02.999+0000 7f8561207640 1 -- start start 2023-11-13T11:04:03.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.001+0000 7f8561207640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f855c0ce550 0x7f855c0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:03.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.001+0000 7f8561207640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855c0ceef0 con 0x7f855c144570 2023-11-13T11:04:03.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.001+0000 7f8561207640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855c144950 con 0x7f855c0b19d0 2023-11-13T11:04:03.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.001+0000 7f8561207640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855c0ce300 con 0x7f855c0ce550 2023-11-13T11:04:03.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.001+0000 7f855ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f855c144570 0x7f855c15f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44200/0 (socket says 172.21.15.16:44200) 2023-11-13T11:04:03.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.001+0000 7f855ad9d640 1 -- 172.21.15.16:0/4115843498 learned_addr learned my addr 172.21.15.16:0/4115843498 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:03.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f855a59c640 1 --2- 172.21.15.16:0/4115843498 >> v2:172.21.15.184:3300/0 conn(0x7f855c0ce550 0x7f855c0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:03.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752730494 0 0) 0x7f855c0ceef0 con 0x7f855c144570 2023-11-13T11:04:03.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8548003520 con 0x7f855c144570 2023-11-13T11:04:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3587278019 0 0) 0x7f8548003520 con 0x7f855c144570 2023-11-13T11:04:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 >> v2:172.21.15.184:3300/0 conn(0x7f855c0ce550 msgr2=0x7f855c0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 --2- 172.21.15.16:0/4115843498 >> v2:172.21.15.184:3300/0 conn(0x7f855c0ce550 0x7f855c0ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 >> v1:172.21.15.184:6789/0 conn(0x7f855c0b19d0 legacy=0x7f855c0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c0cd260 con 0x7f855c144570 2023-11-13T11:04:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f854c001670 con 0x7f855c144570 2023-11-13T11:04:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 >> v1:172.21.15.16:6789/0 conn(0x7f855c144570 legacy=0x7f855c15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.002+0000 7f8559d9b640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85480039c0 0x7f8548003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f8559d9b640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480047c0 0x7f8548004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f8559d9b640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8548005180 0x7f854800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0ce300 con 0x7f85480047c0 2023-11-13T11:04:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854800c080 con 0x7f85480039c0 2023-11-13T11:04:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854800c240 con 0x7f8548005180 2023-11-13T11:04:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855a59c640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480047c0 0x7f8548004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855ad9d640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85480039c0 0x7f8548003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855b59e640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8548005180 0x7f854800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855a59c640 1 -- 172.21.15.16:0/4115843498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8548005180 msgr2=0x7f854800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855a59c640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8548005180 0x7f854800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855a59c640 1 -- 172.21.15.16:0/4115843498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85480039c0 msgr2=0x7f8548003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855a59c640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85480039c0 0x7f8548003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855a59c640 1 -- 172.21.15.16:0/4115843498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548003520 con 0x7f85480047c0 2023-11-13T11:04:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855b59e640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8548005180 0x7f854800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.003+0000 7f855b59e640 1 -- 172.21.15.16:0/4115843498 reap_dead start 2023-11-13T11:04:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.005+0000 7f855a59c640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480047c0 0x7f8548004c40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f8544009fc0 tx=0x7f85480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817be88d55d3b01e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.005+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854402cc20 con 0x7f85480047c0 2023-11-13T11:04:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.005+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854400f040 con 0x7f85480047c0 2023-11-13T11:04:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.005+0000 7f8559d9b640 1 -- 172.21.15.16:0/4115843498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544034440 con 0x7f85480047c0 2023-11-13T11:04:03.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.005+0000 7f8561207640 1 -- 172.21.15.16:0/4115843498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480047c0 msgr2=0x7f8548004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:03.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.005+0000 7f8561207640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480047c0 0x7f8548004c40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f8544009fc0 tx=0x7f85480057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.006+0000 7f8561207640 1 -- 172.21.15.16:0/4115843498 shutdown_connections 2023-11-13T11:04:03.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.006+0000 7f8561207640 1 --2- 172.21.15.16:0/4115843498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85480047c0 0x7f8548004c40 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.006+0000 7f8561207640 1 -- 172.21.15.16:0/4115843498 >> 172.21.15.16:0/4115843498 conn(0x7f855c06a190 msgr2=0x7f855c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:03.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.006+0000 7f8561207640 1 -- 172.21.15.16:0/4115843498 shutdown_connections 2023-11-13T11:04:03.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.006+0000 7f8561207640 1 -- 172.21.15.16:0/4115843498 wait complete. 2023-11-13T11:04:03.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.007+0000 7f8561207640 1 Processor -- start 2023-11-13T11:04:03.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.008+0000 7f8561207640 1 -- start start 2023-11-13T11:04:03.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f8561207640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85480047c0 0x7f855c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f8561207640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f855c13a140 0x7f855c154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f8561207640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855c138470 0x7f855c151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f8561207640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c154f90 con 0x7f855c138470 2023-11-13T11:04:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f8561207640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c154cb0 con 0x7f85480047c0 2023-11-13T11:04:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f8561207640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855c141c90 con 0x7f855c13a140 2023-11-13T11:04:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f855b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855c138470 0x7f855c151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:03.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f855b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855c138470 0x7f855c151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51010/0 (socket says 172.21.15.16:51010) 2023-11-13T11:04:03.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f855b59e640 1 -- 172.21.15.16:0/2456306046 learned_addr learned my addr 172.21.15.16:0/2456306046 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:03.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f855b59e640 1 -- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f855c13a140 msgr2=0x7f855c154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:03.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f855b59e640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f855c13a140 0x7f855c154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f855b59e640 1 -- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85480047c0 msgr2=0x7f855c158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f855b59e640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85480047c0 0x7f855c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.009+0000 7f855b59e640 1 -- 172.21.15.16:0/2456306046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c141f70 con 0x7f855c138470 2023-11-13T11:04:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.010+0000 7f855b59e640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855c138470 0x7f855c151c60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f855c0b30c0 tx=0x7f855400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.010+0000 7f85537fe640 1 -- 172.21.15.16:0/2456306046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400b070 con 0x7f855c138470 2023-11-13T11:04:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.010+0000 7f85537fe640 1 -- 172.21.15.16:0/2456306046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85540035f0 con 0x7f855c138470 2023-11-13T11:04:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.010+0000 7f85537fe640 1 -- 172.21.15.16:0/2456306046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554007ea0 con 0x7f855c138470 2023-11-13T11:04:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.010+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c142250 con 0x7f855c138470 2023-11-13T11:04:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.010+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c138920 con 0x7f855c138470 2023-11-13T11:04:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.011+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c142440 con 0x7f855c138470 2023-11-13T11:04:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.012+0000 7f85537fe640 1 -- 172.21.15.16:0/2456306046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85540039a0 con 0x7f855c138470 2023-11-13T11:04:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.013+0000 7f85537fe640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f853c06ae40 0x7f853c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.013+0000 7f85537fe640 1 -- 172.21.15.16:0/2456306046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f8554088e20 con 0x7f855c138470 2023-11-13T11:04:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.014+0000 7f855ad9d640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f853c06ae40 0x7f853c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.015+0000 7f855ad9d640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f853c06ae40 0x7f853c06d300 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f854c0058b0 tx=0x7f854c000ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.018+0000 7f85537fe640 1 -- 172.21.15.16:0/2456306046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8554053c40 con 0x7f855c138470 2023-11-13T11:04:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.251+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f855c072520 con 0x7f855c138470 2023-11-13T11:04:03.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.758+0000 7f85537fe640 1 -- 172.21.15.16:0/2456306046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85540180a0 con 0x7f855c138470 2023-11-13T11:04:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f853c06ae40 msgr2=0x7f853c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f853c06ae40 0x7f853c06d300 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f854c0058b0 tx=0x7f854c000ed0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855c138470 msgr2=0x7f855c151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855c138470 0x7f855c151c60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f855c0b30c0 tx=0x7f855400d890 comp rx=0 tx=0).stop 2023-11-13T11:04:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 shutdown_connections 2023-11-13T11:04:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f853c06ae40 0x7f853c06d300 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855c138470 0x7f855c151c60 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f855c13a140 0x7f855c154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 --2- 172.21.15.16:0/2456306046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85480047c0 0x7f855c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 >> 172.21.15.16:0/2456306046 conn(0x7f855c06a190 msgr2=0x7f855c15e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 shutdown_connections 2023-11-13T11:04:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:03.762+0000 7f8561207640 1 -- 172.21.15.16:0/2456306046 wait complete. 2023-11-13T11:04:03.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-13T11:04:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.007+0000 7f66cd3d0640 1 Processor -- start 2023-11-13T11:04:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.007+0000 7f66cd3d0640 1 -- start start 2023-11-13T11:04:04.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.009+0000 7f66cd3d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66c81584d0 0x7f66c80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.009+0000 7f66cd3d0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 0x7f66c80adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.009+0000 7f66cd3d0640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f66c80712d0 0x7f66c8071760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.009+0000 7f66cd3d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c815b080 con 0x7f66c81584d0 2023-11-13T11:04:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.009+0000 7f66cd3d0640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c8071ca0 con 0x7f66c80712d0 2023-11-13T11:04:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.009+0000 7f66cd3d0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c815b2c0 con 0x7f66c81588b0 2023-11-13T11:04:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.009+0000 7f66c7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66c81584d0 0x7f66c80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.009+0000 7f66c7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66c81584d0 0x7f66c80ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51014/0 (socket says 172.21.15.16:51014) 2023-11-13T11:04:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.009+0000 7f66c7fff640 1 -- 172.21.15.16:0/3987219355 learned_addr learned my addr 172.21.15.16:0/3987219355 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.010+0000 7f66c77fe640 1 --2- 172.21.15.16:0/3987219355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 0x7f66c80adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.010+0000 7f66ccbcf640 1 --2- 172.21.15.16:0/3987219355 >> v2:172.21.15.184:3300/0 conn(0x7f66c80712d0 0x7f66c8071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.010+0000 7f66c7fff640 1 -- 172.21.15.16:0/3987219355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 msgr2=0x7f66c80adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.010+0000 7f66c7fff640 1 --2- 172.21.15.16:0/3987219355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 0x7f66c80adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.010+0000 7f66c7fff640 1 -- 172.21.15.16:0/3987219355 >> v2:172.21.15.184:3300/0 conn(0x7f66c80712d0 msgr2=0x7f66c8071760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.010+0000 7f66c7fff640 1 --2- 172.21.15.16:0/3987219355 >> v2:172.21.15.184:3300/0 conn(0x7f66c80712d0 0x7f66c8071760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.010+0000 7f66c7fff640 1 -- 172.21.15.16:0/3987219355 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c815b5a0 con 0x7f66c81584d0 2023-11-13T11:04:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.012+0000 7f66c7fff640 1 --2- 172.21.15.16:0/3987219355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 0x7f66c80ad730 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f66c806de90 tx=0x7f66b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec87b237cf9312d server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.012+0000 7f66c6ffd640 1 -- 172.21.15.16:0/3987219355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8031870 con 0x7f66c81584d0 2023-11-13T11:04:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.012+0000 7f66c6ffd640 1 -- 172.21.15.16:0/3987219355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b80319d0 con 0x7f66c81584d0 2023-11-13T11:04:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.012+0000 7f66c6ffd640 1 -- 172.21.15.16:0/3987219355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8031d60 con 0x7f66c81584d0 2023-11-13T11:04:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.013+0000 7f66cd3d0640 1 -- 172.21.15.16:0/3987219355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 msgr2=0x7f66c80ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.013+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/3987219355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 0x7f66c80ad730 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f66c806de90 tx=0x7f66b802b910 comp rx=0 tx=0).stop 2023-11-13T11:04:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.013+0000 7f66cd3d0640 1 -- 172.21.15.16:0/3987219355 shutdown_connections 2023-11-13T11:04:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.013+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/3987219355 >> v2:172.21.15.184:3300/0 conn(0x7f66c80712d0 0x7f66c8071760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.013+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/3987219355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 0x7f66c80adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.013+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/3987219355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 0x7f66c80ad730 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.013+0000 7f66cd3d0640 1 -- 172.21.15.16:0/3987219355 >> 172.21.15.16:0/3987219355 conn(0x7f66c806a190 msgr2=0x7f66c806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:04.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.013+0000 7f66cd3d0640 1 -- 172.21.15.16:0/3987219355 shutdown_connections 2023-11-13T11:04:04.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.013+0000 7f66cd3d0640 1 -- 172.21.15.16:0/3987219355 wait complete. 2023-11-13T11:04:04.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.015+0000 7f66cd3d0640 1 Processor -- start 2023-11-13T11:04:04.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.015+0000 7f66cd3d0640 1 -- start start 2023-11-13T11:04:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.016+0000 7f66cd3d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66c80712d0 0x7f66c8151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.016+0000 7f66cd3d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 0x7f66c8153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.016+0000 7f66cd3d0640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 0x7f66c80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.016+0000 7f66cd3d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8154a30 con 0x7f66c81584d0 2023-11-13T11:04:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.016+0000 7f66cd3d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c806cbc0 con 0x7f66c80712d0 2023-11-13T11:04:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.016+0000 7f66cd3d0640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8154750 con 0x7f66c81588b0 2023-11-13T11:04:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.016+0000 7f66c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 0x7f66c8153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.016+0000 7f66c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 0x7f66c8153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51022/0 (socket says 172.21.15.16:51022) 2023-11-13T11:04:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.016+0000 7f66c77fe640 1 -- 172.21.15.16:0/2330855190 learned_addr learned my addr 172.21.15.16:0/2330855190 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66c7fff640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66c80712d0 0x7f66c8151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66ccbcf640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 0x7f66c80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66c77fe640 1 -- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 msgr2=0x7f66c80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66c77fe640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 0x7f66c80ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66c77fe640 1 -- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66c80712d0 msgr2=0x7f66c8151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66c77fe640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66c80712d0 0x7f66c8151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66c77fe640 1 -- 172.21.15.16:0/2330855190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80094d0 con 0x7f66c81584d0 2023-11-13T11:04:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66ccbcf640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 0x7f66c80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:04.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66c77fe640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 0x7f66c8153d70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00d5f0 tx=0x7f66bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:04.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.017+0000 7f66c4ff9640 1 -- 172.21.15.16:0/2330855190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0125f0 con 0x7f66c81584d0 2023-11-13T11:04:04.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.018+0000 7f66c4ff9640 1 -- 172.21.15.16:0/2330855190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc012750 con 0x7f66c81584d0 2023-11-13T11:04:04.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.018+0000 7f66c4ff9640 1 -- 172.21.15.16:0/2330855190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc012b00 con 0x7f66c81584d0 2023-11-13T11:04:04.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.018+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c806ce60 con 0x7f66c81584d0 2023-11-13T11:04:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.018+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c81526a0 con 0x7f66c81584d0 2023-11-13T11:04:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.019+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c8152830 con 0x7f66c81584d0 2023-11-13T11:04:04.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.020+0000 7f66c4ff9640 1 -- 172.21.15.16:0/2330855190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66bc012c60 con 0x7f66c81584d0 2023-11-13T11:04:04.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.021+0000 7f66c4ff9640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66b406ae40 0x7f66b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:04.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.021+0000 7f66c4ff9640 1 -- 172.21.15.16:0/2330855190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f66bc089cb0 con 0x7f66c81584d0 2023-11-13T11:04:04.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.021+0000 7f66c7fff640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66b406ae40 0x7f66b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.023+0000 7f66c7fff640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66b406ae40 0x7f66b406d300 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f66b8036000 tx=0x7f66b803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.025+0000 7f66c4ff9640 1 -- 172.21.15.16:0/2330855190 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66bc054b60 con 0x7f66c81584d0 2023-11-13T11:04:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:04.214+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66c804e4d0 con 0x7f66c81584d0 2023-11-13T11:04:05.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.119+0000 7f66c4ff9640 1 -- 172.21.15.16:0/2330855190 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) v1 ==== 181+0+0 (secure 0 0 0) 0x7f66bc058180 con 0x7f66c81584d0 2023-11-13T11:04:05.124 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-13T11:04:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.122+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66b406ae40 msgr2=0x7f66b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.122+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66b406ae40 0x7f66b406d300 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f66b8036000 tx=0x7f66b803c040 comp rx=0 tx=0).stop 2023-11-13T11:04:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.122+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 msgr2=0x7f66c8153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.122+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 0x7f66c8153d70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00d5f0 tx=0x7f66bc00dab0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.123+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 shutdown_connections 2023-11-13T11:04:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.123+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f66b406ae40 0x7f66b406d300 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.123+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66c81588b0 0x7f66c80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.123+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66c81584d0 0x7f66c8153d70 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.123+0000 7f66cd3d0640 1 --2- 172.21.15.16:0/2330855190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66c80712d0 0x7f66c8151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.123+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 >> 172.21.15.16:0/2330855190 conn(0x7f66c806a190 msgr2=0x7f66c8158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.123+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 shutdown_connections 2023-11-13T11:04:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.123+0000 7f66cd3d0640 1 -- 172.21.15.16:0/2330855190 wait complete. 2023-11-13T11:04:05.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-13T11:04:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.384+0000 7f32a7154640 1 Processor -- start 2023-11-13T11:04:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.385+0000 7f32a7154640 1 -- start start 2023-11-13T11:04:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.386+0000 7f32a7154640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f32a00cf580 0x7f32a00cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.387+0000 7f32a7154640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00cfea0 con 0x7f32a00ce4c0 2023-11-13T11:04:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.387+0000 7f32a7154640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00cf1f0 con 0x7f32a00b66b0 2023-11-13T11:04:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.387+0000 7f32a7154640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a00b59c0 con 0x7f32a00cf580 2023-11-13T11:04:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.387+0000 7f32a5951640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32a00ce4c0 0x7f32a00ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44202/0 (socket says 172.21.15.16:44202) 2023-11-13T11:04:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.387+0000 7f32a5951640 1 -- 172.21.15.16:0/1752175477 learned_addr learned my addr 172.21.15.16:0/1752175477 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.387+0000 7f32a6152640 1 --2- 172.21.15.16:0/1752175477 >> v2:172.21.15.184:3300/0 conn(0x7f32a00cf580 0x7f32a00cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.387+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734159958 0 0) 0x7f32a00cfea0 con 0x7f32a00ce4c0 2023-11-13T11:04:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.387+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f328c003520 con 0x7f32a00ce4c0 2023-11-13T11:04:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434623016 0 0) 0x7f328c003520 con 0x7f32a00ce4c0 2023-11-13T11:04:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 >> v2:172.21.15.184:3300/0 conn(0x7f32a00cf580 msgr2=0x7f32a00cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 --2- 172.21.15.16:0/1752175477 >> v2:172.21.15.184:3300/0 conn(0x7f32a00cf580 0x7f32a00cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 >> v1:172.21.15.184:6789/0 conn(0x7f32a00b66b0 legacy=0x7f32a00b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00b5bd0 con 0x7f32a00ce4c0 2023-11-13T11:04:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f3290001670 con 0x7f32a00ce4c0 2023-11-13T11:04:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 >> v1:172.21.15.16:6789/0 conn(0x7f32a00ce4c0 legacy=0x7f32a00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 --2- 172.21.15.16:0/1752175477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328c0039c0 0x7f328c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 --2- 172.21.15.16:0/1752175477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f328c0047c0 0x7f328c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 --2- 172.21.15.16:0/1752175477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328c005180 0x7f328c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00b59c0 con 0x7f328c0039c0 2023-11-13T11:04:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c00c080 con 0x7f328c005180 2023-11-13T11:04:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c00c240 con 0x7f328c0047c0 2023-11-13T11:04:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.388+0000 7f32a6152640 1 --2- 172.21.15.16:0/1752175477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328c0039c0 0x7f328c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.389+0000 7f32a6152640 1 -- 172.21.15.16:0/1752175477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f328c0047c0 msgr2=0x7f328c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.389+0000 7f32a6152640 1 --2- 172.21.15.16:0/1752175477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f328c0047c0 0x7f328c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.389+0000 7f32a6152640 1 -- 172.21.15.16:0/1752175477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328c005180 msgr2=0x7f328c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.389+0000 7f32a6152640 1 --2- 172.21.15.16:0/1752175477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328c005180 0x7f328c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.389+0000 7f32a6152640 1 -- 172.21.15.16:0/1752175477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c003520 con 0x7f328c0039c0 2023-11-13T11:04:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.389+0000 7f32a6953640 1 -- 172.21.15.16:0/1752175477 reap_dead start 2023-11-13T11:04:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.390+0000 7f32a6152640 1 --2- 172.21.15.16:0/1752175477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328c0039c0 0x7f328c003de0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f329c009ac0 tx=0x7f328c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9571d47c72a58653 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.390+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c02c710 con 0x7f328c0039c0 2023-11-13T11:04:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.390+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c02c870 con 0x7f328c0039c0 2023-11-13T11:04:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.390+0000 7f32a5150640 1 -- 172.21.15.16:0/1752175477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c02cc00 con 0x7f328c0039c0 2023-11-13T11:04:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.391+0000 7f32a7154640 1 -- 172.21.15.16:0/1752175477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328c0039c0 msgr2=0x7f328c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.391+0000 7f32a7154640 1 --2- 172.21.15.16:0/1752175477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328c0039c0 0x7f328c003de0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f329c009ac0 tx=0x7f328c0057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.391+0000 7f32a7154640 1 -- 172.21.15.16:0/1752175477 shutdown_connections 2023-11-13T11:04:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.391+0000 7f32a7154640 1 --2- 172.21.15.16:0/1752175477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328c0039c0 0x7f328c003de0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.391+0000 7f32a7154640 1 -- 172.21.15.16:0/1752175477 >> 172.21.15.16:0/1752175477 conn(0x7f32a00d2b70 msgr2=0x7f32a00d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.391+0000 7f32a7154640 1 -- 172.21.15.16:0/1752175477 shutdown_connections 2023-11-13T11:04:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.391+0000 7f32a7154640 1 -- 172.21.15.16:0/1752175477 wait complete. 2023-11-13T11:04:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.393+0000 7f32a7154640 1 Processor -- start 2023-11-13T11:04:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.393+0000 7f32a7154640 1 -- start start 2023-11-13T11:04:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.394+0000 7f32a7154640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328c0039c0 0x7f32a009fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.394+0000 7f32a7154640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a009d650 0x7f32a014bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.394+0000 7f32a7154640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32a00cbe10 0x7f32a00cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.394+0000 7f32a7154640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00cc800 con 0x7f32a009d650 2023-11-13T11:04:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.394+0000 7f32a7154640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00a0070 con 0x7f328c0039c0 2023-11-13T11:04:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.394+0000 7f32a7154640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00a02a0 con 0x7f32a00cbe10 2023-11-13T11:04:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.394+0000 7f32a5951640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a009d650 0x7f32a014bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.394+0000 7f32a5951640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a009d650 0x7f32a014bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51032/0 (socket says 172.21.15.16:51032) 2023-11-13T11:04:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.394+0000 7f32a5951640 1 -- 172.21.15.16:0/2710868513 learned_addr learned my addr 172.21.15.16:0/2710868513 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a6152640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328c0039c0 0x7f32a009fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a6953640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32a00cbe10 0x7f32a00cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a5951640 1 -- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32a00cbe10 msgr2=0x7f32a00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a5951640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32a00cbe10 0x7f32a00cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a5951640 1 -- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328c0039c0 msgr2=0x7f32a009fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a5951640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328c0039c0 0x7f32a009fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a5951640 1 -- 172.21.15.16:0/2710868513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00a0580 con 0x7f32a009d650 2023-11-13T11:04:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a6953640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32a00cbe10 0x7f32a00cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a5951640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a009d650 0x7f32a014bf10 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f329000c700 tx=0x7f329000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f3296ffd640 1 -- 172.21.15.16:0/2710868513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32900072d0 con 0x7f32a009d650 2023-11-13T11:04:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f3296ffd640 1 -- 172.21.15.16:0/2710868513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3290007430 con 0x7f32a009d650 2023-11-13T11:04:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f3296ffd640 1 -- 172.21.15.16:0/2710868513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32900047e0 con 0x7f32a009d650 2023-11-13T11:04:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.395+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a00a0860 con 0x7f32a009d650 2023-11-13T11:04:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.396+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a00c7d40 con 0x7f32a009d650 2023-11-13T11:04:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.396+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a00c7f60 con 0x7f32a009d650 2023-11-13T11:04:05.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.398+0000 7f3296ffd640 1 -- 172.21.15.16:0/2710868513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3290004940 con 0x7f32a009d650 2023-11-13T11:04:05.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.398+0000 7f3296ffd640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f328406ae70 0x7f328406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:05.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.399+0000 7f3296ffd640 1 -- 172.21.15.16:0/2710868513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f3290088a90 con 0x7f32a009d650 2023-11-13T11:04:05.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.399+0000 7f32a6152640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f328406ae70 0x7f328406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:05.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.400+0000 7f32a6152640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f328406ae70 0x7f328406d330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f329c036000 tx=0x7f329c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:05.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.403+0000 7f3296ffd640 1 -- 172.21.15.16:0/2710868513 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3290053a70 con 0x7f32a009d650 2023-11-13T11:04:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:05.596+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32a004e4d0 con 0x7f32a009d650 2023-11-13T11:04:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.122+0000 7f3296ffd640 1 -- 172.21.15.16:0/2710868513 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3290018030 con 0x7f32a009d650 2023-11-13T11:04:06.128 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-13T11:04:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.126+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f328406ae70 msgr2=0x7f328406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.126+0000 7f32a7154640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f328406ae70 0x7f328406d330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f329c036000 tx=0x7f329c030000 comp rx=0 tx=0).stop 2023-11-13T11:04:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.126+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a009d650 msgr2=0x7f32a014bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.126+0000 7f32a7154640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a009d650 0x7f32a014bf10 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f329000c700 tx=0x7f329000cbc0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.127+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 shutdown_connections 2023-11-13T11:04:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.127+0000 7f32a7154640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f328406ae70 0x7f328406d330 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.127+0000 7f32a7154640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32a00cbe10 0x7f32a00cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.127+0000 7f32a7154640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a009d650 0x7f32a014bf10 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.127+0000 7f32a7154640 1 --2- 172.21.15.16:0/2710868513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f328c0039c0 0x7f32a009fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.127+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 >> 172.21.15.16:0/2710868513 conn(0x7f32a00d2b70 msgr2=0x7f32a015c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.128+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 shutdown_connections 2023-11-13T11:04:06.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.128+0000 7f32a7154640 1 -- 172.21.15.16:0/2710868513 wait complete. 2023-11-13T11:04:06.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_split_bits 2023-11-13T11:04:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.356+0000 7f2b4d141640 1 Processor -- start 2023-11-13T11:04:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.356+0000 7f2b4d141640 1 -- start start 2023-11-13T11:04:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.357+0000 7f2b4d141640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b4815cf90 0x7f2b480b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.357+0000 7f2b4d141640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b4806c3b0 0x7f2b480b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b4d141640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2b480b39c0 0x7f2b480ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b4d141640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b480b3e50 con 0x7f2b4815cf90 2023-11-13T11:04:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b4d141640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b480b2e00 con 0x7f2b480b39c0 2023-11-13T11:04:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b4d141640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b480b21f0 con 0x7f2b4806c3b0 2023-11-13T11:04:06.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b46d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b4815cf90 0x7f2b480b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b46d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b4815cf90 0x7f2b480b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51042/0 (socket says 172.21.15.16:51042) 2023-11-13T11:04:06.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b46d9d640 1 -- 172.21.15.16:0/1192642365 learned_addr learned my addr 172.21.15.16:0/1192642365 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b3e59c640 1 --2- 172.21.15.16:0/1192642365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b4806c3b0 0x7f2b480b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b4759e640 1 --2- 172.21.15.16:0/1192642365 >> v2:172.21.15.184:3300/0 conn(0x7f2b480b39c0 0x7f2b480ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b46d9d640 1 -- 172.21.15.16:0/1192642365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b4806c3b0 msgr2=0x7f2b480b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b46d9d640 1 --2- 172.21.15.16:0/1192642365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b4806c3b0 0x7f2b480b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b46d9d640 1 -- 172.21.15.16:0/1192642365 >> v2:172.21.15.184:3300/0 conn(0x7f2b480b39c0 msgr2=0x7f2b480ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b46d9d640 1 --2- 172.21.15.16:0/1192642365 >> v2:172.21.15.184:3300/0 conn(0x7f2b480b39c0 0x7f2b480ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.358+0000 7f2b46d9d640 1 -- 172.21.15.16:0/1192642365 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480b24d0 con 0x7f2b4815cf90 2023-11-13T11:04:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.359+0000 7f2b4759e640 1 --2- 172.21.15.16:0/1192642365 >> v2:172.21.15.184:3300/0 conn(0x7f2b480b39c0 0x7f2b480ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.360+0000 7f2b46d9d640 1 --2- 172.21.15.16:0/1192642365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 0x7f2b480b1c10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2b4806dbb0 tx=0x7f2b3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b632eb98ba87a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.360+0000 7f2b4659c640 1 -- 172.21.15.16:0/1192642365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38031870 con 0x7f2b4815cf90 2023-11-13T11:04:06.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.360+0000 7f2b4659c640 1 -- 172.21.15.16:0/1192642365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b380319d0 con 0x7f2b4815cf90 2023-11-13T11:04:06.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.360+0000 7f2b4659c640 1 -- 172.21.15.16:0/1192642365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38031d60 con 0x7f2b4815cf90 2023-11-13T11:04:06.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.361+0000 7f2b4d141640 1 -- 172.21.15.16:0/1192642365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 msgr2=0x7f2b480b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.361+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1192642365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 0x7f2b480b1c10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2b4806dbb0 tx=0x7f2b3802b910 comp rx=0 tx=0).stop 2023-11-13T11:04:06.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.361+0000 7f2b4d141640 1 -- 172.21.15.16:0/1192642365 shutdown_connections 2023-11-13T11:04:06.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.361+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1192642365 >> v2:172.21.15.184:3300/0 conn(0x7f2b480b39c0 0x7f2b480ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.361+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1192642365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b4806c3b0 0x7f2b480b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.361+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1192642365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 0x7f2b480b1c10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.361+0000 7f2b4d141640 1 -- 172.21.15.16:0/1192642365 >> 172.21.15.16:0/1192642365 conn(0x7f2b4806a190 msgr2=0x7f2b4806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:06.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.361+0000 7f2b4d141640 1 -- 172.21.15.16:0/1192642365 shutdown_connections 2023-11-13T11:04:06.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.361+0000 7f2b4d141640 1 -- 172.21.15.16:0/1192642365 wait complete. 2023-11-13T11:04:06.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.363+0000 7f2b4d141640 1 Processor -- start 2023-11-13T11:04:06.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.363+0000 7f2b4d141640 1 -- start start 2023-11-13T11:04:06.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b4d141640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b4806c3b0 0x7f2b48156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b4d141640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b480b39c0 0x7f2b48156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b4d141640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 0x7f2b4815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b4d141640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4815dca0 con 0x7f2b4815cf90 2023-11-13T11:04:06.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b4d141640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48156960 con 0x7f2b4806c3b0 2023-11-13T11:04:06.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b4d141640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481e64c0 con 0x7f2b480b39c0 2023-11-13T11:04:06.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b4759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 0x7f2b4815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b4759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 0x7f2b4815f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51054/0 (socket says 172.21.15.16:51054) 2023-11-13T11:04:06.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b4759e640 1 -- 172.21.15.16:0/1961121160 learned_addr learned my addr 172.21.15.16:0/1961121160 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:06.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b46d9d640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b4806c3b0 0x7f2b48156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.364+0000 7f2b3e59c640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b480b39c0 0x7f2b48156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b4759e640 1 -- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b480b39c0 msgr2=0x7f2b48156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b4759e640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b480b39c0 0x7f2b48156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b4759e640 1 -- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b4806c3b0 msgr2=0x7f2b48156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b4759e640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b4806c3b0 0x7f2b48156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b4759e640 1 -- 172.21.15.16:0/1961121160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b380094d0 con 0x7f2b4815cf90 2023-11-13T11:04:06.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b46d9d640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b4806c3b0 0x7f2b48156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:06.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b3e59c640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b480b39c0 0x7f2b48156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:06.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b4759e640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 0x7f2b4815f9f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000cef0 tx=0x7f2b4000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:06.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b3ffff640 1 -- 172.21.15.16:0/1961121160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000d840 con 0x7f2b4815cf90 2023-11-13T11:04:06.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b3ffff640 1 -- 172.21.15.16:0/1961121160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4000d9c0 con 0x7f2b4815cf90 2023-11-13T11:04:06.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b3ffff640 1 -- 172.21.15.16:0/1961121160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000dd70 con 0x7f2b4815cf90 2023-11-13T11:04:06.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4815d370 con 0x7f2b4815cf90 2023-11-13T11:04:06.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.365+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4815d930 con 0x7f2b4815cf90 2023-11-13T11:04:06.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.367+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4815de50 con 0x7f2b4815cf90 2023-11-13T11:04:06.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.368+0000 7f2b3ffff640 1 -- 172.21.15.16:0/1961121160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b4001e3b0 con 0x7f2b4815cf90 2023-11-13T11:04:06.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.369+0000 7f2b3ffff640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b2806ae40 0x7f2b2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.369+0000 7f2b3ffff640 1 -- 172.21.15.16:0/1961121160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f2b40089f90 con 0x7f2b4815cf90 2023-11-13T11:04:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.369+0000 7f2b46d9d640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b2806ae40 0x7f2b2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.370+0000 7f2b46d9d640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b2806ae40 0x7f2b2806d300 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2b38036000 tx=0x7f2b3803c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:06.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.373+0000 7f2b3ffff640 1 -- 172.21.15.16:0/1961121160 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b4815de50 con 0x7f2b4815cf90 2023-11-13T11:04:06.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.567+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_split_bits"} v 0) v1 -- 0x7f2b4806dd20 con 0x7f2b4815cf90 2023-11-13T11:04:06.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.572+0000 7f2b3ffff640 1 -- 172.21.15.16:0/1961121160 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_split_bits"}]=0 v20) v1 ==== 101+0+0 (secure 0 0 0) 0x7f2b4001a0a0 con 0x7f2b4815cf90 2023-11-13T11:04:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b2806ae40 msgr2=0x7f2b2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b2806ae40 0x7f2b2806d300 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2b38036000 tx=0x7f2b3803c040 comp rx=0 tx=0).stop 2023-11-13T11:04:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 msgr2=0x7f2b4815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 0x7f2b4815f9f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000cef0 tx=0x7f2b4000a710 comp rx=0 tx=0).stop 2023-11-13T11:04:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 shutdown_connections 2023-11-13T11:04:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f2b2806ae40 0x7f2b2806d300 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b4815cf90 0x7f2b4815f9f0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b480b39c0 0x7f2b48156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 --2- 172.21.15.16:0/1961121160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b4806c3b0 0x7f2b48156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.575+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 >> 172.21.15.16:0/1961121160 conn(0x7f2b4806a190 msgr2=0x7f2b48071660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.576+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 shutdown_connections 2023-11-13T11:04:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.576+0000 7f2b4d141640 1 -- 172.21.15.16:0/1961121160 wait complete. 2023-11-13T11:04:06.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_merge_size 2023-11-13T11:04:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.824+0000 7f4c4f59e640 1 Processor -- start 2023-11-13T11:04:06.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.824+0000 7f4c4f59e640 1 -- start start 2023-11-13T11:04:06.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.826+0000 7f4c4f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5015bcc0 0x7f4c500cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.826+0000 7f4c4f59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4c500ce090 0x7f4c500cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.826+0000 7f4c4f59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5006d010 con 0x7f4c5015b8e0 2023-11-13T11:04:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.826+0000 7f4c4f59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c500ce540 con 0x7f4c5015bcc0 2023-11-13T11:04:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.826+0000 7f4c4f59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5006b860 con 0x7f4c500ce090 2023-11-13T11:04:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.826+0000 7f4c4dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5015bcc0 0x7f4c500cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.826+0000 7f4c4dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c5015bcc0 0x7f4c500cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51056/0 (socket says 172.21.15.16:51056) 2023-11-13T11:04:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.826+0000 7f4c4dd9b640 1 -- 172.21.15.16:0/3094757348 learned_addr learned my addr 172.21.15.16:0/3094757348 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.827+0000 7f4c4ed9d640 1 --2- 172.21.15.16:0/3094757348 >> v2:172.21.15.184:3300/0 conn(0x7f4c500ce090 0x7f4c500cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.827+0000 7f4c4dd9b640 1 -- 172.21.15.16:0/3094757348 >> v2:172.21.15.184:3300/0 conn(0x7f4c500ce090 msgr2=0x7f4c500cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:06.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.827+0000 7f4c4dd9b640 1 --2- 172.21.15.16:0/3094757348 >> v2:172.21.15.184:3300/0 conn(0x7f4c500ce090 0x7f4c500cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.827+0000 7f4c4dd9b640 1 -- 172.21.15.16:0/3094757348 >> v1:172.21.15.184:6789/0 conn(0x7f4c5015b8e0 legacy=0x7f4c5006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:06.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.827+0000 7f4c4dd9b640 1 -- 172.21.15.16:0/3094757348 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5006bb40 con 0x7f4c5015bcc0 2023-11-13T11:04:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.827+0000 7f4c4ed9d640 1 --2- 172.21.15.16:0/3094757348 >> v2:172.21.15.184:3300/0 conn(0x7f4c500ce090 0x7f4c500cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:04:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.828+0000 7f4c4dd9b640 1 --2- 172.21.15.16:0/3094757348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5015bcc0 0x7f4c500cdb50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4c500cc830 tx=0x7f4c4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e252bd9742f2ccef server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.829+0000 7f4c4d59a640 1 -- 172.21.15.16:0/3094757348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44031870 con 0x7f4c5015bcc0 2023-11-13T11:04:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.829+0000 7f4c4d59a640 1 -- 172.21.15.16:0/3094757348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c440319d0 con 0x7f4c5015bcc0 2023-11-13T11:04:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.829+0000 7f4c4d59a640 1 -- 172.21.15.16:0/3094757348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44031d60 con 0x7f4c5015bcc0 2023-11-13T11:04:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.829+0000 7f4c4f59e640 1 -- 172.21.15.16:0/3094757348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5015bcc0 msgr2=0x7f4c500cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.829+0000 7f4c4f59e640 1 --2- 172.21.15.16:0/3094757348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5015bcc0 0x7f4c500cdb50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4c500cc830 tx=0x7f4c4402b910 comp rx=0 tx=0).stop 2023-11-13T11:04:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.829+0000 7f4c4f59e640 1 -- 172.21.15.16:0/3094757348 shutdown_connections 2023-11-13T11:04:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.829+0000 7f4c4f59e640 1 --2- 172.21.15.16:0/3094757348 >> v2:172.21.15.184:3300/0 conn(0x7f4c500ce090 0x7f4c500cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.829+0000 7f4c4f59e640 1 --2- 172.21.15.16:0/3094757348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5015bcc0 0x7f4c500cdb50 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.829+0000 7f4c4f59e640 1 -- 172.21.15.16:0/3094757348 >> 172.21.15.16:0/3094757348 conn(0x7f4c500b0970 msgr2=0x7f4c500b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.830+0000 7f4c4f59e640 1 -- 172.21.15.16:0/3094757348 shutdown_connections 2023-11-13T11:04:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.830+0000 7f4c4f59e640 1 -- 172.21.15.16:0/3094757348 wait complete. 2023-11-13T11:04:06.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.831+0000 7f4c4f59e640 1 Processor -- start 2023-11-13T11:04:06.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.832+0000 7f4c4f59e640 1 -- start start 2023-11-13T11:04:06.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4f59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c500ce090 0x7f4c50070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4f59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c5015b8e0 0x7f4c5013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5006f940 0x7f4c5015d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4f59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5006fe90 con 0x7f4c5006f940 2023-11-13T11:04:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4f59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50070170 con 0x7f4c500ce090 2023-11-13T11:04:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4f59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50070450 con 0x7f4c5015b8e0 2023-11-13T11:04:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5006f940 0x7f4c5015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5006f940 0x7f4c5015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51072/0 (socket says 172.21.15.16:51072) 2023-11-13T11:04:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4ed9d640 1 -- 172.21.15.16:0/1635844196 learned_addr learned my addr 172.21.15.16:0/1635844196 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4e59c640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c500ce090 0x7f4c50070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4dd9b640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c5015b8e0 0x7f4c5013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.833+0000 7f4c4ed9d640 1 -- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c5015b8e0 msgr2=0x7f4c5013f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c4ed9d640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c5015b8e0 0x7f4c5013f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c4ed9d640 1 -- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c500ce090 msgr2=0x7f4c50070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c4ed9d640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c500ce090 0x7f4c50070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c4ed9d640 1 -- 172.21.15.16:0/1635844196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440094d0 con 0x7f4c5006f940 2023-11-13T11:04:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c4dd9b640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c5015b8e0 0x7f4c5013f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:06.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c4ed9d640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5006f940 0x7f4c5015d690 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f4c48007ef0 tx=0x7f4c4800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:06.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c3effd640 1 -- 172.21.15.16:0/1635844196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48003b90 con 0x7f4c5006f940 2023-11-13T11:04:06.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c3effd640 1 -- 172.21.15.16:0/1635844196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4800f040 con 0x7f4c5006f940 2023-11-13T11:04:06.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c3effd640 1 -- 172.21.15.16:0/1635844196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800de80 con 0x7f4c5006f940 2023-11-13T11:04:06.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c50070730 con 0x7f4c5006f940 2023-11-13T11:04:06.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.834+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5015dbd0 con 0x7f4c5006f940 2023-11-13T11:04:06.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.836+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c50071470 con 0x7f4c5006f940 2023-11-13T11:04:06.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.837+0000 7f4c3effd640 1 -- 172.21.15.16:0/1635844196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c48004030 con 0x7f4c5006f940 2023-11-13T11:04:06.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.837+0000 7f4c3effd640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4c3006ae40 0x7f4c3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:06.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.838+0000 7f4c3effd640 1 -- 172.21.15.16:0/1635844196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f4c480896e0 con 0x7f4c5006f940 2023-11-13T11:04:06.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.838+0000 7f4c4e59c640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4c3006ae40 0x7f4c3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.839+0000 7f4c4e59c640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4c3006ae40 0x7f4c3006d300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4c40004ea0 tx=0x7f4c40009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:06.842+0000 7f4c3effd640 1 -- 172.21.15.16:0/1635844196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c480549a0 con 0x7f4c5006f940 2023-11-13T11:04:07.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.037+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_merge_size"} v 0) v1 -- 0x7f4c50070c50 con 0x7f4c5006f940 2023-11-13T11:04:07.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.042+0000 7f4c3effd640 1 -- 172.21.15.16:0/1635844196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_merge_size"}]=0 v21) v1 ==== 101+0+0 (secure 0 0 0) 0x7f4c50070c50 con 0x7f4c5006f940 2023-11-13T11:04:07.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4c3006ae40 msgr2=0x7f4c3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4c3006ae40 0x7f4c3006d300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4c40004ea0 tx=0x7f4c40009040 comp rx=0 tx=0).stop 2023-11-13T11:04:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5006f940 msgr2=0x7f4c5015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5006f940 0x7f4c5015d690 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f4c48007ef0 tx=0x7f4c4800d4a0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 shutdown_connections 2023-11-13T11:04:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4c3006ae40 0x7f4c3006d300 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c5006f940 0x7f4c5015d690 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c5015b8e0 0x7f4c5013f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 --2- 172.21.15.16:0/1635844196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c500ce090 0x7f4c50070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 >> 172.21.15.16:0/1635844196 conn(0x7f4c500b0970 msgr2=0x7f4c500b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:07.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.045+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 shutdown_connections 2023-11-13T11:04:07.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.046+0000 7f4c4f59e640 1 -- 172.21.15.16:0/1635844196 wait complete. 2023-11-13T11:04:07.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-13T11:04:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.303+0000 7fe0b9490640 1 Processor -- start 2023-11-13T11:04:07.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.304+0000 7fe0b9490640 1 -- start start 2023-11-13T11:04:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.305+0000 7fe0b9490640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b40ce1d0 0x7fe0b40ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.305+0000 7fe0b9490640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b415d8d0 con 0x7fe0b415e390 2023-11-13T11:04:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.305+0000 7fe0b9490640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b415af00 con 0x7fe0b40ce5b0 2023-11-13T11:04:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.305+0000 7fe0b9490640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b415db90 con 0x7fe0b40ce1d0 2023-11-13T11:04:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.306+0000 7fe0b37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe0b415e390 0x7fe0b40b2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44212/0 (socket says 172.21.15.16:44212) 2023-11-13T11:04:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.306+0000 7fe0b37fe640 1 -- 172.21.15.16:0/1375100993 learned_addr learned my addr 172.21.15.16:0/1375100993 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.306+0000 7fe0b2ffd640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b40ce1d0 0x7fe0b40ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.306+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117981015 0 0) 0x7fe0b415d8d0 con 0x7fe0b415e390 2023-11-13T11:04:07.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.306+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a0005730 con 0x7fe0b415e390 2023-11-13T11:04:07.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1125385670 0 0) 0x7fe0a0005730 con 0x7fe0b415e390 2023-11-13T11:04:07.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b40ce1d0 msgr2=0x7fe0b40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:07.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b40ce1d0 0x7fe0b40ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 >> v1:172.21.15.184:6789/0 conn(0x7fe0b40ce5b0 legacy=0x7fe0b415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b415de70 con 0x7fe0b415e390 2023-11-13T11:04:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fe0ac001670 con 0x7fe0b415e390 2023-11-13T11:04:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 >> v1:172.21.15.16:6789/0 conn(0x7fe0b415e390 legacy=0x7fe0b40b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a0005bb0 0x7fe0a0005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b40ce1d0 0x7fe0a00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0a0006f10 0x7fe0a0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b415db90 con 0x7fe0a0005bb0 2023-11-13T11:04:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a000db90 con 0x7fe0a0006f10 2023-11-13T11:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a000dd50 con 0x7fe0b40ce1d0 2023-11-13T11:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.307+0000 7fe0b2ffd640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a0005bb0 0x7fe0a0005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.308+0000 7fe0b2ffd640 1 -- 172.21.15.16:0/1375100993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b40ce1d0 msgr2=0x7fe0a00069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.308+0000 7fe0b2ffd640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0b40ce1d0 0x7fe0a00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.308+0000 7fe0b2ffd640 1 -- 172.21.15.16:0/1375100993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0a0006f10 msgr2=0x7fe0a0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.308+0000 7fe0b2ffd640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0a0006f10 0x7fe0a0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.308+0000 7fe0b2ffd640 1 -- 172.21.15.16:0/1375100993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a0005730 con 0x7fe0a0005bb0 2023-11-13T11:04:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.309+0000 7fe0b2ffd640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a0005bb0 0x7fe0a0005f90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009ac0 tx=0x7fe0a0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e58bac10d391c58 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.309+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a402c630 con 0x7fe0a0005bb0 2023-11-13T11:04:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.309+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a402c790 con 0x7fe0a0005bb0 2023-11-13T11:04:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.309+0000 7fe0b1ffb640 1 -- 172.21.15.16:0/1375100993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a402cb20 con 0x7fe0a0005bb0 2023-11-13T11:04:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.310+0000 7fe0b9490640 1 -- 172.21.15.16:0/1375100993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a0005bb0 msgr2=0x7fe0a0005f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.310+0000 7fe0b9490640 1 --2- 172.21.15.16:0/1375100993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a0005bb0 0x7fe0a0005f90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009ac0 tx=0x7fe0a0007a00 comp rx=0 tx=0).stop 2023-11-13T11:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.310+0000 7fe0b37fe640 1 -- 172.21.15.16:0/1375100993 reap_dead start 2023-11-13T11:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.310+0000 7fe0b9490640 1 -- 172.21.15.16:0/1375100993 shutdown_connections 2023-11-13T11:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.310+0000 7fe0b9490640 1 -- 172.21.15.16:0/1375100993 >> 172.21.15.16:0/1375100993 conn(0x7fe0b406a190 msgr2=0x7fe0b406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.310+0000 7fe0b9490640 1 -- 172.21.15.16:0/1375100993 shutdown_connections 2023-11-13T11:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.310+0000 7fe0b9490640 1 -- 172.21.15.16:0/1375100993 wait complete. 2023-11-13T11:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.312+0000 7fe0b9490640 1 Processor -- start 2023-11-13T11:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.312+0000 7fe0b9490640 1 -- start start 2023-11-13T11:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.313+0000 7fe0b9490640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0a0005bb0 0x7fe0b4156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.313+0000 7fe0b9490640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0b40c3bb0 0x7fe0b406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.313+0000 7fe0b9490640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0b40c4030 0x7fe0b406c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.313+0000 7fe0b9490640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b413ff30 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.313+0000 7fe0b9490640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b406c5b0 con 0x7fe0b40c4030 2023-11-13T11:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.313+0000 7fe0b9490640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b41401a0 con 0x7fe0a0005bb0 2023-11-13T11:04:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.313+0000 7fe0b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0b40c3bb0 0x7fe0b406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0b40c3bb0 0x7fe0b406bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51078/0 (socket says 172.21.15.16:51078) 2023-11-13T11:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b27fc640 1 -- 172.21.15.16:0/2862258973 learned_addr learned my addr 172.21.15.16:0/2862258973 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b37fe640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0b40c4030 0x7fe0b406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b2ffd640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0a0005bb0 0x7fe0b4156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b27fc640 1 -- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0a0005bb0 msgr2=0x7fe0b4156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b27fc640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0a0005bb0 0x7fe0b4156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b27fc640 1 -- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0b40c4030 msgr2=0x7fe0b406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b27fc640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0b40c4030 0x7fe0b406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b27fc640 1 -- 172.21.15.16:0/2862258973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b4140480 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.314+0000 7fe0b27fc640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0b40c3bb0 0x7fe0b406bb30 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00cb20 tx=0x7fe09c0020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.315+0000 7fe0ab7fe640 1 -- 172.21.15.16:0/2862258973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c003da0 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.315+0000 7fe0ab7fe640 1 -- 172.21.15.16:0/2862258973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c006b30 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.315+0000 7fe0ab7fe640 1 -- 172.21.15.16:0/2862258973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c00a6e0 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.315+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b4140760 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.315+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b4140f40 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.316+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b4072520 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.317+0000 7fe0ab7fe640 1 -- 172.21.15.16:0/2862258973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe09c00a840 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.318+0000 7fe0ab7fe640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe09406ae90 0x7fe09406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.318+0000 7fe0ab7fe640 1 -- 172.21.15.16:0/2862258973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fe09c088f00 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.319+0000 7fe0b2ffd640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe09406ae90 0x7fe09406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.320+0000 7fe0b2ffd640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe09406ae90 0x7fe09406d350 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4036000 tx=0x7fe0a4030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.322+0000 7fe0ab7fe640 1 -- 172.21.15.16:0/2862258973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe09c0540f0 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.524+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe0b404e5b0 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.530+0000 7fe0ab7fe640 1 -- 172.21.15.16:0/2862258973 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe09c057710 con 0x7fe0b40c3bb0 2023-11-13T11:04:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.532+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe09406ae90 msgr2=0x7fe09406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.532+0000 7fe0b9490640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe09406ae90 0x7fe09406d350 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4036000 tx=0x7fe0a4030000 comp rx=0 tx=0).stop 2023-11-13T11:04:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0b40c3bb0 msgr2=0x7fe0b406bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0b40c3bb0 0x7fe0b406bb30 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00cb20 tx=0x7fe09c0020b0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 shutdown_connections 2023-11-13T11:04:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe09406ae90 0x7fe09406d350 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe0b40c4030 0x7fe0b406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0b40c3bb0 0x7fe0b406bb30 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 --2- 172.21.15.16:0/2862258973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0a0005bb0 0x7fe0b4156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 >> 172.21.15.16:0/2862258973 conn(0x7fe0b406a190 msgr2=0x7fe0b40cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 shutdown_connections 2023-11-13T11:04:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.533+0000 7fe0b9490640 1 -- 172.21.15.16:0/2862258973 wait complete. 2023-11-13T11:04:07.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_split_size 2023-11-13T11:04:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.775+0000 7feecf59e640 1 Processor -- start 2023-11-13T11:04:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.775+0000 7feecf59e640 1 -- start start 2023-11-13T11:04:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feecf59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feed0071160 0x7feed015bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feecf59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006bbc0 0x7feed006a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feecf59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed0158a80 con 0x7feed01586a0 2023-11-13T11:04:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feecf59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed006c360 con 0x7feed0071160 2023-11-13T11:04:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feecf59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00b2a30 con 0x7feed006bbc0 2023-11-13T11:04:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feece59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feed01586a0 0x7feed0070a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44224/0 (socket says 172.21.15.16:44224) 2023-11-13T11:04:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feece59c640 1 -- 172.21.15.16:0/4145452617 learned_addr learned my addr 172.21.15.16:0/4145452617 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feecdd9b640 1 --2- 172.21.15.16:0/4145452617 >> v2:172.21.15.184:3300/0 conn(0x7feed0071160 0x7feed015bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feeced9d640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006bbc0 0x7feed006a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950724912 0 0) 0x7feec0001490 con 0x7feed01586a0 2023-11-13T11:04:07.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.777+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feebc003520 con 0x7feed01586a0 2023-11-13T11:04:07.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3682914440 0 0) 0x7feebc003520 con 0x7feed01586a0 2023-11-13T11:04:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006bbc0 msgr2=0x7feed006a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006bbc0 0x7feed006a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 >> v2:172.21.15.184:3300/0 conn(0x7feed0071160 msgr2=0x7feed015bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 --2- 172.21.15.16:0/4145452617 >> v2:172.21.15.184:3300/0 conn(0x7feed0071160 0x7feed015bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00b2d10 con 0x7feed01586a0 2023-11-13T11:04:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7feec0001b80 con 0x7feed01586a0 2023-11-13T11:04:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 >> v1:172.21.15.16:6789/0 conn(0x7feed01586a0 legacy=0x7feed0070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc003970 0x7feebc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feebc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006bbc0 0x7feebc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed006c360 con 0x7feebc0047c0 2023-11-13T11:04:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00b2a30 con 0x7feebc003970 2023-11-13T11:04:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc00ba30 con 0x7feed006bbc0 2023-11-13T11:04:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.778+0000 7feecdd9b640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feebc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.779+0000 7feece59c640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc003970 0x7feebc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.779+0000 7feecdd9b640 1 -- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006bbc0 msgr2=0x7feebc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.779+0000 7feecdd9b640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006bbc0 0x7feebc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.779+0000 7feecdd9b640 1 -- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc003970 msgr2=0x7feebc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.779+0000 7feecdd9b640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc003970 0x7feebc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.779+0000 7feecdd9b640 1 -- 172.21.15.16:0/4145452617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc003520 con 0x7feebc0047c0 2023-11-13T11:04:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.780+0000 7feecdd9b640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feebc004c40 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009a90 tx=0x7feebc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f578060ddfceb7f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.780+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb802c6b0 con 0x7feebc0047c0 2023-11-13T11:04:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.780+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb802c810 con 0x7feebc0047c0 2023-11-13T11:04:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.780+0000 7feecd59a640 1 -- 172.21.15.16:0/4145452617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb802cba0 con 0x7feebc0047c0 2023-11-13T11:04:07.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.781+0000 7feecf59e640 1 -- 172.21.15.16:0/4145452617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 msgr2=0x7feebc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.781+0000 7feecf59e640 1 --2- 172.21.15.16:0/4145452617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feebc004c40 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009a90 tx=0x7feebc0057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.781+0000 7feeced9d640 1 -- 172.21.15.16:0/4145452617 reap_dead start 2023-11-13T11:04:07.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.781+0000 7feecf59e640 1 -- 172.21.15.16:0/4145452617 shutdown_connections 2023-11-13T11:04:07.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.781+0000 7feecf59e640 1 -- 172.21.15.16:0/4145452617 >> 172.21.15.16:0/4145452617 conn(0x7feed00b1600 msgr2=0x7feed00b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.781+0000 7feecf59e640 1 -- 172.21.15.16:0/4145452617 shutdown_connections 2023-11-13T11:04:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.781+0000 7feecf59e640 1 -- 172.21.15.16:0/4145452617 wait complete. 2023-11-13T11:04:07.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.783+0000 7feecf59e640 1 Processor -- start 2023-11-13T11:04:07.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.783+0000 7feecf59e640 1 -- start start 2023-11-13T11:04:07.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.784+0000 7feecf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feed0151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.784+0000 7feecf59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed0155d50 0x7feed006d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.784+0000 7feecf59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006de60 0x7feed00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.784+0000 7feecf59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed006e2e0 con 0x7feebc0047c0 2023-11-13T11:04:07.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.784+0000 7feecf59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00aeca0 con 0x7feed0155d50 2023-11-13T11:04:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.784+0000 7feecf59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed0151b50 con 0x7feed006de60 2023-11-13T11:04:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feece59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feed0151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feece59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feed0151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51096/0 (socket says 172.21.15.16:51096) 2023-11-13T11:04:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feece59c640 1 -- 172.21.15.16:0/623000875 learned_addr learned my addr 172.21.15.16:0/623000875 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feece59c640 1 -- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006de60 msgr2=0x7feed00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feeced9d640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006de60 0x7feed00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feece59c640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006de60 0x7feed00ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feece59c640 1 -- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed0155d50 msgr2=0x7feed006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feece59c640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed0155d50 0x7feed006d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feece59c640 1 -- 172.21.15.16:0/623000875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00ae090 con 0x7feebc0047c0 2023-11-13T11:04:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.785+0000 7feece59c640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feed0151610 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7feec0007a80 tx=0x7feec000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.786+0000 7feec6ffd640 1 -- 172.21.15.16:0/623000875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec000eeb0 con 0x7feebc0047c0 2023-11-13T11:04:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.786+0000 7feec6ffd640 1 -- 172.21.15.16:0/623000875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0013420 con 0x7feebc0047c0 2023-11-13T11:04:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.786+0000 7feec6ffd640 1 -- 172.21.15.16:0/623000875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec00137d0 con 0x7feebc0047c0 2023-11-13T11:04:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.786+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed00ae370 con 0x7feebc0047c0 2023-11-13T11:04:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.786+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed00ae930 con 0x7feebc0047c0 2023-11-13T11:04:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.787+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed00aeac0 con 0x7feebc0047c0 2023-11-13T11:04:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.788+0000 7feec6ffd640 1 -- 172.21.15.16:0/623000875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feec0013930 con 0x7feebc0047c0 2023-11-13T11:04:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.789+0000 7feec6ffd640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feeb006ae40 0x7feeb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.789+0000 7feec6ffd640 1 -- 172.21.15.16:0/623000875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7feec0089b40 con 0x7feebc0047c0 2023-11-13T11:04:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.789+0000 7feecdd9b640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feeb006ae40 0x7feeb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.791+0000 7feecdd9b640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feeb006ae40 0x7feeb006d300 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7feeb8036000 tx=0x7feeb802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.793+0000 7feec6ffd640 1 -- 172.21.15.16:0/623000875 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feec0054d80 con 0x7feebc0047c0 2023-11-13T11:04:07.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.987+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_split_size"} v 0) v1 -- 0x7feed0159c00 con 0x7feebc0047c0 2023-11-13T11:04:07.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.993+0000 7feec6ffd640 1 -- 172.21.15.16:0/623000875 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_split_size"}]=0 v23) v1 ==== 101+0+0 (secure 0 0 0) 0x7feed0159c00 con 0x7feebc0047c0 2023-11-13T11:04:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feeb006ae40 msgr2=0x7feeb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feeb006ae40 0x7feeb006d300 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7feeb8036000 tx=0x7feeb802e000 comp rx=0 tx=0).stop 2023-11-13T11:04:08.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 msgr2=0x7feed0151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:08.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feed0151610 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7feec0007a80 tx=0x7feec000e530 comp rx=0 tx=0).stop 2023-11-13T11:04:08.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 shutdown_connections 2023-11-13T11:04:08.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feeb006ae40 0x7feeb006d300 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed006de60 0x7feed00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed0155d50 0x7feed006d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 --2- 172.21.15.16:0/623000875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feebc0047c0 0x7feed0151610 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 >> 172.21.15.16:0/623000875 conn(0x7feed00b1600 msgr2=0x7feed006a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:08.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 shutdown_connections 2023-11-13T11:04:08.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:07.996+0000 7feecf59e640 1 -- 172.21.15.16:0/623000875 wait complete. 2023-11-13T11:04:08.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags' 2023-11-13T11:04:08.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.252+0000 7f4b5c052640 1 Processor -- start 2023-11-13T11:04:08.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.252+0000 7f4b5c052640 1 -- start start 2023-11-13T11:04:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.254+0000 7f4b5c052640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b54155410 0x7f4b540ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.254+0000 7f4b5c052640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5415b500 con 0x7f4b540ce990 2023-11-13T11:04:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.254+0000 7f4b5c052640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5415b7e0 con 0x7f4b54071b50 2023-11-13T11:04:08.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.254+0000 7f4b5c052640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5415bac0 con 0x7f4b54155410 2023-11-13T11:04:08.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.254+0000 7f4b595ed640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4b540ce990 0x7f4b54071440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44226/0 (socket says 172.21.15.16:44226) 2023-11-13T11:04:08.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.254+0000 7f4b595ed640 1 -- 172.21.15.16:0/2327382678 learned_addr learned my addr 172.21.15.16:0/2327382678 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:08.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b59dee640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b54155410 0x7f4b540ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:08.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584505291 0 0) 0x7f4b5415b500 con 0x7f4b540ce990 2023-11-13T11:04:08.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b44003520 con 0x7f4b540ce990 2023-11-13T11:04:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029585244 0 0) 0x7f4b44003520 con 0x7f4b540ce990 2023-11-13T11:04:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b54155410 msgr2=0x7f4b540ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b54155410 0x7f4b540ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 >> v1:172.21.15.184:6789/0 conn(0x7f4b54071b50 legacy=0x7f4b5406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5415bda0 con 0x7f4b540ce990 2023-11-13T11:04:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f4b40001670 con 0x7f4b540ce990 2023-11-13T11:04:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 >> v1:172.21.15.16:6789/0 conn(0x7f4b540ce990 legacy=0x7f4b54071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b54155410 0x7f4b440039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b44003f70 0x7f4b440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b44004d00 0x7f4b44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5415bac0 con 0x7f4b44003f70 2023-11-13T11:04:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4400bac0 con 0x7f4b44004d00 2023-11-13T11:04:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.255+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4400bc80 con 0x7f4b54155410 2023-11-13T11:04:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.256+0000 7f4b595ed640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b44003f70 0x7f4b440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.256+0000 7f4b59dee640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b54155410 0x7f4b440039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.256+0000 7f4b595ed640 1 -- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b54155410 msgr2=0x7f4b440039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.256+0000 7f4b595ed640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b54155410 0x7f4b440039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.256+0000 7f4b595ed640 1 -- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b44004d00 msgr2=0x7f4b44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.256+0000 7f4b595ed640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b44004d00 0x7f4b44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.256+0000 7f4b595ed640 1 -- 172.21.15.16:0/2327382678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b44003520 con 0x7f4b44003f70 2023-11-13T11:04:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.257+0000 7f4b595ed640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b44003f70 0x7f4b440047c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000c0e0 tx=0x7f4b440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a527efef30b7a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40005ad0 con 0x7f4b44003f70 2023-11-13T11:04:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b40005c30 con 0x7f4b44003f70 2023-11-13T11:04:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b58dec640 1 -- 172.21.15.16:0/2327382678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b400072b0 con 0x7f4b44003f70 2023-11-13T11:04:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b5c052640 1 -- 172.21.15.16:0/2327382678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b44003f70 msgr2=0x7f4b440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b5c052640 1 --2- 172.21.15.16:0/2327382678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b44003f70 0x7f4b440047c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000c0e0 tx=0x7f4b440057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b5a5ef640 1 -- 172.21.15.16:0/2327382678 reap_dead start 2023-11-13T11:04:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b5c052640 1 -- 172.21.15.16:0/2327382678 shutdown_connections 2023-11-13T11:04:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b5c052640 1 -- 172.21.15.16:0/2327382678 >> 172.21.15.16:0/2327382678 conn(0x7f4b5406a190 msgr2=0x7f4b5406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b5c052640 1 -- 172.21.15.16:0/2327382678 shutdown_connections 2023-11-13T11:04:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.258+0000 7f4b5c052640 1 -- 172.21.15.16:0/2327382678 wait complete. 2023-11-13T11:04:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.260+0000 7f4b5c052640 1 Processor -- start 2023-11-13T11:04:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.261+0000 7f4b5c052640 1 -- start start 2023-11-13T11:04:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5c052640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b44003f70 0x7f4b54153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5c052640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b54155870 0x7f4b54157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5c052640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b541f2550 0x7f4b541f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5c052640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541f2130 con 0x7f4b541f2550 2023-11-13T11:04:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5c052640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54156640 con 0x7f4b54155870 2023-11-13T11:04:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5c052640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54155da0 con 0x7f4b44003f70 2023-11-13T11:04:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5a5ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b541f2550 0x7f4b541f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5a5ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b541f2550 0x7f4b541f29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51116/0 (socket says 172.21.15.16:51116) 2023-11-13T11:04:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5a5ef640 1 -- 172.21.15.16:0/926260845 learned_addr learned my addr 172.21.15.16:0/926260845 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b59dee640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b44003f70 0x7f4b54153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b595ed640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b54155870 0x7f4b54157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5a5ef640 1 -- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b44003f70 msgr2=0x7f4b54153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5a5ef640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b44003f70 0x7f4b54153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5a5ef640 1 -- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b54155870 msgr2=0x7f4b54157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5a5ef640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b54155870 0x7f4b54157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.262+0000 7f4b5a5ef640 1 -- 172.21.15.16:0/926260845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54156080 con 0x7f4b541f2550 2023-11-13T11:04:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.263+0000 7f4b5a5ef640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b541f2550 0x7f4b541f29d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4b540cf600 tx=0x7f4b4c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.263+0000 7f4b527fc640 1 -- 172.21.15.16:0/926260845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004220 con 0x7f4b541f2550 2023-11-13T11:04:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.263+0000 7f4b527fc640 1 -- 172.21.15.16:0/926260845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c0043a0 con 0x7f4b541f2550 2023-11-13T11:04:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.263+0000 7f4b527fc640 1 -- 172.21.15.16:0/926260845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c007310 con 0x7f4b541f2550 2023-11-13T11:04:08.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.263+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b54156360 con 0x7f4b541f2550 2023-11-13T11:04:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.263+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b54158520 con 0x7f4b541f2550 2023-11-13T11:04:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.264+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b541f2270 con 0x7f4b541f2550 2023-11-13T11:04:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.265+0000 7f4b527fc640 1 -- 172.21.15.16:0/926260845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b4c007470 con 0x7f4b541f2550 2023-11-13T11:04:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.267+0000 7f4b527fc640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b3806add0 0x7f4b3806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.267+0000 7f4b527fc640 1 -- 172.21.15.16:0/926260845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f4b4c0885b0 con 0x7f4b541f2550 2023-11-13T11:04:08.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.268+0000 7f4b59dee640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b3806add0 0x7f4b3806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:08.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.269+0000 7f4b59dee640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b3806add0 0x7f4b3806d290 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4b48007520 tx=0x7f4b4800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:08.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.272+0000 7f4b527fc640 1 -- 172.21.15.16:0/926260845 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b541f2270 con 0x7f4b541f2550 2023-11-13T11:04:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:08.539+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags"]} v 0) v1 -- 0x7f4b541f2270 con 0x7f4b541f2550 2023-11-13T11:04:09.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.156+0000 7f4b527fc640 1 -- 172.21.15.16:0/926260845 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_merge_dirfrags"]}]=0 v398) v1 ==== 151+0+0 (secure 0 0 0) 0x7f4b541f2270 con 0x7f4b541f2550 2023-11-13T11:04:09.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.159+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b3806add0 msgr2=0x7f4b3806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:09.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.159+0000 7f4b5c052640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b3806add0 0x7f4b3806d290 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4b48007520 tx=0x7f4b4800b040 comp rx=0 tx=0).stop 2023-11-13T11:04:09.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.159+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b541f2550 msgr2=0x7f4b541f29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:09.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.159+0000 7f4b5c052640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b541f2550 0x7f4b541f29d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4b540cf600 tx=0x7f4b4c00c820 comp rx=0 tx=0).stop 2023-11-13T11:04:09.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.160+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 shutdown_connections 2023-11-13T11:04:09.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.160+0000 7f4b5c052640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f4b3806add0 0x7f4b3806d290 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:09.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.160+0000 7f4b5c052640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b541f2550 0x7f4b541f29d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.160+0000 7f4b5c052640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b54155870 0x7f4b54157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.160+0000 7f4b5c052640 1 --2- 172.21.15.16:0/926260845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b44003f70 0x7f4b54153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.160+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 >> 172.21.15.16:0/926260845 conn(0x7f4b5406a190 msgr2=0x7f4b54070090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.160+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 shutdown_connections 2023-11-13T11:04:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:09.160+0000 7f4b5c052640 1 -- 172.21.15.16:0/926260845 wait complete. 2023-11-13T11:04:09.188 INFO:tasks.cephfs_test_runner:That a directory is merged during scrub. ... FAIL 2023-11-13T11:04:09.189 INFO:tasks.cephfs_test_runner: 2023-11-13T11:04:09.189 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-13T11:04:09.189 INFO:tasks.cephfs_test_runner:FAIL: test_scrub_merge_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-13T11:04:09.190 INFO:tasks.cephfs_test_runner:That a directory is merged during scrub. 2023-11-13T11:04:09.190 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-13T11:04:09.191 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-13T11:04:09.191 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_35f0debb3cc28955b7132ef4c509cd138a09d56b/qa/tasks/cephfs/test_scrub_checks.py", line 399, in test_scrub_merge_dirfrags 2023-11-13T11:04:09.192 INFO:tasks.cephfs_test_runner: self.assertEqual(len(self.get_dsplits(dir_ino)), 1) 2023-11-13T11:04:09.192 INFO:tasks.cephfs_test_runner:AssertionError: 2 != 1 2023-11-13T11:04:09.193 INFO:tasks.cephfs_test_runner: 2023-11-13T11:04:09.193 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-13T11:04:09.193 INFO:tasks.cephfs_test_runner:Ran 3 tests in 454.998s 2023-11-13T11:04:09.194 INFO:tasks.cephfs_test_runner: 2023-11-13T11:04:09.194 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2023-11-13T11:04:09.194 INFO:tasks.cephfs_test_runner: 2023-11-13T11:04:09.195 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-13T11:04:09.195 INFO:tasks.cephfs_test_runner:FAIL: test_scrub_merge_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-13T11:04:09.195 INFO:tasks.cephfs_test_runner:That a directory is merged during scrub. 2023-11-13T11:04:09.195 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-13T11:04:09.196 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-13T11:04:09.196 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_35f0debb3cc28955b7132ef4c509cd138a09d56b/qa/tasks/cephfs/test_scrub_checks.py", line 399, in test_scrub_merge_dirfrags 2023-11-13T11:04:09.197 INFO:tasks.cephfs_test_runner: self.assertEqual(len(self.get_dsplits(dir_ino)), 1) 2023-11-13T11:04:09.197 INFO:tasks.cephfs_test_runner:AssertionError: 2 != 1 2023-11-13T11:04:09.197 INFO:tasks.cephfs_test_runner: 2023-11-13T11:04:09.198 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_35f0debb3cc28955b7132ef4c509cd138a09d56b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_merge_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-13T11:04:09.262 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a212fc8f00504ae4ad8e7416eadc0050 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_35f0debb3cc28955b7132ef4c509cd138a09d56b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_merge_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-13T11:04:09.264 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-13T11:04:09.274 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-13T11:04:09.281 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-13T11:04:09.282 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:04:09.282 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:04:09.358 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:04:09.359 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-13T11:04:09.359 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:04:09.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-13T11:04:09.417 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:04:09.417 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:04:09.478 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:04:09.479 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-11-13T11:04:09.479 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-13T11:04:09.533 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-13T11:04:09.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:04:09.534 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-11-13T11:04:09.588 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-13T11:04:09.588 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-13T11:04:09.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:04:09.590 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-13T11:04:09.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:04:09.689 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-13T11:04:09.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:04:09.786 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-11-13T11:04:09.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:04:09.908 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-13T11:04:09.975 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-13T11:04:09.976 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-13T11:04:09.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:04:09.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-13T11:04:10.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:04:10.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-13T11:04:10.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-13T11:04:10.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2023-11-13T11:04:10.343 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-13T11:04:10.359 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-13T11:04:10.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:04:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.561+0000 7f845e2f8640 1 Processor -- start 2023-11-13T11:04:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.562+0000 7f845e2f8640 1 -- start start 2023-11-13T11:04:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.563+0000 7f845e2f8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f84580cb190 0x7f845815b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.563+0000 7f845e2f8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84580716b0 0x7f845806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.563+0000 7f845e2f8640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8458158c20 con 0x7f845806a190 2023-11-13T11:04:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.563+0000 7f845e2f8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84580b1830 con 0x7f84580716b0 2023-11-13T11:04:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.563+0000 7f845e2f8640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845815b780 con 0x7f84580cb190 2023-11-13T11:04:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.563+0000 7f845caf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84580716b0 0x7f845806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.563+0000 7f845caf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84580716b0 0x7f845806f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51122/0 (socket says 172.21.15.16:51122) 2023-11-13T11:04:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.563+0000 7f845caf5640 1 -- 172.21.15.16:0/3285392341 learned_addr learned my addr 172.21.15.16:0/3285392341 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.564+0000 7f845d2f6640 1 --2- 172.21.15.16:0/3285392341 >> v2:172.21.15.184:3300/0 conn(0x7f84580cb190 0x7f845815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.564+0000 7f845caf5640 1 -- 172.21.15.16:0/3285392341 >> v2:172.21.15.184:3300/0 conn(0x7f84580cb190 msgr2=0x7f845815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.564+0000 7f845caf5640 1 --2- 172.21.15.16:0/3285392341 >> v2:172.21.15.184:3300/0 conn(0x7f84580cb190 0x7f845815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.564+0000 7f845caf5640 1 -- 172.21.15.16:0/3285392341 >> v1:172.21.15.184:6789/0 conn(0x7f845806a190 legacy=0x7f84580b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:10.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.564+0000 7f845caf5640 1 -- 172.21.15.16:0/3285392341 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845815ba60 con 0x7f84580716b0 2023-11-13T11:04:10.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.564+0000 7f845d2f6640 1 --2- 172.21.15.16:0/3285392341 >> v2:172.21.15.184:3300/0 conn(0x7f84580cb190 0x7f845815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.565+0000 7f845caf5640 1 --2- 172.21.15.16:0/3285392341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 0x7f845806f970 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f84580b1560 tx=0x7f844802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60eb3527e0323f93 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.565+0000 7f844ffff640 1 -- 172.21.15.16:0/3285392341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448031870 con 0x7f84580716b0 2023-11-13T11:04:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.565+0000 7f844ffff640 1 -- 172.21.15.16:0/3285392341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84480319d0 con 0x7f84580716b0 2023-11-13T11:04:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.565+0000 7f844ffff640 1 -- 172.21.15.16:0/3285392341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448031d60 con 0x7f84580716b0 2023-11-13T11:04:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.566+0000 7f845e2f8640 1 -- 172.21.15.16:0/3285392341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 msgr2=0x7f845806f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:10.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.566+0000 7f845e2f8640 1 --2- 172.21.15.16:0/3285392341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 0x7f845806f970 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f84580b1560 tx=0x7f844802b910 comp rx=0 tx=0).stop 2023-11-13T11:04:10.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.566+0000 7f845e2f8640 1 -- 172.21.15.16:0/3285392341 shutdown_connections 2023-11-13T11:04:10.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.566+0000 7f845e2f8640 1 --2- 172.21.15.16:0/3285392341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 0x7f845806f970 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.566+0000 7f845e2f8640 1 --2- 172.21.15.16:0/3285392341 >> v2:172.21.15.184:3300/0 conn(0x7f84580cb190 0x7f845815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.566+0000 7f845e2f8640 1 -- 172.21.15.16:0/3285392341 >> 172.21.15.16:0/3285392341 conn(0x7f84580b04a0 msgr2=0x7f84580b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.566+0000 7f845e2f8640 1 -- 172.21.15.16:0/3285392341 shutdown_connections 2023-11-13T11:04:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.566+0000 7f845e2f8640 1 -- 172.21.15.16:0/3285392341 wait complete. 2023-11-13T11:04:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.568+0000 7f845e2f8640 1 Processor -- start 2023-11-13T11:04:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.568+0000 7f845e2f8640 1 -- start start 2023-11-13T11:04:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.569+0000 7f845e2f8640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f845806a190 0x7f845815da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.569+0000 7f845e2f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 0x7f8458144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.569+0000 7f845e2f8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84580cb190 0x7f8458149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.569+0000 7f845e2f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8458146c10 con 0x7f84580716b0 2023-11-13T11:04:10.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.569+0000 7f845e2f8640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8458146ef0 con 0x7f84580cb190 2023-11-13T11:04:10.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.569+0000 7f845e2f8640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84581471d0 con 0x7f845806a190 2023-11-13T11:04:10.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.569+0000 7f845caf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 0x7f8458144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:10.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.569+0000 7f845caf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 0x7f8458144880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51126/0 (socket says 172.21.15.16:51126) 2023-11-13T11:04:10.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.570+0000 7f845caf5640 1 -- 172.21.15.16:0/145022832 learned_addr learned my addr 172.21.15.16:0/145022832 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:10.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.570+0000 7f845d2f6640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f845806a190 0x7f845815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:10.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.570+0000 7f845daf7640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84580cb190 0x7f8458149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:10.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.570+0000 7f845caf5640 1 -- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f845806a190 msgr2=0x7f845815da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:10.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.570+0000 7f845caf5640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f845806a190 0x7f845815da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.570+0000 7f845caf5640 1 -- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84580cb190 msgr2=0x7f8458149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:10.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.570+0000 7f845caf5640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84580cb190 0x7f8458149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.570+0000 7f845caf5640 1 -- 172.21.15.16:0/145022832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84480094d0 con 0x7f84580716b0 2023-11-13T11:04:10.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.570+0000 7f845caf5640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 0x7f8458144880 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f844802b950 tx=0x7f84480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.571+0000 7f844dffb640 1 -- 172.21.15.16:0/145022832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448031860 con 0x7f84580716b0 2023-11-13T11:04:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.571+0000 7f844dffb640 1 -- 172.21.15.16:0/145022832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84480443f0 con 0x7f84580716b0 2023-11-13T11:04:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.571+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84581474b0 con 0x7f84580716b0 2023-11-13T11:04:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.571+0000 7f844dffb640 1 -- 172.21.15.16:0/145022832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84480447a0 con 0x7f84580716b0 2023-11-13T11:04:10.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.571+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845815cf00 con 0x7f84580716b0 2023-11-13T11:04:10.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.572+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84581479d0 con 0x7f84580716b0 2023-11-13T11:04:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.573+0000 7f844dffb640 1 -- 172.21.15.16:0/145022832 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f844804d450 con 0x7f84580716b0 2023-11-13T11:04:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.574+0000 7f844dffb640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.574+0000 7f844dffb640 1 -- 172.21.15.16:0/145022832 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f844802d040 con 0x7f84580716b0 2023-11-13T11:04:10.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.575+0000 7f845d2f6640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:10.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.576+0000 7f845d2f6640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8454007510 tx=0x7f84540074a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:10.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.578+0000 7f844dffb640 1 -- 172.21.15.16:0/145022832 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84581479d0 con 0x7f84580716b0 2023-11-13T11:04:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.772+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84580cb570 con 0x7f843c06ae40 2023-11-13T11:04:10.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.778+0000 7f844dffb640 1 -- 172.21.15.16:0/145022832 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48376 (secure 0 0 0) 0x7f8454013070 con 0x7f843c06ae40 2023-11-13T11:04:10.785 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:10.786 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:04:10.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 msgr2=0x7f843c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8454007510 tx=0x7f84540074a0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 msgr2=0x7f8458144880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 0x7f8458144880 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f844802b950 tx=0x7f84480046c0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 shutdown_connections 2023-11-13T11:04:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f843c06ae40 0x7f843c06d300 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84580cb190 0x7f8458149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580716b0 0x7f8458144880 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 --2- 172.21.15.16:0/145022832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f845806a190 0x7f845815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 >> 172.21.15.16:0/145022832 conn(0x7f84580b04a0 msgr2=0x7f845806a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 shutdown_connections 2023-11-13T11:04:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:10.782+0000 7f845e2f8640 1 -- 172.21.15.16:0/145022832 wait complete. 2023-11-13T11:04:10.807 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2023-11-13T11:04:10.174513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":325840,"kb_used_data":8360,"kb_used_omap":12,"kb_used_meta":317363,"kb_avail":754648880,"statfs":{"total":773094113280,"available":772760453120,"internally_reserved":0,"allocated":8560640,"data_stored":4506712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":324980273},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-170919,"num_objects":-66,"num_object_clones":0,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-66,"num_whiteouts":0,"num_read":-83,"num_read_kb":-82,"num_write":-215,"num_write_kb":-289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.413162"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-13T11:04:02.760062+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:04:02.760062+0000","last_peered":"2023-11-13T11:04:02.760062+0000","last_clean":"2023-11-13T11:04:02.760062+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:04:02.760062+0000","last_undegraded":"2023-11-13T11:04:02.760062+0000","last_fullsized":"2023-11-13T11:04:02.760062+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.129,"15min":1.121},"min":{"1min":0.74399999999999999,"5min":0.66500000000000004,"15min":0.63100000000000001},"max":{"1min":1.649,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2210000000000001,"15min":1.141},"min":{"1min":0.78400000000000003,"5min":0.75700000000000001,"15min":0.68500000000000005},"max":{"1min":1.806,"5min":1.865,"15min":2.2879999999999998},"last":1.216}]},{"osd":1,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2569999999999999,"15min":1.1839999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.56399999999999995},"max":{"1min":1.724,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.3100000000000001,"15min":1.2769999999999999},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4710000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.145}]},{"osd":2,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.319,"15min":1.2729999999999999},"min":{"1min":0.86699999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.72,"5min":2.153,"15min":2.1680000000000001},"last":1.425},{"interface":"front","average":{"1min":1.373,"5min":1.379,"15min":1.3480000000000001},"min":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8360000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.341}]},{"osd":3,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4350000000000001,"15min":1.375},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.63800000000000001},"max":{"1min":1.7390000000000001,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.276},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.46,"15min":1.4339999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.71299999999999997},"max":{"1min":1.8600000000000001,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.546}]},{"osd":4,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4379999999999999,"15min":1.536},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.833,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.482},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.476,"15min":1.5549999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.804,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.722}]},{"osd":5,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5669999999999999,"15min":1.625},"min":{"1min":0.94799999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.9379999999999999,"5min":2.7210000000000001,"15min":3.286},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.7250000000000001,"15min":1.74},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.04,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.762}]},{"osd":7,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.671,"15min":1.637},"min":{"1min":0.92000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.524},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.631,"15min":1.631},"min":{"1min":1.0349999999999999,"5min":0.96499999999999997,"15min":0.93700000000000006},"max":{"1min":1.9279999999999999,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.657}]}]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46548,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325292,"statfs":{"total":96636764160,"available":96589099008,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.091,"15min":1.0249999999999999},"min":{"1min":0.66600000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.643,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":2.746},{"interface":"front","average":{"1min":1.151,"5min":1.2270000000000001,"15min":1.198},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7749999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.222}]},{"osd":1,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3089999999999999,"15min":1.238},"min":{"1min":0.878,"5min":0.79100000000000004,"15min":0.65400000000000003},"max":{"1min":1.7390000000000001,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.282,"15min":1.238},"min":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.0219999999999998,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.589}]},{"osd":2,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3740000000000001,"15min":1.2649999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9139999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.718},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3149999999999999,"15min":1.2410000000000001},"min":{"1min":0.81000000000000005,"5min":0.75600000000000001,"15min":0.70799999999999996},"max":{"1min":1.8320000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":2.8439999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.413,"15min":1.3919999999999999},"min":{"1min":1.0720000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.881,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":4.3170000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.502,"15min":1.478},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8400000000000001,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":3.6230000000000002}]},{"osd":4,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.331,"15min":1.3220000000000001},"min":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.55500000000000005},"max":{"1min":1.7669999999999999,"5min":3.484,"15min":3.484},"last":3.1859999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.494,"15min":1.5189999999999999},"min":{"1min":0.95299999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.9590000000000001,"5min":3.653,"15min":3.653},"last":0.94899999999999995}]},{"osd":6,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5469999999999999,"15min":1.5900000000000001},"min":{"1min":1.077,"5min":0.92000000000000004,"15min":0.78000000000000003},"max":{"1min":1.9690000000000001,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":3.7069999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.516,"15min":1.5169999999999999},"min":{"1min":0.92400000000000004,"5min":0.87,"15min":0.61799999999999999},"max":{"1min":1.9910000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":2.9460000000000002}]},{"osd":7,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.6299999999999999,"15min":1.6559999999999999},"min":{"1min":1.1359999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.984,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":4.407},{"interface":"front","average":{"1min":1.595,"5min":1.7130000000000001,"15min":1.704},"min":{"1min":1.292,"5min":1.1200000000000001,"15min":0.69499999999999995},"max":{"1min":2.173,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":3.532}]}]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54288,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317552,"statfs":{"total":96636764160,"available":96581173248,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.173,"15min":1.1020000000000001},"min":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.738,"5min":1.986,"15min":1.986},"last":1.54},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.1539999999999999,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":1.542,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.93799999999999994}]},{"osd":1,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.2230000000000001,"15min":1.1599999999999999},"min":{"1min":0.93300000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.335,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.5},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.256},"min":{"1min":0.872,"5min":0.76200000000000001,"15min":0.745},"max":{"1min":1.5009999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.238}]},{"osd":2,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2869999999999999,"15min":1.238},"min":{"1min":0.89200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.488,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.363,"15min":1.3460000000000001},"min":{"1min":0.85199999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6479999999999999,"5min":1.952,"15min":1.952},"last":1.6339999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4790000000000001,"15min":1.421},"min":{"1min":1.0349999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.841,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.2929999999999999},"min":{"1min":0.92900000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5880000000000001,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.0940000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.464,"15min":1.4830000000000001},"min":{"1min":1,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.996,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.71},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.94399999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":1.7809999999999999,"5min":2.089,"15min":2.089},"last":1.2989999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.613,"15min":1.6180000000000001},"min":{"1min":1.327,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":2.1030000000000002,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.806},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.468,"15min":1.512},"min":{"1min":1.0569999999999999,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":2.0310000000000001,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.3540000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5660000000000001},"min":{"1min":1.157,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":1.9039999999999999,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.5740000000000001,"15min":1.5980000000000001},"min":{"1min":1.1930000000000001,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":2.1499999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.579}]}]},{"osd":3,"up_from":15,"seq":64424509550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44560,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327280,"statfs":{"total":96636764160,"available":96591134720,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34300000000000003},"max":{"1min":1.4099999999999999,"5min":1.867,"15min":1.867},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.04,"15min":1.018},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4790000000000001,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":1.8360000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0369999999999999,"15min":1.0389999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.534,"5min":1.966,"15min":3.71},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1459999999999999,"15min":1.1020000000000001},"min":{"1min":0.59999999999999998,"5min":0.51000000000000001,"15min":0.46200000000000002},"max":{"1min":1.738,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":1.163}]},{"osd":2,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1679999999999999,"15min":1.1000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.772},{"interface":"front","average":{"1min":1.016,"5min":1.1679999999999999,"15min":1.26},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.571,"5min":1.857,"15min":3.5710000000000002},"last":1.218}]},{"osd":4,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3500000000000001,"15min":1.4039999999999999},"min":{"1min":0.754,"5min":0.63600000000000001,"15min":0.497},"max":{"1min":2.0779999999999998,"5min":2.411,"15min":3.5030000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3979999999999999,"15min":1.3480000000000001},"min":{"1min":0.83499999999999996,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.615,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.333}]},{"osd":5,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.83199999999999996,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.627,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.46,"15min":1.496},"min":{"1min":0.82999999999999996,"5min":0.66100000000000003,"15min":0.53700000000000003},"max":{"1min":2.21,"5min":2.673,"15min":3.3809999999999998},"last":1.4830000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.486,"15min":1.476},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.67800000000000005},"max":{"1min":1.794,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.589},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.4870000000000001,"15min":1.5629999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.778,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.6879999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.5700000000000001,"15min":1.5960000000000001},"min":{"1min":0.92900000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1579999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.603,"15min":1.55},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.73099999999999998},"max":{"1min":1.8959999999999999,"5min":2.75,"15min":2.75},"last":1.893}]}]},{"osd":2,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50128,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.92800000000000005,"15min":0.91700000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2889999999999999,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.02,"15min":0.92700000000000005},"min":{"1min":0.51700000000000002,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":1.7330000000000001,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":1.452}]},{"osd":1,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":1.008,"15min":0.95199999999999996},"min":{"1min":0.47699999999999998,"5min":0.42599999999999999,"15min":0.376},"max":{"1min":1.3180000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":0.627},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.1870000000000001,"15min":1.1279999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5980000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.0349999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.2070000000000001,"15min":1.2210000000000001},"min":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":1.6899999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.625,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.524,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.5169999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.95899999999999996,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.7070000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.4019999999999999,"15min":1.393},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.63100000000000001},"max":{"1min":1.873,"5min":3.121,"15min":3.121},"last":0.97499999999999998}]},{"osd":5,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.4430000000000001,"15min":1.4610000000000001},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.605,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.597,"15min":1.5549999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.80500000000000005},"max":{"1min":1.786,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.6599999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.498,"15min":1.5249999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7629999999999999,"5min":3.395,"15min":4.7569999999999997},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.5940000000000001,"15min":1.581},"min":{"1min":0.84299999999999997,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":1.7509999999999999,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.5880000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5349999999999999,"15min":1.5760000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6599999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.629},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.617,"15min":1.603},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.7929999999999999,"5min":3.456,"15min":3.456},"last":1.399}]}]},{"osd":1,"up_from":15,"seq":64424509549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.60799999999999998,"5min":0.40500000000000003,"15min":0.35799999999999998},"max":{"1min":1.369,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.99399999999999999,"15min":0.96099999999999997},"min":{"1min":0.61799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6379999999999999,"5min":1.869,"15min":1.869},"last":1.028}]},{"osd":2,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.117,"15min":1.008},"min":{"1min":0.66900000000000004,"5min":0.44800000000000001,"15min":0.40000000000000002},"max":{"1min":1.6339999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.133,"15min":1.024},"min":{"1min":0.75900000000000001,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":1.619,"5min":2.915,"15min":2.915},"last":1.1599999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.839,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.036,"5min":1.083,"15min":1.0600000000000001},"min":{"1min":0.56100000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.601,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3120000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.361,"15min":1.3140000000000001},"min":{"1min":0.95699999999999996,"5min":0.76200000000000001,"15min":0.58599999999999997},"max":{"1min":1.6990000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.4730000000000001,"15min":1.395},"min":{"1min":1.0029999999999999,"5min":0.69599999999999995,"15min":0.60599999999999998},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.417}]},{"osd":5,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.448,"15min":1.4139999999999999},"min":{"1min":0.874,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.274},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.47,"15min":1.4179999999999999},"min":{"1min":1.3400000000000001,"5min":0.78600000000000003,"15min":0.70899999999999996},"max":{"1min":1.9339999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.125}]},{"osd":6,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.367,"15min":1.401},"min":{"1min":0.79800000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.702,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.379},{"interface":"front","average":{"1min":1.526,"5min":1.5629999999999999,"15min":1.5069999999999999},"min":{"1min":0.93799999999999994,"5min":0.85699999999999998,"15min":0.752},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.534}]},{"osd":7,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.544,"15min":1.5049999999999999},"min":{"1min":1.292,"5min":0.88,"15min":0.82899999999999996},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.498,"15min":1.4590000000000001},"min":{"1min":0.90900000000000003,"5min":0.80400000000000005,"15min":0.63500000000000001},"max":{"1min":1.891,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.452}]}]},{"osd":0,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.56100000000000005,"5min":0.36699999999999999,"15min":0.32100000000000001},"max":{"1min":1.464,"5min":1.74,"15min":1.74},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.589,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.7230000000000001,"5min":2.105,"15min":2.105},"last":1.226},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.371},"max":{"1min":1.4179999999999999,"5min":1.968,"15min":2.1669999999999998},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1559999999999999,"15min":1.105},"min":{"1min":0.66700000000000004,"5min":0.54000000000000004,"15min":0.45200000000000001},"max":{"1min":1.571,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.3009999999999999,"15min":1.3140000000000001},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.8460000000000001,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.0780000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.425,"15min":1.363},"min":{"1min":0.81299999999999994,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.786,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.202},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3680000000000001,"15min":1.4159999999999999},"min":{"1min":0.81999999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5309999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.369}]},{"osd":5,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.492,"15min":1.528},"min":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.611,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.468,"15min":1.482},"min":{"1min":0.91700000000000004,"5min":0.78600000000000003,"15min":0.747},"max":{"1min":1.746,"5min":2.706,"15min":2.706},"last":1.1759999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.597,"15min":1.5329999999999999},"min":{"1min":1.1020000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.669,"5min":4.617,"15min":4.617},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.5860000000000001,"15min":1.573},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.72899999999999998},"max":{"1min":2.0249999999999999,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.4059999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.643,"15min":1.6619999999999999},"min":{"1min":0.85799999999999998,"5min":0.82299999999999995,"15min":0.76700000000000002},"max":{"1min":1.9430000000000001,"5min":2.742,"15min":3.3540000000000001},"last":1.512},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.6699999999999999,"15min":1.6180000000000001},"min":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.77400000000000002},"max":{"1min":1.776,"5min":3.976,"15min":3.976},"last":1.3069999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45972,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325868,"statfs":{"total":96636764160,"available":96589688832,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.105,"15min":1.113},"min":{"1min":0.71399999999999997,"5min":0.65600000000000003,"15min":0.59899999999999998},"max":{"1min":1.9430000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":1.371},{"interface":"front","average":{"1min":1.3,"5min":1.1359999999999999,"15min":1.0880000000000001},"min":{"1min":1.0029999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.266},"last":1.5229999999999999}]},{"osd":1,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.198,"15min":1.0840000000000001},"min":{"1min":0.77900000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7729999999999999,"5min":1.784,"15min":2.0339999999999998},"last":1.141},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2649999999999999,"15min":1.2350000000000001},"min":{"1min":0.83899999999999997,"5min":0.70099999999999996,"15min":0.64700000000000002},"max":{"1min":1.677,"5min":1.893,"15min":2.1899999999999999},"last":1.478}]},{"osd":2,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.337,"15min":1.306},"min":{"1min":1.0229999999999999,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8600000000000001,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.335,"15min":1.2390000000000001},"min":{"1min":1.0449999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.7030000000000001,"5min":1.79,"15min":1.79},"last":1.425}]},{"osd":3,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.351,"15min":1.2709999999999999},"min":{"1min":0.90200000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.379,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.516,"15min":1.4670000000000001},"min":{"1min":1.004,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.8159999999999998},"last":1.075}]},{"osd":4,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.3660000000000001,"15min":1.3340000000000001},"min":{"1min":0.93400000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":3.0190000000000001},"last":1.29},{"interface":"front","average":{"1min":1.494,"5min":1.429,"15min":1.385},"min":{"1min":0.96199999999999997,"5min":0.439,"15min":0.439},"max":{"1min":2.5760000000000001,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.246}]},{"osd":5,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5289999999999999,"15min":1.534},"min":{"1min":0.995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.9620000000000002,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.6659999999999999,"5min":1.542,"15min":1.494},"min":{"1min":1.1459999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.3620000000000001},"last":1.6100000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.8280000000000001,"5min":1.653,"15min":1.591},"min":{"1min":1.409,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":3.383,"5min":3.383,"15min":3.383},"last":1.675},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5840000000000001,"15min":1.5229999999999999},"min":{"1min":1.0669999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.5990000000000002,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.454}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:04:10.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:04:11.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.026+0000 7ff5ac3cc640 1 Processor -- start 2023-11-13T11:04:11.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.026+0000 7ff5ac3cc640 1 -- start start 2023-11-13T11:04:11.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.028+0000 7ff5ac3cc640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5a40ae030 0x7ff5a406c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.028+0000 7ff5ac3cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff5a4070650 0x7ff5a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.028+0000 7ff5ac3cc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a40b3a80 con 0x7ff5a406fb60 2023-11-13T11:04:11.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.028+0000 7ff5ac3cc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a4161880 con 0x7ff5a4070650 2023-11-13T11:04:11.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.028+0000 7ff5ac3cc640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a4161b60 con 0x7ff5a40ae030 2023-11-13T11:04:11.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.028+0000 7ff5aa969640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff5a4070650 0x7ff5a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.028+0000 7ff5aa969640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff5a4070650 0x7ff5a406c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51134/0 (socket says 172.21.15.16:51134) 2023-11-13T11:04:11.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.028+0000 7ff5aa969640 1 -- 172.21.15.16:0/1739347398 learned_addr learned my addr 172.21.15.16:0/1739347398 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:11.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.028+0000 7ff5aa969640 1 -- 172.21.15.16:0/1739347398 >> v2:172.21.15.184:3300/0 conn(0x7ff5a40ae030 msgr2=0x7ff5a406c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:11.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.029+0000 7ff5aa969640 1 --2- 172.21.15.16:0/1739347398 >> v2:172.21.15.184:3300/0 conn(0x7ff5a40ae030 0x7ff5a406c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.029+0000 7ff5aa969640 1 -- 172.21.15.16:0/1739347398 >> v1:172.21.15.16:6789/0 conn(0x7ff5a406fb60 legacy=0x7ff5a406ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.029+0000 7ff5aa969640 1 -- 172.21.15.16:0/1739347398 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4161e40 con 0x7ff5a4070650 2023-11-13T11:04:11.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.029+0000 7ff5a9166640 1 -- 172.21.15.16:0/1739347398 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500267423 0 0) 0x7ff5a40b3a80 con 0x7ff5a406fb60 2023-11-13T11:04:11.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.030+0000 7ff5aa969640 1 --2- 172.21.15.16:0/1739347398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 0x7ff5a406c5a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40cf510 tx=0x7ff59c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f8b640372b38d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:11.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.030+0000 7ff5a9166640 1 -- 172.21.15.16:0/1739347398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c02ca80 con 0x7ff5a4070650 2023-11-13T11:04:11.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.030+0000 7ff5a9166640 1 -- 172.21.15.16:0/1739347398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c02cbe0 con 0x7ff5a4070650 2023-11-13T11:04:11.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.030+0000 7ff5a9166640 1 -- 172.21.15.16:0/1739347398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0343f0 con 0x7ff5a4070650 2023-11-13T11:04:11.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.031+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1739347398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 msgr2=0x7ff5a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.031+0000 7ff5ac3cc640 1 --2- 172.21.15.16:0/1739347398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 0x7ff5a406c5a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40cf510 tx=0x7ff59c02bee0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.031+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1739347398 shutdown_connections 2023-11-13T11:04:11.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.031+0000 7ff5ac3cc640 1 --2- 172.21.15.16:0/1739347398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 0x7ff5a406c5a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.031+0000 7ff5ac3cc640 1 --2- 172.21.15.16:0/1739347398 >> v2:172.21.15.184:3300/0 conn(0x7ff5a40ae030 0x7ff5a406c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.031+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1739347398 >> 172.21.15.16:0/1739347398 conn(0x7ff5a406a190 msgr2=0x7ff5a406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.031+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1739347398 shutdown_connections 2023-11-13T11:04:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.031+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1739347398 wait complete. 2023-11-13T11:04:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.033+0000 7ff5ac3cc640 1 Processor -- start 2023-11-13T11:04:11.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.033+0000 7ff5ac3cc640 1 -- start start 2023-11-13T11:04:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.034+0000 7ff5ac3cc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406fb60 0x7ff5a4150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.034+0000 7ff5ac3cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 0x7ff5a40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.034+0000 7ff5ac3cc640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40ae030 0x7ff5a4163810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.034+0000 7ff5ac3cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4150fd0 con 0x7ff5a4070650 2023-11-13T11:04:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.034+0000 7ff5ac3cc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41f7060 con 0x7ff5a406fb60 2023-11-13T11:04:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.034+0000 7ff5ac3cc640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40cd710 con 0x7ff5a40ae030 2023-11-13T11:04:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.034+0000 7ff5a9967640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 0x7ff5a40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.034+0000 7ff5a9967640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 0x7ff5a40ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51142/0 (socket says 172.21.15.16:51142) 2023-11-13T11:04:11.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.034+0000 7ff5a9967640 1 -- 172.21.15.16:0/1162327168 learned_addr learned my addr 172.21.15.16:0/1162327168 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:11.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff5aa168640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406fb60 0x7ff5a4150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff5aa969640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40ae030 0x7ff5a4163810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff5a9967640 1 -- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40ae030 msgr2=0x7ff5a4163810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff5a9967640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40ae030 0x7ff5a4163810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff5a9967640 1 -- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406fb60 msgr2=0x7ff5a4150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff5a9967640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406fb60 0x7ff5a4150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff5a9967640 1 -- 172.21.15.16:0/1162327168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c009b30 con 0x7ff5a4070650 2023-11-13T11:04:11.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff5a9967640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 0x7ff5a40ccdc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff59800b410 tx=0x7ff59800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:11.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff596ffd640 1 -- 172.21.15.16:0/1162327168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598011830 con 0x7ff5a4070650 2023-11-13T11:04:11.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff596ffd640 1 -- 172.21.15.16:0/1162327168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff598011990 con 0x7ff5a4070650 2023-11-13T11:04:11.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff596ffd640 1 -- 172.21.15.16:0/1162327168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598011d40 con 0x7ff5a4070650 2023-11-13T11:04:11.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.035+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a40cd9f0 con 0x7ff5a4070650 2023-11-13T11:04:11.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.036+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a40cdfb0 con 0x7ff5a4070650 2023-11-13T11:04:11.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.037+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a41f7210 con 0x7ff5a4070650 2023-11-13T11:04:11.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.038+0000 7ff596ffd640 1 -- 172.21.15.16:0/1162327168 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff59801a400 con 0x7ff5a4070650 2023-11-13T11:04:11.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.038+0000 7ff596ffd640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff58806ae40 0x7ff58806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.039+0000 7ff596ffd640 1 -- 172.21.15.16:0/1162327168 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7ff598089760 con 0x7ff5a4070650 2023-11-13T11:04:11.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.039+0000 7ff5aa168640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff58806ae40 0x7ff58806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.040+0000 7ff5aa168640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff58806ae40 0x7ff58806d300 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0000f60 tx=0x7ff5a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:11.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.043+0000 7ff596ffd640 1 -- 172.21.15.16:0/1162327168 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5a41f7210 con 0x7ff5a4070650 2023-11-13T11:04:11.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.236+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5a4162640 con 0x7ff58806ae40 2023-11-13T11:04:11.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.242+0000 7ff596ffd640 1 -- 172.21.15.16:0/1162327168 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48376 (secure 0 0 0) 0x7ff5a0001cf0 con 0x7ff58806ae40 2023-11-13T11:04:11.248 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:11.249 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:04:11.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff58806ae40 msgr2=0x7ff58806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff58806ae40 0x7ff58806d300 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0000f60 tx=0x7ff5a0009040 comp rx=0 tx=0).stop 2023-11-13T11:04:11.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 msgr2=0x7ff5a40ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 0x7ff5a40ccdc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff59800b410 tx=0x7ff59800b8d0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 shutdown_connections 2023-11-13T11:04:11.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7ff58806ae40 0x7ff58806d300 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5a40ae030 0x7ff5a4163810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a4070650 0x7ff5a40ccdc0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 --2- 172.21.15.16:0/1162327168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5a406fb60 0x7ff5a4150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 >> 172.21.15.16:0/1162327168 conn(0x7ff5a406a190 msgr2=0x7ff5a406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:11.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.245+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 shutdown_connections 2023-11-13T11:04:11.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.246+0000 7ff5ac3cc640 1 -- 172.21.15.16:0/1162327168 wait complete. 2023-11-13T11:04:11.270 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2023-11-13T11:04:10.174513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":325840,"kb_used_data":8360,"kb_used_omap":12,"kb_used_meta":317363,"kb_avail":754648880,"statfs":{"total":773094113280,"available":772760453120,"internally_reserved":0,"allocated":8560640,"data_stored":4506712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":324980273},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-170919,"num_objects":-66,"num_object_clones":0,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-66,"num_whiteouts":0,"num_read":-83,"num_read_kb":-82,"num_write":-215,"num_write_kb":-289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.413162"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-13T11:04:02.760062+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:04:02.760062+0000","last_peered":"2023-11-13T11:04:02.760062+0000","last_clean":"2023-11-13T11:04:02.760062+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:04:02.760062+0000","last_undegraded":"2023-11-13T11:04:02.760062+0000","last_fullsized":"2023-11-13T11:04:02.760062+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.129,"15min":1.121},"min":{"1min":0.74399999999999999,"5min":0.66500000000000004,"15min":0.63100000000000001},"max":{"1min":1.649,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2210000000000001,"15min":1.141},"min":{"1min":0.78400000000000003,"5min":0.75700000000000001,"15min":0.68500000000000005},"max":{"1min":1.806,"5min":1.865,"15min":2.2879999999999998},"last":1.216}]},{"osd":1,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2569999999999999,"15min":1.1839999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.56399999999999995},"max":{"1min":1.724,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.3100000000000001,"15min":1.2769999999999999},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4710000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.145}]},{"osd":2,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.319,"15min":1.2729999999999999},"min":{"1min":0.86699999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.72,"5min":2.153,"15min":2.1680000000000001},"last":1.425},{"interface":"front","average":{"1min":1.373,"5min":1.379,"15min":1.3480000000000001},"min":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8360000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.341}]},{"osd":3,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4350000000000001,"15min":1.375},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.63800000000000001},"max":{"1min":1.7390000000000001,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.276},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.46,"15min":1.4339999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.71299999999999997},"max":{"1min":1.8600000000000001,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.546}]},{"osd":4,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4379999999999999,"15min":1.536},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.833,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.482},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.476,"15min":1.5549999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.804,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.722}]},{"osd":5,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5669999999999999,"15min":1.625},"min":{"1min":0.94799999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.9379999999999999,"5min":2.7210000000000001,"15min":3.286},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.7250000000000001,"15min":1.74},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.04,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.762}]},{"osd":7,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.671,"15min":1.637},"min":{"1min":0.92000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.524},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.631,"15min":1.631},"min":{"1min":1.0349999999999999,"5min":0.96499999999999997,"15min":0.93700000000000006},"max":{"1min":1.9279999999999999,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.657}]}]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46548,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325292,"statfs":{"total":96636764160,"available":96589099008,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.091,"15min":1.0249999999999999},"min":{"1min":0.66600000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.643,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":2.746},{"interface":"front","average":{"1min":1.151,"5min":1.2270000000000001,"15min":1.198},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7749999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.222}]},{"osd":1,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3089999999999999,"15min":1.238},"min":{"1min":0.878,"5min":0.79100000000000004,"15min":0.65400000000000003},"max":{"1min":1.7390000000000001,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.282,"15min":1.238},"min":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.0219999999999998,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.589}]},{"osd":2,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3740000000000001,"15min":1.2649999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9139999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.718},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3149999999999999,"15min":1.2410000000000001},"min":{"1min":0.81000000000000005,"5min":0.75600000000000001,"15min":0.70799999999999996},"max":{"1min":1.8320000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":2.8439999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.413,"15min":1.3919999999999999},"min":{"1min":1.0720000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.881,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":4.3170000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.502,"15min":1.478},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8400000000000001,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":3.6230000000000002}]},{"osd":4,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.331,"15min":1.3220000000000001},"min":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.55500000000000005},"max":{"1min":1.7669999999999999,"5min":3.484,"15min":3.484},"last":3.1859999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.494,"15min":1.5189999999999999},"min":{"1min":0.95299999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.9590000000000001,"5min":3.653,"15min":3.653},"last":0.94899999999999995}]},{"osd":6,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5469999999999999,"15min":1.5900000000000001},"min":{"1min":1.077,"5min":0.92000000000000004,"15min":0.78000000000000003},"max":{"1min":1.9690000000000001,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":3.7069999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.516,"15min":1.5169999999999999},"min":{"1min":0.92400000000000004,"5min":0.87,"15min":0.61799999999999999},"max":{"1min":1.9910000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":2.9460000000000002}]},{"osd":7,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.6299999999999999,"15min":1.6559999999999999},"min":{"1min":1.1359999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.984,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":4.407},{"interface":"front","average":{"1min":1.595,"5min":1.7130000000000001,"15min":1.704},"min":{"1min":1.292,"5min":1.1200000000000001,"15min":0.69499999999999995},"max":{"1min":2.173,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":3.532}]}]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54288,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317552,"statfs":{"total":96636764160,"available":96581173248,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.173,"15min":1.1020000000000001},"min":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.738,"5min":1.986,"15min":1.986},"last":1.54},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.1539999999999999,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":1.542,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.93799999999999994}]},{"osd":1,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.2230000000000001,"15min":1.1599999999999999},"min":{"1min":0.93300000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.335,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.5},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.256},"min":{"1min":0.872,"5min":0.76200000000000001,"15min":0.745},"max":{"1min":1.5009999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.238}]},{"osd":2,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2869999999999999,"15min":1.238},"min":{"1min":0.89200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.488,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.363,"15min":1.3460000000000001},"min":{"1min":0.85199999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6479999999999999,"5min":1.952,"15min":1.952},"last":1.6339999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4790000000000001,"15min":1.421},"min":{"1min":1.0349999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.841,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.2929999999999999},"min":{"1min":0.92900000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5880000000000001,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.0940000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.464,"15min":1.4830000000000001},"min":{"1min":1,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.996,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.71},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.94399999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":1.7809999999999999,"5min":2.089,"15min":2.089},"last":1.2989999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.613,"15min":1.6180000000000001},"min":{"1min":1.327,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":2.1030000000000002,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.806},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.468,"15min":1.512},"min":{"1min":1.0569999999999999,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":2.0310000000000001,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.3540000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5660000000000001},"min":{"1min":1.157,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":1.9039999999999999,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.5740000000000001,"15min":1.5980000000000001},"min":{"1min":1.1930000000000001,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":2.1499999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.579}]}]},{"osd":3,"up_from":15,"seq":64424509550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44560,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327280,"statfs":{"total":96636764160,"available":96591134720,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34300000000000003},"max":{"1min":1.4099999999999999,"5min":1.867,"15min":1.867},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.04,"15min":1.018},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4790000000000001,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":1.8360000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0369999999999999,"15min":1.0389999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.534,"5min":1.966,"15min":3.71},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1459999999999999,"15min":1.1020000000000001},"min":{"1min":0.59999999999999998,"5min":0.51000000000000001,"15min":0.46200000000000002},"max":{"1min":1.738,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":1.163}]},{"osd":2,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1679999999999999,"15min":1.1000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.772},{"interface":"front","average":{"1min":1.016,"5min":1.1679999999999999,"15min":1.26},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.571,"5min":1.857,"15min":3.5710000000000002},"last":1.218}]},{"osd":4,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3500000000000001,"15min":1.4039999999999999},"min":{"1min":0.754,"5min":0.63600000000000001,"15min":0.497},"max":{"1min":2.0779999999999998,"5min":2.411,"15min":3.5030000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3979999999999999,"15min":1.3480000000000001},"min":{"1min":0.83499999999999996,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.615,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.333}]},{"osd":5,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.83199999999999996,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.627,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.46,"15min":1.496},"min":{"1min":0.82999999999999996,"5min":0.66100000000000003,"15min":0.53700000000000003},"max":{"1min":2.21,"5min":2.673,"15min":3.3809999999999998},"last":1.4830000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.486,"15min":1.476},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.67800000000000005},"max":{"1min":1.794,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.589},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.4870000000000001,"15min":1.5629999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.778,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.6879999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.5700000000000001,"15min":1.5960000000000001},"min":{"1min":0.92900000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1579999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.603,"15min":1.55},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.73099999999999998},"max":{"1min":1.8959999999999999,"5min":2.75,"15min":2.75},"last":1.893}]}]},{"osd":2,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50128,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.92800000000000005,"15min":0.91700000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2889999999999999,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.02,"15min":0.92700000000000005},"min":{"1min":0.51700000000000002,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":1.7330000000000001,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":1.452}]},{"osd":1,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":1.008,"15min":0.95199999999999996},"min":{"1min":0.47699999999999998,"5min":0.42599999999999999,"15min":0.376},"max":{"1min":1.3180000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":0.627},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.1870000000000001,"15min":1.1279999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5980000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.0349999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.2070000000000001,"15min":1.2210000000000001},"min":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":1.6899999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.625,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.524,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.5169999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.95899999999999996,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.7070000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.4019999999999999,"15min":1.393},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.63100000000000001},"max":{"1min":1.873,"5min":3.121,"15min":3.121},"last":0.97499999999999998}]},{"osd":5,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.4430000000000001,"15min":1.4610000000000001},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.605,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.597,"15min":1.5549999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.80500000000000005},"max":{"1min":1.786,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.6599999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.498,"15min":1.5249999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7629999999999999,"5min":3.395,"15min":4.7569999999999997},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.5940000000000001,"15min":1.581},"min":{"1min":0.84299999999999997,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":1.7509999999999999,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.5880000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5349999999999999,"15min":1.5760000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6599999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.629},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.617,"15min":1.603},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.7929999999999999,"5min":3.456,"15min":3.456},"last":1.399}]}]},{"osd":1,"up_from":15,"seq":64424509549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.60799999999999998,"5min":0.40500000000000003,"15min":0.35799999999999998},"max":{"1min":1.369,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.99399999999999999,"15min":0.96099999999999997},"min":{"1min":0.61799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6379999999999999,"5min":1.869,"15min":1.869},"last":1.028}]},{"osd":2,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.117,"15min":1.008},"min":{"1min":0.66900000000000004,"5min":0.44800000000000001,"15min":0.40000000000000002},"max":{"1min":1.6339999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.133,"15min":1.024},"min":{"1min":0.75900000000000001,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":1.619,"5min":2.915,"15min":2.915},"last":1.1599999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.839,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.036,"5min":1.083,"15min":1.0600000000000001},"min":{"1min":0.56100000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.601,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3120000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.361,"15min":1.3140000000000001},"min":{"1min":0.95699999999999996,"5min":0.76200000000000001,"15min":0.58599999999999997},"max":{"1min":1.6990000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.4730000000000001,"15min":1.395},"min":{"1min":1.0029999999999999,"5min":0.69599999999999995,"15min":0.60599999999999998},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.417}]},{"osd":5,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.448,"15min":1.4139999999999999},"min":{"1min":0.874,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.274},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.47,"15min":1.4179999999999999},"min":{"1min":1.3400000000000001,"5min":0.78600000000000003,"15min":0.70899999999999996},"max":{"1min":1.9339999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.125}]},{"osd":6,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.367,"15min":1.401},"min":{"1min":0.79800000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.702,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.379},{"interface":"front","average":{"1min":1.526,"5min":1.5629999999999999,"15min":1.5069999999999999},"min":{"1min":0.93799999999999994,"5min":0.85699999999999998,"15min":0.752},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.534}]},{"osd":7,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.544,"15min":1.5049999999999999},"min":{"1min":1.292,"5min":0.88,"15min":0.82899999999999996},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.498,"15min":1.4590000000000001},"min":{"1min":0.90900000000000003,"5min":0.80400000000000005,"15min":0.63500000000000001},"max":{"1min":1.891,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.452}]}]},{"osd":0,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.56100000000000005,"5min":0.36699999999999999,"15min":0.32100000000000001},"max":{"1min":1.464,"5min":1.74,"15min":1.74},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.589,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.7230000000000001,"5min":2.105,"15min":2.105},"last":1.226},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.371},"max":{"1min":1.4179999999999999,"5min":1.968,"15min":2.1669999999999998},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1559999999999999,"15min":1.105},"min":{"1min":0.66700000000000004,"5min":0.54000000000000004,"15min":0.45200000000000001},"max":{"1min":1.571,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.3009999999999999,"15min":1.3140000000000001},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.8460000000000001,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.0780000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.425,"15min":1.363},"min":{"1min":0.81299999999999994,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.786,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.202},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3680000000000001,"15min":1.4159999999999999},"min":{"1min":0.81999999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5309999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.369}]},{"osd":5,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.492,"15min":1.528},"min":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.611,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.468,"15min":1.482},"min":{"1min":0.91700000000000004,"5min":0.78600000000000003,"15min":0.747},"max":{"1min":1.746,"5min":2.706,"15min":2.706},"last":1.1759999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.597,"15min":1.5329999999999999},"min":{"1min":1.1020000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.669,"5min":4.617,"15min":4.617},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.5860000000000001,"15min":1.573},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.72899999999999998},"max":{"1min":2.0249999999999999,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.4059999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.643,"15min":1.6619999999999999},"min":{"1min":0.85799999999999998,"5min":0.82299999999999995,"15min":0.76700000000000002},"max":{"1min":1.9430000000000001,"5min":2.742,"15min":3.3540000000000001},"last":1.512},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.6699999999999999,"15min":1.6180000000000001},"min":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.77400000000000002},"max":{"1min":1.776,"5min":3.976,"15min":3.976},"last":1.3069999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45972,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325868,"statfs":{"total":96636764160,"available":96589688832,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.105,"15min":1.113},"min":{"1min":0.71399999999999997,"5min":0.65600000000000003,"15min":0.59899999999999998},"max":{"1min":1.9430000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":1.371},{"interface":"front","average":{"1min":1.3,"5min":1.1359999999999999,"15min":1.0880000000000001},"min":{"1min":1.0029999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.266},"last":1.5229999999999999}]},{"osd":1,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.198,"15min":1.0840000000000001},"min":{"1min":0.77900000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7729999999999999,"5min":1.784,"15min":2.0339999999999998},"last":1.141},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2649999999999999,"15min":1.2350000000000001},"min":{"1min":0.83899999999999997,"5min":0.70099999999999996,"15min":0.64700000000000002},"max":{"1min":1.677,"5min":1.893,"15min":2.1899999999999999},"last":1.478}]},{"osd":2,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.337,"15min":1.306},"min":{"1min":1.0229999999999999,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8600000000000001,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.335,"15min":1.2390000000000001},"min":{"1min":1.0449999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.7030000000000001,"5min":1.79,"15min":1.79},"last":1.425}]},{"osd":3,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.351,"15min":1.2709999999999999},"min":{"1min":0.90200000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.379,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.516,"15min":1.4670000000000001},"min":{"1min":1.004,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.8159999999999998},"last":1.075}]},{"osd":4,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.3660000000000001,"15min":1.3340000000000001},"min":{"1min":0.93400000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":3.0190000000000001},"last":1.29},{"interface":"front","average":{"1min":1.494,"5min":1.429,"15min":1.385},"min":{"1min":0.96199999999999997,"5min":0.439,"15min":0.439},"max":{"1min":2.5760000000000001,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.246}]},{"osd":5,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5289999999999999,"15min":1.534},"min":{"1min":0.995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.9620000000000002,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.6659999999999999,"5min":1.542,"15min":1.494},"min":{"1min":1.1459999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.3620000000000001},"last":1.6100000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.8280000000000001,"5min":1.653,"15min":1.591},"min":{"1min":1.409,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":3.383,"5min":3.383,"15min":3.383},"last":1.675},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5840000000000001,"15min":1.5229999999999999},"min":{"1min":1.0669999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.5990000000000002,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.454}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:04:11.272 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-13T11:04:11.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:04:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.497+0000 7f8c4162e640 1 Processor -- start 2023-11-13T11:04:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.498+0000 7f8c4162e640 1 -- start start 2023-11-13T11:04:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.499+0000 7f8c4162e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c3c0cb190 0x7f8c3c15b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.499+0000 7f8c4162e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3c158c20 con 0x7f8c3c06a190 2023-11-13T11:04:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.499+0000 7f8c4162e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3c0b1830 con 0x7f8c3c0716b0 2023-11-13T11:04:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.499+0000 7f8c4162e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c15b780 con 0x7f8c3c0cb190 2023-11-13T11:04:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.500+0000 7f8c3bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c3c0cb190 0x7f8c3c15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.500+0000 7f8c40e2d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c3c06a190 0x7f8c3c0b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44244/0 (socket says 172.21.15.16:44244) 2023-11-13T11:04:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.500+0000 7f8c40e2d640 1 -- 172.21.15.16:0/809864019 learned_addr learned my addr 172.21.15.16:0/809864019 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.500+0000 7f8c3bfff640 1 -- 172.21.15.16:0/809864019 >> v1:172.21.15.184:6789/0 conn(0x7f8c3c0716b0 legacy=0x7f8c3c06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.500+0000 7f8c3bfff640 1 -- 172.21.15.16:0/809864019 >> v1:172.21.15.16:6789/0 conn(0x7f8c3c06a190 legacy=0x7f8c3c0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.500+0000 7f8c3bfff640 1 -- 172.21.15.16:0/809864019 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c15ba60 con 0x7f8c3c0cb190 2023-11-13T11:04:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.500+0000 7f8c3b7fe640 1 -- 172.21.15.16:0/809864019 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927442181 0 0) 0x7f8c3c158c20 con 0x7f8c3c06a190 2023-11-13T11:04:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.501+0000 7f8c3bfff640 1 --2- 172.21.15.16:0/809864019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c0cb190 0x7f8c3c15b190 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0097a0 tx=0x7f8c2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa381e3d0bda733 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.502+0000 7f8c3b7fe640 1 -- 172.21.15.16:0/809864019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c031830 con 0x7f8c3c0cb190 2023-11-13T11:04:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.502+0000 7f8c3b7fe640 1 -- 172.21.15.16:0/809864019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c031990 con 0x7f8c3c0cb190 2023-11-13T11:04:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.502+0000 7f8c3b7fe640 1 -- 172.21.15.16:0/809864019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c031d20 con 0x7f8c3c0cb190 2023-11-13T11:04:11.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.502+0000 7f8c4162e640 1 -- 172.21.15.16:0/809864019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c0cb190 msgr2=0x7f8c3c15b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.502+0000 7f8c4162e640 1 --2- 172.21.15.16:0/809864019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c0cb190 0x7f8c3c15b190 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0097a0 tx=0x7f8c2c02b8d0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.503+0000 7f8c4162e640 1 -- 172.21.15.16:0/809864019 shutdown_connections 2023-11-13T11:04:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.503+0000 7f8c4162e640 1 --2- 172.21.15.16:0/809864019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c0cb190 0x7f8c3c15b190 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.503+0000 7f8c4162e640 1 -- 172.21.15.16:0/809864019 >> 172.21.15.16:0/809864019 conn(0x7f8c3c0b04a0 msgr2=0x7f8c3c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.503+0000 7f8c4162e640 1 -- 172.21.15.16:0/809864019 shutdown_connections 2023-11-13T11:04:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.503+0000 7f8c4162e640 1 -- 172.21.15.16:0/809864019 wait complete. 2023-11-13T11:04:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.505+0000 7f8c4162e640 1 Processor -- start 2023-11-13T11:04:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.505+0000 7f8c4162e640 1 -- start start 2023-11-13T11:04:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.506+0000 7f8c4162e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c3c06a190 0x7f8c3c0ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.506+0000 7f8c4162e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c3c0716b0 0x7f8c3c1481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.506+0000 7f8c4162e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c06b9d0 0x7f8c3c1f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.506+0000 7f8c4162e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c06cb10 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.506+0000 7f8c4162e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0ca750 con 0x7f8c3c06a190 2023-11-13T11:04:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.506+0000 7f8c4162e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c148720 con 0x7f8c3c0716b0 2023-11-13T11:04:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.506+0000 7f8c40e2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c06b9d0 0x7f8c3c1f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.506+0000 7f8c40e2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c06b9d0 0x7f8c3c1f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51156/0 (socket says 172.21.15.16:51156) 2023-11-13T11:04:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.506+0000 7f8c40e2d640 1 -- 172.21.15.16:0/403276653 learned_addr learned my addr 172.21.15.16:0/403276653 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c3bfff640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c3c06a190 0x7f8c3c0ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c33fff640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c3c0716b0 0x7f8c3c1481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c40e2d640 1 -- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c3c0716b0 msgr2=0x7f8c3c1481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c40e2d640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c3c0716b0 0x7f8c3c1481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c40e2d640 1 -- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c3c06a190 msgr2=0x7f8c3c0ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c40e2d640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c3c06a190 0x7f8c3c0ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c40e2d640 1 -- 172.21.15.16:0/403276653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c009400 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c33fff640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c3c0716b0 0x7f8c3c1481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c40e2d640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c06b9d0 0x7f8c3c1f2ad0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c810 tx=0x7f8c3400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c397fa640 1 -- 172.21.15.16:0/403276653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400a640 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c397fa640 1 -- 172.21.15.16:0/403276653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3400a7c0 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c397fa640 1 -- 172.21.15.16:0/403276653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400ab70 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c06be90 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.507+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c06c450 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.509+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c06cc50 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.510+0000 7f8c397fa640 1 -- 172.21.15.16:0/403276653 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c3400acd0 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.510+0000 7f8c397fa640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c1c06ae40 0x7f8c1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.511+0000 7f8c397fa640 1 -- 172.21.15.16:0/403276653 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f8c34088a00 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.511+0000 7f8c3bfff640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c1c06ae40 0x7f8c1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.512+0000 7f8c3bfff640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c1c06ae40 0x7f8c1c06d300 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c036000 tx=0x7f8c2c03c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.515+0000 7f8c397fa640 1 -- 172.21.15.16:0/403276653 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c3c06cc50 con 0x7f8c3c06b9d0 2023-11-13T11:04:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.707+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c3c15c020 con 0x7f8c1c06ae40 2023-11-13T11:04:11.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.714+0000 7f8c397fa640 1 -- 172.21.15.16:0/403276653 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48376 (secure 0 0 0) 0x7f8c2c045410 con 0x7f8c1c06ae40 2023-11-13T11:04:11.719 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:04:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.716+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c1c06ae40 msgr2=0x7f8c1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c1c06ae40 0x7f8c1c06d300 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c036000 tx=0x7f8c2c03c040 comp rx=0 tx=0).stop 2023-11-13T11:04:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c06b9d0 msgr2=0x7f8c3c1f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c06b9d0 0x7f8c3c1f2ad0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c810 tx=0x7f8c3400ccd0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 shutdown_connections 2023-11-13T11:04:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c1c06ae40 0x7f8c1c06d300 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3c06b9d0 0x7f8c3c1f2ad0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c3c0716b0 0x7f8c3c1481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 --2- 172.21.15.16:0/403276653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c3c06a190 0x7f8c3c0ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 >> 172.21.15.16:0/403276653 conn(0x7f8c3c0b04a0 msgr2=0x7f8c3c15e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 shutdown_connections 2023-11-13T11:04:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.717+0000 7f8c4162e640 1 -- 172.21.15.16:0/403276653 wait complete. 2023-11-13T11:04:11.740 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2023-11-13T11:04:10.174513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":325840,"kb_used_data":8360,"kb_used_omap":12,"kb_used_meta":317363,"kb_avail":754648880,"statfs":{"total":773094113280,"available":772760453120,"internally_reserved":0,"allocated":8560640,"data_stored":4506712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":324980273},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-170919,"num_objects":-66,"num_object_clones":0,"num_object_copies":-132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-66,"num_whiteouts":0,"num_read":-83,"num_read_kb":-82,"num_write":-215,"num_write_kb":-289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.413162"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-13T11:04:02.760062+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:04:02.760062+0000","last_peered":"2023-11-13T11:04:02.760062+0000","last_clean":"2023-11-13T11:04:02.760062+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:04:02.760062+0000","last_undegraded":"2023-11-13T11:04:02.760062+0000","last_fullsized":"2023-11-13T11:04:02.760062+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.129,"15min":1.121},"min":{"1min":0.74399999999999999,"5min":0.66500000000000004,"15min":0.63100000000000001},"max":{"1min":1.649,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2210000000000001,"15min":1.141},"min":{"1min":0.78400000000000003,"5min":0.75700000000000001,"15min":0.68500000000000005},"max":{"1min":1.806,"5min":1.865,"15min":2.2879999999999998},"last":1.216}]},{"osd":1,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2569999999999999,"15min":1.1839999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.56399999999999995},"max":{"1min":1.724,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.3100000000000001,"15min":1.2769999999999999},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4710000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.145}]},{"osd":2,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.319,"15min":1.2729999999999999},"min":{"1min":0.86699999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.72,"5min":2.153,"15min":2.1680000000000001},"last":1.425},{"interface":"front","average":{"1min":1.373,"5min":1.379,"15min":1.3480000000000001},"min":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8360000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.341}]},{"osd":3,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4350000000000001,"15min":1.375},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.63800000000000001},"max":{"1min":1.7390000000000001,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.276},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.46,"15min":1.4339999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.71299999999999997},"max":{"1min":1.8600000000000001,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.546}]},{"osd":4,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4379999999999999,"15min":1.536},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.833,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.482},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.476,"15min":1.5549999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.804,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.722}]},{"osd":5,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5669999999999999,"15min":1.625},"min":{"1min":0.94799999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.9379999999999999,"5min":2.7210000000000001,"15min":3.286},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.7250000000000001,"15min":1.74},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.04,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.762}]},{"osd":7,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.671,"15min":1.637},"min":{"1min":0.92000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.524},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.631,"15min":1.631},"min":{"1min":1.0349999999999999,"5min":0.96499999999999997,"15min":0.93700000000000006},"max":{"1min":1.9279999999999999,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.657}]}]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46548,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325292,"statfs":{"total":96636764160,"available":96589099008,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.091,"15min":1.0249999999999999},"min":{"1min":0.66600000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.643,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":2.746},{"interface":"front","average":{"1min":1.151,"5min":1.2270000000000001,"15min":1.198},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7749999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.222}]},{"osd":1,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3089999999999999,"15min":1.238},"min":{"1min":0.878,"5min":0.79100000000000004,"15min":0.65400000000000003},"max":{"1min":1.7390000000000001,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.282,"15min":1.238},"min":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.0219999999999998,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.589}]},{"osd":2,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3740000000000001,"15min":1.2649999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9139999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.718},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3149999999999999,"15min":1.2410000000000001},"min":{"1min":0.81000000000000005,"5min":0.75600000000000001,"15min":0.70799999999999996},"max":{"1min":1.8320000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":2.8439999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.413,"15min":1.3919999999999999},"min":{"1min":1.0720000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.881,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":4.3170000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.502,"15min":1.478},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8400000000000001,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":3.6230000000000002}]},{"osd":4,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.331,"15min":1.3220000000000001},"min":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.55500000000000005},"max":{"1min":1.7669999999999999,"5min":3.484,"15min":3.484},"last":3.1859999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.494,"15min":1.5189999999999999},"min":{"1min":0.95299999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.9590000000000001,"5min":3.653,"15min":3.653},"last":0.94899999999999995}]},{"osd":6,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5469999999999999,"15min":1.5900000000000001},"min":{"1min":1.077,"5min":0.92000000000000004,"15min":0.78000000000000003},"max":{"1min":1.9690000000000001,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":3.7069999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.516,"15min":1.5169999999999999},"min":{"1min":0.92400000000000004,"5min":0.87,"15min":0.61799999999999999},"max":{"1min":1.9910000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":2.9460000000000002}]},{"osd":7,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.6299999999999999,"15min":1.6559999999999999},"min":{"1min":1.1359999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.984,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":4.407},{"interface":"front","average":{"1min":1.595,"5min":1.7130000000000001,"15min":1.704},"min":{"1min":1.292,"5min":1.1200000000000001,"15min":0.69499999999999995},"max":{"1min":2.173,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":3.532}]}]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54288,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317552,"statfs":{"total":96636764160,"available":96581173248,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.173,"15min":1.1020000000000001},"min":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.738,"5min":1.986,"15min":1.986},"last":1.54},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.1539999999999999,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":1.542,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.93799999999999994}]},{"osd":1,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.2230000000000001,"15min":1.1599999999999999},"min":{"1min":0.93300000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.335,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.5},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.256},"min":{"1min":0.872,"5min":0.76200000000000001,"15min":0.745},"max":{"1min":1.5009999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.238}]},{"osd":2,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2869999999999999,"15min":1.238},"min":{"1min":0.89200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.488,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.363,"15min":1.3460000000000001},"min":{"1min":0.85199999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6479999999999999,"5min":1.952,"15min":1.952},"last":1.6339999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4790000000000001,"15min":1.421},"min":{"1min":1.0349999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.841,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.2929999999999999},"min":{"1min":0.92900000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5880000000000001,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.0940000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.464,"15min":1.4830000000000001},"min":{"1min":1,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.996,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.71},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.94399999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":1.7809999999999999,"5min":2.089,"15min":2.089},"last":1.2989999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.613,"15min":1.6180000000000001},"min":{"1min":1.327,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":2.1030000000000002,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.806},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.468,"15min":1.512},"min":{"1min":1.0569999999999999,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":2.0310000000000001,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.3540000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5660000000000001},"min":{"1min":1.157,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":1.9039999999999999,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.5740000000000001,"15min":1.5980000000000001},"min":{"1min":1.1930000000000001,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":2.1499999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.579}]}]},{"osd":3,"up_from":15,"seq":64424509550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44560,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327280,"statfs":{"total":96636764160,"available":96591134720,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34300000000000003},"max":{"1min":1.4099999999999999,"5min":1.867,"15min":1.867},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.04,"15min":1.018},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4790000000000001,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":1.8360000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0369999999999999,"15min":1.0389999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.534,"5min":1.966,"15min":3.71},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1459999999999999,"15min":1.1020000000000001},"min":{"1min":0.59999999999999998,"5min":0.51000000000000001,"15min":0.46200000000000002},"max":{"1min":1.738,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":1.163}]},{"osd":2,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1679999999999999,"15min":1.1000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.772},{"interface":"front","average":{"1min":1.016,"5min":1.1679999999999999,"15min":1.26},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.571,"5min":1.857,"15min":3.5710000000000002},"last":1.218}]},{"osd":4,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3500000000000001,"15min":1.4039999999999999},"min":{"1min":0.754,"5min":0.63600000000000001,"15min":0.497},"max":{"1min":2.0779999999999998,"5min":2.411,"15min":3.5030000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3979999999999999,"15min":1.3480000000000001},"min":{"1min":0.83499999999999996,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.615,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.333}]},{"osd":5,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.83199999999999996,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.627,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.46,"15min":1.496},"min":{"1min":0.82999999999999996,"5min":0.66100000000000003,"15min":0.53700000000000003},"max":{"1min":2.21,"5min":2.673,"15min":3.3809999999999998},"last":1.4830000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.486,"15min":1.476},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.67800000000000005},"max":{"1min":1.794,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.589},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.4870000000000001,"15min":1.5629999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.778,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.6879999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.5700000000000001,"15min":1.5960000000000001},"min":{"1min":0.92900000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1579999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.603,"15min":1.55},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.73099999999999998},"max":{"1min":1.8959999999999999,"5min":2.75,"15min":2.75},"last":1.893}]}]},{"osd":2,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50128,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.92800000000000005,"15min":0.91700000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2889999999999999,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.02,"15min":0.92700000000000005},"min":{"1min":0.51700000000000002,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":1.7330000000000001,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":1.452}]},{"osd":1,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":1.008,"15min":0.95199999999999996},"min":{"1min":0.47699999999999998,"5min":0.42599999999999999,"15min":0.376},"max":{"1min":1.3180000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":0.627},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.1870000000000001,"15min":1.1279999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5980000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.0349999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.2070000000000001,"15min":1.2210000000000001},"min":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":1.6899999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.625,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.524,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.5169999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.95899999999999996,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.7070000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.4019999999999999,"15min":1.393},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.63100000000000001},"max":{"1min":1.873,"5min":3.121,"15min":3.121},"last":0.97499999999999998}]},{"osd":5,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.4430000000000001,"15min":1.4610000000000001},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.605,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.597,"15min":1.5549999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.80500000000000005},"max":{"1min":1.786,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.6599999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.498,"15min":1.5249999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7629999999999999,"5min":3.395,"15min":4.7569999999999997},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.5940000000000001,"15min":1.581},"min":{"1min":0.84299999999999997,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":1.7509999999999999,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.5880000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5349999999999999,"15min":1.5760000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6599999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.629},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.617,"15min":1.603},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.7929999999999999,"5min":3.456,"15min":3.456},"last":1.399}]}]},{"osd":1,"up_from":15,"seq":64424509549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.60799999999999998,"5min":0.40500000000000003,"15min":0.35799999999999998},"max":{"1min":1.369,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.99399999999999999,"15min":0.96099999999999997},"min":{"1min":0.61799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6379999999999999,"5min":1.869,"15min":1.869},"last":1.028}]},{"osd":2,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.117,"15min":1.008},"min":{"1min":0.66900000000000004,"5min":0.44800000000000001,"15min":0.40000000000000002},"max":{"1min":1.6339999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.133,"15min":1.024},"min":{"1min":0.75900000000000001,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":1.619,"5min":2.915,"15min":2.915},"last":1.1599999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.839,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.036,"5min":1.083,"15min":1.0600000000000001},"min":{"1min":0.56100000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.601,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3120000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.361,"15min":1.3140000000000001},"min":{"1min":0.95699999999999996,"5min":0.76200000000000001,"15min":0.58599999999999997},"max":{"1min":1.6990000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.4730000000000001,"15min":1.395},"min":{"1min":1.0029999999999999,"5min":0.69599999999999995,"15min":0.60599999999999998},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.417}]},{"osd":5,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.448,"15min":1.4139999999999999},"min":{"1min":0.874,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.274},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.47,"15min":1.4179999999999999},"min":{"1min":1.3400000000000001,"5min":0.78600000000000003,"15min":0.70899999999999996},"max":{"1min":1.9339999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.125}]},{"osd":6,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.367,"15min":1.401},"min":{"1min":0.79800000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.702,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.379},{"interface":"front","average":{"1min":1.526,"5min":1.5629999999999999,"15min":1.5069999999999999},"min":{"1min":0.93799999999999994,"5min":0.85699999999999998,"15min":0.752},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.534}]},{"osd":7,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.544,"15min":1.5049999999999999},"min":{"1min":1.292,"5min":0.88,"15min":0.82899999999999996},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.498,"15min":1.4590000000000001},"min":{"1min":0.90900000000000003,"5min":0.80400000000000005,"15min":0.63500000000000001},"max":{"1min":1.891,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.452}]}]},{"osd":0,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.56100000000000005,"5min":0.36699999999999999,"15min":0.32100000000000001},"max":{"1min":1.464,"5min":1.74,"15min":1.74},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.589,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.7230000000000001,"5min":2.105,"15min":2.105},"last":1.226},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.371},"max":{"1min":1.4179999999999999,"5min":1.968,"15min":2.1669999999999998},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1559999999999999,"15min":1.105},"min":{"1min":0.66700000000000004,"5min":0.54000000000000004,"15min":0.45200000000000001},"max":{"1min":1.571,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.3009999999999999,"15min":1.3140000000000001},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.8460000000000001,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.0780000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.425,"15min":1.363},"min":{"1min":0.81299999999999994,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.786,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.202},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3680000000000001,"15min":1.4159999999999999},"min":{"1min":0.81999999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5309999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.369}]},{"osd":5,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.492,"15min":1.528},"min":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.611,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.468,"15min":1.482},"min":{"1min":0.91700000000000004,"5min":0.78600000000000003,"15min":0.747},"max":{"1min":1.746,"5min":2.706,"15min":2.706},"last":1.1759999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.597,"15min":1.5329999999999999},"min":{"1min":1.1020000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.669,"5min":4.617,"15min":4.617},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.5860000000000001,"15min":1.573},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.72899999999999998},"max":{"1min":2.0249999999999999,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.4059999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.643,"15min":1.6619999999999999},"min":{"1min":0.85799999999999998,"5min":0.82299999999999995,"15min":0.76700000000000002},"max":{"1min":1.9430000000000001,"5min":2.742,"15min":3.3540000000000001},"last":1.512},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.6699999999999999,"15min":1.6180000000000001},"min":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.77400000000000002},"max":{"1min":1.776,"5min":3.976,"15min":3.976},"last":1.3069999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45972,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325868,"statfs":{"total":96636764160,"available":96589688832,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.105,"15min":1.113},"min":{"1min":0.71399999999999997,"5min":0.65600000000000003,"15min":0.59899999999999998},"max":{"1min":1.9430000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":1.371},{"interface":"front","average":{"1min":1.3,"5min":1.1359999999999999,"15min":1.0880000000000001},"min":{"1min":1.0029999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.266},"last":1.5229999999999999}]},{"osd":1,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.198,"15min":1.0840000000000001},"min":{"1min":0.77900000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7729999999999999,"5min":1.784,"15min":2.0339999999999998},"last":1.141},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2649999999999999,"15min":1.2350000000000001},"min":{"1min":0.83899999999999997,"5min":0.70099999999999996,"15min":0.64700000000000002},"max":{"1min":1.677,"5min":1.893,"15min":2.1899999999999999},"last":1.478}]},{"osd":2,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.337,"15min":1.306},"min":{"1min":1.0229999999999999,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8600000000000001,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.335,"15min":1.2390000000000001},"min":{"1min":1.0449999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.7030000000000001,"5min":1.79,"15min":1.79},"last":1.425}]},{"osd":3,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.351,"15min":1.2709999999999999},"min":{"1min":0.90200000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.379,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.516,"15min":1.4670000000000001},"min":{"1min":1.004,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.8159999999999998},"last":1.075}]},{"osd":4,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.3660000000000001,"15min":1.3340000000000001},"min":{"1min":0.93400000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":3.0190000000000001},"last":1.29},{"interface":"front","average":{"1min":1.494,"5min":1.429,"15min":1.385},"min":{"1min":0.96199999999999997,"5min":0.439,"15min":0.439},"max":{"1min":2.5760000000000001,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.246}]},{"osd":5,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5289999999999999,"15min":1.534},"min":{"1min":0.995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.9620000000000002,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.6659999999999999,"5min":1.542,"15min":1.494},"min":{"1min":1.1459999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.3620000000000001},"last":1.6100000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.8280000000000001,"5min":1.653,"15min":1.591},"min":{"1min":1.409,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":3.383,"5min":3.383,"15min":3.383},"last":1.675},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5840000000000001,"15min":1.5229999999999999},"min":{"1min":1.0669999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.5990000000000002,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.454}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:04:11.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-13T11:04:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.955+0000 7f18fb48a640 1 Processor -- start 2023-11-13T11:04:11.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.955+0000 7f18fb48a640 1 -- start start 2023-11-13T11:04:11.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.957+0000 7f18fb48a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40cb190 0x7f18f415b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.957+0000 7f18fb48a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18f406a190 0x7f18f406a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.957+0000 7f18fb48a640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f4158c20 con 0x7f18f40716b0 2023-11-13T11:04:11.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.957+0000 7f18fb48a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f40b1830 con 0x7f18f406a190 2023-11-13T11:04:11.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.957+0000 7f18fb48a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f406ab10 con 0x7f18f40cb190 2023-11-13T11:04:11.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.957+0000 7f18fac89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18f406a190 0x7f18f406a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.957+0000 7f18fac89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18f406a190 0x7f18f406a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51168/0 (socket says 172.21.15.16:51168) 2023-11-13T11:04:11.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.957+0000 7f18fac89640 1 -- 172.21.15.16:0/170334297 learned_addr learned my addr 172.21.15.16:0/170334297 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.957+0000 7f18fa488640 1 --2- 172.21.15.16:0/170334297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40cb190 0x7f18f415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.958+0000 7f18fac89640 1 -- 172.21.15.16:0/170334297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40cb190 msgr2=0x7f18f415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.958+0000 7f18fac89640 1 --2- 172.21.15.16:0/170334297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40cb190 0x7f18f415b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.958+0000 7f18fac89640 1 -- 172.21.15.16:0/170334297 >> v1:172.21.15.184:6789/0 conn(0x7f18f40716b0 legacy=0x7f18f406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:11.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.958+0000 7f18fac89640 1 -- 172.21.15.16:0/170334297 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f415b780 con 0x7f18f406a190 2023-11-13T11:04:11.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.958+0000 7f18fa488640 1 --2- 172.21.15.16:0/170334297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40cb190 0x7f18f415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.959+0000 7f18fac89640 1 --2- 172.21.15.16:0/170334297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 0x7f18f406a5d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009e50 tx=0x7f18ec02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6052bee8336dd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.959+0000 7f18f9486640 1 -- 172.21.15.16:0/170334297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec02ca80 con 0x7f18f406a190 2023-11-13T11:04:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.959+0000 7f18f9486640 1 -- 172.21.15.16:0/170334297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec02cbe0 con 0x7f18f406a190 2023-11-13T11:04:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.959+0000 7f18f9486640 1 -- 172.21.15.16:0/170334297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec0343f0 con 0x7f18f406a190 2023-11-13T11:04:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.960+0000 7f18fb48a640 1 -- 172.21.15.16:0/170334297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 msgr2=0x7f18f406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.960+0000 7f18fb48a640 1 --2- 172.21.15.16:0/170334297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 0x7f18f406a5d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009e50 tx=0x7f18ec02bee0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.960+0000 7f18fb48a640 1 -- 172.21.15.16:0/170334297 shutdown_connections 2023-11-13T11:04:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.960+0000 7f18fb48a640 1 --2- 172.21.15.16:0/170334297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 0x7f18f406a5d0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.960+0000 7f18fb48a640 1 --2- 172.21.15.16:0/170334297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40cb190 0x7f18f415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.960+0000 7f18fb48a640 1 -- 172.21.15.16:0/170334297 >> 172.21.15.16:0/170334297 conn(0x7f18f40b04a0 msgr2=0x7f18f40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.960+0000 7f18fb48a640 1 -- 172.21.15.16:0/170334297 shutdown_connections 2023-11-13T11:04:11.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.960+0000 7f18fb48a640 1 -- 172.21.15.16:0/170334297 wait complete. 2023-11-13T11:04:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.962+0000 7f18fb48a640 1 Processor -- start 2023-11-13T11:04:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.962+0000 7f18fb48a640 1 -- start start 2023-11-13T11:04:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.963+0000 7f18fb48a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 0x7f18f41f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.963+0000 7f18fb48a640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40716b0 0x7f18f415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.963+0000 7f18fb48a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f40cb190 0x7f18f4154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.963+0000 7f18fb48a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f415dfd0 con 0x7f18f406a190 2023-11-13T11:04:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.963+0000 7f18fb48a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f415d3c0 con 0x7f18f40cb190 2023-11-13T11:04:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.963+0000 7f18fb48a640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f415d6a0 con 0x7f18f40716b0 2023-11-13T11:04:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.963+0000 7f18fa488640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 0x7f18f41f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.964+0000 7f18fa488640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 0x7f18f41f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51170/0 (socket says 172.21.15.16:51170) 2023-11-13T11:04:11.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.964+0000 7f18fa488640 1 -- 172.21.15.16:0/1213546416 learned_addr learned my addr 172.21.15.16:0/1213546416 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:11.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.964+0000 7f18fa488640 1 -- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40716b0 msgr2=0x7f18f415c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.964+0000 7f18fa488640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40716b0 0x7f18f415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.964+0000 7f18fa488640 1 -- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f40cb190 msgr2=0x7f18f4154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:11.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.964+0000 7f18fa488640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f40cb190 0x7f18f4154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:11.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.964+0000 7f18fa488640 1 -- 172.21.15.16:0/1213546416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec009b30 con 0x7f18f406a190 2023-11-13T11:04:11.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.964+0000 7f18fa488640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 0x7f18f41f1ce0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f18f000c7f0 tx=0x7f18f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:11.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.965+0000 7f18eaffd640 1 -- 172.21.15.16:0/1213546416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0007820 con 0x7f18f406a190 2023-11-13T11:04:11.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.965+0000 7f18eaffd640 1 -- 172.21.15.16:0/1213546416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f0007980 con 0x7f18f406a190 2023-11-13T11:04:11.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.965+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f415d980 con 0x7f18f406a190 2023-11-13T11:04:11.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.965+0000 7f18eaffd640 1 -- 172.21.15.16:0/1213546416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0007cf0 con 0x7f18f406a190 2023-11-13T11:04:11.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.965+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f41f2220 con 0x7f18f406a190 2023-11-13T11:04:11.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.966+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f404e4d0 con 0x7f18f406a190 2023-11-13T11:04:11.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.967+0000 7f18eaffd640 1 -- 172.21.15.16:0/1213546416 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18f00113f0 con 0x7f18f406a190 2023-11-13T11:04:11.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.968+0000 7f18eaffd640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:11.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.968+0000 7f18eaffd640 1 -- 172.21.15.16:0/1213546416 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f18f0089860 con 0x7f18f406a190 2023-11-13T11:04:11.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.968+0000 7f18f9c87640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.970+0000 7f18f9c87640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f18f415c970 tx=0x7f18e40079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:11.972+0000 7f18eaffd640 1 -- 172.21.15.16:0/1213546416 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18f00549d0 con 0x7f18f406a190 2023-11-13T11:04:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.165+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18f40b3100 con 0x7f18f406a190 2023-11-13T11:04:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.167+0000 7f18eaffd640 1 -- 172.21.15.16:0/1213546416 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+13334 (secure 0 0 0) 0x7f18f0057ff0 con 0x7f18f406a190 2023-11-13T11:04:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:12.173 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":76,"fsid":"130fee2f-fe7b-4ef9-80e1-9d0c3fe381f9","created":"2023-11-13T10:55:01.003667+0000","modified":"2023-11-13T11:04:06.120008+0000","last_up_change":"2023-11-13T10:55:13.875746+0000","last_in_change":"2023-11-13T10:55:04.873821+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-13T10:55:14.038047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24c47695-0622-4dd1-a2a3-8337282f0c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6811","nonce":4203265327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6815","nonce":4203265327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6823","nonce":4203265327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":4203265327},{"type":"v1","addr":"172.21.15.16:6819","nonce":4203265327}]},"public_addr":"172.21.15.16:6811/4203265327","cluster_addr":"172.21.15.16:6815/4203265327","heartbeat_back_addr":"172.21.15.16:6823/4203265327","heartbeat_front_addr":"172.21.15.16:6819/4203265327","state":["exists","up"]},{"osd":1,"uuid":"83b02569-c028-44b6-9662-38d2cf5099b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6801","nonce":2377015218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6803","nonce":2377015218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6807","nonce":2377015218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2377015218},{"type":"v1","addr":"172.21.15.16:6805","nonce":2377015218}]},"public_addr":"172.21.15.16:6801/2377015218","cluster_addr":"172.21.15.16:6803/2377015218","heartbeat_back_addr":"172.21.15.16:6807/2377015218","heartbeat_front_addr":"172.21.15.16:6805/2377015218","state":["exists","up"]},{"osd":2,"uuid":"3fb0a6a5-4391-4978-a71d-a08a66c60637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6810","nonce":3899450571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6814","nonce":3899450571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6822","nonce":3899450571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3899450571},{"type":"v1","addr":"172.21.15.16:6818","nonce":3899450571}]},"public_addr":"172.21.15.16:6810/3899450571","cluster_addr":"172.21.15.16:6814/3899450571","heartbeat_back_addr":"172.21.15.16:6822/3899450571","heartbeat_front_addr":"172.21.15.16:6818/3899450571","state":["exists","up"]},{"osd":3,"uuid":"e6dc6d27-7fa0-4ad5-9c9d-dbc555379cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6825","nonce":3562446965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6827","nonce":3562446965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6831","nonce":3562446965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3562446965},{"type":"v1","addr":"172.21.15.16:6829","nonce":3562446965}]},"public_addr":"172.21.15.16:6825/3562446965","cluster_addr":"172.21.15.16:6827/3562446965","heartbeat_back_addr":"172.21.15.16:6831/3562446965","heartbeat_front_addr":"172.21.15.16:6829/3562446965","state":["exists","up"]},{"osd":4,"uuid":"c52a2136-acd5-4f9c-90da-d67149500b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6801","nonce":2936315873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6803","nonce":2936315873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6807","nonce":2936315873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2936315873},{"type":"v1","addr":"172.21.15.184:6805","nonce":2936315873}]},"public_addr":"172.21.15.184:6801/2936315873","cluster_addr":"172.21.15.184:6803/2936315873","heartbeat_back_addr":"172.21.15.184:6807/2936315873","heartbeat_front_addr":"172.21.15.184:6805/2936315873","state":["exists","up"]},{"osd":5,"uuid":"a9764441-fde9-439e-8bbd-76f0f53f31d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6809","nonce":1150854014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6811","nonce":1150854014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6815","nonce":1150854014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1150854014},{"type":"v1","addr":"172.21.15.184:6813","nonce":1150854014}]},"public_addr":"172.21.15.184:6809/1150854014","cluster_addr":"172.21.15.184:6811/1150854014","heartbeat_back_addr":"172.21.15.184:6815/1150854014","heartbeat_front_addr":"172.21.15.184:6813/1150854014","state":["exists","up"]},{"osd":6,"uuid":"21cbef70-c6b7-4ca0-ab55-1fc2ebdde49c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6817","nonce":1720475067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6819","nonce":1720475067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6823","nonce":1720475067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":1720475067},{"type":"v1","addr":"172.21.15.184:6821","nonce":1720475067}]},"public_addr":"172.21.15.184:6817/1720475067","cluster_addr":"172.21.15.184:6819/1720475067","heartbeat_back_addr":"172.21.15.184:6823/1720475067","heartbeat_front_addr":"172.21.15.184:6821/1720475067","state":["exists","up"]},{"osd":7,"uuid":"1ba7a631-0655-4a4a-b869-fbf4747f9ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6825","nonce":3047573298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6827","nonce":3047573298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6831","nonce":3047573298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":3047573298},{"type":"v1","addr":"172.21.15.184:6829","nonce":3047573298}]},"public_addr":"172.21.15.184:6825/3047573298","cluster_addr":"172.21.15.184:6827/3047573298","heartbeat_back_addr":"172.21.15.184:6831/3047573298","heartbeat_front_addr":"172.21.15.184:6829/3047573298","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.478256+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.406892+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.225958+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.486632+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.203300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.173302+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.282811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-13T10:55:10.043584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.184:6836/2632887594":"2023-11-14T11:04:02.753073+0000","172.21.15.184:6834/2632887594":"2023-11-14T11:04:02.753073+0000","172.21.15.184:6837/60343961":"2023-11-14T11:03:50.019659+0000","172.21.15.184:6835/60343961":"2023-11-14T11:03:50.019659+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-13T11:04:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.169+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 msgr2=0x7f18d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.169+0000 7f18fb48a640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f18f415c970 tx=0x7f18e40079b0 comp rx=0 tx=0).stop 2023-11-13T11:04:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.169+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 msgr2=0x7f18f41f1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.169+0000 7f18fb48a640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 0x7f18f41f1ce0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f18f000c7f0 tx=0x7f18f000ccb0 comp rx=0 tx=0).stop 2023-11-13T11:04:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.170+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 shutdown_connections 2023-11-13T11:04:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.170+0000 7f18fb48a640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f18d806ae40 0x7f18d806d300 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.170+0000 7f18fb48a640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18f40cb190 0x7f18f4154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.170+0000 7f18fb48a640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18f40716b0 0x7f18f415c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.170+0000 7f18fb48a640 1 --2- 172.21.15.16:0/1213546416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f406a190 0x7f18f41f1ce0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.170+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 >> 172.21.15.16:0/1213546416 conn(0x7f18f40b04a0 msgr2=0x7f18f415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.170+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 shutdown_connections 2023-11-13T11:04:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:12.170+0000 7f18fb48a640 1 -- 172.21.15.16:0/1213546416 wait complete. 2023-11-13T11:04:13.193 INFO:tasks.ceph:Scrubbing osd.0 2023-11-13T11:04:13.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-13T11:04:13.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.429+0000 7f1c7244f640 1 Processor -- start 2023-11-13T11:04:13.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.429+0000 7f1c7244f640 1 -- start start 2023-11-13T11:04:13.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c7244f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c6c15aaf0 0x7f1c6c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c7244f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0b18f0 0x7f1c6c071790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c7244f640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c0b1df0 con 0x7f1c6c0ce720 2023-11-13T11:04:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c7244f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6c0b20d0 con 0x7f1c6c15aaf0 2023-11-13T11:04:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c7244f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0b23b0 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c6bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c6c15aaf0 0x7f1c6c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c6bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c6c15aaf0 0x7f1c6c0ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55962/0 (socket says 172.21.15.16:55962) 2023-11-13T11:04:13.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c6bfff640 1 -- 172.21.15.16:0/2645038852 learned_addr learned my addr 172.21.15.16:0/2645038852 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:13.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/2645038852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0b18f0 0x7f1c6c071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c6bfff640 1 -- 172.21.15.16:0/2645038852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0b18f0 msgr2=0x7f1c6c071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:13.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/2645038852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0b18f0 0x7f1c6c071790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.431+0000 7f1c6bfff640 1 -- 172.21.15.16:0/2645038852 >> v1:172.21.15.184:6789/0 conn(0x7f1c6c0ce720 legacy=0x7f1c6c06ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:13.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.432+0000 7f1c6bfff640 1 -- 172.21.15.16:0/2645038852 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0b2690 con 0x7f1c6c15aaf0 2023-11-13T11:04:13.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.432+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/2645038852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0b18f0 0x7f1c6c071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.433+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/2645038852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c15aaf0 0x7f1c6c0ccc50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009e50 tx=0x7f1c5c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5b226aeaf81503 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.433+0000 7f1c6affd640 1 -- 172.21.15.16:0/2645038852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c02c9e0 con 0x7f1c6c15aaf0 2023-11-13T11:04:13.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.433+0000 7f1c6affd640 1 -- 172.21.15.16:0/2645038852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c02cb40 con 0x7f1c6c15aaf0 2023-11-13T11:04:13.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.433+0000 7f1c6affd640 1 -- 172.21.15.16:0/2645038852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0343f0 con 0x7f1c6c15aaf0 2023-11-13T11:04:13.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.434+0000 7f1c7244f640 1 -- 172.21.15.16:0/2645038852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c15aaf0 msgr2=0x7f1c6c0ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.434+0000 7f1c7244f640 1 --2- 172.21.15.16:0/2645038852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c15aaf0 0x7f1c6c0ccc50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009e50 tx=0x7f1c5c02be40 comp rx=0 tx=0).stop 2023-11-13T11:04:13.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.434+0000 7f1c7244f640 1 -- 172.21.15.16:0/2645038852 shutdown_connections 2023-11-13T11:04:13.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.434+0000 7f1c7244f640 1 --2- 172.21.15.16:0/2645038852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0b18f0 0x7f1c6c071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.434+0000 7f1c7244f640 1 --2- 172.21.15.16:0/2645038852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c15aaf0 0x7f1c6c0ccc50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.434+0000 7f1c7244f640 1 -- 172.21.15.16:0/2645038852 >> 172.21.15.16:0/2645038852 conn(0x7f1c6c06a190 msgr2=0x7f1c6c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.434+0000 7f1c7244f640 1 -- 172.21.15.16:0/2645038852 shutdown_connections 2023-11-13T11:04:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.434+0000 7f1c7244f640 1 -- 172.21.15.16:0/2645038852 wait complete. 2023-11-13T11:04:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.436+0000 7f1c7244f640 1 Processor -- start 2023-11-13T11:04:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.436+0000 7f1c7244f640 1 -- start start 2023-11-13T11:04:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.437+0000 7f1c7244f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b18f0 0x7f1c6c14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.437+0000 7f1c7244f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0ce720 0x7f1c6c1537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.437+0000 7f1c7244f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c6c15aaf0 0x7f1c6c150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.437+0000 7f1c7244f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c154140 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.437+0000 7f1c7244f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0aca80 con 0x7f1c6c15aaf0 2023-11-13T11:04:13.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.437+0000 7f1c7244f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0acd60 con 0x7f1c6c0ce720 2023-11-13T11:04:13.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.437+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b18f0 0x7f1c6c14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.437+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b18f0 0x7f1c6c14b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55972/0 (socket says 172.21.15.16:55972) 2023-11-13T11:04:13.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.437+0000 7f1c6bfff640 1 -- 172.21.15.16:0/3162449208 learned_addr learned my addr 172.21.15.16:0/3162449208 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:13.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.438+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0ce720 0x7f1c6c1537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.438+0000 7f1c709ec640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c6c15aaf0 0x7f1c6c150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.438+0000 7f1c6bfff640 1 -- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0ce720 msgr2=0x7f1c6c1537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.438+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0ce720 0x7f1c6c1537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.438+0000 7f1c6bfff640 1 -- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c6c15aaf0 msgr2=0x7f1c6c150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.438+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c6c15aaf0 0x7f1c6c150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.438+0000 7f1c6bfff640 1 -- 172.21.15.16:0/3162449208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c009b30 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.438+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c6c0ce720 0x7f1c6c1537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:13.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.438+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b18f0 0x7f1c6c14b4d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009e50 tx=0x7f1c5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:13.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.439+0000 7f1c68ff9640 1 -- 172.21.15.16:0/3162449208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c03e450 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.439+0000 7f1c68ff9640 1 -- 172.21.15.16:0/3162449208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c03e5f0 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.439+0000 7f1c68ff9640 1 -- 172.21.15.16:0/3162449208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c03e9a0 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.439+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c0ad040 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.439+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c14ba10 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.440+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c6c0cebb0 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.441+0000 7f1c68ff9640 1 -- 172.21.15.16:0/3162449208 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c5c03eb00 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.442+0000 7f1c68ff9640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c5806ae40 0x7f1c5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.442+0000 7f1c68ff9640 1 -- 172.21.15.16:0/3162449208 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f1c5c0ac5d0 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.442+0000 7f1c68ff9640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7f1c580711e0 0x7f1c58073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.442+0000 7f1c68ff9640 1 -- 172.21.15.16:0/3162449208 --> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c58073cf0 con 0x7f1c580711e0 2023-11-13T11:04:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.442+0000 7f1c68ff9640 1 -- 172.21.15.16:0/3162449208 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c5c0aebb0 con 0x7f1c6c0b18f0 2023-11-13T11:04:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.443+0000 7f1c709ec640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7f1c580711e0 0x7f1c58073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.443+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c5806ae40 0x7f1c5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.443+0000 7f1c709ec640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7f1c580711e0 0x7f1c58073620 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.444+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c5806ae40 0x7f1c5806d300 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1c60006fd0 tx=0x7f1c60008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.444+0000 7f1c68ff9640 1 -- 172.21.15.16:0/3162449208 <== osd.0 v2:172.21.15.16:6809/4203265327 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f1c58073cf0 con 0x7f1c580711e0 2023-11-13T11:04:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 --> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1c6c067090 con 0x7f1c580711e0 2023-11-13T11:04:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c68ff9640 1 -- 172.21.15.16:0/3162449208 <== osd.0 v2:172.21.15.16:6809/4203265327 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1c6c067090 con 0x7f1c580711e0 2023-11-13T11:04:13.465 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:04:13.465 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-13T11:04:13.466 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:04:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7f1c580711e0 msgr2=0x7f1c58073620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.16:6809/4203265327,v1:172.21.15.16:6811/4203265327] conn(0x7f1c580711e0 0x7f1c58073620 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c5806ae40 msgr2=0x7f1c5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f1c5806ae40 0x7f1c5806d300 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1c60006fd0 tx=0x7f1c60008040 comp rx=0 tx=0).stop 2023-11-13T11:04:13.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b18f0 msgr2=0x7f1c6c14b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 --2- 172.21.15.16:0/3162449208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b18f0 0x7f1c6c14b4d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009e50 tx=0x7f1c5c004a40 comp rx=0 tx=0).stop 2023-11-13T11:04:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c709ec640 1 -- 172.21.15.16:0/3162449208 reap_dead start 2023-11-13T11:04:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 shutdown_connections 2023-11-13T11:04:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 >> 172.21.15.16:0/3162449208 conn(0x7f1c6c06a190 msgr2=0x7f1c6c15bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 shutdown_connections 2023-11-13T11:04:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.460+0000 7f1c7244f640 1 -- 172.21.15.16:0/3162449208 wait complete. 2023-11-13T11:04:13.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-13T11:04:13.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.703+0000 7fc86ab4e640 1 Processor -- start 2023-11-13T11:04:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.703+0000 7fc86ab4e640 1 -- start start 2023-11-13T11:04:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc86ab4e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc864160340 0x7fc8640cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc86ab4e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8640d0820 con 0x7fc8640b5970 2023-11-13T11:04:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc86ab4e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8640ce1a0 con 0x7fc8640b3b70 2023-11-13T11:04:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc86ab4e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8640ce480 con 0x7fc864160340 2023-11-13T11:04:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc86934b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc8640b5970 0x7fc8640cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33822/0 (socket says 172.21.15.16:33822) 2023-11-13T11:04:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc86934b640 1 -- 172.21.15.16:0/995709289 learned_addr learned my addr 172.21.15.16:0/995709289 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc869b4c640 1 --2- 172.21.15.16:0/995709289 >> v2:172.21.15.184:3300/0 conn(0x7fc864160340 0x7fc8640cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635794907 0 0) 0x7fc8640d0820 con 0x7fc8640b5970 2023-11-13T11:04:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc850003520 con 0x7fc8640b5970 2023-11-13T11:04:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 109657898 0 0) 0x7fc850003520 con 0x7fc8640b5970 2023-11-13T11:04:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 >> v2:172.21.15.184:3300/0 conn(0x7fc864160340 msgr2=0x7fc8640cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc868b4a640 1 --2- 172.21.15.16:0/995709289 >> v2:172.21.15.184:3300/0 conn(0x7fc864160340 0x7fc8640cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.705+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 >> v1:172.21.15.184:6789/0 conn(0x7fc8640b3b70 legacy=0x7fc8640d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640ce760 con 0x7fc8640b5970 2023-11-13T11:04:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fc854001670 con 0x7fc8640b5970 2023-11-13T11:04:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 >> v1:172.21.15.16:6789/0 conn(0x7fc8640b5970 legacy=0x7fc8640cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc868b4a640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc850003970 0x7fc850003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc868b4a640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8500047c0 0x7fc850004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc868b4a640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850005180 0x7fc85000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640ce480 con 0x7fc850005180 2023-11-13T11:04:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000c020 con 0x7fc8500047c0 2023-11-13T11:04:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000c1e0 con 0x7fc850003970 2023-11-13T11:04:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc86a34d640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850005180 0x7fc85000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc86934b640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8500047c0 0x7fc850004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.706+0000 7fc869b4c640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc850003970 0x7fc850003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.707+0000 7fc86a34d640 1 -- 172.21.15.16:0/995709289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc850003970 msgr2=0x7fc850003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.707+0000 7fc86a34d640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc850003970 0x7fc850003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.707+0000 7fc86a34d640 1 -- 172.21.15.16:0/995709289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8500047c0 msgr2=0x7fc850004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.707+0000 7fc86a34d640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8500047c0 0x7fc850004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.707+0000 7fc86a34d640 1 -- 172.21.15.16:0/995709289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850003520 con 0x7fc850005180 2023-11-13T11:04:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.708+0000 7fc86a34d640 1 -- 172.21.15.16:0/995709289 reap_dead start 2023-11-13T11:04:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.708+0000 7fc86a34d640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850005180 0x7fc85000b980 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc85c001e20 tx=0x7fc8500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd49b4cdeed7837e server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:13.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.708+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c003b70 con 0x7fc850005180 2023-11-13T11:04:13.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.708+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c003cd0 con 0x7fc850005180 2023-11-13T11:04:13.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.708+0000 7fc868b4a640 1 -- 172.21.15.16:0/995709289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c005310 con 0x7fc850005180 2023-11-13T11:04:13.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.709+0000 7fc86ab4e640 1 -- 172.21.15.16:0/995709289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850005180 msgr2=0x7fc85000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.709+0000 7fc86ab4e640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850005180 0x7fc85000b980 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc85c001e20 tx=0x7fc8500057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.709+0000 7fc86ab4e640 1 -- 172.21.15.16:0/995709289 shutdown_connections 2023-11-13T11:04:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.709+0000 7fc86ab4e640 1 --2- 172.21.15.16:0/995709289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850005180 0x7fc85000b980 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.709+0000 7fc86ab4e640 1 -- 172.21.15.16:0/995709289 >> 172.21.15.16:0/995709289 conn(0x7fc86409be00 msgr2=0x7fc8640d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.709+0000 7fc86ab4e640 1 -- 172.21.15.16:0/995709289 shutdown_connections 2023-11-13T11:04:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.709+0000 7fc86ab4e640 1 -- 172.21.15.16:0/995709289 wait complete. 2023-11-13T11:04:13.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86ab4e640 1 Processor -- start 2023-11-13T11:04:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86ab4e640 1 -- start start 2023-11-13T11:04:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86ab4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc850005180 0x7fc8640ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86ab4e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86415aac0 0x7fc8640b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86ab4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86415af40 0x7fc8640b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86ab4e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86415a130 con 0x7fc86415af40 2023-11-13T11:04:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86ab4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640cc7a0 con 0x7fc850005180 2023-11-13T11:04:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86ab4e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640cbb90 con 0x7fc86415aac0 2023-11-13T11:04:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86a34d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86415af40 0x7fc8640b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86a34d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86415af40 0x7fc8640b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55996/0 (socket says 172.21.15.16:55996) 2023-11-13T11:04:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86a34d640 1 -- 172.21.15.16:0/3851195443 learned_addr learned my addr 172.21.15.16:0/3851195443 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc869b4c640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc850005180 0x7fc8640ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.712+0000 7fc86934b640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86415aac0 0x7fc8640b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc86a34d640 1 -- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86415aac0 msgr2=0x7fc8640b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc86a34d640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86415aac0 0x7fc8640b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc86a34d640 1 -- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc850005180 msgr2=0x7fc8640ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc86a34d640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc850005180 0x7fc8640ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc86a34d640 1 -- 172.21.15.16:0/3851195443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640cbe70 con 0x7fc86415af40 2023-11-13T11:04:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc869b4c640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc850005180 0x7fc8640ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:04:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc86a34d640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86415af40 0x7fc8640b2d40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc85c001f40 tx=0x7fc85c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc85a7fc640 1 -- 172.21.15.16:0/3851195443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c006d50 con 0x7fc86415af40 2023-11-13T11:04:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc85a7fc640 1 -- 172.21.15.16:0/3851195443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c007030 con 0x7fc86415af40 2023-11-13T11:04:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc85a7fc640 1 -- 172.21.15.16:0/3851195443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0073c0 con 0x7fc86415af40 2023-11-13T11:04:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.713+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8640cc150 con 0x7fc86415af40 2023-11-13T11:04:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.714+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8640ccc30 con 0x7fc86415af40 2023-11-13T11:04:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.715+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86404e4d0 con 0x7fc86415af40 2023-11-13T11:04:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.716+0000 7fc85a7fc640 1 -- 172.21.15.16:0/3851195443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc85c007520 con 0x7fc86415af40 2023-11-13T11:04:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.717+0000 7fc85a7fc640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc84806ae40 0x7fc84806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.717+0000 7fc85a7fc640 1 -- 172.21.15.16:0/3851195443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fc85c0ab010 con 0x7fc86415af40 2023-11-13T11:04:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.718+0000 7fc869b4c640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc84806ae40 0x7fc84806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.719+0000 7fc869b4c640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc84806ae40 0x7fc84806d300 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc860009a50 tx=0x7fc860008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.721+0000 7fc85a7fc640 1 -- 172.21.15.16:0/3851195443 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc85c0762b0 con 0x7fc86415af40 2023-11-13T11:04:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.922+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8640cd730 con 0x7fc84806ae40 2023-11-13T11:04:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.924+0000 7fc85a7fc640 1 -- 172.21.15.16:0/3851195443 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc860013a20 con 0x7fc84806ae40 2023-11-13T11:04:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2023-11-13T11:04:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc84806ae40 msgr2=0x7fc84806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc84806ae40 0x7fc84806d300 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc860009a50 tx=0x7fc860008040 comp rx=0 tx=0).stop 2023-11-13T11:04:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86415af40 msgr2=0x7fc8640b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86415af40 0x7fc8640b2d40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc85c001f40 tx=0x7fc85c0014a0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 shutdown_connections 2023-11-13T11:04:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fc84806ae40 0x7fc84806d300 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86415af40 0x7fc8640b2d40 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86415aac0 0x7fc8640b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 --2- 172.21.15.16:0/3851195443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc850005180 0x7fc8640ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 >> 172.21.15.16:0/3851195443 conn(0x7fc86409be00 msgr2=0x7fc8640d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.926+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 shutdown_connections 2023-11-13T11:04:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:13.927+0000 7fc86ab4e640 1 -- 172.21.15.16:0/3851195443 wait complete. 2023-11-13T11:04:13.952 INFO:tasks.ceph:Scrubbing osd.1 2023-11-13T11:04:13.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-13T11:04:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.177+0000 7f6da2caa640 1 Processor -- start 2023-11-13T11:04:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.177+0000 7f6da2caa640 1 -- start start 2023-11-13T11:04:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.178+0000 7f6da2caa640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d9c0ce4c0 0x7f6d9c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.178+0000 7f6da2caa640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9c0cf1f0 con 0x7f6d9c0b66b0 2023-11-13T11:04:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.178+0000 7f6da2caa640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9c0b5bd0 con 0x7f6d9c0cf580 2023-11-13T11:04:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.178+0000 7f6da2caa640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9c0b59c0 con 0x7f6d9c0ce4c0 2023-11-13T11:04:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da24a9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6d9c0b66b0 0x7f6d9c0b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33836/0 (socket says 172.21.15.16:33836) 2023-11-13T11:04:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da24a9640 1 -- 172.21.15.16:0/559417943 learned_addr learned my addr 172.21.15.16:0/559417943 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da14a7640 1 --2- 172.21.15.16:0/559417943 >> v2:172.21.15.184:3300/0 conn(0x7f6d9c0ce4c0 0x7f6d9c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414787276 0 0) 0x7f6d9c0cf1f0 con 0x7f6d9c0b66b0 2023-11-13T11:04:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d88003520 con 0x7f6d9c0b66b0 2023-11-13T11:04:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1620737706 0 0) 0x7f6d88003520 con 0x7f6d9c0b66b0 2023-11-13T11:04:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 >> v2:172.21.15.184:3300/0 conn(0x7f6d9c0ce4c0 msgr2=0x7f6d9c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 --2- 172.21.15.16:0/559417943 >> v2:172.21.15.184:3300/0 conn(0x7f6d9c0ce4c0 0x7f6d9c0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 >> v1:172.21.15.184:6789/0 conn(0x7f6d9c0cf580 legacy=0x7f6d9c0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0b5eb0 con 0x7f6d9c0b66b0 2023-11-13T11:04:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f6d940019f0 con 0x7f6d9c0b66b0 2023-11-13T11:04:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 >> v1:172.21.15.16:6789/0 conn(0x7f6d9c0b66b0 legacy=0x7f6d9c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d88003970 0x7f6d88003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d880047c0 0x7f6d88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0b59c0 con 0x7f6d88005180 2023-11-13T11:04:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8800c020 con 0x7f6d880047c0 2023-11-13T11:04:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8800c1e0 con 0x7f6d88003970 2023-11-13T11:04:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.179+0000 7f6da24a9640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.181+0000 7f6da14a7640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d880047c0 0x7f6d88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.181+0000 7f6da24a9640 1 -- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d88003970 msgr2=0x7f6d88003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.181+0000 7f6da1ca8640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d88003970 0x7f6d88003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.181+0000 7f6da24a9640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d88003970 0x7f6d88003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.181+0000 7f6da24a9640 1 -- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d880047c0 msgr2=0x7f6d88004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.181+0000 7f6da24a9640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d880047c0 0x7f6d88004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.181+0000 7f6da24a9640 1 -- 172.21.15.16:0/559417943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88003520 con 0x7f6d88005180 2023-11-13T11:04:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.181+0000 7f6da14a7640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d880047c0 0x7f6d88004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.181+0000 7f6da1ca8640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d88003970 0x7f6d88003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.182+0000 7f6da24a9640 1 -- 172.21.15.16:0/559417943 reap_dead start 2023-11-13T11:04:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.182+0000 7f6da24a9640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d8800b980 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400a830 tx=0x7f6d880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0aa1b4d3d27a15f server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.185+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d940072d0 con 0x7f6d88005180 2023-11-13T11:04:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.185+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9402f070 con 0x7f6d88005180 2023-11-13T11:04:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.185+0000 7f6da0ca6640 1 -- 172.21.15.16:0/559417943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d940076b0 con 0x7f6d88005180 2023-11-13T11:04:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.185+0000 7f6da2caa640 1 -- 172.21.15.16:0/559417943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 msgr2=0x7f6d8800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.185+0000 7f6da2caa640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d8800b980 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400a830 tx=0x7f6d880057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.185+0000 7f6da2caa640 1 -- 172.21.15.16:0/559417943 shutdown_connections 2023-11-13T11:04:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.185+0000 7f6da2caa640 1 --2- 172.21.15.16:0/559417943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d8800b980 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.185+0000 7f6da2caa640 1 -- 172.21.15.16:0/559417943 >> 172.21.15.16:0/559417943 conn(0x7f6d9c0d2b70 msgr2=0x7f6d9c0d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.185+0000 7f6da2caa640 1 -- 172.21.15.16:0/559417943 shutdown_connections 2023-11-13T11:04:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.186+0000 7f6da2caa640 1 -- 172.21.15.16:0/559417943 wait complete. 2023-11-13T11:04:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.186+0000 7f6da2caa640 1 Processor -- start 2023-11-13T11:04:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.186+0000 7f6da2caa640 1 -- start start 2023-11-13T11:04:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.188+0000 7f6da2caa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d9c1e7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.188+0000 7f6da2caa640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d9c0cc850 0x7f6d9c148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da2caa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d9c0c9d90 0x7f6d9c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da2caa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1e7550 con 0x7f6d88005180 2023-11-13T11:04:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da2caa640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0cc3c0 con 0x7f6d9c0c9d90 2023-11-13T11:04:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da2caa640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0cb530 con 0x7f6d9c0cc850 2023-11-13T11:04:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da1ca8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d9c1e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da1ca8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d9c1e7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56014/0 (socket says 172.21.15.16:56014) 2023-11-13T11:04:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da1ca8640 1 -- 172.21.15.16:0/3234420718 learned_addr learned my addr 172.21.15.16:0/3234420718 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da1ca8640 1 -- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d9c0cc850 msgr2=0x7f6d9c148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da14a7640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d9c0cc850 0x7f6d9c148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da24a9640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d9c0c9d90 0x7f6d9c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da1ca8640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d9c0cc850 0x7f6d9c148620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da1ca8640 1 -- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d9c0c9d90 msgr2=0x7f6d9c158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da1ca8640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d9c0c9d90 0x7f6d9c158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.189+0000 7f6da1ca8640 1 -- 172.21.15.16:0/3234420718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0cb7b0 con 0x7f6d88005180 2023-11-13T11:04:14.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.190+0000 7f6da1ca8640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d9c1e7010 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800d770 tx=0x7f6d9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.191+0000 7f6d927fc640 1 -- 172.21.15.16:0/3234420718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98005270 con 0x7f6d88005180 2023-11-13T11:04:14.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.191+0000 7f6d927fc640 1 -- 172.21.15.16:0/3234420718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d980053d0 con 0x7f6d88005180 2023-11-13T11:04:14.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.191+0000 7f6d927fc640 1 -- 172.21.15.16:0/3234420718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800b890 con 0x7f6d88005180 2023-11-13T11:04:14.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.191+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c0cba90 con 0x7f6d88005180 2023-11-13T11:04:14.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.191+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c0cc050 con 0x7f6d88005180 2023-11-13T11:04:14.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.191+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d9c0cf960 con 0x7f6d88005180 2023-11-13T11:04:14.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.193+0000 7f6d927fc640 1 -- 172.21.15.16:0/3234420718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d9800b9f0 con 0x7f6d88005180 2023-11-13T11:04:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.194+0000 7f6d927fc640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8006ae40 0x7f6d8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.194+0000 7f6d927fc640 1 -- 172.21.15.16:0/3234420718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f6d98089310 con 0x7f6d88005180 2023-11-13T11:04:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.195+0000 7f6d927fc640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f6d800711e0 0x7f6d80073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.195+0000 7f6d927fc640 1 -- 172.21.15.16:0/3234420718 --> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d80073cf0 con 0x7f6d800711e0 2023-11-13T11:04:14.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.195+0000 7f6d927fc640 1 -- 172.21.15.16:0/3234420718 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9808b9d0 con 0x7f6d88005180 2023-11-13T11:04:14.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.195+0000 7f6da24a9640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f6d800711e0 0x7f6d80073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.195+0000 7f6da14a7640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8006ae40 0x7f6d8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.195+0000 7f6da24a9640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f6d800711e0 0x7f6d80073620 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.195+0000 7f6da14a7640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8006ae40 0x7f6d8006d300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0cccb0 tx=0x7f6d8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.195+0000 7f6d927fc640 1 -- 172.21.15.16:0/3234420718 <== osd.1 v2:172.21.15.16:6800/2377015218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f6d80073cf0 con 0x7f6d800711e0 2023-11-13T11:04:14.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.210+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 --> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6d9c0cc500 con 0x7f6d800711e0 2023-11-13T11:04:14.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.210+0000 7f6d927fc640 1 -- 172.21.15.16:0/3234420718 <== osd.1 v2:172.21.15.16:6800/2377015218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6d94001730 con 0x7f6d800711e0 2023-11-13T11:04:14.216 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:04:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-13T11:04:14.217 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:04:14.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f6d800711e0 msgr2=0x7f6d80073620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.16:6800/2377015218,v1:172.21.15.16:6801/2377015218] conn(0x7f6d800711e0 0x7f6d80073620 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8006ae40 msgr2=0x7f6d8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6d8006ae40 0x7f6d8006d300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0cccb0 tx=0x7f6d8c008040 comp rx=0 tx=0).stop 2023-11-13T11:04:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 msgr2=0x7f6d9c1e7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 --2- 172.21.15.16:0/3234420718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88005180 0x7f6d9c1e7010 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800d770 tx=0x7f6d9800dc30 comp rx=0 tx=0).stop 2023-11-13T11:04:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da24a9640 1 -- 172.21.15.16:0/3234420718 reap_dead start 2023-11-13T11:04:14.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 shutdown_connections 2023-11-13T11:04:14.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 >> 172.21.15.16:0/3234420718 conn(0x7f6d9c0d2b70 msgr2=0x7f6d9c0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:14.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 shutdown_connections 2023-11-13T11:04:14.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.211+0000 7f6da2caa640 1 -- 172.21.15.16:0/3234420718 wait complete. 2023-11-13T11:04:14.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-13T11:04:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.440+0000 7fe6bb59e640 1 Processor -- start 2023-11-13T11:04:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.440+0000 7fe6bb59e640 1 -- start start 2023-11-13T11:04:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.442+0000 7fe6bb59e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe6bc15ece0 0x7fe6bc071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.442+0000 7fe6bb59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc06c070 con 0x7fe6bc155080 2023-11-13T11:04:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.442+0000 7fe6bb59e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc071bc0 con 0x7fe6bc070330 2023-11-13T11:04:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.442+0000 7fe6bb59e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6bc0707e0 con 0x7fe6bc15ece0 2023-11-13T11:04:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.442+0000 7fe6ba59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe6bc155080 0x7fe6bc0ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33840/0 (socket says 172.21.15.16:33840) 2023-11-13T11:04:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.442+0000 7fe6ba59c640 1 -- 172.21.15.16:0/362163576 learned_addr learned my addr 172.21.15.16:0/362163576 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6bad9d640 1 --2- 172.21.15.16:0/362163576 >> v2:172.21.15.184:3300/0 conn(0x7fe6bc15ece0 0x7fe6bc071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37954166 0 0) 0x7fe6bc06c070 con 0x7fe6bc155080 2023-11-13T11:04:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a8003520 con 0x7fe6bc155080 2023-11-13T11:04:14.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 935727539 0 0) 0x7fe6a8003520 con 0x7fe6bc155080 2023-11-13T11:04:14.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 >> v2:172.21.15.184:3300/0 conn(0x7fe6bc15ece0 msgr2=0x7fe6bc071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:14.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 --2- 172.21.15.16:0/362163576 >> v2:172.21.15.184:3300/0 conn(0x7fe6bc15ece0 0x7fe6bc071680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 >> v1:172.21.15.184:6789/0 conn(0x7fe6bc070330 legacy=0x7fe6bc15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0709c0 con 0x7fe6bc155080 2023-11-13T11:04:14.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fe6ac001670 con 0x7fe6bc155080 2023-11-13T11:04:14.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 >> v1:172.21.15.16:6789/0 conn(0x7fe6bc155080 legacy=0x7fe6bc0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6a8003970 0x7fe6a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.443+0000 7fe6b9d9b640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a80047c0 0x7fe6a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b9d9b640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a8005180 0x7fe6a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0707e0 con 0x7fe6a80047c0 2023-11-13T11:04:14.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800c020 con 0x7fe6a8005180 2023-11-13T11:04:14.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800c1e0 con 0x7fe6a8003970 2023-11-13T11:04:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b3fff640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a80047c0 0x7fe6a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6bad9d640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a8005180 0x7fe6a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6ba59c640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6a8003970 0x7fe6a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b3fff640 1 -- 172.21.15.16:0/362163576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6a8003970 msgr2=0x7fe6a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:14.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b3fff640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6a8003970 0x7fe6a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b3fff640 1 -- 172.21.15.16:0/362163576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a8005180 msgr2=0x7fe6a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:14.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b3fff640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a8005180 0x7fe6a800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6b3fff640 1 -- 172.21.15.16:0/362163576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a8003520 con 0x7fe6a80047c0 2023-11-13T11:04:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.444+0000 7fe6bad9d640 1 -- 172.21.15.16:0/362163576 reap_dead start 2023-11-13T11:04:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.446+0000 7fe6b3fff640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a80047c0 0x7fe6a8004c40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4001750 tx=0x7fe6a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675f00467e9bda28 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.446+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40052d0 con 0x7fe6a80047c0 2023-11-13T11:04:14.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.446+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a402e050 con 0x7fe6a80047c0 2023-11-13T11:04:14.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.446+0000 7fe6b9d9b640 1 -- 172.21.15.16:0/362163576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4003830 con 0x7fe6a80047c0 2023-11-13T11:04:14.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.447+0000 7fe6bb59e640 1 -- 172.21.15.16:0/362163576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a80047c0 msgr2=0x7fe6a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.447+0000 7fe6bb59e640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a80047c0 0x7fe6a8004c40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4001750 tx=0x7fe6a80057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.447+0000 7fe6bb59e640 1 -- 172.21.15.16:0/362163576 shutdown_connections 2023-11-13T11:04:14.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.447+0000 7fe6bb59e640 1 --2- 172.21.15.16:0/362163576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a80047c0 0x7fe6a8004c40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.447+0000 7fe6bb59e640 1 -- 172.21.15.16:0/362163576 >> 172.21.15.16:0/362163576 conn(0x7fe6bc0b0580 msgr2=0x7fe6bc0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.447+0000 7fe6bb59e640 1 -- 172.21.15.16:0/362163576 shutdown_connections 2023-11-13T11:04:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.447+0000 7fe6bb59e640 1 -- 172.21.15.16:0/362163576 wait complete. 2023-11-13T11:04:14.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.449+0000 7fe6bb59e640 1 Processor -- start 2023-11-13T11:04:14.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.449+0000 7fe6bb59e640 1 -- start start 2023-11-13T11:04:14.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6bb59e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a80047c0 0x7fe6bc157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6bb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6bc143420 0x7fe6bc1466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6bb59e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc0cdac0 0x7fe6bc0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6bb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0ce480 con 0x7fe6bc143420 2023-11-13T11:04:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6bb59e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc146be0 con 0x7fe6a80047c0 2023-11-13T11:04:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6bb59e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc157c80 con 0x7fe6bc0cdac0 2023-11-13T11:04:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6bc143420 0x7fe6bc1466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6bc143420 0x7fe6bc1466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56036/0 (socket says 172.21.15.16:56036) 2023-11-13T11:04:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6b3fff640 1 -- 172.21.15.16:0/42036484 learned_addr learned my addr 172.21.15.16:0/42036484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.450+0000 7fe6ba59c640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a80047c0 0x7fe6bc157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.451+0000 7fe6bad9d640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc0cdac0 0x7fe6bc0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.451+0000 7fe6b3fff640 1 -- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc0cdac0 msgr2=0x7fe6bc0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.451+0000 7fe6b3fff640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc0cdac0 0x7fe6bc0cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.451+0000 7fe6b3fff640 1 -- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a80047c0 msgr2=0x7fe6bc157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.451+0000 7fe6b3fff640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a80047c0 0x7fe6bc157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.451+0000 7fe6b3fff640 1 -- 172.21.15.16:0/42036484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc1f5d30 con 0x7fe6bc143420 2023-11-13T11:04:14.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.451+0000 7fe6ba59c640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a80047c0 0x7fe6bc157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:04:14.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.451+0000 7fe6b3fff640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6bc143420 0x7fe6bc1466a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4005040 tx=0x7fe6a4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.452+0000 7fe6b2ffd640 1 -- 172.21.15.16:0/42036484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4007a30 con 0x7fe6bc143420 2023-11-13T11:04:14.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.452+0000 7fe6b2ffd640 1 -- 172.21.15.16:0/42036484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a40069e0 con 0x7fe6bc143420 2023-11-13T11:04:14.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.452+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc1f6010 con 0x7fe6bc143420 2023-11-13T11:04:14.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.452+0000 7fe6b2ffd640 1 -- 172.21.15.16:0/42036484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4006d70 con 0x7fe6bc143420 2023-11-13T11:04:14.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.452+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc1f65d0 con 0x7fe6bc143420 2023-11-13T11:04:14.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.453+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc1f6760 con 0x7fe6bc143420 2023-11-13T11:04:14.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.454+0000 7fe6b2ffd640 1 -- 172.21.15.16:0/42036484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6a40035f0 con 0x7fe6bc143420 2023-11-13T11:04:14.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.455+0000 7fe6b2ffd640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe69c06ae40 0x7fe69c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.455+0000 7fe6b2ffd640 1 -- 172.21.15.16:0/42036484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fe6a40ab570 con 0x7fe6bc143420 2023-11-13T11:04:14.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.456+0000 7fe6ba59c640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe69c06ae40 0x7fe69c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.457+0000 7fe6ba59c640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe69c06ae40 0x7fe69c06d300 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac001050 tx=0x7fe6ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.459+0000 7fe6b2ffd640 1 -- 172.21.15.16:0/42036484 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6a4076770 con 0x7fe6bc143420 2023-11-13T11:04:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.653+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6bc0ce480 con 0x7fe69c06ae40 2023-11-13T11:04:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.654+0000 7fe6b2ffd640 1 -- 172.21.15.16:0/42036484 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe6bc0ce480 con 0x7fe69c06ae40 2023-11-13T11:04:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2023-11-13T11:04:14.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe69c06ae40 msgr2=0x7fe69c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe69c06ae40 0x7fe69c06d300 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac001050 tx=0x7fe6ac009040 comp rx=0 tx=0).stop 2023-11-13T11:04:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6bc143420 msgr2=0x7fe6bc1466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6bc143420 0x7fe6bc1466a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4005040 tx=0x7fe6a4001120 comp rx=0 tx=0).stop 2023-11-13T11:04:14.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 shutdown_connections 2023-11-13T11:04:14.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe69c06ae40 0x7fe69c06d300 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6bc0cdac0 0x7fe6bc0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6bc143420 0x7fe6bc1466a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 --2- 172.21.15.16:0/42036484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6a80047c0 0x7fe6bc157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 >> 172.21.15.16:0/42036484 conn(0x7fe6bc0b0580 msgr2=0x7fe6bc0b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 shutdown_connections 2023-11-13T11:04:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.657+0000 7fe6bb59e640 1 -- 172.21.15.16:0/42036484 wait complete. 2023-11-13T11:04:14.680 INFO:tasks.ceph:Scrubbing osd.2 2023-11-13T11:04:14.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-13T11:04:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.904+0000 7f8c7de25640 1 Processor -- start 2023-11-13T11:04:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.904+0000 7f8c7de25640 1 -- start start 2023-11-13T11:04:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.905+0000 7f8c7de25640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c780b76c0 0x7f8c78167880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.905+0000 7f8c7de25640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c780cf1f0 con 0x7f8c780cf580 2023-11-13T11:04:14.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.905+0000 7f8c7de25640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c780b5bd0 con 0x7f8c780ce4c0 2023-11-13T11:04:14.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.905+0000 7f8c7de25640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c780b59c0 con 0x7f8c780b76c0 2023-11-13T11:04:14.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.906+0000 7f8c7ce23640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c780cf580 0x7f8c780cf960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33850/0 (socket says 172.21.15.16:33850) 2023-11-13T11:04:14.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.906+0000 7f8c7ce23640 1 -- 172.21.15.16:0/2595370557 learned_addr learned my addr 172.21.15.16:0/2595370557 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:14.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.906+0000 7f8c7d624640 1 --2- 172.21.15.16:0/2595370557 >> v2:172.21.15.16:3300/0 conn(0x7f8c780b76c0 0x7f8c78167880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.906+0000 7f8c777fe640 1 -- 172.21.15.16:0/2595370557 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589716132 0 0) 0x7f8c780cf1f0 con 0x7f8c780cf580 2023-11-13T11:04:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.906+0000 7f8c777fe640 1 -- 172.21.15.16:0/2595370557 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c68003520 con 0x7f8c780cf580 2023-11-13T11:04:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.906+0000 7f8c7d624640 1 -- 172.21.15.16:0/2595370557 >> v1:172.21.15.184:6789/0 conn(0x7f8c780ce4c0 legacy=0x7f8c780ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:14.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.906+0000 7f8c7d624640 1 -- 172.21.15.16:0/2595370557 >> v1:172.21.15.16:6789/0 conn(0x7f8c780cf580 legacy=0x7f8c780cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.907+0000 7f8c7d624640 1 -- 172.21.15.16:0/2595370557 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c780b5eb0 con 0x7f8c780b76c0 2023-11-13T11:04:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.908+0000 7f8c7d624640 1 --2- 172.21.15.16:0/2595370557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780b76c0 0x7f8c78167880 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009d90 tx=0x7f8c7002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d236d138d474c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.908+0000 7f8c777fe640 1 -- 172.21.15.16:0/2595370557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7002cc60 con 0x7f8c780b76c0 2023-11-13T11:04:14.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.909+0000 7f8c777fe640 1 -- 172.21.15.16:0/2595370557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7002cdc0 con 0x7f8c780b76c0 2023-11-13T11:04:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.909+0000 7f8c777fe640 1 -- 172.21.15.16:0/2595370557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70034640 con 0x7f8c780b76c0 2023-11-13T11:04:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.909+0000 7f8c7de25640 1 -- 172.21.15.16:0/2595370557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780b76c0 msgr2=0x7f8c78167880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.909+0000 7f8c7de25640 1 --2- 172.21.15.16:0/2595370557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780b76c0 0x7f8c78167880 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009d90 tx=0x7f8c7002beb0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.909+0000 7f8c7de25640 1 -- 172.21.15.16:0/2595370557 shutdown_connections 2023-11-13T11:04:14.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.909+0000 7f8c7de25640 1 --2- 172.21.15.16:0/2595370557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780b76c0 0x7f8c78167880 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.909+0000 7f8c7de25640 1 -- 172.21.15.16:0/2595370557 >> 172.21.15.16:0/2595370557 conn(0x7f8c780d2b70 msgr2=0x7f8c780d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:14.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.910+0000 7f8c7de25640 1 -- 172.21.15.16:0/2595370557 shutdown_connections 2023-11-13T11:04:14.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.910+0000 7f8c7de25640 1 -- 172.21.15.16:0/2595370557 wait complete. 2023-11-13T11:04:14.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.911+0000 7f8c7de25640 1 Processor -- start 2023-11-13T11:04:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.911+0000 7f8c7de25640 1 -- start start 2023-11-13T11:04:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.912+0000 7f8c7de25640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c780b76c0 0x7f8c780c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.912+0000 7f8c7de25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780ce4c0 0x7f8c78155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.912+0000 7f8c7de25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c780cf580 0x7f8c7814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.912+0000 7f8c7de25640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780c9cd0 con 0x7f8c780ce4c0 2023-11-13T11:04:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.912+0000 7f8c7de25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781519a0 con 0x7f8c780cf580 2023-11-13T11:04:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.912+0000 7f8c7de25640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78151c80 con 0x7f8c780b76c0 2023-11-13T11:04:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780ce4c0 0x7f8c78155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780ce4c0 0x7f8c78155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56058/0 (socket says 172.21.15.16:56058) 2023-11-13T11:04:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c77fff640 1 -- 172.21.15.16:0/1295808853 learned_addr learned my addr 172.21.15.16:0/1295808853 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c7d624640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c780cf580 0x7f8c7814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c7ce23640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c780b76c0 0x7f8c780c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c77fff640 1 -- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c780b76c0 msgr2=0x7f8c780c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c77fff640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c780b76c0 0x7f8c780c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c77fff640 1 -- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c780cf580 msgr2=0x7f8c7814b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c77fff640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c780cf580 0x7f8c7814b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.913+0000 7f8c77fff640 1 -- 172.21.15.16:0/1295808853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70009a70 con 0x7f8c780ce4c0 2023-11-13T11:04:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.914+0000 7f8c77fff640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780ce4c0 0x7f8c78155d20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400c430 tx=0x7f8c6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.914+0000 7f8c757fa640 1 -- 172.21.15.16:0/1295808853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011790 con 0x7f8c780ce4c0 2023-11-13T11:04:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.914+0000 7f8c757fa640 1 -- 172.21.15.16:0/1295808853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c640118f0 con 0x7f8c780ce4c0 2023-11-13T11:04:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.914+0000 7f8c757fa640 1 -- 172.21.15.16:0/1295808853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011ce0 con 0x7f8c780ce4c0 2023-11-13T11:04:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.914+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c78151f60 con 0x7f8c780ce4c0 2023-11-13T11:04:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.914+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c781515d0 con 0x7f8c780ce4c0 2023-11-13T11:04:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.915+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c78167dc0 con 0x7f8c780ce4c0 2023-11-13T11:04:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.916+0000 7f8c757fa640 1 -- 172.21.15.16:0/1295808853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c6401c460 con 0x7f8c780ce4c0 2023-11-13T11:04:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.917+0000 7f8c757fa640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c5c06ae40 0x7f8c5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.917+0000 7f8c757fa640 1 -- 172.21.15.16:0/1295808853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f8c640894f0 con 0x7f8c780ce4c0 2023-11-13T11:04:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.918+0000 7f8c757fa640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f8c5c0711e0 0x7f8c5c073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.918+0000 7f8c757fa640 1 -- 172.21.15.16:0/1295808853 --> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c5c073cf0 con 0x7f8c5c0711e0 2023-11-13T11:04:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.918+0000 7f8c757fa640 1 -- 172.21.15.16:0/1295808853 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c6408bad0 con 0x7f8c780ce4c0 2023-11-13T11:04:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.918+0000 7f8c7d624640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f8c5c0711e0 0x7f8c5c073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.918+0000 7f8c7ce23640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c5c06ae40 0x7f8c5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.919+0000 7f8c7d624640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f8c5c0711e0 0x7f8c5c073620 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.919+0000 7f8c7ce23640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c5c06ae40 0x7f8c5c06d300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0011e0 tx=0x7f8c6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.920+0000 7f8c757fa640 1 -- 172.21.15.16:0/1295808853 <== osd.2 v2:172.21.15.16:6808/3899450571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f8c5c073cf0 con 0x7f8c5c0711e0 2023-11-13T11:04:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.934+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 --> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8c7804e4d0 con 0x7f8c5c0711e0 2023-11-13T11:04:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c757fa640 1 -- 172.21.15.16:0/1295808853 <== osd.2 v2:172.21.15.16:6808/3899450571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8c7804e4d0 con 0x7f8c5c0711e0 2023-11-13T11:04:14.939 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:04:14.940 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-13T11:04:14.940 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:04:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f8c5c0711e0 msgr2=0x7f8c5c073620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.16:6808/3899450571,v1:172.21.15.16:6810/3899450571] conn(0x7f8c5c0711e0 0x7f8c5c073620 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c5c06ae40 msgr2=0x7f8c5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8c5c06ae40 0x7f8c5c06d300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0011e0 tx=0x7f8c6c009040 comp rx=0 tx=0).stop 2023-11-13T11:04:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780ce4c0 msgr2=0x7f8c78155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 --2- 172.21.15.16:0/1295808853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c780ce4c0 0x7f8c78155d20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400c430 tx=0x7f8c6400c8f0 comp rx=0 tx=0).stop 2023-11-13T11:04:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7d624640 1 -- 172.21.15.16:0/1295808853 reap_dead start 2023-11-13T11:04:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 shutdown_connections 2023-11-13T11:04:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 >> 172.21.15.16:0/1295808853 conn(0x7f8c780d2b70 msgr2=0x7f8c780d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 shutdown_connections 2023-11-13T11:04:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:14.935+0000 7f8c7de25640 1 -- 172.21.15.16:0/1295808853 wait complete. 2023-11-13T11:04:14.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-13T11:04:15.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.162+0000 7f7179ded640 1 Processor -- start 2023-11-13T11:04:15.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.162+0000 7f7179ded640 1 -- start start 2023-11-13T11:04:15.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.164+0000 7f7179ded640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174155080 0x7f7174070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.164+0000 7f7179ded640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f717415e5d0 0x7f717415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.164+0000 7f7179ded640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717406c070 con 0x7f7174071680 2023-11-13T11:04:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.164+0000 7f7179ded640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71740cd1c0 con 0x7f717415e5d0 2023-11-13T11:04:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.164+0000 7f7179ded640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717415efa0 con 0x7f7174155080 2023-11-13T11:04:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.164+0000 7f71795ec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f717415e5d0 0x7f717415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.164+0000 7f71795ec640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f717415e5d0 0x7f717415ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.16:34246/0 (socket says 172.21.15.16:34246) 2023-11-13T11:04:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.164+0000 7f71795ec640 1 -- 172.21.15.16:0/1095948727 learned_addr learned my addr 172.21.15.16:0/1095948727 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.164+0000 7f716bfff640 1 --1- 172.21.15.16:0/1095948727 >> v1:172.21.15.184:6789/0 conn(0x7f7174071680 0x7f71740ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.16:47090/0 (socket says 172.21.15.16:47090) 2023-11-13T11:04:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.165+0000 7f7178deb640 1 --2- 172.21.15.16:0/1095948727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174155080 0x7f7174070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.165+0000 7f71795ec640 1 -- 172.21.15.16:0/1095948727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174155080 msgr2=0x7f7174070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.165+0000 7f71795ec640 1 --2- 172.21.15.16:0/1095948727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174155080 0x7f7174070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.165+0000 7f71795ec640 1 -- 172.21.15.16:0/1095948727 >> v1:172.21.15.184:6789/0 conn(0x7f7174071680 legacy=0x7f71740ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.165+0000 7f716b7fe640 1 -- 172.21.15.16:0/1095948727 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523553103 0 0) 0x7f717406c070 con 0x7f7174071680 2023-11-13T11:04:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.165+0000 7f71795ec640 1 -- 172.21.15.16:0/1095948727 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71740709c0 con 0x7f717415e5d0 2023-11-13T11:04:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.167+0000 7f71795ec640 1 --2- 172.21.15.16:0/1095948727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f717415e5d0 0x7f717415ea60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f716c009870 tx=0x7f716c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50ba350d1efa5629 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.167+0000 7f716b7fe640 1 -- 172.21.15.16:0/1095948727 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c030840 con 0x7f717415e5d0 2023-11-13T11:04:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.167+0000 7f716b7fe640 1 -- 172.21.15.16:0/1095948727 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c0309a0 con 0x7f717415e5d0 2023-11-13T11:04:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.167+0000 7f716b7fe640 1 -- 172.21.15.16:0/1095948727 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c030d30 con 0x7f717415e5d0 2023-11-13T11:04:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.168+0000 7f7179ded640 1 -- 172.21.15.16:0/1095948727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f717415e5d0 msgr2=0x7f717415ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.168+0000 7f7179ded640 1 --2- 172.21.15.16:0/1095948727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f717415e5d0 0x7f717415ea60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f716c009870 tx=0x7f716c02b980 comp rx=0 tx=0).stop 2023-11-13T11:04:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.168+0000 7f7179ded640 1 -- 172.21.15.16:0/1095948727 shutdown_connections 2023-11-13T11:04:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.168+0000 7f7179ded640 1 --2- 172.21.15.16:0/1095948727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f717415e5d0 0x7f717415ea60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.168+0000 7f7179ded640 1 --2- 172.21.15.16:0/1095948727 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174155080 0x7f7174070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.168+0000 7f7179ded640 1 -- 172.21.15.16:0/1095948727 >> 172.21.15.16:0/1095948727 conn(0x7f71740b0580 msgr2=0x7f71740b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:15.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.168+0000 7f7179ded640 1 -- 172.21.15.16:0/1095948727 shutdown_connections 2023-11-13T11:04:15.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.168+0000 7f7179ded640 1 -- 172.21.15.16:0/1095948727 wait complete. 2023-11-13T11:04:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.170+0000 7f7179ded640 1 Processor -- start 2023-11-13T11:04:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.170+0000 7f7179ded640 1 -- start start 2023-11-13T11:04:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.171+0000 7f7179ded640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174071680 0x7f7174150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.171+0000 7f7179ded640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7174155080 0x7f7174149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.171+0000 7f7179ded640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f717415e5d0 0x7f71741f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.171+0000 7f7179ded640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7174149db0 con 0x7f717415e5d0 2023-11-13T11:04:15.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.171+0000 7f7179ded640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71741f6ab0 con 0x7f7174155080 2023-11-13T11:04:15.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.171+0000 7f7179ded640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717414fc40 con 0x7f7174071680 2023-11-13T11:04:15.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.171+0000 7f71795ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f717415e5d0 0x7f71741f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.171+0000 7f71795ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f717415e5d0 0x7f71741f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56060/0 (socket says 172.21.15.16:56060) 2023-11-13T11:04:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.171+0000 7f71795ec640 1 -- 172.21.15.16:0/2997021747 learned_addr learned my addr 172.21.15.16:0/2997021747 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f7178deb640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174071680 0x7f7174150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f716bfff640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7174155080 0x7f7174149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f71795ec640 1 -- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174071680 msgr2=0x7f7174150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f71795ec640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174071680 0x7f7174150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f71795ec640 1 -- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7174155080 msgr2=0x7f7174149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f71795ec640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7174155080 0x7f7174149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f71795ec640 1 -- 172.21.15.16:0/2997021747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c0094d0 con 0x7f717415e5d0 2023-11-13T11:04:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f716bfff640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7174155080 0x7f7174149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f71795ec640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f717415e5d0 0x7f71741f5f10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f716c009870 tx=0x7f716c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f71697fa640 1 -- 172.21.15.16:0/2997021747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c039a10 con 0x7f717415e5d0 2023-11-13T11:04:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f71697fa640 1 -- 172.21.15.16:0/2997021747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c039b70 con 0x7f717415e5d0 2023-11-13T11:04:15.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.172+0000 7f71697fa640 1 -- 172.21.15.16:0/2997021747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c043410 con 0x7f717415e5d0 2023-11-13T11:04:15.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.173+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717414ff20 con 0x7f717415e5d0 2023-11-13T11:04:15.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.173+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71741f6d90 con 0x7f717415e5d0 2023-11-13T11:04:15.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.174+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7174150440 con 0x7f717415e5d0 2023-11-13T11:04:15.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.175+0000 7f71697fa640 1 -- 172.21.15.16:0/2997021747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f716c04c3b0 con 0x7f717415e5d0 2023-11-13T11:04:15.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.176+0000 7f71697fa640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f716006ae40 0x7f716006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.176+0000 7f71697fa640 1 -- 172.21.15.16:0/2997021747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f716c0ab9d0 con 0x7f717415e5d0 2023-11-13T11:04:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.177+0000 7f7178deb640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f716006ae40 0x7f716006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.178+0000 7f7178deb640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f716006ae40 0x7f716006d300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f71700074a0 tx=0x7f717000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.181+0000 7f71697fa640 1 -- 172.21.15.16:0/2997021747 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7174150440 con 0x7f717415e5d0 2023-11-13T11:04:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.376+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f717414fac0 con 0x7f716006ae40 2023-11-13T11:04:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.377+0000 7f71697fa640 1 -- 172.21.15.16:0/2997021747 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f717000f050 con 0x7f716006ae40 2023-11-13T11:04:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2023-11-13T11:04:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.379+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f716006ae40 msgr2=0x7f716006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.379+0000 7f7179ded640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f716006ae40 0x7f716006d300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f71700074a0 tx=0x7f717000b040 comp rx=0 tx=0).stop 2023-11-13T11:04:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.379+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f717415e5d0 msgr2=0x7f71741f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.379+0000 7f7179ded640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f717415e5d0 0x7f71741f5f10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f716c009870 tx=0x7f716c004730 comp rx=0 tx=0).stop 2023-11-13T11:04:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.380+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 shutdown_connections 2023-11-13T11:04:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.380+0000 7f7179ded640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f716006ae40 0x7f716006d300 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.380+0000 7f7179ded640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f717415e5d0 0x7f71741f5f10 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.380+0000 7f7179ded640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7174155080 0x7f7174149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.380+0000 7f7179ded640 1 --2- 172.21.15.16:0/2997021747 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7174071680 0x7f7174150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.380+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 >> 172.21.15.16:0/2997021747 conn(0x7f71740b0580 msgr2=0x7f7174068050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:15.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.380+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 shutdown_connections 2023-11-13T11:04:15.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.380+0000 7f7179ded640 1 -- 172.21.15.16:0/2997021747 wait complete. 2023-11-13T11:04:15.403 INFO:tasks.ceph:Scrubbing osd.3 2023-11-13T11:04:15.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-13T11:04:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.636+0000 7f533fc4b640 1 Processor -- start 2023-11-13T11:04:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.637+0000 7f533fc4b640 1 -- start start 2023-11-13T11:04:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.638+0000 7f533fc4b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 0x7f53380cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.639+0000 7f533fc4b640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53380cef80 0x7f533806ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.639+0000 7f533fc4b640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5338159ec0 con 0x7f53380adbc0 2023-11-13T11:04:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.639+0000 7f533fc4b640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533815dc20 con 0x7f53380cef80 2023-11-13T11:04:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.639+0000 7f533fc4b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533815df00 con 0x7f5338159ae0 2023-11-13T11:04:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.639+0000 7f533d9e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 0x7f53380cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.639+0000 7f533e1e8640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53380cef80 0x7f533806ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.639+0000 7f533d9e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 0x7f53380cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.16:44672/0 (socket says 172.21.15.16:44672) 2023-11-13T11:04:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.639+0000 7f533d9e7640 1 -- 172.21.15.16:0/2164300359 learned_addr learned my addr 172.21.15.16:0/2164300359 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.640+0000 7f533d9e7640 1 -- 172.21.15.16:0/2164300359 >> v2:172.21.15.184:3300/0 conn(0x7f53380cef80 msgr2=0x7f533806ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.640+0000 7f533d9e7640 1 --2- 172.21.15.16:0/2164300359 >> v2:172.21.15.184:3300/0 conn(0x7f53380cef80 0x7f533806ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.640+0000 7f533d9e7640 1 -- 172.21.15.16:0/2164300359 >> v1:172.21.15.184:6789/0 conn(0x7f53380adbc0 legacy=0x7f53380ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.640+0000 7f533d9e7640 1 -- 172.21.15.16:0/2164300359 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533815e1e0 con 0x7f5338159ae0 2023-11-13T11:04:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.642+0000 7f533d9e7640 1 --2- 172.21.15.16:0/2164300359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 0x7f53380cbb90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5334009870 tx=0x7f533402db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec137f29b7b8c2c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.642+0000 7f533c9e5640 1 -- 172.21.15.16:0/2164300359 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533402e940 con 0x7f5338159ae0 2023-11-13T11:04:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.642+0000 7f533c9e5640 1 -- 172.21.15.16:0/2164300359 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533402eaa0 con 0x7f5338159ae0 2023-11-13T11:04:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.642+0000 7f533c9e5640 1 -- 172.21.15.16:0/2164300359 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533402ee30 con 0x7f5338159ae0 2023-11-13T11:04:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.643+0000 7f533fc4b640 1 -- 172.21.15.16:0/2164300359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 msgr2=0x7f53380cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.643+0000 7f533fc4b640 1 --2- 172.21.15.16:0/2164300359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 0x7f53380cbb90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5334009870 tx=0x7f533402db90 comp rx=0 tx=0).stop 2023-11-13T11:04:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.643+0000 7f533fc4b640 1 -- 172.21.15.16:0/2164300359 shutdown_connections 2023-11-13T11:04:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.643+0000 7f533fc4b640 1 --2- 172.21.15.16:0/2164300359 >> v2:172.21.15.184:3300/0 conn(0x7f53380cef80 0x7f533806ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.643+0000 7f533fc4b640 1 --2- 172.21.15.16:0/2164300359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 0x7f53380cbb90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.643+0000 7f533fc4b640 1 -- 172.21.15.16:0/2164300359 >> 172.21.15.16:0/2164300359 conn(0x7f533806a190 msgr2=0x7f533806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.644+0000 7f533fc4b640 1 -- 172.21.15.16:0/2164300359 shutdown_connections 2023-11-13T11:04:15.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.644+0000 7f533fc4b640 1 -- 172.21.15.16:0/2164300359 wait complete. 2023-11-13T11:04:15.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.646+0000 7f533fc4b640 1 Processor -- start 2023-11-13T11:04:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.647+0000 7f533fc4b640 1 -- start start 2023-11-13T11:04:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.648+0000 7f533fc4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53380adbc0 0x7f5338142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.648+0000 7f533fc4b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53380cef80 0x7f53381e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.648+0000 7f533fc4b640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 0x7f5338153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.648+0000 7f533fc4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533815b4f0 con 0x7f53380adbc0 2023-11-13T11:04:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.648+0000 7f533fc4b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53381573b0 con 0x7f53380cef80 2023-11-13T11:04:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.648+0000 7f533fc4b640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5338142ac0 con 0x7f5338159ae0 2023-11-13T11:04:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.648+0000 7f533d9e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53380adbc0 0x7f5338142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.648+0000 7f533d9e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53380adbc0 0x7f5338142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56076/0 (socket says 172.21.15.16:56076) 2023-11-13T11:04:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.648+0000 7f533d9e7640 1 -- 172.21.15.16:0/1621223988 learned_addr learned my addr 172.21.15.16:0/1621223988 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.649+0000 7f533d1e6640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53380cef80 0x7f53381e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.649+0000 7f533e1e8640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 0x7f5338153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.649+0000 7f533d9e7640 1 -- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 msgr2=0x7f5338153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.649+0000 7f533d9e7640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5338159ae0 0x7f5338153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.649+0000 7f533d9e7640 1 -- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53380cef80 msgr2=0x7f53381e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.649+0000 7f533d9e7640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53380cef80 0x7f53381e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.649+0000 7f533d9e7640 1 -- 172.21.15.16:0/1621223988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53340094d0 con 0x7f53380adbc0 2023-11-13T11:04:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.649+0000 7f533d9e7640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53380adbc0 0x7f5338142580 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5334000c00 tx=0x7f53340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.650+0000 7f532e7fc640 1 -- 172.21.15.16:0/1621223988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334040600 con 0x7f53380adbc0 2023-11-13T11:04:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.650+0000 7f532e7fc640 1 -- 172.21.15.16:0/1621223988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5334040780 con 0x7f53380adbc0 2023-11-13T11:04:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.650+0000 7f532e7fc640 1 -- 172.21.15.16:0/1621223988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334040b30 con 0x7f53380adbc0 2023-11-13T11:04:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.650+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5338156320 con 0x7f53380adbc0 2023-11-13T11:04:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.650+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53381568e0 con 0x7f53380adbc0 2023-11-13T11:04:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.651+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f533806c4a0 con 0x7f53380adbc0 2023-11-13T11:04:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.652+0000 7f532e7fc640 1 -- 172.21.15.16:0/1621223988 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5334040c90 con 0x7f53380adbc0 2023-11-13T11:04:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.653+0000 7f532e7fc640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f531c06ae40 0x7f531c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.653+0000 7f532e7fc640 1 -- 172.21.15.16:0/1621223988 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f53340abcc0 con 0x7f53380adbc0 2023-11-13T11:04:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.654+0000 7f532e7fc640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7f531c0711e0 0x7f531c073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.654+0000 7f532e7fc640 1 -- 172.21.15.16:0/1621223988 --> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f531c073cf0 con 0x7f531c0711e0 2023-11-13T11:04:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.654+0000 7f532e7fc640 1 -- 172.21.15.16:0/1621223988 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53340ae2a0 con 0x7f53380adbc0 2023-11-13T11:04:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.654+0000 7f533e1e8640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7f531c0711e0 0x7f531c073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.654+0000 7f533d1e6640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f531c06ae40 0x7f531c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.655+0000 7f533e1e8640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7f531c0711e0 0x7f531c073620 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.655+0000 7f533d1e6640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f531c06ae40 0x7f531c06d300 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f53380cec30 tx=0x7f5328009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.655+0000 7f532e7fc640 1 -- 172.21.15.16:0/1621223988 <== osd.3 v2:172.21.15.16:6824/3562446965 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f531c073cf0 con 0x7f531c0711e0 2023-11-13T11:04:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.670+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 --> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f533806c920 con 0x7f531c0711e0 2023-11-13T11:04:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.671+0000 7f532e7fc640 1 -- 172.21.15.16:0/1621223988 <== osd.3 v2:172.21.15.16:6824/3562446965 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f533000f040 con 0x7f531c0711e0 2023-11-13T11:04:15.675 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:04:15.675 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-13T11:04:15.676 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:04:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.671+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7f531c0711e0 msgr2=0x7f531c073620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.671+0000 7f533fc4b640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.16:6824/3562446965,v1:172.21.15.16:6825/3562446965] conn(0x7f531c0711e0 0x7f531c073620 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.671+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f531c06ae40 msgr2=0x7f531c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.671+0000 7f533fc4b640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f531c06ae40 0x7f531c06d300 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f53380cec30 tx=0x7f5328009040 comp rx=0 tx=0).stop 2023-11-13T11:04:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.671+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53380adbc0 msgr2=0x7f5338142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.671+0000 7f533fc4b640 1 --2- 172.21.15.16:0/1621223988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53380adbc0 0x7f5338142580 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5334000c00 tx=0x7f53340046c0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.671+0000 7f533e1e8640 1 -- 172.21.15.16:0/1621223988 reap_dead start 2023-11-13T11:04:15.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.672+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 shutdown_connections 2023-11-13T11:04:15.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.672+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 >> 172.21.15.16:0/1621223988 conn(0x7f533806a190 msgr2=0x7f5338070500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:15.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.672+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 shutdown_connections 2023-11-13T11:04:15.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.672+0000 7f533fc4b640 1 -- 172.21.15.16:0/1621223988 wait complete. 2023-11-13T11:04:15.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-13T11:04:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.919+0000 7feb47347640 1 Processor -- start 2023-11-13T11:04:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.919+0000 7feb47347640 1 -- start start 2023-11-13T11:04:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.921+0000 7feb47347640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40155080 0x7feb40070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.921+0000 7feb47347640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb40071680 0x7feb400ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.921+0000 7feb47347640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4006c070 con 0x7feb4015e5d0 2023-11-13T11:04:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.921+0000 7feb47347640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb400709c0 con 0x7feb40071680 2023-11-13T11:04:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.921+0000 7feb47347640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb40155460 con 0x7feb40155080 2023-11-13T11:04:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb46b46640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feb4015e5d0 0x7feb4006c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33862/0 (socket says 172.21.15.16:33862) 2023-11-13T11:04:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb46b46640 1 -- 172.21.15.16:0/149027839 learned_addr learned my addr 172.21.15.16:0/149027839 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb45b44640 1 --2- 172.21.15.16:0/149027839 >> v2:172.21.15.16:3300/0 conn(0x7feb40071680 0x7feb400ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb46345640 1 --2- 172.21.15.16:0/149027839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40155080 0x7feb40070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb45343640 1 -- 172.21.15.16:0/149027839 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671168449 0 0) 0x7feb4006c070 con 0x7feb4015e5d0 2023-11-13T11:04:15.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb45343640 1 -- 172.21.15.16:0/149027839 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb2c003520 con 0x7feb4015e5d0 2023-11-13T11:04:15.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb45b44640 1 -- 172.21.15.16:0/149027839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40155080 msgr2=0x7feb40070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb45b44640 1 --2- 172.21.15.16:0/149027839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40155080 0x7feb40070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb45b44640 1 -- 172.21.15.16:0/149027839 >> v1:172.21.15.16:6789/0 conn(0x7feb4015e5d0 legacy=0x7feb4006c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.922+0000 7feb45b44640 1 -- 172.21.15.16:0/149027839 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4006b740 con 0x7feb40071680 2023-11-13T11:04:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.924+0000 7feb45b44640 1 --2- 172.21.15.16:0/149027839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40071680 0x7feb400ccab0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7feb300097a0 tx=0x7feb3002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7035d3bc7aea6555 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.924+0000 7feb45343640 1 -- 172.21.15.16:0/149027839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3002d040 con 0x7feb40071680 2023-11-13T11:04:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.924+0000 7feb45343640 1 -- 172.21.15.16:0/149027839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30031810 con 0x7feb40071680 2023-11-13T11:04:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.924+0000 7feb45343640 1 -- 172.21.15.16:0/149027839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30031ba0 con 0x7feb40071680 2023-11-13T11:04:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.925+0000 7feb47347640 1 -- 172.21.15.16:0/149027839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40071680 msgr2=0x7feb400ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.925+0000 7feb47347640 1 --2- 172.21.15.16:0/149027839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40071680 0x7feb400ccab0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7feb300097a0 tx=0x7feb3002b8b0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.925+0000 7feb47347640 1 -- 172.21.15.16:0/149027839 shutdown_connections 2023-11-13T11:04:15.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.925+0000 7feb47347640 1 --2- 172.21.15.16:0/149027839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40071680 0x7feb400ccab0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.925+0000 7feb47347640 1 --2- 172.21.15.16:0/149027839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40155080 0x7feb40070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.925+0000 7feb47347640 1 -- 172.21.15.16:0/149027839 >> 172.21.15.16:0/149027839 conn(0x7feb400b0580 msgr2=0x7feb400b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.925+0000 7feb47347640 1 -- 172.21.15.16:0/149027839 shutdown_connections 2023-11-13T11:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.925+0000 7feb47347640 1 -- 172.21.15.16:0/149027839 wait complete. 2023-11-13T11:04:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.927+0000 7feb47347640 1 Processor -- start 2023-11-13T11:04:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.927+0000 7feb47347640 1 -- start start 2023-11-13T11:04:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb47347640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40071680 0x7feb40141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb47347640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40155080 0x7feb4014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb47347640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb4015e5d0 0x7feb400b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb47347640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40150760 con 0x7feb40155080 2023-11-13T11:04:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb47347640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40150a40 con 0x7feb4015e5d0 2023-11-13T11:04:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb47347640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb40150d20 con 0x7feb40071680 2023-11-13T11:04:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb45b44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40155080 0x7feb4014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb45b44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40155080 0x7feb4014f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56080/0 (socket says 172.21.15.16:56080) 2023-11-13T11:04:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb45b44640 1 -- 172.21.15.16:0/3752880826 learned_addr learned my addr 172.21.15.16:0/3752880826 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb46345640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40071680 0x7feb40141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb45b44640 1 -- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40071680 msgr2=0x7feb40141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb45b44640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40071680 0x7feb40141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.928+0000 7feb45b44640 1 -- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb4015e5d0 msgr2=0x7feb400b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.929+0000 7feb45b44640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb4015e5d0 0x7feb400b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.929+0000 7feb45b44640 1 -- 172.21.15.16:0/3752880826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb30009400 con 0x7feb40155080 2023-11-13T11:04:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.929+0000 7feb45b44640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40155080 0x7feb4014f0d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7feb300098c0 tx=0x7feb300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.929+0000 7feb36ffd640 1 -- 172.21.15.16:0/3752880826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3003ad60 con 0x7feb40155080 2023-11-13T11:04:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.929+0000 7feb36ffd640 1 -- 172.21.15.16:0/3752880826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30044400 con 0x7feb40155080 2023-11-13T11:04:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.929+0000 7feb36ffd640 1 -- 172.21.15.16:0/3752880826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb300445b0 con 0x7feb40155080 2023-11-13T11:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.929+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb40151000 con 0x7feb40155080 2023-11-13T11:04:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.929+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4014f610 con 0x7feb40155080 2023-11-13T11:04:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.931+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb40151520 con 0x7feb40155080 2023-11-13T11:04:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.932+0000 7feb36ffd640 1 -- 172.21.15.16:0/3752880826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb3004d440 con 0x7feb40155080 2023-11-13T11:04:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.933+0000 7feb36ffd640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb2406ae40 0x7feb2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.933+0000 7feb36ffd640 1 -- 172.21.15.16:0/3752880826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7feb300ab7e0 con 0x7feb40155080 2023-11-13T11:04:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.933+0000 7feb46345640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb2406ae40 0x7feb2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.934+0000 7feb46345640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb2406ae40 0x7feb2406d300 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009a10 tx=0x7feb3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:15.937+0000 7feb36ffd640 1 -- 172.21.15.16:0/3752880826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb40151520 con 0x7feb40155080 2023-11-13T11:04:16.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.132+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7feb40150bb0 con 0x7feb2406ae40 2023-11-13T11:04:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.133+0000 7feb36ffd640 1 -- 172.21.15.16:0/3752880826 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7feb40150bb0 con 0x7feb2406ae40 2023-11-13T11:04:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2023-11-13T11:04:16.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb2406ae40 msgr2=0x7feb2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb2406ae40 0x7feb2406d300 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009a10 tx=0x7feb3c008040 comp rx=0 tx=0).stop 2023-11-13T11:04:16.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40155080 msgr2=0x7feb4014f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40155080 0x7feb4014f0d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7feb300098c0 tx=0x7feb300046c0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 shutdown_connections 2023-11-13T11:04:16.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7feb2406ae40 0x7feb2406d300 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb4015e5d0 0x7feb400b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb40155080 0x7feb4014f0d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 --2- 172.21.15.16:0/3752880826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb40071680 0x7feb40141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 >> 172.21.15.16:0/3752880826 conn(0x7feb400b0580 msgr2=0x7feb400cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 shutdown_connections 2023-11-13T11:04:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.136+0000 7feb47347640 1 -- 172.21.15.16:0/3752880826 wait complete. 2023-11-13T11:04:16.161 INFO:tasks.ceph:Scrubbing osd.4 2023-11-13T11:04:16.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-13T11:04:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.388+0000 7f6bcac4f640 1 Processor -- start 2023-11-13T11:04:16.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.388+0000 7f6bcac4f640 1 -- start start 2023-11-13T11:04:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.390+0000 7f6bcac4f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 0x7f6bc40ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.390+0000 7f6bcac4f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6bc40b18f0 0x7f6bc4071790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.390+0000 7f6bcac4f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc40b1df0 con 0x7f6bc40ce720 2023-11-13T11:04:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.390+0000 7f6bcac4f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc40b20d0 con 0x7f6bc40b18f0 2023-11-13T11:04:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.390+0000 7f6bcac4f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40b23b0 con 0x7f6bc415aaf0 2023-11-13T11:04:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.390+0000 7f6bc3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6bc40b18f0 0x7f6bc4071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.391+0000 7f6bc3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6bc40b18f0 0x7f6bc4071790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56090/0 (socket says 172.21.15.16:56090) 2023-11-13T11:04:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.391+0000 7f6bc3fff640 1 -- 172.21.15.16:0/679931498 learned_addr learned my addr 172.21.15.16:0/679931498 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.391+0000 7f6bc91ec640 1 --1- 172.21.15.16:0/679931498 >> v1:172.21.15.16:6789/0 conn(0x7f6bc40ce720 0x7f6bc406ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33874/0 (socket says 172.21.15.16:33874) 2023-11-13T11:04:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.391+0000 7f6bc89eb640 1 --2- 172.21.15.16:0/679931498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 0x7f6bc40ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.391+0000 7f6bc3fff640 1 -- 172.21.15.16:0/679931498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 msgr2=0x7f6bc40ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.391+0000 7f6bc3fff640 1 --2- 172.21.15.16:0/679931498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 0x7f6bc40ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.391+0000 7f6bc3fff640 1 -- 172.21.15.16:0/679931498 >> v1:172.21.15.16:6789/0 conn(0x7f6bc40ce720 legacy=0x7f6bc406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.391+0000 7f6bc3fff640 1 -- 172.21.15.16:0/679931498 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc40b2690 con 0x7f6bc40b18f0 2023-11-13T11:04:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.392+0000 7f6bc3fff640 1 --2- 172.21.15.16:0/679931498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 0x7f6bc4071790 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1defdace6474c97 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.393+0000 7f6bc37fe640 1 -- 172.21.15.16:0/679931498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0031640 con 0x7f6bc40b18f0 2023-11-13T11:04:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.393+0000 7f6bc37fe640 1 -- 172.21.15.16:0/679931498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb00317a0 con 0x7f6bc40b18f0 2023-11-13T11:04:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.393+0000 7f6bc37fe640 1 -- 172.21.15.16:0/679931498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0031b30 con 0x7f6bc40b18f0 2023-11-13T11:04:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.393+0000 7f6bcac4f640 1 -- 172.21.15.16:0/679931498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 msgr2=0x7f6bc4071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.393+0000 7f6bcac4f640 1 --2- 172.21.15.16:0/679931498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 0x7f6bc4071790 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb002b8f0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.393+0000 7f6bcac4f640 1 -- 172.21.15.16:0/679931498 shutdown_connections 2023-11-13T11:04:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.393+0000 7f6bcac4f640 1 --2- 172.21.15.16:0/679931498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 0x7f6bc4071790 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.393+0000 7f6bcac4f640 1 --2- 172.21.15.16:0/679931498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 0x7f6bc40ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.393+0000 7f6bcac4f640 1 -- 172.21.15.16:0/679931498 >> 172.21.15.16:0/679931498 conn(0x7f6bc406a190 msgr2=0x7f6bc406a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:16.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.394+0000 7f6bcac4f640 1 -- 172.21.15.16:0/679931498 shutdown_connections 2023-11-13T11:04:16.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.394+0000 7f6bcac4f640 1 -- 172.21.15.16:0/679931498 wait complete. 2023-11-13T11:04:16.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.395+0000 7f6bcac4f640 1 Processor -- start 2023-11-13T11:04:16.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.395+0000 7f6bcac4f640 1 -- start start 2023-11-13T11:04:16.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.396+0000 7f6bcac4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 0x7f6bc414c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.396+0000 7f6bcac4f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bc40ce720 0x7f6bc414f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.396+0000 7f6bcac4f640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 0x7f6bc4152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.396+0000 7f6bcac4f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc41519a0 con 0x7f6bc40b18f0 2023-11-13T11:04:16.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.396+0000 7f6bcac4f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc414c560 con 0x7f6bc40ce720 2023-11-13T11:04:16.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.396+0000 7f6bcac4f640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc414fcd0 con 0x7f6bc415aaf0 2023-11-13T11:04:16.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc89eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 0x7f6bc414c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc89eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 0x7f6bc414c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56098/0 (socket says 172.21.15.16:56098) 2023-11-13T11:04:16.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc89eb640 1 -- 172.21.15.16:0/2326175040 learned_addr learned my addr 172.21.15.16:0/2326175040 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:16.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc91ec640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 0x7f6bc4152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc89eb640 1 -- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 msgr2=0x7f6bc4152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc3fff640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bc40ce720 0x7f6bc414f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc89eb640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 0x7f6bc4152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc89eb640 1 -- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bc40ce720 msgr2=0x7f6bc414f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc89eb640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bc40ce720 0x7f6bc414f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc89eb640 1 -- 172.21.15.16:0/2326175040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00094d0 con 0x7f6bc40b18f0 2023-11-13T11:04:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.397+0000 7f6bc91ec640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6bc415aaf0 0x7f6bc4152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:16.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.398+0000 7f6bc89eb640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 0x7f6bc414c020 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800d5f0 tx=0x7f6bb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:16.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.398+0000 7f6bc17fa640 1 -- 172.21.15.16:0/2326175040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80125f0 con 0x7f6bc40b18f0 2023-11-13T11:04:16.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.398+0000 7f6bc17fa640 1 -- 172.21.15.16:0/2326175040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb8012750 con 0x7f6bc40b18f0 2023-11-13T11:04:16.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.398+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc414ffb0 con 0x7f6bc40b18f0 2023-11-13T11:04:16.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.398+0000 7f6bc17fa640 1 -- 172.21.15.16:0/2326175040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8012b00 con 0x7f6bc40b18f0 2023-11-13T11:04:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.398+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc4151f20 con 0x7f6bc40b18f0 2023-11-13T11:04:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.399+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6bc40cebb0 con 0x7f6bc40b18f0 2023-11-13T11:04:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.400+0000 7f6bc17fa640 1 -- 172.21.15.16:0/2326175040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bb8012c60 con 0x7f6bc40b18f0 2023-11-13T11:04:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.401+0000 7f6bc17fa640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ba806ae40 0x7f6ba806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.401+0000 7f6bc17fa640 1 -- 172.21.15.16:0/2326175040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f6bb808a500 con 0x7f6bc40b18f0 2023-11-13T11:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.401+0000 7f6bc17fa640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f6ba80711e0 0x7f6ba8073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.401+0000 7f6bc17fa640 1 -- 172.21.15.16:0/2326175040 --> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ba8073cf0 con 0x7f6ba80711e0 2023-11-13T11:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.401+0000 7f6bc17fa640 1 -- 172.21.15.16:0/2326175040 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bb808a870 con 0x7f6bc40b18f0 2023-11-13T11:04:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.402+0000 7f6bc3fff640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ba806ae40 0x7f6ba806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.402+0000 7f6bc91ec640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f6ba80711e0 0x7f6ba8073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.403+0000 7f6bc3fff640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ba806ae40 0x7f6ba806d300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0036000 tx=0x7f6bb003c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.403+0000 7f6bc91ec640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f6ba80711e0 0x7f6ba8073620 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.405+0000 7f6bc17fa640 1 -- 172.21.15.16:0/2326175040 <== osd.4 v2:172.21.15.184:6800/2936315873 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f6ba8073cf0 con 0x7f6ba80711e0 2023-11-13T11:04:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.420+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 --> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6bc4067090 con 0x7f6ba80711e0 2023-11-13T11:04:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.421+0000 7f6bc17fa640 1 -- 172.21.15.16:0/2326175040 <== osd.4 v2:172.21.15.184:6800/2936315873 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6bc4067090 con 0x7f6ba80711e0 2023-11-13T11:04:16.425 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:04:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-13T11:04:16.426 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:04:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.421+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f6ba80711e0 msgr2=0x7f6ba8073620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.421+0000 7f6bcac4f640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6800/2936315873,v1:172.21.15.184:6801/2936315873] conn(0x7f6ba80711e0 0x7f6ba8073620 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.421+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ba806ae40 msgr2=0x7f6ba806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.421+0000 7f6bcac4f640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f6ba806ae40 0x7f6ba806d300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0036000 tx=0x7f6bb003c040 comp rx=0 tx=0).stop 2023-11-13T11:04:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.421+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 msgr2=0x7f6bc414c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.421+0000 7f6bcac4f640 1 --2- 172.21.15.16:0/2326175040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc40b18f0 0x7f6bc414c020 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800d5f0 tx=0x7f6bb800dab0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.422+0000 7f6bc91ec640 1 -- 172.21.15.16:0/2326175040 reap_dead start 2023-11-13T11:04:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.422+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 shutdown_connections 2023-11-13T11:04:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.422+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 >> 172.21.15.16:0/2326175040 conn(0x7f6bc406a190 msgr2=0x7f6bc415daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.422+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 shutdown_connections 2023-11-13T11:04:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.422+0000 7f6bcac4f640 1 -- 172.21.15.16:0/2326175040 wait complete. 2023-11-13T11:04:16.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-13T11:04:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.667+0000 7f88b359e640 1 Processor -- start 2023-11-13T11:04:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.667+0000 7f88b359e640 1 -- start start 2023-11-13T11:04:16.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.669+0000 7f88b359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88b415cde0 0x7f88b40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.669+0000 7f88b359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f88b40cc610 0x7f88b40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.669+0000 7f88b359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b415d1c0 con 0x7f88b406ce70 2023-11-13T11:04:16.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.669+0000 7f88b359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b40cc1f0 con 0x7f88b415cde0 2023-11-13T11:04:16.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.669+0000 7f88b359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b40cb890 con 0x7f88b40cc610 2023-11-13T11:04:16.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.669+0000 7f88b259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88b415cde0 0x7f88b40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.669+0000 7f88b259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88b415cde0 0x7f88b40cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56112/0 (socket says 172.21.15.16:56112) 2023-11-13T11:04:16.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.669+0000 7f88b259c640 1 -- 172.21.15.16:0/1162106726 learned_addr learned my addr 172.21.15.16:0/1162106726 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.670+0000 7f88b1d9b640 1 --2- 172.21.15.16:0/1162106726 >> v2:172.21.15.184:3300/0 conn(0x7f88b40cc610 0x7f88b40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.670+0000 7f88b259c640 1 -- 172.21.15.16:0/1162106726 >> v2:172.21.15.184:3300/0 conn(0x7f88b40cc610 msgr2=0x7f88b40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.670+0000 7f88b259c640 1 --2- 172.21.15.16:0/1162106726 >> v2:172.21.15.184:3300/0 conn(0x7f88b40cc610 0x7f88b40cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.670+0000 7f88b259c640 1 -- 172.21.15.16:0/1162106726 >> v1:172.21.15.184:6789/0 conn(0x7f88b406ce70 legacy=0x7f88b406d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.670+0000 7f88b259c640 1 -- 172.21.15.16:0/1162106726 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b406db40 con 0x7f88b415cde0 2023-11-13T11:04:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.672+0000 7f88b259c640 1 --2- 172.21.15.16:0/1162106726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b415cde0 0x7f88b40cf3f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f88b406dd30 tx=0x7f88a402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559f24cc2fd6a70a server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.672+0000 7f88b159a640 1 -- 172.21.15.16:0/1162106726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a402c950 con 0x7f88b415cde0 2023-11-13T11:04:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.672+0000 7f88b159a640 1 -- 172.21.15.16:0/1162106726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a402cab0 con 0x7f88b415cde0 2023-11-13T11:04:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.672+0000 7f88b159a640 1 -- 172.21.15.16:0/1162106726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a402ce40 con 0x7f88b415cde0 2023-11-13T11:04:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.673+0000 7f88b359e640 1 -- 172.21.15.16:0/1162106726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b415cde0 msgr2=0x7f88b40cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.673+0000 7f88b359e640 1 --2- 172.21.15.16:0/1162106726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b415cde0 0x7f88b40cf3f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f88b406dd30 tx=0x7f88a402bdb0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.673+0000 7f88b359e640 1 -- 172.21.15.16:0/1162106726 shutdown_connections 2023-11-13T11:04:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.673+0000 7f88b359e640 1 --2- 172.21.15.16:0/1162106726 >> v2:172.21.15.184:3300/0 conn(0x7f88b40cc610 0x7f88b40cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.673+0000 7f88b359e640 1 --2- 172.21.15.16:0/1162106726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b415cde0 0x7f88b40cf3f0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.673+0000 7f88b359e640 1 -- 172.21.15.16:0/1162106726 >> 172.21.15.16:0/1162106726 conn(0x7f88b406a5b0 msgr2=0x7f88b406a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:16.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.673+0000 7f88b359e640 1 -- 172.21.15.16:0/1162106726 shutdown_connections 2023-11-13T11:04:16.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.673+0000 7f88b359e640 1 -- 172.21.15.16:0/1162106726 wait complete. 2023-11-13T11:04:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.675+0000 7f88b359e640 1 Processor -- start 2023-11-13T11:04:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.675+0000 7f88b359e640 1 -- start start 2023-11-13T11:04:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.676+0000 7f88b359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b406ce70 0x7f88b414aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.676+0000 7f88b359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b40cc610 0x7f88b40abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.676+0000 7f88b359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b415cde0 0x7f88b4070d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.676+0000 7f88b359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b414b0e0 con 0x7f88b406ce70 2023-11-13T11:04:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.676+0000 7f88b359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40ad900 con 0x7f88b415cde0 2023-11-13T11:04:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.676+0000 7f88b359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40ad470 con 0x7f88b40cc610 2023-11-13T11:04:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.676+0000 7f88b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b406ce70 0x7f88b414aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.676+0000 7f88b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b406ce70 0x7f88b414aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56120/0 (socket says 172.21.15.16:56120) 2023-11-13T11:04:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.676+0000 7f88b259c640 1 -- 172.21.15.16:0/3849401516 learned_addr learned my addr 172.21.15.16:0/3849401516 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88b1d9b640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b40cc610 0x7f88b40abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88b2d9d640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b415cde0 0x7f88b4070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88b259c640 1 -- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b40cc610 msgr2=0x7f88b40abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88b259c640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b40cc610 0x7f88b40abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88b259c640 1 -- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b415cde0 msgr2=0x7f88b4070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88b259c640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b415cde0 0x7f88b4070d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88b259c640 1 -- 172.21.15.16:0/3849401516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4009b30 con 0x7f88b406ce70 2023-11-13T11:04:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88b259c640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b406ce70 0x7f88b414aba0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f88a4036040 tx=0x7f88a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:16.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88aaffd640 1 -- 172.21.15.16:0/3849401516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4009510 con 0x7f88b406ce70 2023-11-13T11:04:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88aaffd640 1 -- 172.21.15.16:0/3849401516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a403e450 con 0x7f88b406ce70 2023-11-13T11:04:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88aaffd640 1 -- 172.21.15.16:0/3849401516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a403e800 con 0x7f88b406ce70 2023-11-13T11:04:16.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.677+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b40ac860 con 0x7f88b406ce70 2023-11-13T11:04:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.678+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b40ace20 con 0x7f88b406ce70 2023-11-13T11:04:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.679+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b404e4d0 con 0x7f88b406ce70 2023-11-13T11:04:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.680+0000 7f88aaffd640 1 -- 172.21.15.16:0/3849401516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88a403e960 con 0x7f88b406ce70 2023-11-13T11:04:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.681+0000 7f88aaffd640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f889406ae70 0x7f889406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.681+0000 7f88aaffd640 1 -- 172.21.15.16:0/3849401516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f88a40abf90 con 0x7f88b406ce70 2023-11-13T11:04:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.681+0000 7f88b1d9b640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f889406ae70 0x7f889406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.683+0000 7f88b1d9b640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f889406ae70 0x7f889406d330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f88b4155910 tx=0x7f889c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.685+0000 7f88aaffd640 1 -- 172.21.15.16:0/3849401516 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88a40771b0 con 0x7f88b406ce70 2023-11-13T11:04:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.881+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f88b40cbe10 con 0x7f889406ae70 2023-11-13T11:04:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.883+0000 7f88aaffd640 1 -- 172.21.15.16:0/3849401516 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f889c009d80 con 0x7f889406ae70 2023-11-13T11:04:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 4 to deep-scrub 2023-11-13T11:04:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f889406ae70 msgr2=0x7f889406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f889406ae70 0x7f889406d330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f88b4155910 tx=0x7f889c008040 comp rx=0 tx=0).stop 2023-11-13T11:04:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b406ce70 msgr2=0x7f88b414aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b406ce70 0x7f88b414aba0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f88a4036040 tx=0x7f88a4004a40 comp rx=0 tx=0).stop 2023-11-13T11:04:16.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 shutdown_connections 2023-11-13T11:04:16.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f889406ae70 0x7f889406d330 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88b415cde0 0x7f88b4070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b40cc610 0x7f88b40abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 --2- 172.21.15.16:0/3849401516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b406ce70 0x7f88b414aba0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.885+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 >> 172.21.15.16:0/3849401516 conn(0x7f88b406a5b0 msgr2=0x7f88b406d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.886+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 shutdown_connections 2023-11-13T11:04:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:16.886+0000 7f88b359e640 1 -- 172.21.15.16:0/3849401516 wait complete. 2023-11-13T11:04:16.907 INFO:tasks.ceph:Scrubbing osd.5 2023-11-13T11:04:16.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-13T11:04:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.133+0000 7fcad5432640 1 Processor -- start 2023-11-13T11:04:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.133+0000 7fcad5432640 1 -- start start 2023-11-13T11:04:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.135+0000 7fcad5432640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcad01442a0 0x7fcad00b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.135+0000 7fcad5432640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00ce300 0x7fcad00cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.135+0000 7fcad5432640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad00b2930 con 0x7fcad00cbbe0 2023-11-13T11:04:17.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.135+0000 7fcad5432640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad00cc5b0 con 0x7fcad01442a0 2023-11-13T11:04:17.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.135+0000 7fcad5432640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad015bfb0 con 0x7fcad00ce300 2023-11-13T11:04:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.135+0000 7fcacffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcad01442a0 0x7fcad00b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.135+0000 7fcacffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcad01442a0 0x7fcad00b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56136/0 (socket says 172.21.15.16:56136) 2023-11-13T11:04:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.135+0000 7fcacffff640 1 -- 172.21.15.16:0/3580053293 learned_addr learned my addr 172.21.15.16:0/3580053293 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.135+0000 7fcacf7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcad00cbbe0 0x7fcad00cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33890/0 (socket says 172.21.15.16:33890) 2023-11-13T11:04:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.136+0000 7fcad4c31640 1 --2- 172.21.15.16:0/3580053293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00ce300 0x7fcad00cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.136+0000 7fcacffff640 1 -- 172.21.15.16:0/3580053293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00ce300 msgr2=0x7fcad00cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:17.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.136+0000 7fcacffff640 1 --2- 172.21.15.16:0/3580053293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00ce300 0x7fcad00cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.136+0000 7fcacffff640 1 -- 172.21.15.16:0/3580053293 >> v1:172.21.15.16:6789/0 conn(0x7fcad00cbbe0 legacy=0x7fcad00cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.136+0000 7fcacffff640 1 -- 172.21.15.16:0/3580053293 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad00b2c10 con 0x7fcad01442a0 2023-11-13T11:04:17.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.136+0000 7fcaceffd640 1 -- 172.21.15.16:0/3580053293 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188539469 0 0) 0x7fcad00b2930 con 0x7fcad00cbbe0 2023-11-13T11:04:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.137+0000 7fcacffff640 1 --2- 172.21.15.16:0/3580053293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 0x7fcad00b39d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcac00098a0 tx=0x7fcac002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f7e54b80007ef server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.137+0000 7fcaceffd640 1 -- 172.21.15.16:0/3580053293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0030840 con 0x7fcad01442a0 2023-11-13T11:04:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.137+0000 7fcaceffd640 1 -- 172.21.15.16:0/3580053293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac00309a0 con 0x7fcad01442a0 2023-11-13T11:04:17.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.138+0000 7fcaceffd640 1 -- 172.21.15.16:0/3580053293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0030d30 con 0x7fcad01442a0 2023-11-13T11:04:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.138+0000 7fcad5432640 1 -- 172.21.15.16:0/3580053293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 msgr2=0x7fcad00b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.138+0000 7fcad5432640 1 --2- 172.21.15.16:0/3580053293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 0x7fcad00b39d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcac00098a0 tx=0x7fcac002b980 comp rx=0 tx=0).stop 2023-11-13T11:04:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.138+0000 7fcad5432640 1 -- 172.21.15.16:0/3580053293 shutdown_connections 2023-11-13T11:04:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.138+0000 7fcad5432640 1 --2- 172.21.15.16:0/3580053293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00ce300 0x7fcad00cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.138+0000 7fcad5432640 1 --2- 172.21.15.16:0/3580053293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 0x7fcad00b39d0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.138+0000 7fcad5432640 1 -- 172.21.15.16:0/3580053293 >> 172.21.15.16:0/3580053293 conn(0x7fcad00b0970 msgr2=0x7fcad00b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.138+0000 7fcad5432640 1 -- 172.21.15.16:0/3580053293 shutdown_connections 2023-11-13T11:04:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.138+0000 7fcad5432640 1 -- 172.21.15.16:0/3580053293 wait complete. 2023-11-13T11:04:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.140+0000 7fcad5432640 1 Processor -- start 2023-11-13T11:04:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.140+0000 7fcad5432640 1 -- start start 2023-11-13T11:04:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.141+0000 7fcad5432640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00cbbe0 0x7fcad0140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.141+0000 7fcad5432640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcad00ce300 0x7fcad01311f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.141+0000 7fcad5432640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 0x7fcad0131730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.141+0000 7fcad5432640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad01410b0 con 0x7fcad01442a0 2023-11-13T11:04:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.141+0000 7fcad5432640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad012f1b0 con 0x7fcad00ce300 2023-11-13T11:04:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.141+0000 7fcad5432640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad0131d90 con 0x7fcad00cbbe0 2023-11-13T11:04:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.141+0000 7fcad4c31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 0x7fcad0131730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.141+0000 7fcad4c31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 0x7fcad0131730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56140/0 (socket says 172.21.15.16:56140) 2023-11-13T11:04:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.141+0000 7fcad4c31640 1 -- 172.21.15.16:0/2585691842 learned_addr learned my addr 172.21.15.16:0/2585691842 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.142+0000 7fcacf7fe640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcad00ce300 0x7fcad01311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.142+0000 7fcacffff640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00cbbe0 0x7fcad0140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.142+0000 7fcad4c31640 1 -- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00cbbe0 msgr2=0x7fcad0140b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.142+0000 7fcad4c31640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00cbbe0 0x7fcad0140b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.142+0000 7fcad4c31640 1 -- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcad00ce300 msgr2=0x7fcad01311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.142+0000 7fcad4c31640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcad00ce300 0x7fcad01311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.142+0000 7fcad4c31640 1 -- 172.21.15.16:0/2585691842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac00094d0 con 0x7fcad01442a0 2023-11-13T11:04:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.142+0000 7fcacffff640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcad00cbbe0 0x7fcad0140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.142+0000 7fcad4c31640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 0x7fcad0131730 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fcac8007e40 tx=0x7fcac800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.143+0000 7fcaccff9640 1 -- 172.21.15.16:0/2585691842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac8014620 con 0x7fcad01442a0 2023-11-13T11:04:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.143+0000 7fcaccff9640 1 -- 172.21.15.16:0/2585691842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac800ee90 con 0x7fcad01442a0 2023-11-13T11:04:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.143+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad0132070 con 0x7fcad01442a0 2023-11-13T11:04:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.143+0000 7fcaccff9640 1 -- 172.21.15.16:0/2585691842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac80149e0 con 0x7fcad01442a0 2023-11-13T11:04:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.143+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad012f750 con 0x7fcad01442a0 2023-11-13T11:04:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.144+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcad00b34b0 con 0x7fcad01442a0 2023-11-13T11:04:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.145+0000 7fcaccff9640 1 -- 172.21.15.16:0/2585691842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcac8014b40 con 0x7fcad01442a0 2023-11-13T11:04:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.146+0000 7fcaccff9640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcabc06ae40 0x7fcabc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.146+0000 7fcaccff9640 1 -- 172.21.15.16:0/2585691842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fcac8089f90 con 0x7fcad01442a0 2023-11-13T11:04:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.146+0000 7fcaccff9640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7fcabc0711e0 0x7fcabc073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.146+0000 7fcaccff9640 1 -- 172.21.15.16:0/2585691842 --> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcabc073cf0 con 0x7fcabc0711e0 2023-11-13T11:04:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.146+0000 7fcaccff9640 1 -- 172.21.15.16:0/2585691842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcac808c570 con 0x7fcad01442a0 2023-11-13T11:04:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.147+0000 7fcacffff640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcabc06ae40 0x7fcabc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.147+0000 7fcacf7fe640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7fcabc0711e0 0x7fcabc073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.148+0000 7fcacffff640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcabc06ae40 0x7fcabc06d300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcac0036000 tx=0x7fcac003b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.148+0000 7fcacf7fe640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7fcabc0711e0 0x7fcabc073620 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.150+0000 7fcaccff9640 1 -- 172.21.15.16:0/2585691842 <== osd.5 v2:172.21.15.184:6808/1150854014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fcabc073cf0 con 0x7fcabc0711e0 2023-11-13T11:04:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.165+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 --> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcad00b31d0 con 0x7fcabc0711e0 2023-11-13T11:04:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.166+0000 7fcaccff9640 1 -- 172.21.15.16:0/2585691842 <== osd.5 v2:172.21.15.184:6808/1150854014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcac4007170 con 0x7fcabc0711e0 2023-11-13T11:04:17.171 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:04:17.171 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-13T11:04:17.171 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:04:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7fcabc0711e0 msgr2=0x7fcabc073620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6808/1150854014,v1:172.21.15.184:6809/1150854014] conn(0x7fcabc0711e0 0x7fcabc073620 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcabc06ae40 msgr2=0x7fcabc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcabc06ae40 0x7fcabc06d300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcac0036000 tx=0x7fcac003b040 comp rx=0 tx=0).stop 2023-11-13T11:04:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 msgr2=0x7fcad0131730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 --2- 172.21.15.16:0/2585691842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcad01442a0 0x7fcad0131730 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fcac8007e40 tx=0x7fcac800e4b0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad4c31640 1 -- 172.21.15.16:0/2585691842 reap_dead start 2023-11-13T11:04:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 shutdown_connections 2023-11-13T11:04:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 >> 172.21.15.16:0/2585691842 conn(0x7fcad00b0970 msgr2=0x7fcad0141be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 shutdown_connections 2023-11-13T11:04:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.167+0000 7fcad5432640 1 -- 172.21.15.16:0/2585691842 wait complete. 2023-11-13T11:04:17.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-13T11:04:17.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.406+0000 7fdc5e990640 1 Processor -- start 2023-11-13T11:04:17.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.407+0000 7fdc5e990640 1 -- start start 2023-11-13T11:04:17.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.408+0000 7fdc5e990640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc5815aaf0 0x7fdc580ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.408+0000 7fdc5e990640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc580ce720 0x7fdc580cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.408+0000 7fdc5e990640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc58071ea0 con 0x7fdc580b18f0 2023-11-13T11:04:17.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.408+0000 7fdc5e990640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc580cf0f0 con 0x7fdc5815aaf0 2023-11-13T11:04:17.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.409+0000 7fdc5e990640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc580b1df0 con 0x7fdc580ce720 2023-11-13T11:04:17.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.409+0000 7fdc57fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc5815aaf0 0x7fdc580ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.409+0000 7fdc57fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc5815aaf0 0x7fdc580ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56142/0 (socket says 172.21.15.16:56142) 2023-11-13T11:04:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.409+0000 7fdc57fff640 1 -- 172.21.15.16:0/1285377243 learned_addr learned my addr 172.21.15.16:0/1285377243 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.409+0000 7fdc5cf2d640 1 --2- 172.21.15.16:0/1285377243 >> v2:172.21.15.184:3300/0 conn(0x7fdc580ce720 0x7fdc580cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.410+0000 7fdc57fff640 1 -- 172.21.15.16:0/1285377243 >> v2:172.21.15.184:3300/0 conn(0x7fdc580ce720 msgr2=0x7fdc580cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.410+0000 7fdc57fff640 1 --2- 172.21.15.16:0/1285377243 >> v2:172.21.15.184:3300/0 conn(0x7fdc580ce720 0x7fdc580cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.410+0000 7fdc57fff640 1 -- 172.21.15.16:0/1285377243 >> v1:172.21.15.184:6789/0 conn(0x7fdc580b18f0 legacy=0x7fdc58071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.410+0000 7fdc57fff640 1 -- 172.21.15.16:0/1285377243 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc580b20d0 con 0x7fdc5815aaf0 2023-11-13T11:04:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.411+0000 7fdc57fff640 1 --2- 172.21.15.16:0/1285377243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 0x7fdc580ccc50 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fdc580ae4d0 tx=0x7fdc4802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad28f9fa05edf04 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.411+0000 7fdc577fe640 1 -- 172.21.15.16:0/1285377243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480317e0 con 0x7fdc5815aaf0 2023-11-13T11:04:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.412+0000 7fdc577fe640 1 -- 172.21.15.16:0/1285377243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc48031940 con 0x7fdc5815aaf0 2023-11-13T11:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.412+0000 7fdc577fe640 1 -- 172.21.15.16:0/1285377243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48031cd0 con 0x7fdc5815aaf0 2023-11-13T11:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.412+0000 7fdc5e990640 1 -- 172.21.15.16:0/1285377243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 msgr2=0x7fdc580ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.412+0000 7fdc5e990640 1 --2- 172.21.15.16:0/1285377243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 0x7fdc580ccc50 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fdc580ae4d0 tx=0x7fdc4802b880 comp rx=0 tx=0).stop 2023-11-13T11:04:17.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.412+0000 7fdc5e990640 1 -- 172.21.15.16:0/1285377243 shutdown_connections 2023-11-13T11:04:17.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.412+0000 7fdc5e990640 1 --2- 172.21.15.16:0/1285377243 >> v2:172.21.15.184:3300/0 conn(0x7fdc580ce720 0x7fdc580cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.412+0000 7fdc5e990640 1 --2- 172.21.15.16:0/1285377243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 0x7fdc580ccc50 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.412+0000 7fdc5e990640 1 -- 172.21.15.16:0/1285377243 >> 172.21.15.16:0/1285377243 conn(0x7fdc5806a190 msgr2=0x7fdc5806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.412+0000 7fdc5e990640 1 -- 172.21.15.16:0/1285377243 shutdown_connections 2023-11-13T11:04:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.413+0000 7fdc5e990640 1 -- 172.21.15.16:0/1285377243 wait complete. 2023-11-13T11:04:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.414+0000 7fdc5e990640 1 Processor -- start 2023-11-13T11:04:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.414+0000 7fdc5e990640 1 -- start start 2023-11-13T11:04:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.415+0000 7fdc5e990640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc580b18f0 0x7fdc5813c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.415+0000 7fdc5e990640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc580ce720 0x7fdc58155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.415+0000 7fdc5e990640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 0x7fdc58152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.415+0000 7fdc5e990640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5814fd60 con 0x7fdc5815aaf0 2023-11-13T11:04:17.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.415+0000 7fdc5e990640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc581501f0 con 0x7fdc580b18f0 2023-11-13T11:04:17.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.415+0000 7fdc5e990640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5814f150 con 0x7fdc580ce720 2023-11-13T11:04:17.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc5cf2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 0x7fdc58152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc5cf2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 0x7fdc58152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56146/0 (socket says 172.21.15.16:56146) 2023-11-13T11:04:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc5cf2d640 1 -- 172.21.15.16:0/2755490600 learned_addr learned my addr 172.21.15.16:0/2755490600 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc57fff640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc580b18f0 0x7fdc5813c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc4ffff640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc580ce720 0x7fdc58155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc5cf2d640 1 -- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc580ce720 msgr2=0x7fdc58155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc5cf2d640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc580ce720 0x7fdc58155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc5cf2d640 1 -- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc580b18f0 msgr2=0x7fdc5813c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc5cf2d640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc580b18f0 0x7fdc5813c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc5cf2d640 1 -- 172.21.15.16:0/2755490600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480094d0 con 0x7fdc5815aaf0 2023-11-13T11:04:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.416+0000 7fdc57fff640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc580b18f0 0x7fdc5813c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.417+0000 7fdc5cf2d640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 0x7fdc58152f40 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fdc50007e40 tx=0x7fdc5000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.417+0000 7fdc557fa640 1 -- 172.21.15.16:0/2755490600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50014620 con 0x7fdc5815aaf0 2023-11-13T11:04:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.417+0000 7fdc557fa640 1 -- 172.21.15.16:0/2755490600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5000ee90 con 0x7fdc5815aaf0 2023-11-13T11:04:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.417+0000 7fdc557fa640 1 -- 172.21.15.16:0/2755490600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500149e0 con 0x7fdc5815aaf0 2023-11-13T11:04:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.417+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5814f430 con 0x7fdc5815aaf0 2023-11-13T11:04:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.417+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5814f9f0 con 0x7fdc5815aaf0 2023-11-13T11:04:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.418+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc58067090 con 0x7fdc5815aaf0 2023-11-13T11:04:17.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.419+0000 7fdc557fa640 1 -- 172.21.15.16:0/2755490600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc50014b40 con 0x7fdc5815aaf0 2023-11-13T11:04:17.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.420+0000 7fdc557fa640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc3806ae40 0x7fdc3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.420+0000 7fdc557fa640 1 -- 172.21.15.16:0/2755490600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fdc50089ee0 con 0x7fdc5815aaf0 2023-11-13T11:04:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.421+0000 7fdc57fff640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc3806ae40 0x7fdc3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.422+0000 7fdc57fff640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc3806ae40 0x7fdc3806d300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdc48036000 tx=0x7fdc4803d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.424+0000 7fdc557fa640 1 -- 172.21.15.16:0/2755490600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc500550e0 con 0x7fdc5815aaf0 2023-11-13T11:04:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.617+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc580b2690 con 0x7fdc3806ae40 2023-11-13T11:04:17.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.618+0000 7fdc557fa640 1 -- 172.21.15.16:0/2755490600 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdc48031b30 con 0x7fdc3806ae40 2023-11-13T11:04:17.622 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 5 to deep-scrub 2023-11-13T11:04:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc3806ae40 msgr2=0x7fdc3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc3806ae40 0x7fdc3806d300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdc48036000 tx=0x7fdc4803d040 comp rx=0 tx=0).stop 2023-11-13T11:04:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 msgr2=0x7fdc58152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 0x7fdc58152f40 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fdc50007e40 tx=0x7fdc5000e4b0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 shutdown_connections 2023-11-13T11:04:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fdc3806ae40 0x7fdc3806d300 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5815aaf0 0x7fdc58152f40 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc580ce720 0x7fdc58155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 --2- 172.21.15.16:0/2755490600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc580b18f0 0x7fdc5813c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.620+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 >> 172.21.15.16:0/2755490600 conn(0x7fdc5806a190 msgr2=0x7fdc5806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.621+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 shutdown_connections 2023-11-13T11:04:17.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.621+0000 7fdc5e990640 1 -- 172.21.15.16:0/2755490600 wait complete. 2023-11-13T11:04:17.644 INFO:tasks.ceph:Scrubbing osd.6 2023-11-13T11:04:17.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-13T11:04:17.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.864+0000 7f932087c640 1 Processor -- start 2023-11-13T11:04:17.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.864+0000 7f932087c640 1 -- start start 2023-11-13T11:04:17.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.865+0000 7f932087c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f931c159ae0 0x7f931c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.865+0000 7f932087c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f931c0adbc0 0x7f931c0ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.865+0000 7f932087c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c15d680 0x7f931c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f932087c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c159ec0 con 0x7f931c0adbc0 2023-11-13T11:04:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f932087c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c15dc20 con 0x7f931c159ae0 2023-11-13T11:04:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f932087c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c15df00 con 0x7f931c15d680 2023-11-13T11:04:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f9313fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f931c0adbc0 0x7f931c0ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f9313fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f931c0adbc0 0x7f931c0ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56154/0 (socket says 172.21.15.16:56154) 2023-11-13T11:04:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f9313fff640 1 -- 172.21.15.16:0/999582663 learned_addr learned my addr 172.21.15.16:0/999582663 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f931a59c640 1 --2- 172.21.15.16:0/999582663 >> v2:172.21.15.184:3300/0 conn(0x7f931c159ae0 0x7f931c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f931ad9d640 1 --2- 172.21.15.16:0/999582663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c15d680 0x7f931c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f9313fff640 1 -- 172.21.15.16:0/999582663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c15d680 msgr2=0x7f931c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f9313fff640 1 --2- 172.21.15.16:0/999582663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c15d680 0x7f931c0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f9313fff640 1 -- 172.21.15.16:0/999582663 >> v2:172.21.15.184:3300/0 conn(0x7f931c159ae0 msgr2=0x7f931c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f9313fff640 1 --2- 172.21.15.16:0/999582663 >> v2:172.21.15.184:3300/0 conn(0x7f931c159ae0 0x7f931c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f9313fff640 1 -- 172.21.15.16:0/999582663 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c15e1e0 con 0x7f931c0adbc0 2023-11-13T11:04:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.866+0000 7f931ad9d640 1 --2- 172.21.15.16:0/999582663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c15d680 0x7f931c0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:17.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.868+0000 7f9313fff640 1 --2- 172.21.15.16:0/999582663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0adbc0 0x7f931c0ce870 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f931c0cf960 tx=0x7f930402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d2eeb81521d242 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.868+0000 7f9319d9b640 1 -- 172.21.15.16:0/999582663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304031880 con 0x7f931c0adbc0 2023-11-13T11:04:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.868+0000 7f9319d9b640 1 -- 172.21.15.16:0/999582663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93040319e0 con 0x7f931c0adbc0 2023-11-13T11:04:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.868+0000 7f9319d9b640 1 -- 172.21.15.16:0/999582663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304031d70 con 0x7f931c0adbc0 2023-11-13T11:04:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.869+0000 7f932087c640 1 -- 172.21.15.16:0/999582663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0adbc0 msgr2=0x7f931c0ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.869+0000 7f932087c640 1 --2- 172.21.15.16:0/999582663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0adbc0 0x7f931c0ce870 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f931c0cf960 tx=0x7f930402b920 comp rx=0 tx=0).stop 2023-11-13T11:04:17.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.869+0000 7f932087c640 1 -- 172.21.15.16:0/999582663 shutdown_connections 2023-11-13T11:04:17.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.869+0000 7f932087c640 1 --2- 172.21.15.16:0/999582663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c15d680 0x7f931c0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.869+0000 7f932087c640 1 --2- 172.21.15.16:0/999582663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c0adbc0 0x7f931c0ce870 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.869+0000 7f932087c640 1 --2- 172.21.15.16:0/999582663 >> v2:172.21.15.184:3300/0 conn(0x7f931c159ae0 0x7f931c0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.869+0000 7f932087c640 1 -- 172.21.15.16:0/999582663 >> 172.21.15.16:0/999582663 conn(0x7f931c06a190 msgr2=0x7f931c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:17.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.869+0000 7f932087c640 1 -- 172.21.15.16:0/999582663 shutdown_connections 2023-11-13T11:04:17.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.869+0000 7f932087c640 1 -- 172.21.15.16:0/999582663 wait complete. 2023-11-13T11:04:17.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.871+0000 7f932087c640 1 Processor -- start 2023-11-13T11:04:17.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.871+0000 7f932087c640 1 -- start start 2023-11-13T11:04:17.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f932087c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c0adbc0 0x7f931c13ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f932087c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c159ae0 0x7f931c15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f932087c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c15d680 0x7f931c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f932087c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c148ad0 con 0x7f931c159ae0 2023-11-13T11:04:17.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f932087c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c148d70 con 0x7f931c15d680 2023-11-13T11:04:17.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f932087c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c149050 con 0x7f931c0adbc0 2023-11-13T11:04:17.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f9313fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c159ae0 0x7f931c15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f9313fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c159ae0 0x7f931c15c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56156/0 (socket says 172.21.15.16:56156) 2023-11-13T11:04:17.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f9313fff640 1 -- 172.21.15.16:0/3442528797 learned_addr learned my addr 172.21.15.16:0/3442528797 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:17.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f931a59c640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c0adbc0 0x7f931c13ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.872+0000 7f931ad9d640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c15d680 0x7f931c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f9313fff640 1 -- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c0adbc0 msgr2=0x7f931c13ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f9313fff640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f931c0adbc0 0x7f931c13ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f9313fff640 1 -- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c15d680 msgr2=0x7f931c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f9313fff640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c15d680 0x7f931c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f9313fff640 1 -- 172.21.15.16:0/3442528797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93040094d0 con 0x7f931c159ae0 2023-11-13T11:04:17.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f931ad9d640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c15d680 0x7f931c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:17.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f9313fff640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c159ae0 0x7f931c15c290 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f930402b960 tx=0x7f93040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f9312ffd640 1 -- 172.21.15.16:0/3442528797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930403ad30 con 0x7f931c159ae0 2023-11-13T11:04:17.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f9312ffd640 1 -- 172.21.15.16:0/3442528797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93040443f0 con 0x7f931c159ae0 2023-11-13T11:04:17.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f9312ffd640 1 -- 172.21.15.16:0/3442528797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040447a0 con 0x7f931c159ae0 2023-11-13T11:04:17.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c149330 con 0x7f931c159ae0 2023-11-13T11:04:17.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.873+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c13f490 con 0x7f931c159ae0 2023-11-13T11:04:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.874+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f931c148ee0 con 0x7f931c159ae0 2023-11-13T11:04:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.876+0000 7f9312ffd640 1 -- 172.21.15.16:0/3442528797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f930404d450 con 0x7f931c159ae0 2023-11-13T11:04:17.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.877+0000 7f9312ffd640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f92fc06ae40 0x7f92fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.877+0000 7f9312ffd640 1 -- 172.21.15.16:0/3442528797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f93040abb50 con 0x7f931c159ae0 2023-11-13T11:04:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.877+0000 7f9312ffd640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f92fc0711e0 0x7f92fc073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.877+0000 7f9312ffd640 1 -- 172.21.15.16:0/3442528797 --> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92fc073cf0 con 0x7f92fc0711e0 2023-11-13T11:04:17.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.877+0000 7f9312ffd640 1 -- 172.21.15.16:0/3442528797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93040b02a0 con 0x7f931c159ae0 2023-11-13T11:04:17.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.878+0000 7f931a59c640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f92fc06ae40 0x7f92fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.878+0000 7f931ad9d640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f92fc0711e0 0x7f92fc073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:17.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.879+0000 7f931a59c640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f92fc06ae40 0x7f92fc06d300 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f930c006fd0 tx=0x7f930c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.879+0000 7f931ad9d640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f92fc0711e0 0x7f92fc073620 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:17.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.881+0000 7f9312ffd640 1 -- 172.21.15.16:0/3442528797 <== osd.6 v2:172.21.15.184:6816/1720475067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f92fc073cf0 con 0x7f92fc0711e0 2023-11-13T11:04:17.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.896+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 --> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f931c06c4a0 con 0x7f92fc0711e0 2023-11-13T11:04:17.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.897+0000 7f9312ffd640 1 -- 172.21.15.16:0/3442528797 <== osd.6 v2:172.21.15.184:6816/1720475067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f931400f450 con 0x7f92fc0711e0 2023-11-13T11:04:17.901 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:04:17.902 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-13T11:04:17.902 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:04:17.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.897+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f92fc0711e0 msgr2=0x7f92fc073620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.897+0000 7f932087c640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6816/1720475067,v1:172.21.15.184:6817/1720475067] conn(0x7f92fc0711e0 0x7f92fc073620 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.898+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f92fc06ae40 msgr2=0x7f92fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.898+0000 7f932087c640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f92fc06ae40 0x7f92fc06d300 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f930c006fd0 tx=0x7f930c008040 comp rx=0 tx=0).stop 2023-11-13T11:04:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.898+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c159ae0 msgr2=0x7f931c15c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.898+0000 7f932087c640 1 --2- 172.21.15.16:0/3442528797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f931c159ae0 0x7f931c15c290 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f930402b960 tx=0x7f93040046c0 comp rx=0 tx=0).stop 2023-11-13T11:04:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.898+0000 7f931ad9d640 1 -- 172.21.15.16:0/3442528797 reap_dead start 2023-11-13T11:04:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.898+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 shutdown_connections 2023-11-13T11:04:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.898+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 >> 172.21.15.16:0/3442528797 conn(0x7f931c06a190 msgr2=0x7f931c157260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.898+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 shutdown_connections 2023-11-13T11:04:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:17.898+0000 7f932087c640 1 -- 172.21.15.16:0/3442528797 wait complete. 2023-11-13T11:04:17.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-13T11:04:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.147+0000 7f824359e640 1 Processor -- start 2023-11-13T11:04:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.147+0000 7f824359e640 1 -- start start 2023-11-13T11:04:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f824359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82440b5970 0x7f82440cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f824359e640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f82440b3b70 0x7f82440cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f824359e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440d0820 con 0x7f8244160340 2023-11-13T11:04:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f824359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440ce1a0 con 0x7f82440b5970 2023-11-13T11:04:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f824359e640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440ce480 con 0x7f82440b3b70 2023-11-13T11:04:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f8241d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82440b5970 0x7f82440cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f8241d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82440b5970 0x7f82440cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56158/0 (socket says 172.21.15.16:56158) 2023-11-13T11:04:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f8241d9b640 1 -- 172.21.15.16:0/3896506996 learned_addr learned my addr 172.21.15.16:0/3896506996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f8242d9d640 1 --2- 172.21.15.16:0/3896506996 >> v2:172.21.15.184:3300/0 conn(0x7f82440b3b70 0x7f82440cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f8241d9b640 1 -- 172.21.15.16:0/3896506996 >> v2:172.21.15.184:3300/0 conn(0x7f82440b3b70 msgr2=0x7f82440cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f8241d9b640 1 --2- 172.21.15.16:0/3896506996 >> v2:172.21.15.184:3300/0 conn(0x7f82440b3b70 0x7f82440cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f8241d9b640 1 -- 172.21.15.16:0/3896506996 >> v1:172.21.15.184:6789/0 conn(0x7f8244160340 legacy=0x7f82440cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.149+0000 7f8241d9b640 1 -- 172.21.15.16:0/3896506996 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440ce760 con 0x7f82440b5970 2023-11-13T11:04:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.151+0000 7f8241d9b640 1 --2- 172.21.15.16:0/3896506996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 0x7f82440cf510 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f824408bc40 tx=0x7f822c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a134d504224951b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.151+0000 7f824159a640 1 -- 172.21.15.16:0/3896506996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c031870 con 0x7f82440b5970 2023-11-13T11:04:18.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.151+0000 7f824159a640 1 -- 172.21.15.16:0/3896506996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c0319d0 con 0x7f82440b5970 2023-11-13T11:04:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.151+0000 7f824159a640 1 -- 172.21.15.16:0/3896506996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c031d60 con 0x7f82440b5970 2023-11-13T11:04:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.151+0000 7f824359e640 1 -- 172.21.15.16:0/3896506996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 msgr2=0x7f82440cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.151+0000 7f824359e640 1 --2- 172.21.15.16:0/3896506996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 0x7f82440cf510 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f824408bc40 tx=0x7f822c02b910 comp rx=0 tx=0).stop 2023-11-13T11:04:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.152+0000 7f824359e640 1 -- 172.21.15.16:0/3896506996 shutdown_connections 2023-11-13T11:04:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.152+0000 7f824359e640 1 --2- 172.21.15.16:0/3896506996 >> v2:172.21.15.184:3300/0 conn(0x7f82440b3b70 0x7f82440cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.152+0000 7f824359e640 1 --2- 172.21.15.16:0/3896506996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 0x7f82440cf510 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.152+0000 7f824359e640 1 -- 172.21.15.16:0/3896506996 >> 172.21.15.16:0/3896506996 conn(0x7f824409be00 msgr2=0x7f82440d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.152+0000 7f824359e640 1 -- 172.21.15.16:0/3896506996 shutdown_connections 2023-11-13T11:04:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.152+0000 7f824359e640 1 -- 172.21.15.16:0/3896506996 wait complete. 2023-11-13T11:04:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.153+0000 7f824359e640 1 Processor -- start 2023-11-13T11:04:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.154+0000 7f824359e640 1 -- start start 2023-11-13T11:04:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 0x7f82440c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824359e640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8244160340 0x7f82440cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824359e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82441599a0 0x7f824415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82441f2f70 con 0x7f82440b5970 2023-11-13T11:04:18.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824359e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824415ee10 con 0x7f82441599a0 2023-11-13T11:04:18.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824359e640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82440c75a0 con 0x7f8244160340 2023-11-13T11:04:18.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 0x7f82440c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 0x7f82440c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56168/0 (socket says 172.21.15.16:56168) 2023-11-13T11:04:18.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824259c640 1 -- 172.21.15.16:0/124759989 learned_addr learned my addr 172.21.15.16:0/124759989 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:18.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f8242d9d640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82441599a0 0x7f824415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824259c640 1 -- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8244160340 msgr2=0x7f82440cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824259c640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8244160340 0x7f82440cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824259c640 1 -- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82441599a0 msgr2=0x7f824415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824259c640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82441599a0 0x7f824415ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.155+0000 7f824259c640 1 -- 172.21.15.16:0/124759989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c0094d0 con 0x7f82440b5970 2023-11-13T11:04:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.156+0000 7f8242d9d640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82441599a0 0x7f824415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.156+0000 7f824259c640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 0x7f82440c7060 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f823400ba50 tx=0x7f823400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.156+0000 7f823affd640 1 -- 172.21.15.16:0/124759989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400cbe0 con 0x7f82440b5970 2023-11-13T11:04:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.156+0000 7f823affd640 1 -- 172.21.15.16:0/124759989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823400cd80 con 0x7f82440b5970 2023-11-13T11:04:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.156+0000 7f823affd640 1 -- 172.21.15.16:0/124759989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234014680 con 0x7f82440b5970 2023-11-13T11:04:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.156+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824415f040 con 0x7f82440b5970 2023-11-13T11:04:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.156+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82440c7dd0 con 0x7f82440b5970 2023-11-13T11:04:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.158+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824415f560 con 0x7f82440b5970 2023-11-13T11:04:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.159+0000 7f823affd640 1 -- 172.21.15.16:0/124759989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82340147e0 con 0x7f82440b5970 2023-11-13T11:04:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.159+0000 7f823affd640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f822406ae70 0x7f822406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.160+0000 7f823affd640 1 -- 172.21.15.16:0/124759989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f8234089450 con 0x7f82440b5970 2023-11-13T11:04:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.160+0000 7f8241d9b640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f822406ae70 0x7f822406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.161+0000 7f8241d9b640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f822406ae70 0x7f822406d330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f822c036000 tx=0x7f822c030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.164+0000 7f823affd640 1 -- 172.21.15.16:0/124759989 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82340545e0 con 0x7f82440b5970 2023-11-13T11:04:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.359+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f82440d20c0 con 0x7f822406ae70 2023-11-13T11:04:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.361+0000 7f823affd640 1 -- 172.21.15.16:0/124759989 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f82440d20c0 con 0x7f822406ae70 2023-11-13T11:04:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 6 to deep-scrub 2023-11-13T11:04:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.363+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f822406ae70 msgr2=0x7f822406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.363+0000 7f824359e640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f822406ae70 0x7f822406d330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f822c036000 tx=0x7f822c030000 comp rx=0 tx=0).stop 2023-11-13T11:04:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.363+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 msgr2=0x7f82440c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.363+0000 7f824359e640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 0x7f82440c7060 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f823400ba50 tx=0x7f823400bf10 comp rx=0 tx=0).stop 2023-11-13T11:04:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.363+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 shutdown_connections 2023-11-13T11:04:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.363+0000 7f824359e640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f822406ae70 0x7f822406d330 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.363+0000 7f824359e640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82441599a0 0x7f824415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.364+0000 7f824359e640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8244160340 0x7f82440cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.364+0000 7f824359e640 1 --2- 172.21.15.16:0/124759989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82440b5970 0x7f82440c7060 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.364+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 >> 172.21.15.16:0/124759989 conn(0x7f824409be00 msgr2=0x7f82440d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.364+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 shutdown_connections 2023-11-13T11:04:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.364+0000 7f824359e640 1 -- 172.21.15.16:0/124759989 wait complete. 2023-11-13T11:04:18.388 INFO:tasks.ceph:Scrubbing osd.7 2023-11-13T11:04:18.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-13T11:04:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.622+0000 7fe8233e9640 1 Processor -- start 2023-11-13T11:04:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.622+0000 7fe8233e9640 1 -- start start 2023-11-13T11:04:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.624+0000 7fe8233e9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe81c0b2a50 0x7fe81c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.624+0000 7fe8233e9640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe81c0cb3b0 0x7fe81c0adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.624+0000 7fe8233e9640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c0b3d50 con 0x7fe81c157720 2023-11-13T11:04:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.624+0000 7fe8233e9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c157b00 con 0x7fe81c0b2a50 2023-11-13T11:04:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.624+0000 7fe8233e9640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c0cdac0 con 0x7fe81c0cb3b0 2023-11-13T11:04:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.624+0000 7fe821be6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe81c0b2a50 0x7fe81c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.624+0000 7fe821be6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe81c0b2a50 0x7fe81c0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56172/0 (socket says 172.21.15.16:56172) 2023-11-13T11:04:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.624+0000 7fe821be6640 1 -- 172.21.15.16:0/2342742936 learned_addr learned my addr 172.21.15.16:0/2342742936 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.625+0000 7fe822be8640 1 --2- 172.21.15.16:0/2342742936 >> v2:172.21.15.184:3300/0 conn(0x7fe81c0cb3b0 0x7fe81c0adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.625+0000 7fe821be6640 1 -- 172.21.15.16:0/2342742936 >> v2:172.21.15.184:3300/0 conn(0x7fe81c0cb3b0 msgr2=0x7fe81c0adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.625+0000 7fe821be6640 1 --2- 172.21.15.16:0/2342742936 >> v2:172.21.15.184:3300/0 conn(0x7fe81c0cb3b0 0x7fe81c0adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.625+0000 7fe821be6640 1 -- 172.21.15.16:0/2342742936 >> v1:172.21.15.184:6789/0 conn(0x7fe81c157720 legacy=0x7fe81c0b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.625+0000 7fe821be6640 1 -- 172.21.15.16:0/2342742936 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0b19b0 con 0x7fe81c0b2a50 2023-11-13T11:04:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.626+0000 7fe821be6640 1 --2- 172.21.15.16:0/2342742936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c0b2a50 0x7fe81c0ad730 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0ae4c0 tx=0x7fe80c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82c154b0ba6f6e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.626+0000 7fe8213e5640 1 -- 172.21.15.16:0/2342742936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c0317e0 con 0x7fe81c0b2a50 2023-11-13T11:04:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.626+0000 7fe8213e5640 1 -- 172.21.15.16:0/2342742936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c031940 con 0x7fe81c0b2a50 2023-11-13T11:04:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.626+0000 7fe8213e5640 1 -- 172.21.15.16:0/2342742936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c031cd0 con 0x7fe81c0b2a50 2023-11-13T11:04:18.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.627+0000 7fe8233e9640 1 -- 172.21.15.16:0/2342742936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c0b2a50 msgr2=0x7fe81c0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.627+0000 7fe8233e9640 1 --2- 172.21.15.16:0/2342742936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c0b2a50 0x7fe81c0ad730 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0ae4c0 tx=0x7fe80c02b880 comp rx=0 tx=0).stop 2023-11-13T11:04:18.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.627+0000 7fe8233e9640 1 -- 172.21.15.16:0/2342742936 shutdown_connections 2023-11-13T11:04:18.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.627+0000 7fe8233e9640 1 --2- 172.21.15.16:0/2342742936 >> v2:172.21.15.184:3300/0 conn(0x7fe81c0cb3b0 0x7fe81c0adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.627+0000 7fe8233e9640 1 --2- 172.21.15.16:0/2342742936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c0b2a50 0x7fe81c0ad730 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.627+0000 7fe8233e9640 1 -- 172.21.15.16:0/2342742936 >> 172.21.15.16:0/2342742936 conn(0x7fe81c0b0970 msgr2=0x7fe81c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.627+0000 7fe8233e9640 1 -- 172.21.15.16:0/2342742936 shutdown_connections 2023-11-13T11:04:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.627+0000 7fe8233e9640 1 -- 172.21.15.16:0/2342742936 wait complete. 2023-11-13T11:04:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.629+0000 7fe8233e9640 1 Processor -- start 2023-11-13T11:04:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.629+0000 7fe8233e9640 1 -- start start 2023-11-13T11:04:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.630+0000 7fe8233e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe81c0b2a50 0x7fe81c15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.630+0000 7fe8233e9640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe81c0cb3b0 0x7fe81c14a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.630+0000 7fe8233e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c157720 0x7fe81c0ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.630+0000 7fe8233e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c14a8b0 con 0x7fe81c157720 2023-11-13T11:04:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.630+0000 7fe8233e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1f30e0 con 0x7fe81c0b2a50 2023-11-13T11:04:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.630+0000 7fe8233e9640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1f27b0 con 0x7fe81c0cb3b0 2023-11-13T11:04:18.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe822be8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c157720 0x7fe81c0ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe822be8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c157720 0x7fe81c0ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56184/0 (socket says 172.21.15.16:56184) 2023-11-13T11:04:18.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe822be8640 1 -- 172.21.15.16:0/2684834961 learned_addr learned my addr 172.21.15.16:0/2684834961 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe821be6640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe81c0cb3b0 0x7fe81c14a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe822be8640 1 -- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe81c0cb3b0 msgr2=0x7fe81c14a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe822be8640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe81c0cb3b0 0x7fe81c14a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe822be8640 1 -- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe81c0b2a50 msgr2=0x7fe81c15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe822be8640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe81c0b2a50 0x7fe81c15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe822be8640 1 -- 172.21.15.16:0/2684834961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c0094d0 con 0x7fe81c157720 2023-11-13T11:04:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.631+0000 7fe822be8640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c157720 0x7fe81c0ae730 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe814007e30 tx=0x7fe81400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.632+0000 7fe812ffd640 1 -- 172.21.15.16:0/2684834961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814014620 con 0x7fe81c157720 2023-11-13T11:04:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.632+0000 7fe812ffd640 1 -- 172.21.15.16:0/2684834961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81400ee90 con 0x7fe81c157720 2023-11-13T11:04:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.632+0000 7fe812ffd640 1 -- 172.21.15.16:0/2684834961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8140149c0 con 0x7fe81c157720 2023-11-13T11:04:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.632+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c1f2a90 con 0x7fe81c157720 2023-11-13T11:04:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.632+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c0aec70 con 0x7fe81c157720 2023-11-13T11:04:18.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.633+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe81c0b2250 con 0x7fe81c157720 2023-11-13T11:04:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.634+0000 7fe812ffd640 1 -- 172.21.15.16:0/2684834961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe814014b20 con 0x7fe81c157720 2023-11-13T11:04:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.635+0000 7fe812ffd640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe80006ae40 0x7fe80006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.635+0000 7fe812ffd640 1 -- 172.21.15.16:0/2684834961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fe81408a840 con 0x7fe81c157720 2023-11-13T11:04:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.635+0000 7fe812ffd640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7fe8000711e0 0x7fe800073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.635+0000 7fe812ffd640 1 -- 172.21.15.16:0/2684834961 --> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe800073cf0 con 0x7fe8000711e0 2023-11-13T11:04:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.635+0000 7fe812ffd640 1 -- 172.21.15.16:0/2684834961 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe81408abb0 con 0x7fe81c157720 2023-11-13T11:04:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.636+0000 7fe8223e7640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe80006ae40 0x7fe80006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.636+0000 7fe821be6640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7fe8000711e0 0x7fe800073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.637+0000 7fe821be6640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7fe8000711e0 0x7fe800073620 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.637+0000 7fe8223e7640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe80006ae40 0x7fe80006d300 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe818006fd0 tx=0x7fe818006bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.639+0000 7fe812ffd640 1 -- 172.21.15.16:0/2684834961 <== osd.7 v2:172.21.15.184:6824/3047573298 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fe800073cf0 con 0x7fe8000711e0 2023-11-13T11:04:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.654+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 --> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe81c0b1f70 con 0x7fe8000711e0 2023-11-13T11:04:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.654+0000 7fe812ffd640 1 -- 172.21.15.16:0/2684834961 <== osd.7 v2:172.21.15.184:6824/3047573298 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe80c031b30 con 0x7fe8000711e0 2023-11-13T11:04:18.659 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-13T11:04:18.659 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-13T11:04:18.659 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-13T11:04:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.655+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7fe8000711e0 msgr2=0x7fe800073620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.655+0000 7fe8233e9640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6824/3047573298,v1:172.21.15.184:6825/3047573298] conn(0x7fe8000711e0 0x7fe800073620 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.655+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe80006ae40 msgr2=0x7fe80006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.655+0000 7fe8233e9640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fe80006ae40 0x7fe80006d300 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe818006fd0 tx=0x7fe818006bc0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.655+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c157720 msgr2=0x7fe81c0ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.655+0000 7fe8233e9640 1 --2- 172.21.15.16:0/2684834961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c157720 0x7fe81c0ae730 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe814007e30 tx=0x7fe81400e4b0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.655+0000 7fe822be8640 1 -- 172.21.15.16:0/2684834961 reap_dead start 2023-11-13T11:04:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.655+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 shutdown_connections 2023-11-13T11:04:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.655+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 >> 172.21.15.16:0/2684834961 conn(0x7fe81c0b0970 msgr2=0x7fe81c0cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.656+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 shutdown_connections 2023-11-13T11:04:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.656+0000 7fe8233e9640 1 -- 172.21.15.16:0/2684834961 wait complete. 2023-11-13T11:04:18.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-13T11:04:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.902+0000 7f3a714ad640 1 Processor -- start 2023-11-13T11:04:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.902+0000 7f3a714ad640 1 -- start start 2023-11-13T11:04:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.904+0000 7f3a714ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a6c06f920 0x7f3a6c06fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.904+0000 7f3a714ad640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c0adc00 0x7f3a6c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.904+0000 7f3a714ad640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6c070240 con 0x7f3a6c06b6a0 2023-11-13T11:04:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.904+0000 7f3a714ad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c15b070 con 0x7f3a6c06f920 2023-11-13T11:04:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.904+0000 7f3a714ad640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c159e90 con 0x7f3a6c0adc00 2023-11-13T11:04:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.904+0000 7f3a6bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a6c06f920 0x7f3a6c06fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.904+0000 7f3a6bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a6c06f920 0x7f3a6c06fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56186/0 (socket says 172.21.15.16:56186) 2023-11-13T11:04:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.904+0000 7f3a6bfff640 1 -- 172.21.15.16:0/353864699 learned_addr learned my addr 172.21.15.16:0/353864699 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.904+0000 7f3a70cac640 1 --2- 172.21.15.16:0/353864699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c0adc00 0x7f3a6c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.905+0000 7f3a6bfff640 1 -- 172.21.15.16:0/353864699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c0adc00 msgr2=0x7f3a6c15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.905+0000 7f3a6bfff640 1 --2- 172.21.15.16:0/353864699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c0adc00 0x7f3a6c15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.905+0000 7f3a6bfff640 1 -- 172.21.15.16:0/353864699 >> v1:172.21.15.16:6789/0 conn(0x7f3a6c06b6a0 legacy=0x7f3a6c15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.905+0000 7f3a6b7fe640 1 -- 172.21.15.16:0/353864699 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101940040 0 0) 0x7f3a6c070240 con 0x7f3a6c06b6a0 2023-11-13T11:04:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.905+0000 7f3a6bfff640 1 -- 172.21.15.16:0/353864699 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0ad9b0 con 0x7f3a6c06f920 2023-11-13T11:04:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.906+0000 7f3a6bfff640 1 --2- 172.21.15.16:0/353864699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c06f920 0x7f3a6c06fd00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009870 tx=0x7f3a5c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a7a6548096af36 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.906+0000 7f3a6b7fe640 1 -- 172.21.15.16:0/353864699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c030840 con 0x7f3a6c06f920 2023-11-13T11:04:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.906+0000 7f3a6b7fe640 1 -- 172.21.15.16:0/353864699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c0309a0 con 0x7f3a6c06f920 2023-11-13T11:04:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.906+0000 7f3a6b7fe640 1 -- 172.21.15.16:0/353864699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c030d30 con 0x7f3a6c06f920 2023-11-13T11:04:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.907+0000 7f3a714ad640 1 -- 172.21.15.16:0/353864699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c06f920 msgr2=0x7f3a6c06fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.907+0000 7f3a714ad640 1 --2- 172.21.15.16:0/353864699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c06f920 0x7f3a6c06fd00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009870 tx=0x7f3a5c02b980 comp rx=0 tx=0).stop 2023-11-13T11:04:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.907+0000 7f3a714ad640 1 -- 172.21.15.16:0/353864699 shutdown_connections 2023-11-13T11:04:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.907+0000 7f3a714ad640 1 --2- 172.21.15.16:0/353864699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c0adc00 0x7f3a6c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.907+0000 7f3a714ad640 1 --2- 172.21.15.16:0/353864699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c06f920 0x7f3a6c06fd00 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.907+0000 7f3a714ad640 1 -- 172.21.15.16:0/353864699 >> 172.21.15.16:0/353864699 conn(0x7f3a6c0b0580 msgr2=0x7f3a6c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.907+0000 7f3a714ad640 1 -- 172.21.15.16:0/353864699 shutdown_connections 2023-11-13T11:04:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.907+0000 7f3a714ad640 1 -- 172.21.15.16:0/353864699 wait complete. 2023-11-13T11:04:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.909+0000 7f3a714ad640 1 Processor -- start 2023-11-13T11:04:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.909+0000 7f3a714ad640 1 -- start start 2023-11-13T11:04:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a714ad640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c06b6a0 0x7f3a6c14d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a714ad640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6c06f920 0x7f3a6c0b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a714ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c0adc00 0x7f3a6c0b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a714ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c137f80 con 0x7f3a6c0adc00 2023-11-13T11:04:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a714ad640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0b2f60 con 0x7f3a6c06f920 2023-11-13T11:04:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a714ad640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c138220 con 0x7f3a6c06b6a0 2023-11-13T11:04:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a70cac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c0adc00 0x7f3a6c0b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a70cac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c0adc00 0x7f3a6c0b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56194/0 (socket says 172.21.15.16:56194) 2023-11-13T11:04:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a70cac640 1 -- 172.21.15.16:0/693355786 learned_addr learned my addr 172.21.15.16:0/693355786 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.910+0000 7f3a6bfff640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c06b6a0 0x7f3a6c14d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a63fff640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6c06f920 0x7f3a6c0b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a70cac640 1 -- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c06b6a0 msgr2=0x7f3a6c14d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a70cac640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c06b6a0 0x7f3a6c14d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a70cac640 1 -- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6c06f920 msgr2=0x7f3a6c0b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a70cac640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6c06f920 0x7f3a6c0b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a70cac640 1 -- 172.21.15.16:0/693355786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0094d0 con 0x7f3a6c0adc00 2023-11-13T11:04:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a63fff640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6c06f920 0x7f3a6c0b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a70cac640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c0adc00 0x7f3a6c0b2a20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f3a64007e60 tx=0x7f3a6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a697fa640 1 -- 172.21.15.16:0/693355786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64014420 con 0x7f3a6c0adc00 2023-11-13T11:04:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a697fa640 1 -- 172.21.15.16:0/693355786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a640145a0 con 0x7f3a6c0adc00 2023-11-13T11:04:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a697fa640 1 -- 172.21.15.16:0/693355786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64014950 con 0x7f3a6c0adc00 2023-11-13T11:04:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c138500 con 0x7f3a6c0adc00 2023-11-13T11:04:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.911+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c0b3230 con 0x7f3a6c0adc00 2023-11-13T11:04:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.912+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c138a20 con 0x7f3a6c0adc00 2023-11-13T11:04:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.914+0000 7f3a697fa640 1 -- 172.21.15.16:0/693355786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a64014ab0 con 0x7f3a6c0adc00 2023-11-13T11:04:18.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.915+0000 7f3a697fa640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a4c06ae40 0x7f3a4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:18.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.915+0000 7f3a697fa640 1 -- 172.21.15.16:0/693355786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f3a64089d70 con 0x7f3a6c0adc00 2023-11-13T11:04:18.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.915+0000 7f3a6bfff640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a4c06ae40 0x7f3a4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.916+0000 7f3a6bfff640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a4c06ae40 0x7f3a4c06d300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c036000 tx=0x7f3a5c03b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:18.919+0000 7f3a697fa640 1 -- 172.21.15.16:0/693355786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a6c138a20 con 0x7f3a6c0adc00 2023-11-13T11:04:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.117+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a6c071890 con 0x7f3a4c06ae40 2023-11-13T11:04:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.118+0000 7f3a697fa640 1 -- 172.21.15.16:0/693355786 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3a6c071890 con 0x7f3a4c06ae40 2023-11-13T11:04:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 7 to deep-scrub 2023-11-13T11:04:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a4c06ae40 msgr2=0x7f3a4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a4c06ae40 0x7f3a4c06d300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c036000 tx=0x7f3a5c03b040 comp rx=0 tx=0).stop 2023-11-13T11:04:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c0adc00 msgr2=0x7f3a6c0b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c0adc00 0x7f3a6c0b2a20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f3a64007e60 tx=0x7f3a6400e4b0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 shutdown_connections 2023-11-13T11:04:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f3a4c06ae40 0x7f3a4c06d300 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a6c0adc00 0x7f3a6c0b2a20 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a6c06f920 0x7f3a6c0b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 --2- 172.21.15.16:0/693355786 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a6c06b6a0 0x7f3a6c14d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 >> 172.21.15.16:0/693355786 conn(0x7f3a6c0b0580 msgr2=0x7f3a6c15c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 shutdown_connections 2023-11-13T11:04:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.121+0000 7f3a714ad640 1 -- 172.21.15.16:0/693355786 wait complete. 2023-11-13T11:04:19.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:04:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.360+0000 7f91f56e7640 1 Processor -- start 2023-11-13T11:04:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.361+0000 7f91f56e7640 1 -- start start 2023-11-13T11:04:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.363+0000 7f91f56e7640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f91f015cf20 0x7f91f006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.363+0000 7f91f56e7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f00b2900 con 0x7f91f006bb90 2023-11-13T11:04:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.363+0000 7f91f56e7640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f00b1fd0 con 0x7f91f00cecc0 2023-11-13T11:04:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.363+0000 7f91f56e7640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f015d300 con 0x7f91f015cf20 2023-11-13T11:04:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.363+0000 7f91e7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f91f006bb90 0x7f91f00ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33908/0 (socket says 172.21.15.16:33908) 2023-11-13T11:04:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.363+0000 7f91e7fff640 1 -- 172.21.15.16:0/4038837730 learned_addr learned my addr 172.21.15.16:0/4038837730 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91eeffd640 1 --2- 172.21.15.16:0/4038837730 >> v2:172.21.15.184:3300/0 conn(0x7f91f015cf20 0x7f91f006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376863982 0 0) 0x7f91f00b2900 con 0x7f91f006bb90 2023-11-13T11:04:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91dc003520 con 0x7f91f006bb90 2023-11-13T11:04:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2431116063 0 0) 0x7f91dc003520 con 0x7f91f006bb90 2023-11-13T11:04:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 >> v2:172.21.15.184:3300/0 conn(0x7f91f015cf20 msgr2=0x7f91f006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91ee7fc640 1 --2- 172.21.15.16:0/4038837730 >> v2:172.21.15.184:3300/0 conn(0x7f91f015cf20 0x7f91f006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 >> v1:172.21.15.184:6789/0 conn(0x7f91f00cecc0 legacy=0x7f91f0070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00b22b0 con 0x7f91f006bb90 2023-11-13T11:04:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f91d8001670 con 0x7f91f006bb90 2023-11-13T11:04:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.364+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 >> v1:172.21.15.16:6789/0 conn(0x7f91f006bb90 legacy=0x7f91f00ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.365+0000 7f91ee7fc640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc003970 0x7f91dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.365+0000 7f91ee7fc640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.365+0000 7f91ee7fc640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc005180 0x7f91dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.365+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc00bff0 con 0x7f91dc0047c0 2023-11-13T11:04:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.365+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc00c160 con 0x7f91dc005180 2023-11-13T11:04:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.365+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc00c320 con 0x7f91dc003970 2023-11-13T11:04:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.365+0000 7f91e7fff640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.365+0000 7f91eeffd640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc003970 0x7f91dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.365+0000 7f91ef7fe640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc005180 0x7f91dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.366+0000 7f91e7fff640 1 -- 172.21.15.16:0/4038837730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc003970 msgr2=0x7f91dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.366+0000 7f91e7fff640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc003970 0x7f91dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.366+0000 7f91e7fff640 1 -- 172.21.15.16:0/4038837730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc005180 msgr2=0x7f91dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.366+0000 7f91e7fff640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91dc005180 0x7f91dc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.366+0000 7f91e7fff640 1 -- 172.21.15.16:0/4038837730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc003520 con 0x7f91dc0047c0 2023-11-13T11:04:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.366+0000 7f91eeffd640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91dc003970 0x7f91dc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-13T11:04:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.366+0000 7f91ef7fe640 1 -- 172.21.15.16:0/4038837730 reap_dead start 2023-11-13T11:04:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.367+0000 7f91e7fff640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91dc004c40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c210 tx=0x7f91dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eede0baeac36ae7 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8005ad0 con 0x7f91dc0047c0 2023-11-13T11:04:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8005c30 con 0x7f91dc0047c0 2023-11-13T11:04:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91ee7fc640 1 -- 172.21.15.16:0/4038837730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d802cad0 con 0x7f91dc0047c0 2023-11-13T11:04:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91f56e7640 1 -- 172.21.15.16:0/4038837730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 msgr2=0x7f91dc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91f56e7640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91dc004c40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f91d800c210 tx=0x7f91dc0057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91f56e7640 1 -- 172.21.15.16:0/4038837730 shutdown_connections 2023-11-13T11:04:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91f56e7640 1 --2- 172.21.15.16:0/4038837730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91dc004c40 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91f56e7640 1 -- 172.21.15.16:0/4038837730 >> 172.21.15.16:0/4038837730 conn(0x7f91f006a190 msgr2=0x7f91f006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91f56e7640 1 -- 172.21.15.16:0/4038837730 shutdown_connections 2023-11-13T11:04:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.368+0000 7f91f56e7640 1 -- 172.21.15.16:0/4038837730 wait complete. 2023-11-13T11:04:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.370+0000 7f91f56e7640 1 Processor -- start 2023-11-13T11:04:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.371+0000 7f91f56e7640 1 -- start start 2023-11-13T11:04:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.371+0000 7f91f56e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91f0159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91f56e7640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f0149fc0 0x7f91f014a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91f56e7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f0156820 0x7f91f0156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91f56e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f014a980 con 0x7f91dc0047c0 2023-11-13T11:04:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91f56e7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01f6950 con 0x7f91f0156820 2023-11-13T11:04:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91f56e7640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01f5d40 con 0x7f91f0149fc0 2023-11-13T11:04:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91f0159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91f0159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56212/0 (socket says 172.21.15.16:56212) 2023-11-13T11:04:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91eeffd640 1 -- 172.21.15.16:0/2010102225 learned_addr learned my addr 172.21.15.16:0/2010102225 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91ef7fe640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f0156820 0x7f91f0156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91e7fff640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f0149fc0 0x7f91f014a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91eeffd640 1 -- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f0149fc0 msgr2=0x7f91f014a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91eeffd640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f0149fc0 0x7f91f014a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91eeffd640 1 -- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f0156820 msgr2=0x7f91f0156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91eeffd640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f0156820 0x7f91f0156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91eeffd640 1 -- 172.21.15.16:0/2010102225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f01f6020 con 0x7f91dc0047c0 2023-11-13T11:04:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.372+0000 7f91e7fff640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f0149fc0 0x7f91f014a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.373+0000 7f91eeffd640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91f0159e90 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f91e0011510 tx=0x7f91e00119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:19.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.373+0000 7f91e77fe640 1 -- 172.21.15.16:0/2010102225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0015870 con 0x7f91dc0047c0 2023-11-13T11:04:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.373+0000 7f91e77fe640 1 -- 172.21.15.16:0/2010102225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e00159d0 con 0x7f91dc0047c0 2023-11-13T11:04:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.373+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f01f6300 con 0x7f91dc0047c0 2023-11-13T11:04:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.373+0000 7f91e77fe640 1 -- 172.21.15.16:0/2010102225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0015d80 con 0x7f91dc0047c0 2023-11-13T11:04:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.373+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f01571e0 con 0x7f91dc0047c0 2023-11-13T11:04:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.374+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f0067090 con 0x7f91dc0047c0 2023-11-13T11:04:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.376+0000 7f91e77fe640 1 -- 172.21.15.16:0/2010102225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f91e001d400 con 0x7f91dc0047c0 2023-11-13T11:04:19.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.376+0000 7f91e77fe640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f91d006ae70 0x7f91d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.377+0000 7f91e77fe640 1 -- 172.21.15.16:0/2010102225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f91e0089cc0 con 0x7f91dc0047c0 2023-11-13T11:04:19.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.377+0000 7f91e7fff640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f91d006ae70 0x7f91d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.378+0000 7f91e7fff640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f91d006ae70 0x7f91d006d330 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f91d8034000 tx=0x7f91d8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.381+0000 7f91e77fe640 1 -- 172.21.15.16:0/2010102225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91e0054f80 con 0x7f91dc0047c0 2023-11-13T11:04:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.571+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91f00b3540 con 0x7f91d006ae70 2023-11-13T11:04:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.578+0000 7f91e77fe640 1 -- 172.21.15.16:0/2010102225 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48293 (secure 0 0 0) 0x7f91d80014f0 con 0x7f91d006ae70 2023-11-13T11:04:19.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:04:19.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f91d006ae70 msgr2=0x7f91d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:19.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f91d006ae70 0x7f91d006d330 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f91d8034000 tx=0x7f91d8009030 comp rx=0 tx=0).stop 2023-11-13T11:04:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 msgr2=0x7f91f0159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91f0159e90 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f91e0011510 tx=0x7f91e00119d0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 shutdown_connections 2023-11-13T11:04:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f91d006ae70 0x7f91d006d330 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f0156820 0x7f91f0156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f0149fc0 0x7f91f014a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 --2- 172.21.15.16:0/2010102225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0047c0 0x7f91f0159e90 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 >> 172.21.15.16:0/2010102225 conn(0x7f91f006a190 msgr2=0x7f91f015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 shutdown_connections 2023-11-13T11:04:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:19.580+0000 7f91f56e7640 1 -- 172.21.15.16:0/2010102225 wait complete. 2023-11-13T11:04:19.602 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2023-11-13T11:04:18.177335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":325840,"kb_used_data":8360,"kb_used_omap":12,"kb_used_meta":317363,"kb_avail":754648880,"statfs":{"total":773094113280,"available":772760453120,"internally_reserved":0,"allocated":8560640,"data_stored":4506712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":324980273},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003918"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-13T11:04:02.760062+0000","last_change":"2023-11-13T10:55:15.914726+0000","last_active":"2023-11-13T11:04:02.760062+0000","last_peered":"2023-11-13T11:04:02.760062+0000","last_clean":"2023-11-13T11:04:02.760062+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:04:02.760062+0000","last_undegraded":"2023-11-13T11:04:02.760062+0000","last_fullsized":"2023-11-13T11:04:02.760062+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-13T10:55:14.881275+0000","last_clean_scrub_stamp":"2023-11-13T10:55:14.881275+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T21:30:44.924322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.129,"15min":1.121},"min":{"1min":0.74399999999999999,"5min":0.66500000000000004,"15min":0.63100000000000001},"max":{"1min":1.649,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2210000000000001,"15min":1.141},"min":{"1min":0.78400000000000003,"5min":0.75700000000000001,"15min":0.68500000000000005},"max":{"1min":1.806,"5min":1.865,"15min":2.2879999999999998},"last":0.90500000000000003}]},{"osd":1,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2569999999999999,"15min":1.1839999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.56399999999999995},"max":{"1min":1.724,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.286},{"interface":"front","average":{"1min":1.119,"5min":1.3100000000000001,"15min":1.2769999999999999},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4710000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.0569999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.319,"15min":1.2729999999999999},"min":{"1min":0.86699999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.72,"5min":2.153,"15min":2.1680000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.379,"15min":1.3480000000000001},"min":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8360000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.6220000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4350000000000001,"15min":1.375},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.63800000000000001},"max":{"1min":1.7390000000000001,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.238},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.46,"15min":1.4339999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.71299999999999997},"max":{"1min":1.8600000000000001,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.3799999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4379999999999999,"15min":1.536},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.833,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.115},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.476,"15min":1.5549999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.804,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.331}]},{"osd":5,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5669999999999999,"15min":1.625},"min":{"1min":0.94799999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.9379999999999999,"5min":2.7210000000000001,"15min":3.286},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.7250000000000001,"15min":1.74},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.04,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.6990000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.671,"15min":1.637},"min":{"1min":0.92000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.542},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.631,"15min":1.631},"min":{"1min":1.0349999999999999,"5min":0.96499999999999997,"15min":0.93700000000000006},"max":{"1min":1.9279999999999999,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.579}]}]},{"osd":5,"up_from":15,"seq":64424509552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46548,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325292,"statfs":{"total":96636764160,"available":96589099008,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.091,"15min":1.0249999999999999},"min":{"1min":0.66600000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.643,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.2270000000000001,"15min":1.198},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7749999999999999,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.82999999999999996}]},{"osd":1,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3089999999999999,"15min":1.238},"min":{"1min":0.878,"5min":0.79100000000000004,"15min":0.65400000000000003},"max":{"1min":1.7390000000000001,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.351},{"interface":"front","average":{"1min":1.143,"5min":1.282,"15min":1.238},"min":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.0219999999999998,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.2769999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3740000000000001,"15min":1.2649999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9139999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.879},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3149999999999999,"15min":1.2410000000000001},"min":{"1min":0.81000000000000005,"5min":0.75600000000000001,"15min":0.70799999999999996},"max":{"1min":1.8320000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.413,"15min":1.3919999999999999},"min":{"1min":1.0720000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.881,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.75},{"interface":"front","average":{"1min":1.367,"5min":1.502,"15min":1.478},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8400000000000001,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":1.169}]},{"osd":4,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.331,"15min":1.3220000000000001},"min":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.55500000000000005},"max":{"1min":1.7669999999999999,"5min":3.484,"15min":3.484},"last":1.631},{"interface":"front","average":{"1min":1.401,"5min":1.494,"15min":1.5189999999999999},"min":{"1min":0.95299999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.9590000000000001,"5min":3.653,"15min":3.653},"last":1.8839999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5469999999999999,"15min":1.5900000000000001},"min":{"1min":1.077,"5min":0.92000000000000004,"15min":0.78000000000000003},"max":{"1min":1.9690000000000001,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.427},{"interface":"front","average":{"1min":1.49,"5min":1.516,"15min":1.5169999999999999},"min":{"1min":0.92400000000000004,"5min":0.87,"15min":0.61799999999999999},"max":{"1min":1.9910000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.6879999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.6299999999999999,"15min":1.6559999999999999},"min":{"1min":1.1359999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.984,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.9790000000000001},{"interface":"front","average":{"1min":1.595,"5min":1.7130000000000001,"15min":1.704},"min":{"1min":1.292,"5min":1.1200000000000001,"15min":0.69499999999999995},"max":{"1min":2.173,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":1.571}]}]},{"osd":4,"up_from":15,"seq":64424509552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54288,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317552,"statfs":{"total":96636764160,"available":96581173248,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.173,"15min":1.1020000000000001},"min":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.738,"5min":1.986,"15min":1.986},"last":1.776},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.1539999999999999,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":1.542,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.278}]},{"osd":1,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.2230000000000001,"15min":1.1599999999999999},"min":{"1min":0.93300000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.335,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.256},"min":{"1min":0.872,"5min":0.76200000000000001,"15min":0.745},"max":{"1min":1.5009999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.331}]},{"osd":2,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2869999999999999,"15min":1.238},"min":{"1min":0.89200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.488,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.609},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.363,"15min":1.3460000000000001},"min":{"1min":0.85199999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6479999999999999,"5min":1.952,"15min":1.952},"last":1.2270000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4790000000000001,"15min":1.421},"min":{"1min":1.0349999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.841,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.669},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.2929999999999999},"min":{"1min":0.92900000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5880000000000001,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.577}]},{"osd":5,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.464,"15min":1.4830000000000001},"min":{"1min":1,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.996,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.94399999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":1.7809999999999999,"5min":2.089,"15min":2.089},"last":1.4419999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.613,"15min":1.6180000000000001},"min":{"1min":1.327,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":2.1030000000000002,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.468,"15min":1.512},"min":{"1min":1.0569999999999999,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":2.0310000000000001,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.49}]},{"osd":7,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5660000000000001},"min":{"1min":1.157,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":1.9039999999999999,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.633},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.5740000000000001,"15min":1.5980000000000001},"min":{"1min":1.1930000000000001,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":2.1499999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.7170000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44560,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327280,"statfs":{"total":96636764160,"available":96591134720,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34300000000000003},"max":{"1min":1.4099999999999999,"5min":1.867,"15min":1.867},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.04,"15min":1.018},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4790000000000001,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":1.208}]},{"osd":1,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0369999999999999,"15min":1.0389999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.534,"5min":1.966,"15min":3.71},"last":1.27},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1459999999999999,"15min":1.1020000000000001},"min":{"1min":0.59999999999999998,"5min":0.51000000000000001,"15min":0.46200000000000002},"max":{"1min":1.738,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":1.012}]},{"osd":2,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1679999999999999,"15min":1.1000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.1679999999999999,"15min":1.26},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.571,"5min":1.857,"15min":3.5710000000000002},"last":1.155}]},{"osd":4,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3500000000000001,"15min":1.4039999999999999},"min":{"1min":0.754,"5min":0.63600000000000001,"15min":0.497},"max":{"1min":2.0779999999999998,"5min":2.411,"15min":3.5030000000000001},"last":1.111},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3979999999999999,"15min":1.3480000000000001},"min":{"1min":0.83499999999999996,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.615,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.3220000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.83199999999999996,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.627,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.46,"15min":1.496},"min":{"1min":0.82999999999999996,"5min":0.66100000000000003,"15min":0.53700000000000003},"max":{"1min":2.21,"5min":2.673,"15min":3.3809999999999998},"last":1.4510000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.486,"15min":1.476},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.67800000000000005},"max":{"1min":1.794,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.413},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.4870000000000001,"15min":1.5629999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.778,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.5429999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.5700000000000001,"15min":1.5960000000000001},"min":{"1min":0.92900000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1579999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.603,"15min":1.55},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.73099999999999998},"max":{"1min":1.8959999999999999,"5min":2.75,"15min":2.75},"last":1.6140000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50128,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.92800000000000005,"15min":0.91700000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2889999999999999,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.02,"15min":0.92700000000000005},"min":{"1min":0.51700000000000002,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":1.7330000000000001,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":1.0920000000000001}]},{"osd":1,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":1.008,"15min":0.95199999999999996},"min":{"1min":0.47699999999999998,"5min":0.42599999999999999,"15min":0.376},"max":{"1min":1.3180000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.018},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.1870000000000001,"15min":1.1279999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5980000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.2350000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.2070000000000001,"15min":1.2210000000000001},"min":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":1.6899999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.625,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.524,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":1.335}]},{"osd":4,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.95899999999999996,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.7070000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.4019999999999999,"15min":1.393},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.63100000000000001},"max":{"1min":1.873,"5min":3.121,"15min":3.121},"last":1.278}]},{"osd":5,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.4430000000000001,"15min":1.4610000000000001},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.605,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.597,"15min":1.5549999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.80500000000000005},"max":{"1min":1.786,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":1.5389999999999999}]},{"osd":6,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.498,"15min":1.5249999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7629999999999999,"5min":3.395,"15min":4.7569999999999997},"last":1.371},{"interface":"front","average":{"1min":1.359,"5min":1.5940000000000001,"15min":1.581},"min":{"1min":0.84299999999999997,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":1.7509999999999999,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":1.657}]},{"osd":7,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5349999999999999,"15min":1.5760000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6599999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.617,"15min":1.603},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.7929999999999999,"5min":3.456,"15min":3.456},"last":1.464}]}]},{"osd":1,"up_from":15,"seq":64424509550,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.60799999999999998,"5min":0.40500000000000003,"15min":0.35799999999999998},"max":{"1min":1.369,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.99399999999999999,"15min":0.96099999999999997},"min":{"1min":0.61799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6379999999999999,"5min":1.869,"15min":1.869},"last":1.3169999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.117,"15min":1.008},"min":{"1min":0.66900000000000004,"5min":0.44800000000000001,"15min":0.40000000000000002},"max":{"1min":1.6339999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.133,"15min":1.024},"min":{"1min":0.75900000000000001,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":1.619,"5min":2.915,"15min":2.915},"last":0.95599999999999996}]},{"osd":3,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.839,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.083,"15min":1.0600000000000001},"min":{"1min":0.56100000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.601,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.121}]},{"osd":4,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.361,"15min":1.3140000000000001},"min":{"1min":0.95699999999999996,"5min":0.76200000000000001,"15min":0.58599999999999997},"max":{"1min":1.6990000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.4730000000000001,"15min":1.395},"min":{"1min":1.0029999999999999,"5min":0.69599999999999995,"15min":0.60599999999999998},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.3899999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.448,"15min":1.4139999999999999},"min":{"1min":0.874,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.47,"15min":1.4179999999999999},"min":{"1min":1.3400000000000001,"5min":0.78600000000000003,"15min":0.70899999999999996},"max":{"1min":1.9339999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.621}]},{"osd":6,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.367,"15min":1.401},"min":{"1min":0.79800000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.702,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.526,"5min":1.5629999999999999,"15min":1.5069999999999999},"min":{"1min":0.93799999999999994,"5min":0.85699999999999998,"15min":0.752},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.5109999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.544,"15min":1.5049999999999999},"min":{"1min":1.292,"5min":0.88,"15min":0.82899999999999996},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.502,"5min":1.498,"15min":1.4590000000000001},"min":{"1min":0.90900000000000003,"5min":0.80400000000000005,"15min":0.63500000000000001},"max":{"1min":1.891,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.575}]}]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.56100000000000005,"5min":0.36699999999999999,"15min":0.32100000000000001},"max":{"1min":1.464,"5min":1.74,"15min":1.74},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.589,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.107}]},{"osd":2,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.7230000000000001,"5min":2.105,"15min":2.105},"last":1.542},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.371},"max":{"1min":1.4179999999999999,"5min":1.968,"15min":2.1669999999999998},"last":1.214}]},{"osd":3,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.1559999999999999,"15min":1.105},"min":{"1min":0.66700000000000004,"5min":0.54000000000000004,"15min":0.45200000000000001},"max":{"1min":1.571,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.3009999999999999,"15min":1.3140000000000001},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.8460000000000001,"5min":2.2879999999999998,"15min":2.2919999999999998},"last":1.429}]},{"osd":4,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.425,"15min":1.363},"min":{"1min":0.81299999999999994,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.786,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3680000000000001,"15min":1.4159999999999999},"min":{"1min":0.81999999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5309999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.2889999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.492,"15min":1.528},"min":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.611,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.468,"15min":1.482},"min":{"1min":0.91700000000000004,"5min":0.78600000000000003,"15min":0.747},"max":{"1min":1.746,"5min":2.706,"15min":2.706},"last":1.385}]},{"osd":6,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.597,"15min":1.5329999999999999},"min":{"1min":1.1020000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.669,"5min":4.617,"15min":4.617},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.5860000000000001,"15min":1.573},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.72899999999999998},"max":{"1min":2.0249999999999999,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.7130000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.643,"15min":1.6619999999999999},"min":{"1min":0.85799999999999998,"5min":0.82299999999999995,"15min":0.76700000000000002},"max":{"1min":1.9430000000000001,"5min":2.742,"15min":3.3540000000000001},"last":1.819},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.6699999999999999,"15min":1.6180000000000001},"min":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.77400000000000002},"max":{"1min":1.776,"5min":3.976,"15min":3.976},"last":1.855}]}]},{"osd":7,"up_from":14,"seq":60129542255,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45972,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325868,"statfs":{"total":96636764160,"available":96589688832,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.105,"15min":1.113},"min":{"1min":0.71399999999999997,"5min":0.65600000000000003,"15min":0.59899999999999998},"max":{"1min":1.9430000000000001,"5min":1.9590000000000001,"15min":2.3879999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.3,"5min":1.1359999999999999,"15min":1.0880000000000001},"min":{"1min":1.0029999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.266},"last":0.94699999999999995}]},{"osd":1,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.198,"15min":1.0840000000000001},"min":{"1min":0.77900000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7729999999999999,"5min":1.784,"15min":2.0339999999999998},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2649999999999999,"15min":1.2350000000000001},"min":{"1min":0.83899999999999997,"5min":0.70099999999999996,"15min":0.64700000000000002},"max":{"1min":1.677,"5min":1.893,"15min":2.1899999999999999},"last":1.268}]},{"osd":2,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.337,"15min":1.306},"min":{"1min":1.0229999999999999,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.8600000000000001,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.579},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.335,"15min":1.2390000000000001},"min":{"1min":1.0449999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.7030000000000001,"5min":1.79,"15min":1.79},"last":1.385}]},{"osd":3,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.351,"15min":1.2709999999999999},"min":{"1min":0.90200000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.379,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.516,"15min":1.4670000000000001},"min":{"1min":1.004,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.8159999999999998},"last":1.675}]},{"osd":4,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.3660000000000001,"15min":1.3340000000000001},"min":{"1min":0.93400000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":3.0190000000000001},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.429,"15min":1.385},"min":{"1min":0.96199999999999997,"5min":0.439,"15min":0.439},"max":{"1min":2.5760000000000001,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.4390000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5289999999999999,"15min":1.534},"min":{"1min":0.995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.9620000000000002,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.474},{"interface":"front","average":{"1min":1.6659999999999999,"5min":1.542,"15min":1.494},"min":{"1min":1.1459999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.3620000000000001},"last":1.651}]},{"osd":6,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.8280000000000001,"5min":1.653,"15min":1.591},"min":{"1min":1.409,"5min":0.81399999999999995,"15min":0.53700000000000003},"max":{"1min":3.383,"5min":3.383,"15min":3.383},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5840000000000001,"15min":1.5229999999999999},"min":{"1min":1.0669999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.5990000000000002,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.6220000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:04:19.606 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-13T10:55:14.881275+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=13, tm_hour=10, tm_min=55, tm_sec=14, tm_wday=0, tm_yday=317, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=13, tm_hour=11, tm_min=4, tm_sec=12, tm_wday=0, tm_yday=317, tm_isdst=0) 2023-11-13T11:04:19.606 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-13T11:04:20.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:04:20.029 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:04:39.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-13T11:04:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.848+0000 7fbf0958c640 1 Processor -- start 2023-11-13T11:04:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.849+0000 7fbf0958c640 1 -- start start 2023-11-13T11:04:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.850+0000 7fbf0958c640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbf0415aa90 0x7fbf0406a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.850+0000 7fbf0958c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0406a6d0 0x7fbf04070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.850+0000 7fbf0958c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0406d680 con 0x7fbf040708a0 2023-11-13T11:04:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.851+0000 7fbf0958c640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0406ab50 con 0x7fbf0415aa90 2023-11-13T11:04:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.851+0000 7fbf0958c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf04070d30 con 0x7fbf0406a6d0 2023-11-13T11:04:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.851+0000 7fbf08d8b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbf040708a0 0x7fbf040b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58112/0 (socket says 172.21.15.16:58112) 2023-11-13T11:04:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.851+0000 7fbf08d8b640 1 -- 172.21.15.16:0/3682040515 learned_addr learned my addr 172.21.15.16:0/3682040515 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.851+0000 7fbefbfff640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0406a6d0 0x7fbf04070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.851+0000 7fbf03fff640 1 --2- 172.21.15.16:0/3682040515 >> v2:172.21.15.184:3300/0 conn(0x7fbf0415aa90 0x7fbf0406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716163803 0 0) 0x7fbf0406d680 con 0x7fbf040708a0 2023-11-13T11:04:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef0003520 con 0x7fbf040708a0 2023-11-13T11:04:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2190856544 0 0) 0x7fbef0003520 con 0x7fbf040708a0 2023-11-13T11:04:39.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0406a6d0 msgr2=0x7fbf04070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:39.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0406a6d0 0x7fbf04070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:39.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 >> v2:172.21.15.184:3300/0 conn(0x7fbf0415aa90 msgr2=0x7fbf0406a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 --2- 172.21.15.16:0/3682040515 >> v2:172.21.15.184:3300/0 conn(0x7fbf0415aa90 0x7fbf0406a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0415d630 con 0x7fbf040708a0 2023-11-13T11:04:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fbefc0019f0 con 0x7fbf040708a0 2023-11-13T11:04:39.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 >> v1:172.21.15.16:6789/0 conn(0x7fbf040708a0 legacy=0x7fbf040b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbef0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0406a6d0 0x7fbef00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0004d00 0x7fbef0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0406ab50 con 0x7fbef0003970 2023-11-13T11:04:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf04070d30 con 0x7fbef0004d00 2023-11-13T11:04:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.852+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef000fdc0 con 0x7fbf0406a6d0 2023-11-13T11:04:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.853+0000 7fbf03fff640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbef0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.853+0000 7fbf08d8b640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0004d00 0x7fbef0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.853+0000 7fbefbfff640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0406a6d0 0x7fbef00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.853+0000 7fbf03fff640 1 -- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0406a6d0 msgr2=0x7fbef00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.853+0000 7fbf03fff640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0406a6d0 0x7fbef00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.853+0000 7fbf03fff640 1 -- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0004d00 msgr2=0x7fbef0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.853+0000 7fbf03fff640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0004d00 0x7fbef0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.853+0000 7fbf03fff640 1 -- 172.21.15.16:0/3682040515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef0003520 con 0x7fbef0003970 2023-11-13T11:04:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.854+0000 7fbf03fff640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbef0003d90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009ac0 tx=0x7fbef00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51b854a6f2c6dff server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.854+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef402c710 con 0x7fbef0003970 2023-11-13T11:04:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.855+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef402c870 con 0x7fbef0003970 2023-11-13T11:04:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.855+0000 7fbf037fe640 1 -- 172.21.15.16:0/3682040515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef402cc00 con 0x7fbef0003970 2023-11-13T11:04:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.855+0000 7fbf0958c640 1 -- 172.21.15.16:0/3682040515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 msgr2=0x7fbef0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.855+0000 7fbf0958c640 1 --2- 172.21.15.16:0/3682040515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbef0003d90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009ac0 tx=0x7fbef00057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.855+0000 7fbf08d8b640 1 -- 172.21.15.16:0/3682040515 reap_dead start 2023-11-13T11:04:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.855+0000 7fbf0958c640 1 -- 172.21.15.16:0/3682040515 shutdown_connections 2023-11-13T11:04:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.855+0000 7fbf0958c640 1 -- 172.21.15.16:0/3682040515 >> 172.21.15.16:0/3682040515 conn(0x7fbf040b04a0 msgr2=0x7fbf040b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.855+0000 7fbf0958c640 1 -- 172.21.15.16:0/3682040515 shutdown_connections 2023-11-13T11:04:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.855+0000 7fbf0958c640 1 -- 172.21.15.16:0/3682040515 wait complete. 2023-11-13T11:04:39.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.857+0000 7fbf0958c640 1 Processor -- start 2023-11-13T11:04:39.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.857+0000 7fbf0958c640 1 -- start start 2023-11-13T11:04:39.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.858+0000 7fbf0958c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbf04142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:39.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.858+0000 7fbf0958c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf040b35b0 0x7fbf041ee190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:39.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.858+0000 7fbf0958c640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0415ba90 0x7fbf0415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:39.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.858+0000 7fbf0958c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf041427c0 con 0x7fbef0003970 2023-11-13T11:04:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.858+0000 7fbf0958c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0415c450 con 0x7fbf040b35b0 2023-11-13T11:04:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.858+0000 7fbf0958c640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040c3a80 con 0x7fbf0415ba90 2023-11-13T11:04:39.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.859+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbf04142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.859+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbf04142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47352/0 (socket says 172.21.15.16:47352) 2023-11-13T11:04:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.859+0000 7fbf03fff640 1 -- 172.21.15.16:0/821420698 learned_addr learned my addr 172.21.15.16:0/821420698 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.859+0000 7fbf03fff640 1 -- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0415ba90 msgr2=0x7fbf0415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:39.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.859+0000 7fbf03fff640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0415ba90 0x7fbf0415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.859+0000 7fbf03fff640 1 -- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf040b35b0 msgr2=0x7fbf041ee190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.859+0000 7fbf03fff640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf040b35b0 0x7fbf041ee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.859+0000 7fbf03fff640 1 -- 172.21.15.16:0/821420698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf041eea30 con 0x7fbef0003970 2023-11-13T11:04:39.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.859+0000 7fbf03fff640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbf04142280 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000c00 tx=0x7fbef4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:39.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.860+0000 7fbf017fa640 1 -- 172.21.15.16:0/821420698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4003810 con 0x7fbef0003970 2023-11-13T11:04:39.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.860+0000 7fbf017fa640 1 -- 172.21.15.16:0/821420698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef402de90 con 0x7fbef0003970 2023-11-13T11:04:39.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.860+0000 7fbf017fa640 1 -- 172.21.15.16:0/821420698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4003bb0 con 0x7fbef0003970 2023-11-13T11:04:39.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.860+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf041eed10 con 0x7fbef0003970 2023-11-13T11:04:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.860+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf040c3710 con 0x7fbef0003970 2023-11-13T11:04:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.861+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0404e4d0 con 0x7fbef0003970 2023-11-13T11:04:39.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.862+0000 7fbf017fa640 1 -- 172.21.15.16:0/821420698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbef4003d10 con 0x7fbef0003970 2023-11-13T11:04:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.863+0000 7fbf017fa640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbee406ae70 0x7fbee406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.863+0000 7fbf017fa640 1 -- 172.21.15.16:0/821420698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fbef40ab300 con 0x7fbef0003970 2023-11-13T11:04:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.864+0000 7fbefbfff640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbee406ae70 0x7fbee406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:39.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.865+0000 7fbefbfff640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbee406ae70 0x7fbee406d330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbf040b2b10 tx=0x7fbeec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:39.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:39.867+0000 7fbf017fa640 1 -- 172.21.15.16:0/821420698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbef4076510 con 0x7fbef0003970 2023-11-13T11:04:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.060+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 --> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf0415ded0 con 0x7fbee406ae70 2023-11-13T11:04:40.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.067+0000 7fbf017fa640 1 -- 172.21.15.16:0/821420698 <== mgr.4099 v2:172.21.15.184:6832/1022916571 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48502 (secure 0 0 0) 0x7fbeec011860 con 0x7fbee406ae70 2023-11-13T11:04:40.071 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:40.072 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-13T11:04:40.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbee406ae70 msgr2=0x7fbee406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbee406ae70 0x7fbee406d330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbf040b2b10 tx=0x7fbeec008040 comp rx=0 tx=0).stop 2023-11-13T11:04:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 msgr2=0x7fbf04142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbf04142280 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000c00 tx=0x7fbef4004730 comp rx=0 tx=0).stop 2023-11-13T11:04:40.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 shutdown_connections 2023-11-13T11:04:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fbee406ae70 0x7fbee406d330 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf0415ba90 0x7fbf0415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf040b35b0 0x7fbf041ee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 --2- 172.21.15.16:0/821420698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef0003970 0x7fbf04142280 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 >> 172.21.15.16:0/821420698 conn(0x7fbf040b04a0 msgr2=0x7fbf0406c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 shutdown_connections 2023-11-13T11:04:40.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.070+0000 7fbf0958c640 1 -- 172.21.15.16:0/821420698 wait complete. 2023-11-13T11:04:40.095 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2023-11-13T11:04:38.183178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":325832,"kb_used_data":8360,"kb_used_omap":12,"kb_used_meta":317363,"kb_avail":754648888,"statfs":{"total":773094113280,"available":772760461312,"internally_reserved":0,"allocated":8560640,"data_stored":4506712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":324980273},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003583"},"pg_stats":[{"pgid":"1.0","version":"76'119","reported_seq":223,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-13T11:04:16.515747+0000","last_change":"2023-11-13T11:04:16.515782+0000","last_active":"2023-11-13T11:04:16.515747+0000","last_peered":"2023-11-13T11:04:16.515747+0000","last_clean":"2023-11-13T11:04:16.515747+0000","last_became_active":"2023-11-13T10:55:15.913995+0000","last_became_peered":"2023-11-13T10:55:15.913995+0000","last_unstale":"2023-11-13T11:04:16.515747+0000","last_undegraded":"2023-11-13T11:04:16.515747+0000","last_fullsized":"2023-11-13T11:04:16.515747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'119","last_scrub_stamp":"2023-11-13T11:04:16.515666+0000","last_deep_scrub":"76'119","last_deep_scrub_stamp":"2023-11-13T11:04:16.515666+0000","last_clean_scrub_stamp":"2023-11-13T11:04:16.515666+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-14T17:48:04.673259+0000","scrub_duration":0.010938450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.129,"15min":1.121},"min":{"1min":0.74399999999999999,"5min":0.66500000000000004,"15min":0.63100000000000001},"max":{"1min":1.649,"5min":1.9870000000000001,"15min":2.0019999999999998},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2210000000000001,"15min":1.141},"min":{"1min":0.78400000000000003,"5min":0.75700000000000001,"15min":0.68500000000000005},"max":{"1min":1.806,"5min":1.865,"15min":2.2879999999999998},"last":0.93400000000000005}]},{"osd":1,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2569999999999999,"15min":1.1839999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.56399999999999995},"max":{"1min":1.724,"5min":2.0459999999999998,"15min":2.5390000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.3100000000000001,"15min":1.2769999999999999},"min":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4710000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.2569999999999999}]},{"osd":2,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.319,"15min":1.2729999999999999},"min":{"1min":0.86699999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.72,"5min":2.153,"15min":2.1680000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.379,"15min":1.3480000000000001},"min":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8360000000000001,"5min":2.2730000000000001,"15min":2.3610000000000002},"last":1.167}]},{"osd":3,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4350000000000001,"15min":1.375},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.63800000000000001},"max":{"1min":1.7390000000000001,"5min":2.2999999999999998,"15min":2.4609999999999999},"last":1.585},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.46,"15min":1.4339999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.71299999999999997},"max":{"1min":1.8600000000000001,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.464}]},{"osd":4,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4379999999999999,"15min":1.536},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.833,"5min":2.9870000000000001,"15min":4.4740000000000002},"last":1.296},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.476,"15min":1.5549999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.804,"5min":2.5489999999999999,"15min":4.1479999999999997},"last":1.6599999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5669999999999999,"15min":1.625},"min":{"1min":0.94799999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.9379999999999999,"5min":2.7210000000000001,"15min":3.286},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.7250000000000001,"15min":1.74},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.04,"5min":2.8889999999999998,"15min":3.1949999999999998},"last":1.6279999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:39 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.671,"15min":1.637},"min":{"1min":0.92000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.631,"15min":1.631},"min":{"1min":1.0349999999999999,"5min":0.96499999999999997,"15min":0.93700000000000006},"max":{"1min":1.9279999999999999,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.5469999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509556,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46544,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325296,"statfs":{"total":96636764160,"available":96589103104,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:04:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.131,"15min":1.0409999999999999},"min":{"1min":0.751,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.216},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.2210000000000001,"15min":1.1919999999999999},"min":{"1min":0.72099999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.679,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.85299999999999998}]},{"osd":1,"last update":"Mon Nov 13 11:04:33 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3169999999999999,"15min":1.24},"min":{"1min":0.86499999999999999,"5min":0.79100000000000004,"15min":0.65400000000000003},"max":{"1min":1.6879999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.292,"15min":1.2370000000000001},"min":{"1min":0.78800000000000003,"5min":0.73799999999999999,"15min":0.72199999999999998},"max":{"1min":1.8089999999999999,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.1020000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:04:33 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3879999999999999,"15min":1.272},"min":{"1min":0.879,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.718,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.387},{"interface":"front","average":{"1min":1.254,"5min":1.3089999999999999,"15min":1.244},"min":{"1min":0.752,"5min":0.752,"15min":0.70799999999999996},"max":{"1min":2.8439999999999999,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.5049999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:04:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4259999999999999,"15min":1.3939999999999999},"min":{"1min":0.99399999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":4.3170000000000002,"5min":4.3170000000000002,"15min":4.3170000000000002},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.5229999999999999,"15min":1.476},"min":{"1min":1.0249999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.6230000000000002,"5min":4.6959999999999997,"15min":4.6959999999999997},"last":1.2769999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:04:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.29,"15min":1.3169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.55500000000000005},"max":{"1min":3.1859999999999999,"5min":3.484,"15min":3.484},"last":1.651},{"interface":"front","average":{"1min":1.3,"5min":1.4630000000000001,"15min":1.502},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.9059999999999999,"5min":3.653,"15min":3.653},"last":1.151}]},{"osd":6,"last update":"Mon Nov 13 11:04:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.516,"15min":1.5760000000000001},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.78000000000000003},"max":{"1min":3.7069999999999999,"5min":4.8760000000000003,"15min":4.8760000000000003},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.4910000000000001,"15min":1.512},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.9460000000000002,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":1.7989999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:04:33 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.6120000000000001,"15min":1.6479999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":1.341},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.7070000000000001,"15min":1.6990000000000001},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":0.69499999999999995},"max":{"1min":3.532,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":1.7430000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54288,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317552,"statfs":{"total":96636764160,"available":96581173248,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.173,"15min":1.1020000000000001},"min":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.738,"5min":1.986,"15min":1.986},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.1539999999999999,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.47499999999999998,"15min":0.45500000000000002},"max":{"1min":1.542,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.246}]},{"osd":1,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.2230000000000001,"15min":1.1599999999999999},"min":{"1min":0.93300000000000005,"5min":0.502,"15min":0.502},"max":{"1min":1.335,"5min":1.9299999999999999,"15min":1.9319999999999999},"last":1.036},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2829999999999999,"15min":1.256},"min":{"1min":0.872,"5min":0.76200000000000001,"15min":0.745},"max":{"1min":1.5009999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.29}]},{"osd":2,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2869999999999999,"15min":1.238},"min":{"1min":0.89200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.488,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.458},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.363,"15min":1.3460000000000001},"min":{"1min":0.85199999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6479999999999999,"5min":1.952,"15min":1.952},"last":1.679}]},{"osd":3,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4790000000000001,"15min":1.421},"min":{"1min":1.0349999999999999,"5min":0.88600000000000001,"15min":0.80200000000000005},"max":{"1min":1.841,"5min":2.0049999999999999,"15min":2.0310000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.302,"15min":1.2929999999999999},"min":{"1min":0.92900000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5880000000000001,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.5169999999999999}]},{"osd":5,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.464,"15min":1.4830000000000001},"min":{"1min":1,"5min":0.76800000000000002,"15min":0.72399999999999998},"max":{"1min":1.996,"5min":2.0680000000000001,"15min":2.3170000000000002},"last":1.488},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.94399999999999995,"5min":0.52500000000000002,"15min":0.5},"max":{"1min":1.7809999999999999,"5min":2.089,"15min":2.089},"last":1.623}]},{"osd":6,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.613,"15min":1.6180000000000001},"min":{"1min":1.327,"5min":0.90700000000000003,"15min":0.65400000000000003},"max":{"1min":2.1030000000000002,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.468,"15min":1.512},"min":{"1min":1.0569999999999999,"5min":0.63600000000000001,"15min":0.56999999999999995},"max":{"1min":2.0310000000000001,"5min":2.2480000000000002,"15min":2.6480000000000001},"last":1.7450000000000001}]},{"osd":7,"last update":"Mon Nov 13 11:03:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.5129999999999999,"15min":1.5660000000000001},"min":{"1min":1.157,"5min":0.80000000000000004,"15min":0.54100000000000004},"max":{"1min":1.9039999999999999,"5min":2.2010000000000001,"15min":2.2410000000000001},"last":1.881},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.5740000000000001,"15min":1.5980000000000001},"min":{"1min":1.1930000000000001,"5min":0.71699999999999997,"15min":0.63},"max":{"1min":2.1499999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.841}]}]},{"osd":3,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44560,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327280,"statfs":{"total":96636764160,"available":96591134720,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34300000000000003},"max":{"1min":1.4099999999999999,"5min":1.867,"15min":1.867},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.04,"15min":1.018},"min":{"1min":0.47199999999999998,"5min":0.432,"15min":0.35899999999999999},"max":{"1min":1.4790000000000001,"5min":1.9870000000000001,"15min":2.2160000000000002},"last":1.26}]},{"osd":1,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0369999999999999,"15min":1.0389999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.534,"5min":1.966,"15min":3.71},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1459999999999999,"15min":1.1020000000000001},"min":{"1min":0.59999999999999998,"5min":0.51000000000000001,"15min":0.46200000000000002},"max":{"1min":1.738,"5min":1.9359999999999999,"15min":2.0190000000000001},"last":1.8580000000000001}]},{"osd":2,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1679999999999999,"15min":1.1000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.5509999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.1679999999999999,"15min":1.26},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.571,"5min":1.857,"15min":3.5710000000000002},"last":1.4119999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3500000000000001,"15min":1.4039999999999999},"min":{"1min":0.754,"5min":0.63600000000000001,"15min":0.497},"max":{"1min":2.0779999999999998,"5min":2.411,"15min":3.5030000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3979999999999999,"15min":1.3480000000000001},"min":{"1min":0.83499999999999996,"5min":0.77800000000000002,"15min":0.69699999999999995},"max":{"1min":1.615,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.4530000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.83199999999999996,"5min":0.80100000000000005,"15min":0.78200000000000003},"max":{"1min":1.627,"5min":2.0649999999999999,"15min":3.6339999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.46,"15min":1.496},"min":{"1min":0.82999999999999996,"5min":0.66100000000000003,"15min":0.53700000000000003},"max":{"1min":2.21,"5min":2.673,"15min":3.3809999999999998},"last":1.371}]},{"osd":6,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.486,"15min":1.476},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.67800000000000005},"max":{"1min":1.794,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.4870000000000001,"15min":1.5629999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.778,"5min":2.5339999999999998,"15min":3.6720000000000002},"last":1.804}]},{"osd":7,"last update":"Mon Nov 13 11:03:31 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.5700000000000001,"15min":1.5960000000000001},"min":{"1min":0.92900000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1579999999999999,"5min":2.5760000000000001,"15min":3.4540000000000002},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.603,"15min":1.55},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.73099999999999998},"max":{"1min":1.8959999999999999,"5min":2.75,"15min":2.75},"last":1.907}]}]},{"osd":2,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50128,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.92800000000000005,"15min":0.91700000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2889999999999999,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.02,"15min":0.92700000000000005},"min":{"1min":0.51700000000000002,"5min":0.38300000000000001,"15min":0.33100000000000002},"max":{"1min":1.7330000000000001,"5min":3.9660000000000002,"15min":3.9660000000000002},"last":1.276}]},{"osd":1,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":1.008,"15min":0.95199999999999996},"min":{"1min":0.47699999999999998,"5min":0.42599999999999999,"15min":0.376},"max":{"1min":1.3180000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.359},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.1870000000000001,"15min":1.1279999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5980000000000001,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.847}]},{"osd":3,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.2070000000000001,"15min":1.2210000000000001},"min":{"1min":0.61499999999999999,"5min":0.59499999999999997,"15min":0.43099999999999999},"max":{"1min":1.6899999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":1.647},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.254,"15min":1.2589999999999999},"min":{"1min":0.625,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.524,"5min":2.9580000000000002,"15min":2.9580000000000002},"last":2.2000000000000002}]},{"osd":4,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.95899999999999996,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.7070000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":2.484},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.4019999999999999,"15min":1.393},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.63100000000000001},"max":{"1min":1.873,"5min":3.121,"15min":3.121},"last":2.1389999999999998}]},{"osd":5,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.4430000000000001,"15min":1.4610000000000001},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.605,"5min":3.1680000000000001,"15min":3.1680000000000001},"last":2.0859999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.597,"15min":1.5549999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.80500000000000005},"max":{"1min":1.786,"5min":3.7080000000000002,"15min":3.7080000000000002},"last":2.4140000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.498,"15min":1.5249999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7629999999999999,"5min":3.395,"15min":4.7569999999999997},"last":2.4660000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.5940000000000001,"15min":1.581},"min":{"1min":0.84299999999999997,"5min":0.82899999999999996,"15min":0.73499999999999999},"max":{"1min":1.7509999999999999,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":2.5259999999999998}]},{"osd":7,"last update":"Mon Nov 13 11:03:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5349999999999999,"15min":1.5760000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6599999999999999,"5min":4.1269999999999998,"15min":4.9370000000000003},"last":2.3650000000000002},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.617,"15min":1.603},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.7929999999999999,"5min":3.456,"15min":3.456},"last":2.3220000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509554,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1515520,"data_stored":1006115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.60799999999999998,"5min":0.40500000000000003,"15min":0.35799999999999998},"max":{"1min":1.369,"5min":1.7030000000000001,"15min":2.6040000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.99399999999999999,"15min":0.96099999999999997},"min":{"1min":0.61799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6379999999999999,"5min":1.869,"15min":1.869},"last":0.94299999999999995}]},{"osd":2,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.117,"15min":1.008},"min":{"1min":0.66900000000000004,"5min":0.44800000000000001,"15min":0.40000000000000002},"max":{"1min":1.6339999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.133,"15min":1.024},"min":{"1min":0.75900000000000001,"5min":0.45500000000000002,"15min":0.41999999999999998},"max":{"1min":1.619,"5min":2.915,"15min":2.915},"last":1.0960000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.1850000000000001},"min":{"1min":0.91300000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.839,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.083,"15min":1.0600000000000001},"min":{"1min":0.56100000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.601,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.7949999999999999}]},{"osd":4,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.361,"15min":1.3140000000000001},"min":{"1min":0.95699999999999996,"5min":0.76200000000000001,"15min":0.58599999999999997},"max":{"1min":1.6990000000000001,"5min":1.9730000000000001,"15min":2.0569999999999999},"last":1.405},{"interface":"front","average":{"1min":1.516,"5min":1.4730000000000001,"15min":1.395},"min":{"1min":1.0029999999999999,"5min":0.69599999999999995,"15min":0.60599999999999998},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.893}]},{"osd":5,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.448,"15min":1.4139999999999999},"min":{"1min":0.874,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.47,"15min":1.4179999999999999},"min":{"1min":1.3400000000000001,"5min":0.78600000000000003,"15min":0.70899999999999996},"max":{"1min":1.9339999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.3100000000000001}]},{"osd":6,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.367,"15min":1.401},"min":{"1min":0.79800000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.702,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.526,"5min":1.5629999999999999,"15min":1.5069999999999999},"min":{"1min":0.93799999999999994,"5min":0.85699999999999998,"15min":0.752},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":2.0249999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:03:36 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.544,"15min":1.5049999999999999},"min":{"1min":1.292,"5min":0.88,"15min":0.82899999999999996},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.742},{"interface":"front","average":{"1min":1.502,"5min":1.498,"15min":1.4590000000000001},"min":{"1min":0.90900000000000003,"5min":0.80400000000000005,"15min":0.63500000000000001},"max":{"1min":1.891,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.95}]}]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 13 11:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.91100000000000003,"15min":0.89200000000000002},"min":{"1min":0.52700000000000002,"5min":0.37,"15min":0.32100000000000001},"max":{"1min":1.1970000000000001,"5min":1.726,"15min":1.74},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0589999999999999,"15min":1.036},"min":{"1min":0.46999999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.94999999999999996}]},{"osd":2,"last update":"Mon Nov 13 11:04:29 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.238,"15min":1.153},"min":{"1min":0.67600000000000005,"5min":0.441,"15min":0.441},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.0820000000000001,"15min":1.016},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.371},"max":{"1min":1.7210000000000001,"5min":1.968,"15min":2.1669999999999998},"last":1.1200000000000001}]},{"osd":3,"last update":"Mon Nov 13 11:04:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1559999999999999,"15min":1.1020000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.45200000000000001},"max":{"1min":1.7210000000000001,"5min":2.0409999999999999,"15min":2.0630000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.3129999999999999,"15min":1.3049999999999999},"min":{"1min":0.60299999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.8959999999999999,"5min":2.2240000000000002,"15min":2.2919999999999998},"last":1.4490000000000001}]},{"osd":4,"last update":"Mon Nov 13 11:04:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.4239999999999999,"15min":1.359},"min":{"1min":0.86399999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0390000000000001,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.78300000000000003,"5min":0.72699999999999998,"15min":0.71799999999999997},"max":{"1min":1.655,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4850000000000001}]},{"osd":5,"last update":"Mon Nov 13 11:04:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.48,"15min":1.508},"min":{"1min":0.93999999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.9390000000000001,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":1.599},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4610000000000001,"15min":1.466},"min":{"1min":0.88100000000000001,"5min":0.85199999999999998,"15min":0.747},"max":{"1min":1.615,"5min":2.706,"15min":2.706},"last":1.387}]},{"osd":6,"last update":"Mon Nov 13 11:04:29 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.587,"15min":1.522},"min":{"1min":0.90000000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.835,"5min":4.617,"15min":4.617},"last":1.9039999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.585,"15min":1.5629999999999999},"min":{"1min":0.96099999999999997,"5min":0.80200000000000005,"15min":0.72899999999999998},"max":{"1min":1.756,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.6459999999999999}]},{"osd":7,"last update":"Mon Nov 13 11:04:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.639,"15min":1.647},"min":{"1min":0.93799999999999994,"5min":0.82299999999999995,"15min":0.76700000000000002},"max":{"1min":1.819,"5min":2.742,"15min":3.3540000000000001},"last":1.857},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.6519999999999999,"15min":1.605},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.77400000000000002},"max":{"1min":2.1880000000000002,"5min":3.976,"15min":3.976},"last":1.8169999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45972,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325868,"statfs":{"total":96636764160,"available":96589688832,"internally_reserved":0,"allocated":921600,"data_stored":415747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 13 11:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.129,"15min":1.1180000000000001},"min":{"1min":0.79500000000000004,"5min":0.67200000000000004,"15min":0.59899999999999998},"max":{"1min":1.74,"5min":1.9430000000000001,"15min":2.3879999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.153,"15min":1.0920000000000001},"min":{"1min":0.71799999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.546,"5min":2.0190000000000001,"15min":2.266},"last":1.5569999999999999}]},{"osd":1,"last update":"Mon Nov 13 11:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.198,"15min":1.0940000000000001},"min":{"1min":0.90600000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6399999999999999,"5min":1.784,"15min":2.0339999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.278,"15min":1.2370000000000001},"min":{"1min":0.81999999999999995,"5min":0.79800000000000004,"15min":0.64700000000000002},"max":{"1min":1.702,"5min":1.893,"15min":2.1899999999999999},"last":1.534}]},{"osd":2,"last update":"Mon Nov 13 11:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.333,"15min":1.3080000000000001},"min":{"1min":0.81200000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.609,"5min":1.9139999999999999,"15min":2.6560000000000001},"last":1.224},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.343,"15min":1.246},"min":{"1min":0.74099999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.1719999999999999}]},{"osd":3,"last update":"Mon Nov 13 11:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.351,"15min":1.276},"min":{"1min":0.86199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5880000000000001,"5min":2.7389999999999999,"15min":3.0859999999999999},"last":1.274},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.502,"15min":1.468},"min":{"1min":0.90200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.839,"5min":2.0579999999999998,"15min":2.8159999999999998},"last":1.452}]},{"osd":4,"last update":"Mon Nov 13 11:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.333,"15min":1.3320000000000001},"min":{"1min":0.83699999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.419,"5min":2.9980000000000002,"15min":3.0190000000000001},"last":1.48},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3899999999999999,"15min":1.377},"min":{"1min":0.67000000000000004,"5min":0.439,"15min":0.439},"max":{"1min":2.1440000000000001,"5min":2.8780000000000001,"15min":3.2530000000000001},"last":1.129}]},{"osd":5,"last update":"Mon Nov 13 11:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4870000000000001,"15min":1.522},"min":{"1min":0.73099999999999998,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.8819999999999999,"5min":3.1429999999999998,"15min":3.5990000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.528,"15min":1.4950000000000001},"min":{"1min":0.83899999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.3439999999999999,"5min":3.2610000000000001,"15min":3.3620000000000001},"last":1.335}]},{"osd":6,"last update":"Mon Nov 13 11:04:34 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.6319999999999999,"15min":1.5900000000000001},"min":{"1min":0.92400000000000004,"5min":0.84899999999999998,"15min":0.53700000000000003},"max":{"1min":2.2160000000000002,"5min":3.383,"15min":3.383},"last":1.399},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.5589999999999999,"15min":1.5209999999999999},"min":{"1min":0.76400000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.2730000000000001,"5min":2.9740000000000002,"15min":3.3330000000000002},"last":1.423}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-13T11:04:40.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-13T11:04:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.305+0000 7f8f153b6640 1 Processor -- start 2023-11-13T11:04:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.306+0000 7f8f153b6640 1 -- start start 2023-11-13T11:04:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.307+0000 7f8f153b6640 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8f100ce550 0x7f8f100ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.307+0000 7f8f153b6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f100ceef0 con 0x7f8f10144570 2023-11-13T11:04:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.307+0000 7f8f153b6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f10144950 con 0x7f8f100b19d0 2023-11-13T11:04:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.307+0000 7f8f153b6640 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f100ce300 con 0x7f8f100ce550 2023-11-13T11:04:40.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.308+0000 7f8f0effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8f10144570 0x7f8f1015f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58126/0 (socket says 172.21.15.16:58126) 2023-11-13T11:04:40.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.308+0000 7f8f0effd640 1 -- 172.21.15.16:0/2142193383 learned_addr learned my addr 172.21.15.16:0/2142193383 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:40.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.308+0000 7f8f0e7fc640 1 --2- 172.21.15.16:0/2142193383 >> v2:172.21.15.184:3300/0 conn(0x7f8f100ce550 0x7f8f100ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.308+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215810243 0 0) 0x7f8f100ceef0 con 0x7f8f10144570 2023-11-13T11:04:40.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.308+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8efc003520 con 0x7f8f10144570 2023-11-13T11:04:40.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 885636643 0 0) 0x7f8efc003520 con 0x7f8f10144570 2023-11-13T11:04:40.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 >> v2:172.21.15.184:3300/0 conn(0x7f8f100ce550 msgr2=0x7f8f100ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:40.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 --2- 172.21.15.16:0/2142193383 >> v2:172.21.15.184:3300/0 conn(0x7f8f100ce550 0x7f8f100ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 >> v1:172.21.15.184:6789/0 conn(0x7f8f100b19d0 legacy=0x7f8f100b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100cd260 con 0x7f8f10144570 2023-11-13T11:04:40.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7f8f000019f0 con 0x7f8f10144570 2023-11-13T11:04:40.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 >> v1:172.21.15.16:6789/0 conn(0x7f8f10144570 legacy=0x7f8f1015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8efc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8efc0047c0 0x7f8efc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8efc005180 0x7f8efc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100ce300 con 0x7f8efc003970 2023-11-13T11:04:40.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc00c020 con 0x7f8efc005180 2023-11-13T11:04:40.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc00c1e0 con 0x7f8efc0047c0 2023-11-13T11:04:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.309+0000 7f8f0effd640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8efc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.310+0000 7f8f0effd640 1 -- 172.21.15.16:0/2142193383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8efc0047c0 msgr2=0x7f8efc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:40.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.310+0000 7f8f0effd640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8efc0047c0 0x7f8efc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.310+0000 7f8f0effd640 1 -- 172.21.15.16:0/2142193383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8efc005180 msgr2=0x7f8efc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.310+0000 7f8f0f7fe640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8efc005180 0x7f8efc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.310+0000 7f8f0effd640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8efc005180 0x7f8efc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.310+0000 7f8f0effd640 1 -- 172.21.15.16:0/2142193383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc003520 con 0x7f8efc003970 2023-11-13T11:04:40.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.310+0000 7f8f0f7fe640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8efc005180 0x7f8efc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:40.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.310+0000 7f8f0f7fe640 1 -- 172.21.15.16:0/2142193383 reap_dead start 2023-11-13T11:04:40.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.311+0000 7f8f0effd640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8efc003d90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000a820 tx=0x7f8efc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfc936baa76b7e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:40.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.311+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000072d0 con 0x7f8efc003970 2023-11-13T11:04:40.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.311+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0002f070 con 0x7f8efc003970 2023-11-13T11:04:40.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.311+0000 7f8f0dffb640 1 -- 172.21.15.16:0/2142193383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000076f0 con 0x7f8efc003970 2023-11-13T11:04:40.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.312+0000 7f8f153b6640 1 -- 172.21.15.16:0/2142193383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 msgr2=0x7f8efc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.312+0000 7f8f153b6640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8efc003d90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000a820 tx=0x7f8efc0057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.312+0000 7f8f153b6640 1 -- 172.21.15.16:0/2142193383 shutdown_connections 2023-11-13T11:04:40.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.312+0000 7f8f153b6640 1 --2- 172.21.15.16:0/2142193383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8efc003d90 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.312+0000 7f8f153b6640 1 -- 172.21.15.16:0/2142193383 >> 172.21.15.16:0/2142193383 conn(0x7f8f1006a190 msgr2=0x7f8f1006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:40.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.312+0000 7f8f153b6640 1 -- 172.21.15.16:0/2142193383 shutdown_connections 2023-11-13T11:04:40.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.312+0000 7f8f153b6640 1 -- 172.21.15.16:0/2142193383 wait complete. 2023-11-13T11:04:40.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.314+0000 7f8f153b6640 1 Processor -- start 2023-11-13T11:04:40.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.314+0000 7f8f153b6640 1 -- start start 2023-11-13T11:04:40.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.315+0000 7f8f153b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8f10142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.315+0000 7f8f153b6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f101405a0 0x7f8f10139b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.315+0000 7f8f153b6640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f1015c240 0x7f8f1015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.315+0000 7f8f153b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1015cc30 con 0x7f8efc003970 2023-11-13T11:04:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.315+0000 7f8f153b6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f101434f0 con 0x7f8f101405a0 2023-11-13T11:04:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.315+0000 7f8f153b6640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f101437d0 con 0x7f8f1015c240 2023-11-13T11:04:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.315+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8f10142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.315+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8f10142e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47364/0 (socket says 172.21.15.16:47364) 2023-11-13T11:04:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.315+0000 7f8f0effd640 1 -- 172.21.15.16:0/2215566147 learned_addr learned my addr 172.21.15.16:0/2215566147 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.316+0000 7f8f0f7fe640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f1015c240 0x7f8f1015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.316+0000 7f8f0e7fc640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f101405a0 0x7f8f10139b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.316+0000 7f8f0effd640 1 -- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f1015c240 msgr2=0x7f8f1015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.316+0000 7f8f0effd640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f1015c240 0x7f8f1015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.316+0000 7f8f0effd640 1 -- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f101405a0 msgr2=0x7f8f10139b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.316+0000 7f8f0effd640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f101405a0 0x7f8f10139b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.316+0000 7f8f0effd640 1 -- 172.21.15.16:0/2215566147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f10143ab0 con 0x7f8efc003970 2023-11-13T11:04:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.316+0000 7f8f0effd640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8f10142e10 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8f0002cb70 tx=0x7f8f100cc680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.317+0000 7f8f077fe640 1 -- 172.21.15.16:0/2215566147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000070d0 con 0x7f8efc003970 2023-11-13T11:04:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.317+0000 7f8f077fe640 1 -- 172.21.15.16:0/2215566147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f00007230 con 0x7f8efc003970 2023-11-13T11:04:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.317+0000 7f8f077fe640 1 -- 172.21.15.16:0/2215566147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000075a0 con 0x7f8efc003970 2023-11-13T11:04:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.317+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f10158160 con 0x7f8efc003970 2023-11-13T11:04:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.317+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1013a070 con 0x7f8efc003970 2023-11-13T11:04:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.318+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f10072520 con 0x7f8efc003970 2023-11-13T11:04:40.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.319+0000 7f8f077fe640 1 -- 172.21.15.16:0/2215566147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f00031030 con 0x7f8efc003970 2023-11-13T11:04:40.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.320+0000 7f8f077fe640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ef006ae40 0x7f8ef006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.320+0000 7f8f077fe640 1 -- 172.21.15.16:0/2215566147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f8f000ab5d0 con 0x7f8efc003970 2023-11-13T11:04:40.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.321+0000 7f8f0e7fc640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ef006ae40 0x7f8ef006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.322+0000 7f8f0e7fc640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ef006ae40 0x7f8ef006d300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f10067320 tx=0x7f8ef8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:40.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.327+0000 7f8f077fe640 1 -- 172.21.15.16:0/2215566147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f000767e0 con 0x7f8efc003970 2023-11-13T11:04:40.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.527+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f8f10072520 con 0x7f8efc003970 2023-11-13T11:04:40.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.534+0000 7f8f077fe640 1 -- 172.21.15.16:0/2215566147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8f00007700 con 0x7f8efc003970 2023-11-13T11:04:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.537+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ef006ae40 msgr2=0x7f8ef006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.537+0000 7f8f153b6640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ef006ae40 0x7f8ef006d300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f10067320 tx=0x7f8ef8008040 comp rx=0 tx=0).stop 2023-11-13T11:04:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.537+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 msgr2=0x7f8f10142e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.537+0000 7f8f153b6640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8f10142e10 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8f0002cb70 tx=0x7f8f100cc680 comp rx=0 tx=0).stop 2023-11-13T11:04:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.538+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 shutdown_connections 2023-11-13T11:04:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.538+0000 7f8f153b6640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7f8ef006ae40 0x7f8ef006d300 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.538+0000 7f8f153b6640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f1015c240 0x7f8f1015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.538+0000 7f8f153b6640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f101405a0 0x7f8f10139b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.538+0000 7f8f153b6640 1 --2- 172.21.15.16:0/2215566147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8efc003970 0x7f8f10142e10 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.538+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 >> 172.21.15.16:0/2215566147 conn(0x7f8f1006a190 msgr2=0x7f8f1006a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.538+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 shutdown_connections 2023-11-13T11:04:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.538+0000 7f8f153b6640 1 -- 172.21.15.16:0/2215566147 wait complete. 2023-11-13T11:04:40.561 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-13T11:04:40.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-13T11:04:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.769+0000 7fcdcb013640 1 Processor -- start 2023-11-13T11:04:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.770+0000 7fcdcb013640 1 -- start start 2023-11-13T11:04:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.771+0000 7fcdcb013640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc415cfc0 0x7fcdc40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.771+0000 7fcdcb013640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc40b3cf0 con 0x7fcdc40d2ae0 2023-11-13T11:04:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.771+0000 7fcdcb013640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc40b3730 con 0x7fcdc40b30e0 2023-11-13T11:04:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.771+0000 7fcdcb013640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b3a10 con 0x7fcdc415cfc0 2023-11-13T11:04:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.772+0000 7fcdc9810640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcdc40d2ae0 0x7fcdc40b29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58132/0 (socket says 172.21.15.16:58132) 2023-11-13T11:04:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.772+0000 7fcdc9810640 1 -- 172.21.15.16:0/918558202 learned_addr learned my addr 172.21.15.16:0/918558202 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-13T11:04:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.772+0000 7fcdca011640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc415cfc0 0x7fcdc40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.772+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289117162 0 0) 0x7fcdc40b3cf0 con 0x7fcdc40d2ae0 2023-11-13T11:04:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.772+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb0003520 con 0x7fcdc40d2ae0 2023-11-13T11:04:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3724044078 0 0) 0x7fcdb0003520 con 0x7fcdc40d2ae0 2023-11-13T11:04:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc415cfc0 msgr2=0x7fcdc40cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc415cfc0 0x7fcdc40cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 >> v1:172.21.15.184:6789/0 conn(0x7fcdc40b30e0 legacy=0x7fcdc40d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40d0880 con 0x7fcdc40d2ae0 2023-11-13T11:04:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2904252034 0 0) 0x7fcdb4001670 con 0x7fcdc40d2ae0 2023-11-13T11:04:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 >> v1:172.21.15.16:6789/0 conn(0x7fcdc40d2ae0 legacy=0x7fcdc40b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc415cfc0 0x7fcdb00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb0003f20 0x7fcdb00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b3a10 con 0x7fcdb0004d00 2023-11-13T11:04:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb000bb10 con 0x7fcdb0003f20 2023-11-13T11:04:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb000bcd0 con 0x7fcdc415cfc0 2023-11-13T11:04:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdca812640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdc9810640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb0003f20 0x7fcdb00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdca812640 1 -- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc415cfc0 msgr2=0x7fcdb00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdca812640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc415cfc0 0x7fcdb00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdca812640 1 -- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb0003f20 msgr2=0x7fcdb00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-13T11:04:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.773+0000 7fcdca812640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb0003f20 0x7fcdb00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.774+0000 7fcdca812640 1 -- 172.21.15.16:0/918558202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0003520 con 0x7fcdb0004d00 2023-11-13T11:04:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.774+0000 7fcdc9810640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb0003f20 0x7fcdb00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-13T11:04:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.775+0000 7fcdca812640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdb0005180 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc001750 tx=0x7fcdb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55103ecd66d6adc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.775+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0052d0 con 0x7fcdb0004d00 2023-11-13T11:04:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.775+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcdbc02e050 con 0x7fcdb0004d00 2023-11-13T11:04:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.775+0000 7fcdc900f640 1 -- 172.21.15.16:0/918558202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0035f0 con 0x7fcdb0004d00 2023-11-13T11:04:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.775+0000 7fcdcb013640 1 -- 172.21.15.16:0/918558202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 msgr2=0x7fcdb0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.775+0000 7fcdcb013640 1 --2- 172.21.15.16:0/918558202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdb0005180 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc001750 tx=0x7fcdb00057f0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.775+0000 7fcdca812640 1 -- 172.21.15.16:0/918558202 reap_dead start 2023-11-13T11:04:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.775+0000 7fcdcb013640 1 -- 172.21.15.16:0/918558202 shutdown_connections 2023-11-13T11:04:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.775+0000 7fcdcb013640 1 -- 172.21.15.16:0/918558202 >> 172.21.15.16:0/918558202 conn(0x7fcdc408c570 msgr2=0x7fcdc409be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.776+0000 7fcdcb013640 1 -- 172.21.15.16:0/918558202 shutdown_connections 2023-11-13T11:04:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.776+0000 7fcdcb013640 1 -- 172.21.15.16:0/918558202 wait complete. 2023-11-13T11:04:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.777+0000 7fcdcb013640 1 Processor -- start 2023-11-13T11:04:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.778+0000 7fcdcb013640 1 -- start start 2023-11-13T11:04:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdcb013640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdc40b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdcb013640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc414ee70 0x7fcdc415eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdcb013640 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc414f2f0 0x7fcdc415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdcb013640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b1550 con 0x7fcdb0004d00 2023-11-13T11:04:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdcb013640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40cc710 con 0x7fcdc414ee70 2023-11-13T11:04:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdcb013640 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40cbb00 con 0x7fcdc414f2f0 2023-11-13T11:04:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdca011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdc40b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdca011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdc40b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47388/0 (socket says 172.21.15.16:47388) 2023-11-13T11:04:40.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdca011640 1 -- 172.21.15.16:0/808855118 learned_addr learned my addr 172.21.15.16:0/808855118 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-13T11:04:40.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdc9810640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc414ee70 0x7fcdc415eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdca011640 1 -- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc414f2f0 msgr2=0x7fcdc415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdca011640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc414f2f0 0x7fcdc415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdca011640 1 -- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc414ee70 msgr2=0x7fcdc415eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdca011640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc414ee70 0x7fcdc415eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.779+0000 7fcdca011640 1 -- 172.21.15.16:0/808855118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40cbde0 con 0x7fcdb0004d00 2023-11-13T11:04:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.780+0000 7fcdca011640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdc40b0420 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000ef40 tx=0x7fcdc000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:40.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.780+0000 7fcdbaffd640 1 -- 172.21.15.16:0/808855118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0010040 con 0x7fcdb0004d00 2023-11-13T11:04:40.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.780+0000 7fcdbaffd640 1 -- 172.21.15.16:0/808855118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcdc0009e60 con 0x7fcdb0004d00 2023-11-13T11:04:40.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.780+0000 7fcdbaffd640 1 -- 172.21.15.16:0/808855118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc000bea0 con 0x7fcdb0004d00 2023-11-13T11:04:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.780+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc40cc0c0 con 0x7fcdb0004d00 2023-11-13T11:04:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.780+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc40b0960 con 0x7fcdb0004d00 2023-11-13T11:04:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.781+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc40b5f20 con 0x7fcdb0004d00 2023-11-13T11:04:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.782+0000 7fcdbaffd640 1 -- 172.21.15.16:0/808855118 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcdc0003bd0 con 0x7fcdb0004d00 2023-11-13T11:04:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.783+0000 7fcdbaffd640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcda806ae70 0x7fcda806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-13T11:04:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.783+0000 7fcdbaffd640 1 -- 172.21.15.16:0/808855118 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fcdc0089b20 con 0x7fcdb0004d00 2023-11-13T11:04:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.784+0000 7fcdc9810640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcda806ae70 0x7fcda806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-13T11:04:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.785+0000 7fcdc9810640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcda806ae70 0x7fcda806d330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4001eb0 tx=0x7fcdb4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-13T11:04:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:40.788+0000 7fcdbaffd640 1 -- 172.21.15.16:0/808855118 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcdc0054c00 con 0x7fcdb0004d00 2023-11-13T11:04:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.030+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdc40cc850 con 0x7fcdb0004d00 2023-11-13T11:04:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.031+0000 7fcdbaffd640 1 -- 172.21.15.16:0/808855118 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fcdc0058220 con 0x7fcdb0004d00 2023-11-13T11:04:41.036 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-13T11:04:41.036 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":48,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5372,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6836/282558415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":282558415},{"type":"v1","addr":"172.21.15.184:6836","nonce":282558415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/960469256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":960469256},{"type":"v1","addr":"172.21.15.16:6833","nonce":960469256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5854,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/1265200946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6835","nonce":1265200946},{"type":"v1","addr":"172.21.15.184:6837","nonce":1265200946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3768737652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3768737652},{"type":"v1","addr":"172.21.15.16:6835","nonce":3768737652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[]} 2023-11-13T11:04:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 48 2023-11-13T11:04:41.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcda806ae70 msgr2=0x7fcda806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:41.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcda806ae70 0x7fcda806d330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4001eb0 tx=0x7fcdb4009040 comp rx=0 tx=0).stop 2023-11-13T11:04:41.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 msgr2=0x7fcdc40b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-13T11:04:41.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdc40b0420 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000ef40 tx=0x7fcdc000b480 comp rx=0 tx=0).stop 2023-11-13T11:04:41.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 shutdown_connections 2023-11-13T11:04:41.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:6832/1022916571,v1:172.21.15.184:6833/1022916571] conn(0x7fcda806ae70 0x7fcda806d330 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:41.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc414f2f0 0x7fcdc415f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:41.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc414ee70 0x7fcdc415eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:41.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 --2- 172.21.15.16:0/808855118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0004d00 0x7fcdc40b0420 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-13T11:04:41.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 >> 172.21.15.16:0/808855118 conn(0x7fcdc408c570 msgr2=0x7fcdc40ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-13T11:04:41.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 shutdown_connections 2023-11-13T11:04:41.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-13T11:04:41.034+0000 7fcdcb013640 1 -- 172.21.15.16:0/808855118 wait complete. 2023-11-13T11:04:41.057 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-13T11:04:41.058 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-13T11:04:41.058 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-13T11:04:41.059 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.077 INFO:tasks.ceph.mds.a:Stopped 2023-11-13T11:04:41.077 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-13T11:04:41.078 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.147 INFO:tasks.ceph.mds.c:Stopped 2023-11-13T11:04:41.147 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-13T11:04:41.148 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.175 INFO:tasks.ceph.mds.b:Stopped 2023-11-13T11:04:41.176 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-13T11:04:41.176 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.239 INFO:tasks.ceph.mds.d:Stopped 2023-11-13T11:04:41.240 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-13T11:04:41.240 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-13T11:04:41.240 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.305 INFO:tasks.ceph.osd.0:Stopped 2023-11-13T11:04:41.306 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-13T11:04:41.306 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.400 INFO:tasks.ceph.osd.1:Stopped 2023-11-13T11:04:41.400 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-13T11:04:41.401 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.464 INFO:tasks.ceph.osd.2:Stopped 2023-11-13T11:04:41.465 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-13T11:04:41.465 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.539 INFO:tasks.ceph.osd.3:Stopped 2023-11-13T11:04:41.540 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-13T11:04:41.540 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.632 INFO:tasks.ceph.osd.4:Stopped 2023-11-13T11:04:41.632 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-13T11:04:41.633 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.697 INFO:tasks.ceph.osd.5:Stopped 2023-11-13T11:04:41.697 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-13T11:04:41.697 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.762 INFO:tasks.ceph.osd.6:Stopped 2023-11-13T11:04:41.762 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-13T11:04:41.763 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.807 INFO:tasks.ceph.osd.7:Stopped 2023-11-13T11:04:41.808 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-13T11:04:41.808 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-13T11:04:41.808 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.886 INFO:tasks.ceph.mgr.y:Stopped 2023-11-13T11:04:41.886 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-13T11:04:41.886 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:41.955 INFO:tasks.ceph.mgr.x:Stopped 2023-11-13T11:04:41.956 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-13T11:04:41.956 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-13T11:04:41.956 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:42.004 INFO:tasks.ceph.mon.a:Stopped 2023-11-13T11:04:42.005 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-13T11:04:42.005 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:42.043 INFO:tasks.ceph.mon.b:Stopped 2023-11-13T11:04:42.044 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-13T11:04:42.044 INFO:teuthology.orchestra.run:waiting for 300 2023-11-13T11:04:42.127 INFO:tasks.ceph.mon.c:Stopped 2023-11-13T11:04:42.127 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-13T11:04:42.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2023-11-13T11:04:42.248 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:04:42.248 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-13T11:04:42.509 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:04:42.510 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-13T11:04:42.718 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:04:42.718 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-13T11:04:42.918 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:04:42.919 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-13T11:04:43.133 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:04:43.133 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-13T11:04:43.420 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:04:43.421 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-13T11:04:43.582 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:04:43.583 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-13T11:04:43.742 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:04:43.743 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-13T11:04:43.901 INFO:tasks.ceph:Archiving mon data... 2023-11-13T11:04:43.903 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/data/mon.a.tgz 2023-11-13T11:04:43.903 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-11-13T11:04:43.969 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.42urowV5Oe 2023-11-13T11:04:43.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.42urowV5Oe 2023-11-13T11:04:44.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.42urowV5Oe 2023-11-13T11:04:44.396 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.42urowV5Oe is 340KB 2023-11-13T11:04:44.509 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.42urowV5Oe 2023-11-13T11:04:44.571 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/data/mon.b.tgz 2023-11-13T11:04:44.572 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-11-13T11:04:44.633 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.70BbhGU3gp 2023-11-13T11:04:44.633 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.70BbhGU3gp 2023-11-13T11:04:44.824 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.70BbhGU3gp 2023-11-13T11:04:44.977 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.70BbhGU3gp is 345KB 2023-11-13T11:04:45.044 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.70BbhGU3gp 2023-11-13T11:04:45.101 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/data/mon.c.tgz 2023-11-13T11:04:45.102 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-11-13T11:04:45.155 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.czgJnAcmli 2023-11-13T11:04:45.156 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.czgJnAcmli 2023-11-13T11:04:45.328 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.czgJnAcmli 2023-11-13T11:04:45.482 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.czgJnAcmli is 345KB 2023-11-13T11:04:45.591 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.czgJnAcmli 2023-11-13T11:04:45.649 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-13T11:04:45.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-13T11:04:45.660 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-13T11:04:45.763 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-13T11:04:45.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-13T11:04:45.770 DEBUG:teuthology.orchestra.run.smithi184:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-13T11:04:45.873 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-13T11:04:45.877 INFO:teuthology.orchestra.run.smithi184.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-13T11:04:45.891 INFO:tasks.ceph:Archiving crash dumps... 2023-11-13T11:04:45.893 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/remote/smithi016/crash 2023-11-13T11:04:45.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-13T11:04:45.972 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/remote/smithi184/crash 2023-11-13T11:04:45.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-13T11:04:46.073 INFO:tasks.ceph:Shutting down logrotate 2023-11-13T11:04:46.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:04:46.082 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-13T11:04:46.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-13T11:04:46.342 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-13T11:04:46.446 INFO:tasks.ceph:Compressing logs... 2023-11-13T11:04:46.447 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-13T11:04:46.457 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-13T11:04:46.550 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-13T11:04:46.551 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101723.log 2023-11-13T11:04:46.551 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92575.log 2023-11-13T11:04:46.553 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87362.log 2023-11-13T11:04:46.553 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100512.log 2023-11-13T11:04:46.555 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.92575.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-13T11:04:46.555 INFO:teuthology.orchestra.run.smithi184.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.1.92575.log.gz 2023-11-13T11:04:46.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116156.log 2023-11-13T11:04:46.556 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101723.log.gz 2023-11-13T11:04:46.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100866.log 2023-11-13T11:04:46.557 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100512.log.gz 2023-11-13T11:04:46.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116156.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116156.log.gz 2023-11-13T11:04:46.557 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114446.log 2023-11-13T11:04:46.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119868.log 2023-11-13T11:04:46.561 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100866.log.gz 2023-11-13T11:04:46.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115984.log 2023-11-13T11:04:46.562 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114446.log.gz 2023-11-13T11:04:46.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119868.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119868.log.gz 2023-11-13T11:04:46.562 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120120.log 2023-11-13T11:04:46.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93533.log 2023-11-13T11:04:46.563 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115984.log.gz 2023-11-13T11:04:46.563 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.87362.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87362.log.gz 2023-11-13T11:04:46.564 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-13T11:04:46.564 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87322.log 2023-11-13T11:04:46.564 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-13T11:04:46.565 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.87322.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-13T11:04:46.565 INFO:teuthology.orchestra.run.smithi184.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87322.log.gz 2023-11-13T11:04:46.565 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87242.log 2023-11-13T11:04:46.565 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87121.log 2023-11-13T11:04:46.569 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.87242.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87242.log.gz 2023-11-13T11:04:46.570 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87402.log 2023-11-13T11:04:46.570 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.87121.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-13T11:04:46.570 INFO:teuthology.orchestra.run.smithi184.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87121.log.gz 2023-11-13T11:04:46.571 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.87402.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.87402.log.gz 2023-11-13T11:04:46.571 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-13T11:04:46.571 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.93027.log 2023-11-13T11:04:46.571 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-13T11:04:46.574 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120078.log 2023-11-13T11:04:46.574 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120120.log.gz 2023-11-13T11:04:46.575 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93533.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93533.log.gz 2023-11-13T11:04:46.575 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113034.log 2023-11-13T11:04:46.575 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120078.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120078.log.gz 2023-11-13T11:04:46.576 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118425.log 2023-11-13T11:04:46.576 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94765.log 2023-11-13T11:04:46.576 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.113034.log.gz 2023-11-13T11:04:46.576 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116594.log 2023-11-13T11:04:46.577 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118425.log.gz 2023-11-13T11:04:46.577 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94765.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94765.log.gz 2023-11-13T11:04:46.577 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95803.log 2023-11-13T11:04:46.578 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116594.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116594.log.gz 2023-11-13T11:04:46.578 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94407.log 2023-11-13T11:04:46.578 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95803.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95803.log.gz 2023-11-13T11:04:46.578 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-13T11:04:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94407.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94407.log.gz 2023-11-13T11:04:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110682.log 2023-11-13T11:04:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95087.log 2023-11-13T11:04:46.583 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.3.93027.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-13T11:04:46.583 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mgr.x.log: 92.3% -- replaced with /var/log/ceph/ceph-client.3.93027.log.gz 2023-11-13T11:04:46.583 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87202.log 2023-11-13T11:04:46.584 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.87202.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-13T11:04:46.584 INFO:teuthology.orchestra.run.smithi184.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-13T11:04:46.584 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87202.log.gz 2023-11-13T11:04:46.585 INFO:teuthology.orchestra.run.smithi184.stderr: --verbose -- /var/log/ceph/ceph-client.admin.87282.log 2023-11-13T11:04:46.585 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.87282.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-13T11:04:46.589 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110682.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110682.log.gz 2023-11-13T11:04:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119952.log 2023-11-13T11:04:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95087.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95087.log.gz 2023-11-13T11:04:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119826.log 2023-11-13T11:04:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119952.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119952.log.gz 2023-11-13T11:04:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99084.log 2023-11-13T11:04:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119826.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119826.log.gz 2023-11-13T11:04:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110809.log 2023-11-13T11:04:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99084.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99084.log.gz 2023-11-13T11:04:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116384.log 2023-11-13T11:04:46.592 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110809.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110809.log.gz 2023-11-13T11:04:46.593 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100554.log 2023-11-13T11:04:46.593 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116384.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116384.log.gz 2023-11-13T11:04:46.593 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96322.log 2023-11-13T11:04:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100554.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100554.log.gz 2023-11-13T11:04:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101933.log 2023-11-13T11:04:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96322.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96322.log.gz 2023-11-13T11:04:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102209.log 2023-11-13T11:04:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101933.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101933.log.gz 2023-11-13T11:04:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94768.log 2023-11-13T11:04:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102209.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102209.log.gz 2023-11-13T11:04:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94093.log 2023-11-13T11:04:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94768.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94768.log.gz 2023-11-13T11:04:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100764.log 2023-11-13T11:04:46.598 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.d.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87282.log.gz 2023-11-13T11:04:46.598 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87161.log 2023-11-13T11:04:46.599 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-13T11:04:46.599 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.87161.log: /var/log/ceph/ceph-mon.c.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87161.log.gz 2023-11-13T11:04:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94093.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94093.log.gz 2023-11-13T11:04:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118136.log 2023-11-13T11:04:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100764.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100764.log.gz 2023-11-13T11:04:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111664.log 2023-11-13T11:04:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118136.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118136.log.gz 2023-11-13T11:04:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114000.log 2023-11-13T11:04:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111664.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111664.log.gz 2023-11-13T11:04:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116342.log 2023-11-13T11:04:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114000.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.114000.log.gz 2023-11-13T11:04:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109063.log 2023-11-13T11:04:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116342.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116342.log.gz 2023-11-13T11:04:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112352.log 2023-11-13T11:04:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99678.log 2023-11-13T11:04:46.606 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.109063.log.gz 2023-11-13T11:04:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97197.log 2023-11-13T11:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.112352.log.gz 2023-11-13T11:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99678.log.gz 2023-11-13T11:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110426.log 2023-11-13T11:04:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97197.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97197.log.gz 2023-11-13T11:04:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119461.log 2023-11-13T11:04:46.610 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110426.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110426.log.gz 2023-11-13T11:04:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120036.log 2023-11-13T11:04:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119461.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.119461.log.gz 2023-11-13T11:04:46.611 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.95045.log 2023-11-13T11:04:46.611 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120036.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120036.log.gz 2023-11-13T11:04:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119125.log 2023-11-13T11:04:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95045.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95045.log.gz 2023-11-13T11:04:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98484.log 2023-11-13T11:04:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119125.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119125.log.gz 2023-11-13T11:04:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116426.log 2023-11-13T11:04:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98484.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98484.log.gz 2023-11-13T11:04:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114629.log 2023-11-13T11:04:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116426.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116426.log.gz 2023-11-13T11:04:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116468.log 2023-11-13T11:04:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119742.log 2023-11-13T11:04:46.616 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114629.log.gz 2023-11-13T11:04:46.616 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116468.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116468.log.gz 2023-11-13T11:04:46.616 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115203.log 2023-11-13T11:04:46.617 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119742.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119742.log.gz 2023-11-13T11:04:46.618 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99552.log 2023-11-13T11:04:46.618 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115203.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115203.log.gz 2023-11-13T11:04:46.618 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97953.log 2023-11-13T11:04:46.619 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91909.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99552.log.gz 2023-11-13T11:04:46.619 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-11-13T11:04:46.620 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97953.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97953.log.gz 2023-11-13T11:04:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108997.log 2023-11-13T11:04:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102143.log 2023-11-13T11:04:46.621 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91909.log.gz 2023-11-13T11:04:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97827.log 2023-11-13T11:04:46.623 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.108997.log.gz 2023-11-13T11:04:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91558.log 2023-11-13T11:04:46.624 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.102143.log.gz 2023-11-13T11:04:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97827.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97827.log.gz 2023-11-13T11:04:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101365.log 2023-11-13T11:04:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91953.log 2023-11-13T11:04:46.627 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91558.log.gz 2023-11-13T11:04:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101365.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101365.log.gz 2023-11-13T11:04:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99210.log 2023-11-13T11:04:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110989.log 2023-11-13T11:04:46.629 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91953.log.gz 2023-11-13T11:04:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99210.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99210.log.gz 2023-11-13T11:04:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117849.log 2023-11-13T11:04:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111411.log 2023-11-13T11:04:46.632 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110989.log.gz 2023-11-13T11:04:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117849.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117849.log.gz 2023-11-13T11:04:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119503.log 2023-11-13T11:04:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93743.log 2023-11-13T11:04:46.634 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111411.log.gz 2023-11-13T11:04:46.635 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119503.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119503.log.gz 2023-11-13T11:04:46.635 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.114122.log 2023-11-13T11:04:46.636 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117665.log 2023-11-13T11:04:46.637 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93743.log.gz 2023-11-13T11:04:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93257.log 2023-11-13T11:04:46.638 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.114122.log.gz 2023-11-13T11:04:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117665.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.117665.log.gz -- 2023-11-13T11:04:46.639 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.94089.log 2023-11-13T11:04:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93257.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93257.log.gz 2023-11-13T11:04:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93031.log 2023-11-13T11:04:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114272.log 2023-11-13T11:04:46.642 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94089.log.gz 2023-11-13T11:04:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93031.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93031.log.gz 2023-11-13T11:04:46.643 INFO:teuthology.orchestra.run.smithi184.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-13T11:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113696.log 2023-11-13T11:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100470.log 2023-11-13T11:04:46.644 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.114272.log.gz 2023-11-13T11:04:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113696.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.113696.log.gz 2023-11-13T11:04:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112162.log 2023-11-13T11:04:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100134.log 2023-11-13T11:04:46.647 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100470.log.gz 2023-11-13T11:04:46.647 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112162.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112162.log.gz 2023-11-13T11:04:46.648 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96987.log 2023-11-13T11:04:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100134.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100134.log.gz 2023-11-13T11:04:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97029.log 2023-11-13T11:04:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116763.log 2023-11-13T11:04:46.650 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96987.log.gz 2023-11-13T11:04:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97029.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97029.log.gz 2023-11-13T11:04:46.652 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100344.log 2023-11-13T11:04:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116763.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116763.log.gz 2023-11-13T11:04:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117287.log 2023-11-13T11:04:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100344.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100344.log.gz 2023-11-13T11:04:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91727.log 2023-11-13T11:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117287.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117287.log.gzgzip 2023-11-13T11:04:46.655 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.94081.log 2023-11-13T11:04:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91727.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91727.log.gz 2023-11-13T11:04:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109162.log 2023-11-13T11:04:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100680.log 2023-11-13T11:04:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109162.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94081.log.gz 2023-11-13T11:04:46.659 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.109162.log.gz 2023-11-13T11:04:46.660 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113436.log 2023-11-13T11:04:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100680.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100680.log.gz 2023-11-13T11:04:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119377.log 2023-11-13T11:04:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113436.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113436.log.gz 2023-11-13T11:04:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109228.log 2023-11-13T11:04:46.663 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95549.log 2023-11-13T11:04:46.663 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119377.log.gz 2023-11-13T11:04:46.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109964.log 2023-11-13T11:04:46.665 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.109228.log.gz 2023-11-13T11:04:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97071.log 2023-11-13T11:04:46.666 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95549.log.gz 2023-11-13T11:04:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109964.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109964.log.gz 2023-11-13T11:04:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92989.log 2023-11-13T11:04:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97071.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97071.log.gz 2023-11-13T11:04:46.668 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.115390.log 2023-11-13T11:04:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92989.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92989.log.gz 2023-11-13T11:04:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97281.log 2023-11-13T11:04:46.670 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93827.log 2023-11-13T11:04:46.670 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115390.log.gz 2023-11-13T11:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97281.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97281.log.gz 2023-11-13T11:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93383.log 2023-11-13T11:04:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93827.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93827.log.gz 2023-11-13T11:04:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98814.log 2023-11-13T11:04:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93383.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93383.log.gz 2023-11-13T11:04:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120610.log 2023-11-13T11:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117891.log 2023-11-13T11:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98814.log.gz 2023-11-13T11:04:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120610.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120610.log.gz 2023-11-13T11:04:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99882.log 2023-11-13T11:04:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117891.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117891.log.gz 2023-11-13T11:04:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100260.log 2023-11-13T11:04:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99882.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99882.log.gz 2023-11-13T11:04:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95255.log 2023-11-13T11:04:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112418.log 2023-11-13T11:04:46.680 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100260.log.gz 2023-11-13T11:04:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95255.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95255.log.gz 2023-11-13T11:04:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120570.log 2023-11-13T11:04:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119910.log 2023-11-13T11:04:46.683 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112418.log.gz 2023-11-13T11:04:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120570.log: gzip -5 --verbose 88.6% -- -- replaced with /var/log/ceph/ceph-client.admin.120570.log.gz /var/log/ceph/ceph-client.admin.118134.log 2023-11-13T11:04:46.684 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-11-13T11:04:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91238.log 2023-11-13T11:04:46.685 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119910.log.gz 2023-11-13T11:04:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101073.log 2023-11-13T11:04:46.686 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118134.log.gz 2023-11-13T11:04:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91238.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91238.log.gz 2023-11-13T11:04:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108931.log 2023-11-13T11:04:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92733.log 2023-11-13T11:04:46.689 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101073.log.gz 2023-11-13T11:04:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115287.log 2023-11-13T11:04:46.691 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.108931.log.gz 2023-11-13T11:04:46.692 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95591.log 2023-11-13T11:04:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115287.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.92733.log.gz 2023-11-13T11:04:46.694 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118789.log 2023-11-13T11:04:46.694 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115287.log.gz 2023-11-13T11:04:46.695 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95591.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95591.log.gz 2023-11-13T11:04:46.695 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112691.log 2023-11-13T11:04:46.696 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118789.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118789.log.gz 2023-11-13T11:04:46.697 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97533.log 2023-11-13T11:04:46.698 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114867.log 2023-11-13T11:04:46.698 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112691.log.gz 2023-11-13T11:04:46.699 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111031.log 2023-11-13T11:04:46.699 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97533.log.gz 2023-11-13T11:04:46.700 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114867.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.114867.log.gz 2023-11-13T11:04:46.700 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.119251.log 2023-11-13T11:04:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111031.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111031.log.gz 2023-11-13T11:04:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115035.log 2023-11-13T11:04:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93491.log 2023-11-13T11:04:46.703 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119251.log.gz 2023-11-13T11:04:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115035.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115035.log.gz 2023-11-13T11:04:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97869.log 2023-11-13T11:04:46.706 INFO:teuthology.orchestra.run.smithi184.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-13T11:04:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119335.log 2023-11-13T11:04:46.707 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93491.log.gz 2023-11-13T11:04:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97869.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97869.log.gz 2023-11-13T11:04:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120288.log 2023-11-13T11:04:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98300.log 2023-11-13T11:04:46.709 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.119335.log.gz 2023-11-13T11:04:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110521.log 2023-11-13T11:04:46.710 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120288.log.gz 2023-11-13T11:04:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98300.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98300.log.gz 2023-11-13T11:04:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100638.log 2023-11-13T11:04:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110521.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110521.log.gz 2023-11-13T11:04:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113100.log 2023-11-13T11:04:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100638.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-13T11:04:46.714 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100638.log.gz 2023-11-13T11:04:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110342.log 2023-11-13T11:04:46.715 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.113100.log.gz 2023-11-13T11:04:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92163.log 2023-11-13T11:04:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110342.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110342.log.gz 2023-11-13T11:04:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93173.log 2023-11-13T11:04:46.719 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-13T11:04:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92163.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.92163.log.gz --verbose 2023-11-13T11:04:46.720 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.99042.log 2023-11-13T11:04:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.96713.log 2023-11-13T11:04:46.721 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93173.log.gz 2023-11-13T11:04:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96835.log 2023-11-13T11:04:46.722 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99042.log.gz 2023-11-13T11:04:46.723 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.2.96713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100218.log 2023-11-13T11:04:46.724 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96835.log: 92.4% -- replaced with /var/log/ceph/ceph-client.2.96713.log.gz 2023-11-13T11:04:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96835.log.gz -5 2023-11-13T11:04:46.725 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.92205.log 2023-11-13T11:04:46.726 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100909.log 2023-11-13T11:04:46.726 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100218.log.gz 2023-11-13T11:04:46.727 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117807.log 2023-11-13T11:04:46.728 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92205.log.gz 2023-11-13T11:04:46.728 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100909.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100909.log.gz 2023-11-13T11:04:46.729 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100092.log 2023-11-13T11:04:46.730 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120204.log 2023-11-13T11:04:46.730 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117807.log.gz 2023-11-13T11:04:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99326.log 2023-11-13T11:04:46.735 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100092.log.gz 2023-11-13T11:04:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120162.log 2023-11-13T11:04:46.735 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120204.log.gz 2023-11-13T11:04:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99326.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99326.log.gz 2023-11-13T11:04:46.736 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87619.log 2023-11-13T11:04:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116510.log 2023-11-13T11:04:46.737 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120162.log.gz 2023-11-13T11:04:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.87619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111941.log 2023-11-13T11:04:46.738 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87619.log.gz 2023-11-13T11:04:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116510.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116510.log.gz 2023-11-13T11:04:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115642.log 2023-11-13T11:04:46.739 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.111941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111201.log 2023-11-13T11:04:46.741 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100722.log 2023-11-13T11:04:46.741 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115642.log.gz 2023-11-13T11:04:46.742 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118915.log 2023-11-13T11:04:46.742 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111201.log.gz 2023-11-13T11:04:46.743 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100951.log 2023-11-13T11:04:46.744 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100722.log.gz 2023-11-13T11:04:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118915.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118915.log.gz 2023-11-13T11:04:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95339.log 2023-11-13T11:04:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94671.log 2023-11-13T11:04:46.746 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100951.log.gz 2023-11-13T11:04:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93785.log 2023-11-13T11:04:46.748 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95339.log.gz 2023-11-13T11:04:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115600.log 2023-11-13T11:04:46.749 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94671.log.gz 2023-11-13T11:04:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97365.log 2023-11-13T11:04:46.750 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93785.log.gz 2023-11-13T11:04:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101239.log 2023-11-13T11:04:46.752 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115600.log.gz 2023-11-13T11:04:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97365.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97365.log.gz -5 2023-11-13T11:04:46.753 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.101765.log 2023-11-13T11:04:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118575.log 2023-11-13T11:04:46.754 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101239.log.gz 2023-11-13T11:04:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101765.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101765.log.gz 2023-11-13T11:04:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93073.log 2023-11-13T11:04:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119994.log 2023-11-13T11:04:46.757 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118575.log.gz 2023-11-13T11:04:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93073.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.93073.log.gz -5 2023-11-13T11:04:46.759 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.113953.log 2023-11-13T11:04:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91436.log 2023-11-13T11:04:46.760 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119994.log.gz 2023-11-13T11:04:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114056.log 2023-11-13T11:04:46.761 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.113953.log.gz 2023-11-13T11:04:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91436.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91436.log.gz 2023-11-13T11:04:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118130.log 2023-11-13T11:04:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92947.log 2023-11-13T11:04:46.765 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.114056.log.gz 2023-11-13T11:04:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95297.log 2023-11-13T11:04:46.766 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118130.log.gz 2023-11-13T11:04:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111075.log 88.9% 2023-11-13T11:04:46.768 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92947.log.gz 2023-11-13T11:04:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101891.log 2023-11-13T11:04:46.769 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95297.log.gz 2023-11-13T11:04:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118617.log 2023-11-13T11:04:46.770 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111075.log.gz 2023-11-13T11:04:46.772 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119083.log 2023-11-13T11:04:46.772 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101891.log.gz 2023-11-13T11:04:46.773 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118617.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118617.log.gz 2023-11-13T11:04:46.773 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92037.log 2023-11-13T11:04:46.774 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98342.log 2023-11-13T11:04:46.775 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119083.log.gz 2023-11-13T11:04:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102059.log 2023-11-13T11:04:46.776 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92037.log.gz 2023-11-13T11:04:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112608.log 2023-11-13T11:04:46.777 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98342.log.gz 2023-11-13T11:04:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102059.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102059.log.gz 2023-11-13T11:04:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117329.log 2023-11-13T11:04:46.779 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109670.log 2023-11-13T11:04:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117329.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.112608.log.gz 2023-11-13T11:04:46.781 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.117329.log.gz 2023-11-13T11:04:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101509.log 2023-11-13T11:04:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115119.log 2023-11-13T11:04:46.782 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109670.log.gz 2023-11-13T11:04:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101509.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101509.log.gz 2023-11-13T11:04:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93869.log 2023-11-13T11:04:46.785 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95003.log 2023-11-13T11:04:46.785 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115119.log.gz 2023-11-13T11:04:46.786 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93869.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93869.log.gz 2023-11-13T11:04:46.786 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97575.log 2023-11-13T11:04:46.787 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94090.log 2023-11-13T11:04:46.788 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95003.log.gz 2023-11-13T11:04:46.788 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97575.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97575.log.gz 2023-11-13T11:04:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100596.log 2023-11-13T11:04:46.790 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116678.log 2023-11-13T11:04:46.790 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94090.log.gz 2023-11-13T11:04:46.791 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100596.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100596.log.gz 2023-11-13T11:04:46.791 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95423.log 2023-11-13T11:04:46.792 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119167.log 2023-11-13T11:04:46.793 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116678.log.gz 2023-11-13T11:04:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95423.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95423.log.gz 2023-11-13T11:04:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99636.log 2023-11-13T11:04:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91643.log 2023-11-13T11:04:46.795 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119167.log.gz 2023-11-13T11:04:46.796 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99636.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99636.log.gz 2023-11-13T11:04:46.796 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120652.log 2023-11-13T11:04:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109415.log 2023-11-13T11:04:46.798 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91643.log.gz 2023-11-13T11:04:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120652.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120652.log.gz 2023-11-13T11:04:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91337.log 2023-11-13T11:04:46.800 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112968.log 2023-11-13T11:04:46.800 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109415.log.gz 2023-11-13T11:04:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91337.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91337.log.gz 2023-11-13T11:04:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98730.log 2023-11-13T11:04:46.802 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95129.log 2023-11-13T11:04:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98730.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112968.log.gz 2023-11-13T11:04:46.804 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98730.log.gz 2023-11-13T11:04:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110851.log 2023-11-13T11:04:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97785.log 2023-11-13T11:04:46.806 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95129.log.gz 2023-11-13T11:04:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110851.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110851.log.gz 2023-11-13T11:04:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110174.log 2023-11-13T11:04:46.808 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99494.log 2023-11-13T11:04:46.808 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97785.log.gz 2023-11-13T11:04:46.809 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110174.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110174.log.gz 2023-11-13T11:04:46.809 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-13T11:04:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115900.log 2023-11-13T11:04:46.811 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99494.log.gz 2023-11-13T11:04:46.812 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98442.log 2023-11-13T11:04:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115474.log 2023-11-13T11:04:46.814 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115900.log.gz 2023-11-13T11:04:46.815 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117581.log 2023-11-13T11:04:46.815 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98442.log.gz 2023-11-13T11:04:46.816 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117447.log 2023-11-13T11:04:46.817 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115474.log.gz 2023-11-13T11:04:46.818 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101153.log 2023-11-13T11:04:46.819 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117581.log.gz 2023-11-13T11:04:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116112.log 2023-11-13T11:04:46.820 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117447.log.gz 2023-11-13T11:04:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111556.log 2023-11-13T11:04:46.822 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101153.log.gz 2023-11-13T11:04:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100302.log 2023-11-13T11:04:46.823 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116112.log.gz 2023-11-13T11:04:46.825 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113779.log 2023-11-13T11:04:46.825 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111556.log.gz 2023-11-13T11:04:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95465.log 2023-11-13T11:04:46.826 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100302.log.gz 2023-11-13T11:04:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116552.log 2023-11-13T11:04:46.828 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113779.log.gz 2023-11-13T11:04:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114993.log 2023-11-13T11:04:46.830 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95465.log.gz 2023-11-13T11:04:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99368.log 2023-11-13T11:04:46.831 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116552.log.gz 2023-11-13T11:04:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114993.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114993.log.gzgzip 2023-11-13T11:04:46.833 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.91995.log 2023-11-13T11:04:46.834 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120498.log 2023-11-13T11:04:46.834 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99368.log.gz 2023-11-13T11:04:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111327.log 2023-11-13T11:04:46.836 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91995.log.gz 2023-11-13T11:04:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114951.log 2023-11-13T11:04:46.838 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120498.log.gz 2023-11-13T11:04:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119419.log 2023-11-13T11:04:46.839 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111327.log.gz 2023-11-13T11:04:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94425.log 2023-11-13T11:04:46.841 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114951.log.gz 2023-11-13T11:04:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110384.log 2023-11-13T11:04:46.842 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119419.log.gz 2023-11-13T11:04:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118831.log 2023-11-13T11:04:46.844 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94425.log.gz 2023-11-13T11:04:46.845 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94733.log 2023-11-13T11:04:46.845 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110384.log.gz 2023-11-13T11:04:46.847 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111117.log 2023-11-13T11:04:46.847 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118831.log.gz 2023-11-13T11:04:46.848 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115077.log 2023-11-13T11:04:46.849 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94733.log.gz 2023-11-13T11:04:46.850 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101807.log 2023-11-13T11:04:46.850 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111117.log.gz 2023-11-13T11:04:46.852 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99924.log 2023-11-13T11:04:46.852 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115077.log.gz 2023-11-13T11:04:46.853 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101807.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101807.log.gz 2023-11-13T11:04:46.853 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95213.log 2023-11-13T11:04:46.854 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93617.log 2023-11-13T11:04:46.855 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99924.log.gz 2023-11-13T11:04:46.856 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95213.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95213.log.gz 2023-11-13T11:04:46.856 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98037.log 2023-11-13T11:04:46.858 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117707.log 2023-11-13T11:04:46.858 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93617.log.gz 2023-11-13T11:04:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98037.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98037.log.gz 2023-11-13T11:04:46.859 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112786.log 2023-11-13T11:04:46.860 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96877.log 2023-11-13T11:04:46.861 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117707.log.gz 2023-11-13T11:04:46.862 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112786.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112786.log.gz 2023-11-13T11:04:46.862 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93341.log 2023-11-13T11:04:46.864 INFO:teuthology.orchestra.run.smithi184.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-13T11:04:46.864 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96877.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96877.log.gz 2023-11-13T11:04:46.864 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109838.log 2023-11-13T11:04:46.865 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97491.log 2023-11-13T11:04:46.866 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93341.log.gz 2023-11-13T11:04:46.867 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109838.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109838.log.gz 2023-11-13T11:04:46.867 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109754.log 2023-11-13T11:04:46.868 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101637.log 2023-11-13T11:04:46.868 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97491.log.gz 2023-11-13T11:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109754.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109754.log.gz 2023-11-13T11:04:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87702.log 2023-11-13T11:04:46.871 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101637.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101637.log.gz 2023-11-13T11:04:46.871 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110258.log 2023-11-13T11:04:46.872 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.87702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109880.log 2023-11-13T11:04:46.872 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87702.log.gz 2023-11-13T11:04:46.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110258.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110258.log.gz 2023-11-13T11:04:46.874 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118873.log 2023-11-13T11:04:46.875 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109880.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109880.log.gz 2023-11-13T11:04:46.875 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97701.log 2023-11-13T11:04:46.877 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111598.log 2023-11-13T11:04:46.877 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.118873.log.gz 2023-11-13T11:04:46.878 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97701.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97701.log.gz 2023-11-13T11:04:46.879 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101551.log 2023-11-13T11:04:46.880 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98216.log 2023-11-13T11:04:46.880 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101551.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.111598.log.gz 2023-11-13T11:04:46.881 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97155.log 2023-11-13T11:04:46.882 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101551.log.gz 2023-11-13T11:04:46.883 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110724.log 2023-11-13T11:04:46.883 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98216.log.gz 2023-11-13T11:04:46.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97155.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97155.log.gz -5 2023-11-13T11:04:46.885 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.95381.log 2023-11-13T11:04:46.886 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91601.log 2023-11-13T11:04:46.886 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110724.log.gz 2023-11-13T11:04:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95381.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95381.log.gz 2023-11-13T11:04:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92121.log 2023-11-13T11:04:46.889 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112480.log 2023-11-13T11:04:46.889 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91601.log.gz 2023-11-13T11:04:46.890 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92121.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92121.log.gz 2023-11-13T11:04:46.891 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97911.log 2023-11-13T11:04:46.892 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100386.log 2023-11-13T11:04:46.892 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.112480.log.gz 2023-11-13T11:04:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112912.log 2023-11-13T11:04:46.893 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97911.log.gz 2023-11-13T11:04:46.895 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100386.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100386.log.gz 2023-11-13T11:04:46.895 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111159.log 2023-11-13T11:04:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112912.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.112912.log.gz 2023-11-13T11:04:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97449.log 2023-11-13T11:04:46.898 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92313.log 2023-11-13T11:04:46.898 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111159.log.gz 2023-11-13T11:04:46.899 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97449.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97449.log.gz 2023-11-13T11:04:46.899 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98772.log 2023-11-13T11:04:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92313.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.92313.log.gz 2023-11-13T11:04:46.901 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93299.log 2023-11-13T11:04:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98772.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98772.log.gz 2023-11-13T11:04:46.903 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93115.log 2023-11-13T11:04:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93299.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93299.log.gz 2023-11-13T11:04:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-13T11:04:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93115.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.93115.log.gz 2023-11-13T11:04:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101681.log 2023-11-13T11:04:46.907 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94422.log 2023-11-13T11:04:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101681.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101681.log.gz 2023-11-13T11:04:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98384.log 2023-11-13T11:04:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93215.log 2023-11-13T11:04:46.911 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94422.log.gz 2023-11-13T11:04:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98384.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98384.log.gz 2023-11-13T11:04:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101281.log 2023-11-13T11:04:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99452.log 2023-11-13T11:04:46.914 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93215.log.gz 2023-11-13T11:04:46.915 INFO:teuthology.orchestra.run.smithi184.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-13T11:04:46.916 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101281.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101281.log.gz 2023-11-13T11:04:46.916 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98982.log 2023-11-13T11:04:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99452.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99452.log.gz 2023-11-13T11:04:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114671.log 2023-11-13T11:04:46.918 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98610.log 2023-11-13T11:04:46.918 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98982.log.gz 2023-11-13T11:04:46.920 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94416.log 2023-11-13T11:04:46.920 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114671.log.gz 2023-11-13T11:04:46.921 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100806.log 2023-11-13T11:04:46.922 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98610.log.gz 2023-11-13T11:04:46.923 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116721.log 2023-11-13T11:04:46.923 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94416.log.gz 2023-11-13T11:04:46.925 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110767.log 2023-11-13T11:04:46.925 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100806.log.gz 2023-11-13T11:04:46.926 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102677.log 2023-11-13T11:04:46.927 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116721.log.gz 2023-11-13T11:04:46.928 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109922.log 2023-11-13T11:04:46.928 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110767.log.gz 2023-11-13T11:04:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118383.log 2023-11-13T11:04:46.930 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102677.log.gz 2023-11-13T11:04:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110601.log 2023-11-13T11:04:46.931 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109922.log.gz 2023-11-13T11:04:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102488.log 2023-11-13T11:04:46.933 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118383.log.gz 2023-11-13T11:04:46.935 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115738.log 2023-11-13T11:04:46.935 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110601.log.gz 2023-11-13T11:04:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.102488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99762.log 2023-11-13T11:04:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97743.log 2023-11-13T11:04:46.938 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115738.log.gz 2023-11-13T11:04:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117623.log 2023-11-13T11:04:46.940 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99762.log.gz 2023-11-13T11:04:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94427.log 2023-11-13T11:04:46.941 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97743.log.gz 2023-11-13T11:04:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101849.log 2023-11-13T11:04:46.943 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117623.log.gz 2023-11-13T11:04:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110048.log 2023-11-13T11:04:46.944 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94427.log.gz 2023-11-13T11:04:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99410.log 2023-11-13T11:04:46.946 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101849.log.gz 2023-11-13T11:04:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114380.log 2023-11-13T11:04:46.948 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110048.log.gz 2023-11-13T11:04:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110132.log 2023-11-13T11:04:46.949 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99410.log.gz 2023-11-13T11:04:46.950 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109626.log 2023-11-13T11:04:46.951 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.114380.log.gz 2023-11-13T11:04:46.952 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110132.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110132.log.gz -5 2023-11-13T11:04:46.952 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.118011.log 2023-11-13T11:04:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99168.log 2023-11-13T11:04:46.954 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109626.log.gz 2023-11-13T11:04:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98079.log 2023-11-13T11:04:46.956 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.118011.log.gz 2023-11-13T11:04:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100993.log 2023-11-13T11:04:46.957 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99168.log.gz 2023-11-13T11:04:46.958 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98079.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98079.log.gz 2023-11-13T11:04:46.958 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112546.log 2023-11-13T11:04:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109331.log 2023-11-13T11:04:46.960 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100993.log.gz 2023-11-13T11:04:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91516.log 2023-11-13T11:04:46.962 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112546.log.gz 2023-11-13T11:04:46.962 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109331.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109331.log.gz 2023-11-13T11:04:46.963 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99720.log 2023-11-13T11:04:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118747.log 2023-11-13T11:04:46.965 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91516.log.gz 2023-11-13T11:04:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94429.log 2023-11-13T11:04:46.966 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99720.log.gz 2023-11-13T11:04:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112865.log 2023-11-13T11:04:46.968 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118747.log.gz 2023-11-13T11:04:46.969 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97323.log 2023-11-13T11:04:46.969 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94429.log.gz 2023-11-13T11:04:46.971 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115348.log 2023-11-13T11:04:46.971 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.112865.log.gz 2023-11-13T11:04:46.972 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97323.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97323.log.gz 2023-11-13T11:04:46.972 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117539.log 2023-11-13T11:04:46.973 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115348.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115348.log.gz 2023-11-13T11:04:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109457.log 2023-11-13T11:04:46.976 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94082.log 2023-11-13T11:04:46.976 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117539.log.gz 2023-11-13T11:04:46.977 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99284.log 2023-11-13T11:04:46.977 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109457.log.gz 2023-11-13T11:04:46.979 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100008.log 2023-11-13T11:04:46.979 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94082.log.gz 2023-11-13T11:04:46.980 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99284.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81318.log 2023-11-13T11:04:46.980 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99284.log.gz 2023-11-13T11:04:46.982 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113166.log 2023-11-13T11:04:46.982 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100008.log.gz 2023-11-13T11:04:46.983 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.tmp-client.admin.81318.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81318.log.gz 2023-11-13T11:04:46.983 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118302.log 2023-11-13T11:04:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113166.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-13T11:04:46.985 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.113166.log.gz 2023-11-13T11:04:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118302.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118302.log.gz 2023-11-13T11:04:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93575.log 2023-11-13T11:04:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115432.log 2023-11-13T11:04:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93575.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93575.log.gz 2023-11-13T11:04:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111369.log 2023-11-13T11:04:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115432.log: 88.0%gzip -5 --verbose/var/log/ceph/ceph-client.admin.111369.log: -- /var/log/ceph/ceph-client.admin.98526.log 2023-11-13T11:04:46.993 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111369.log.gz -- replaced with /var/log/ceph/ceph-client.admin.115432.log.gz 2023-11-13T11:04:46.993 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-11-13T11:04:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115516.log 2023-11-13T11:04:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98526.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98526.log.gz 2023-11-13T11:04:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110300.log 2023-11-13T11:04:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115516.log: 94.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-13T11:04:46.999 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115516.log.gz 2023-11-13T11:04:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117765.log 2023-11-13T11:04:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110300.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110300.log.gz 2023-11-13T11:04:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111454.log 2023-11-13T11:04:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119041.log 2023-11-13T11:04:47.002 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117765.log.gz 2023-11-13T11:04:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.117148.log 2023-11-13T11:04:47.005 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111454.log.gz 2023-11-13T11:04:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115942.log 2023-11-13T11:04:47.006 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119041.log.gz 2023-11-13T11:04:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.117148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111243.log 2023-11-13T11:04:47.008 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115942.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115942.log.gz 2023-11-13T11:04:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101195.log/var/log/ceph/ceph-client.admin.111243.log: 2023-11-13T11:04:47.010 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111243.log.gz 2023-11-13T11:04:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120414.log 2023-11-13T11:04:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101195.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101195.log.gz 2023-11-13T11:04:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95507.log 2023-11-13T11:04:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120414.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120414.log.gz 2023-11-13T11:04:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101593.log 2023-11-13T11:04:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95507.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95507.log.gz 2023-11-13T11:04:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101975.log 2023-11-13T11:04:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101593.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101593.log.gz 2023-11-13T11:04:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100428.log 2023-11-13T11:04:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101975.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101975.log.gz 2023-11-13T11:04:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97995.log 2023-11-13T11:04:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100428.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100428.log.gz 2023-11-13T11:04:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113874.log 2023-11-13T11:04:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97995.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97995.log.gz 2023-11-13T11:04:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91771.log 2023-11-13T11:04:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113874.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113874.log.gz 2023-11-13T11:04:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112290.log 2023-11-13T11:04:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91771.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91771.log.gz 2023-11-13T11:04:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98258.log 2023-11-13T11:04:47.031 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112290.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112290.log.gz 2023-11-13T11:04:47.033 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116216.log 2023-11-13T11:04:47.033 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98258.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98258.log.gz 2023-11-13T11:04:47.034 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98940.log 2023-11-13T11:04:47.036 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116216.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116216.log.gz 2023-11-13T11:04:47.037 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111285.log 2023-11-13T11:04:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98940.log: 93.0% -- replaced with /var/log/ceph/ceph-client.0.117148.log.gz 2023-11-13T11:04:47.038 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98940.log.gz 2023-11-13T11:04:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114338.log 2023-11-13T11:04:47.040 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111285.log: gzip -5 --verbose -- 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111285.log.gz 2023-11-13T11:04:47.040 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.115818.log 2023-11-13T11:04:47.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114338.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114338.log.gz 2023-11-13T11:04:47.042 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116026.log 2023-11-13T11:04:47.043 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109373.log 2023-11-13T11:04:47.044 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115818.log.gz 2023-11-13T11:04:47.045 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120246.log 2023-11-13T11:04:47.045 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116026.log.gz 2023-11-13T11:04:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109373.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-13T11:04:47.047 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109373.log.gz 2023-11-13T11:04:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113274.log 2023-11-13T11:04:47.048 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120246.log.gz 2023-11-13T11:04:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94086.log 2023-11-13T11:04:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95675.log 2023-11-13T11:04:47.052 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.113274.log.gz 2023-11-13T11:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102017.log 2023-11-13T11:04:47.054 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94086.log.gz 2023-11-13T11:04:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95675.log: /var/log/ceph/ceph-client.admin.102017.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.95675.log.gz 2023-11-13T11:04:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114825.log 2023-11-13T11:04:47.057 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102017.log.gz 2023-11-13T11:04:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.114825.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.99966.log 2023-11-13T11:04:47.059 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114825.log.gz 2023-11-13T11:04:47.060 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94421.log 2023-11-13T11:04:47.062 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99966.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99966.log.gz 2023-11-13T11:04:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118467.log 2023-11-13T11:04:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94421.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94421.log.gz 2023-11-13T11:04:47.065 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110939.log 2023-11-13T11:04:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118467.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.118467.log.gz 2023-11-13T11:04:47.067 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.110939.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.100176.log 2023-11-13T11:04:47.068 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110939.log.gz 2023-11-13T11:04:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113232.log 2023-11-13T11:04:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100176.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100176.log.gz 2023-11-13T11:04:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102101.log 2023-11-13T11:04:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113232.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113232.log.gz 2023-11-13T11:04:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116068.log 2023-11-13T11:04:47.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102101.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102101.log.gz 2023-11-13T11:04:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116805.log 2023-11-13T11:04:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116068.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116068.log.gz 2023-11-13T11:04:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94769.log 2023-11-13T11:04:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116805.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.116805.log.gz 2023-11-13T11:04:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5/var/log/ceph/ceph-client.admin.94769.log: --verbose -- /var/log/ceph/ceph-client.admin.109712.log 2023-11-13T11:04:47.081 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94769.log.gz 2023-11-13T11:04:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120372.log 2023-11-13T11:04:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109712.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109712.log.gz 2023-11-13T11:04:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101323.log 2023-11-13T11:04:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120372.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120372.log.gz 2023-11-13T11:04:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119293.log 2023-11-13T11:04:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101323.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101323.log.gz 2023-11-13T11:04:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92905.log 2023-11-13T11:04:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119293.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119293.log.gz 2023-11-13T11:04:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113630.log 2023-11-13T11:04:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92905.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92905.log.gz 2023-11-13T11:04:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94424.log 2023-11-13T11:04:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113630.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.113630.log.gz 2023-11-13T11:04:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110895.log 2023-11-13T11:04:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94424.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94424.log.gz 2023-11-13T11:04:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98898.log 2023-11-13T11:04:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110895.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110895.log.gz 2023-11-13T11:04:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116258.log 2023-11-13T11:04:47.100 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98898.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98898.log.gz 2023-11-13T11:04:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114713.log 2023-11-13T11:04:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116258.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116258.log.gz 2023-11-13T11:04:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95633.log 2023-11-13T11:04:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114713.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114713.log.gz 2023-11-13T11:04:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94092.log 2023-11-13T11:04:47.106 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95633.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95633.log.gz 2023-11-13T11:04:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114587.log 2023-11-13T11:04:47.108 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94092.log.gz 2023-11-13T11:04:47.109 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-13T11:04:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114587.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114587.log.gz 2023-11-13T11:04:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116636.log 2023-11-13T11:04:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.116636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97113.log 2023-11-13T11:04:47.115 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-13T11:04:47.115 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116636.log.gz 2023-11-13T11:04:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116300.log 2023-11-13T11:04:47.118 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97113.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97113.log.gz 2023-11-13T11:04:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94961.log 2023-11-13T11:04:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116300.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116300.log.gz 2023-11-13T11:04:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116871.log 2023-11-13T11:04:47.122 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94961.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94961.log.gz 2023-11-13T11:04:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118957.log 2023-11-13T11:04:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116871.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116871.log.gz 2023-11-13T11:04:47.126 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118957.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118957.log.gz 2023-11-13T11:04:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-13T11:04:47.128 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100050.log 2023-11-13T11:04:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.100050.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100050.log.gz 2023-11-13T11:04:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97407.log 2023-11-13T11:04:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119209.log 2023-11-13T11:04:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119209.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97407.log.gz 2023-11-13T11:04:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91865.log 2023-11-13T11:04:47.141 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119209.log.gz 2023-11-13T11:04:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118533.log 2023-11-13T11:04:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118533.log: /var/log/ceph/ceph-client.admin.91865.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-13T11:04:47.148 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91865.log.gz 2023-11-13T11:04:47.149 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118533.log.gz 2023-11-13T11:04:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101407.log 2023-11-13T11:04:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101407.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99594.log 2023-11-13T11:04:47.158 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101407.log.gz 2023-11-13T11:04:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101449.log 2023-11-13T11:04:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99594.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99594.log.gz 2023-11-13T11:04:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109796.log 2023-11-13T11:04:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101449.log: /var/log/ceph/ceph-client.admin.109796.log: 88.8% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109796.log.gz 2023-11-13T11:04:47.168 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101449.log.gz 2023-11-13T11:04:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117371.log 2023-11-13T11:04:47.170 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111508.log 2023-11-13T11:04:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117371.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109499.log 2023-11-13T11:04:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111508.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111508.log.gz 2023-11-13T11:04:47.177 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.117371.log.gz 2023-11-13T11:04:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92079.log 2023-11-13T11:04:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94675.log 2023-11-13T11:04:47.182 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109499.log.gz/var/log/ceph/ceph-client.admin.92079.log: 2023-11-13T11:04:47.185 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120456.log 2023-11-13T11:04:47.186 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94675.log: -- replaced with /var/log/ceph/ceph-client.admin.92079.log.gz 2023-11-13T11:04:47.187 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94675.log.gz 2023-11-13T11:04:47.189 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118137.log 2023-11-13T11:04:47.192 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97239.log 2023-11-13T11:04:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118137.log: -- replaced with /var/log/ceph/ceph-client.admin.120456.log.gz 2023-11-13T11:04:47.196 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97239.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118137.log.gz 2023-11-13T11:04:47.196 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97239.log.gz 2023-11-13T11:04:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112224.log 2023-11-13T11:04:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92247.log 2023-11-13T11:04:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115558.log 2023-11-13T11:04:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92247.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.92247.log.gz 2023-11-13T11:04:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113498.log 2023-11-13T11:04:47.207 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.112224.log.gz 2023-11-13T11:04:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115558.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115558.log.gz 2023-11-13T11:04:47.211 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94730.log 2023-11-13T11:04:47.212 INFO:teuthology.orchestra.run.smithi016.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-13T11:04:47.213 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.113498.log.gz 2023-11-13T11:04:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98652.log 2023-11-13T11:04:47.215 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91685.log 2023-11-13T11:04:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94730.log: /var/log/ceph/ceph-client.admin.98652.log: /var/log/ceph/ceph-client.admin.91685.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94730.log.gz 2023-11-13T11:04:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93911.log 2023-11-13T11:04:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93911.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91685.log.gz 2023-11-13T11:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93911.log.gz 2023-11-13T11:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120330.log 2023-11-13T11:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98652.log.gz 2023-11-13T11:04:47.227 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.120330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118999.log 2023-11-13T11:04:47.231 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118999.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120330.log.gz 2023-11-13T11:04:47.232 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93701.log 2023-11-13T11:04:47.234 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 89.3% --verbose -- /var/log/ceph/ceph-client.admin.119700.log 2023-11-13T11:04:47.236 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118999.log.gz 2023-11-13T11:04:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93701.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93701.log.gz 2023-11-13T11:04:47.239 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93659.log 2023-11-13T11:04:47.243 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93659.log: /var/log/ceph/ceph-client.admin.119700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94764.log 2023-11-13T11:04:47.245 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119700.log.gz 2023-11-13T11:04:47.246 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93659.log.gz 2023-11-13T11:04:47.246 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99840.log 2023-11-13T11:04:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94764.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115245.log 2023-11-13T11:04:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99840.log: -- replaced with /var/log/ceph/ceph-client.admin.94764.log.gz 2023-11-13T11:04:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115245.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98568.log 2023-11-13T11:04:47.253 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99840.log.gz 2023-11-13T11:04:47.254 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115245.log.gz 2023-11-13T11:04:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95171.log 2023-11-13T11:04:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98568.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98568.log.gz 2023-11-13T11:04:47.260 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95171.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95171.log.gz 2023-11-13T11:04:47.262 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113564.log 2023-11-13T11:04:47.264 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94078.log 2023-11-13T11:04:47.266 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.113564.log.gz 2023-11-13T11:04:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117933.log 2023-11-13T11:04:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94078.log: /var/log/ceph/ceph-client.admin.117933.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114909.log 2023-11-13T11:04:47.273 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94078.log.gz 2023-11-13T11:04:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114909.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117933.log.gz 2023-11-13T11:04:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119658.log 2023-11-13T11:04:47.280 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96211.log 2023-11-13T11:04:47.280 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114909.log.gz 2023-11-13T11:04:47.282 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114545.log 2023-11-13T11:04:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.96211.log: /var/log/ceph/ceph-client.admin.114545.log: 89.2% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119658.log.gz 2023-11-13T11:04:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99126.log 2023-11-13T11:04:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113340.log 2023-11-13T11:04:47.288 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114545.log.gz 91.3% 2023-11-13T11:04:47.288 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.0.96211.log.gz 2023-11-13T11:04:47.290 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99126.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99126.log.gz 2023-11-13T11:04:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97617.log 2023-11-13T11:04:47.293 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113340.log.gz 2023-11-13T11:04:47.295 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91815.log 2023-11-13T11:04:47.300 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.91815.log: /var/log/ceph/ceph-client.admin.97617.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114206.log 2023-11-13T11:04:47.300 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97617.log.gz 2023-11-13T11:04:47.301 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91815.log.gz 2023-11-13T11:04:47.303 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98174.log 2023-11-13T11:04:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98174.log: /var/log/ceph/ceph-client.admin.114206.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115161.log 2023-11-13T11:04:47.310 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114206.log.gz 2023-11-13T11:04:47.310 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98174.log.gz 2023-11-13T11:04:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110090.log 2023-11-13T11:04:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119784.log 2023-11-13T11:04:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115161.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115161.log.gz 2023-11-13T11:04:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110006.log 2023-11-13T11:04:47.318 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.119784.log: /var/log/ceph/ceph-client.admin.110090.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110090.log.gz 2023-11-13T11:04:47.319 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119784.log.gz 2023-11-13T11:04:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110006.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-13T11:04:47.324 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5%/var/log/ceph/ceph-mds.a.log: -- replaced with /var/log/ceph/ceph-client.admin.110006.log.gz 2023-11-13T11:04:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87335.log 2023-11-13T11:04:47.330 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.87335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98856.log 2023-11-13T11:04:47.331 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110216.log 2023-11-13T11:04:47.333 INFO:teuthology.orchestra.run.smithi016.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.87335.log.gz 2023-11-13T11:04:47.334 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-13T11:04:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97659.log 2023-11-13T11:04:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98856.log: /var/log/ceph/ceph-client.admin.110216.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98856.log.gz 2023-11-13T11:04:47.338 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110216.log.gz 2023-11-13T11:04:47.340 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97659.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97659.log.gz 2023-11-13T11:04:48.603 INFO:teuthology.orchestra.run.smithi016.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.111941.log.gz 2023-11-13T11:04:48.845 INFO:teuthology.orchestra.run.smithi184.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-13T11:04:48.884 INFO:teuthology.orchestra.run.smithi184.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-13T11:04:49.438 INFO:teuthology.orchestra.run.smithi184.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-13T11:04:49.825 INFO:teuthology.orchestra.run.smithi016.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-13T11:04:50.143 INFO:teuthology.orchestra.run.smithi016.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-13T11:04:50.191 INFO:teuthology.orchestra.run.smithi016.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-13T11:04:50.603 INFO:teuthology.orchestra.run.smithi016.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-13T11:04:50.620 INFO:teuthology.orchestra.run.smithi184.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-13T11:04:50.627 INFO:teuthology.orchestra.run.smithi016.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-13T11:04:53.022 INFO:teuthology.orchestra.run.smithi016.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.0.102488.log.gz 2023-11-13T11:04:53.029 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-11-13T11:04:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m6.519s 2023-11-13T11:04:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m23.430s 2023-11-13T11:04:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m2.983s 2023-11-13T11:05:07.228 INFO:teuthology.orchestra.run.smithi184.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-13T11:05:07.232 INFO:teuthology.orchestra.run.smithi184.stderr: 2023-11-13T11:05:07.232 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m20.721s 2023-11-13T11:05:07.232 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m30.645s 2023-11-13T11:05:07.233 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m2.459s 2023-11-13T11:05:07.234 INFO:tasks.ceph:Archiving logs... 2023-11-13T11:05:07.235 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/remote/smithi016/log 2023-11-13T11:05:07.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-13T11:05:10.340 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/remote/smithi184/log 2023-11-13T11:05:10.341 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-13T11:05:14.790 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-13T11:05:14.802 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-13T11:05:14.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-13T11:05:14.812 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-13T11:05:14.979 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-13T11:05:14.991 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-13T11:05:14.991 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-13T11:05:15.001 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-13T11:05:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2023-11-13T11:05:15.062 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2023-11-13T11:05:15.063 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-13T11:05:15.063 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-11-13T11:05:15.063 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 -477us[ -477us] +/- 91ms 2023-11-13T11:05:15.064 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -641us[ -665us] +/- 60ms 2023-11-13T11:05:15.064 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +1195us[+1195us] +/- 70ms 2023-11-13T11:05:15.064 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T11:05:15.073 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-13T11:05:15.073 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2023-11-13T11:05:15.073 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 56 -479us[ -479us] +/- 91ms 2023-11-13T11:05:15.073 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -629us[ -619us] +/- 60ms 2023-11-13T11:05:15.074 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +1204us[+1204us] +/- 70ms 2023-11-13T11:05:15.074 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-13T11:05:15.075 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-13T11:05:15.084 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-13T11:05:15.085 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-13T11:05:15.124 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-13T11:05:15.416 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-13T11:05:15.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-13T11:05:15.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:05:15.799 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-13T11:05:15.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:05:15.903 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-13T11:05:15.920 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-13T11:05:15.948 INFO:teuthology.task.internal:Duration was 1440.350548 seconds 2023-11-13T11:05:15.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-13T11:05:15.966 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-13T11:05:15.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-13T11:05:15.976 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-13T11:05:16.147 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-13T11:05:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-13T11:05:16.558 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-13T11:05:16.559 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2023-11-13T11:05:16.559 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-13T11:05:16.628 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2023-11-13T11:05:16.628 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-13T11:05:16.708 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-13T11:05:16.709 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-13T11:05:16.719 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-13T11:05:16.820 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-13T11:05:16.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-13T11:05:16.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-13T11:05:17.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-13T11:05:17.039 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-13T11:05:17.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-13T11:05:17.146 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-13T11:05:17.247 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-13T11:05:17.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-13T11:05:17.269 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-13T11:05:17.385 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2023-11-13T11:05:17.386 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2023-11-13T11:05:17.448 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-13T11:05:17.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:05:17.503 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-13T11:05:17.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-13T11:05:17.561 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-13T11:05:17.574 INFO:teuthology.task.internal:Transferring archived files... 2023-11-13T11:05:17.574 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/remote/smithi016 2023-11-13T11:05:17.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-13T11:05:17.694 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203/remote/smithi184 2023-11-13T11:05:17.695 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-13T11:05:17.804 INFO:teuthology.task.internal:Removing archive directory... 2023-11-13T11:05:17.805 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-13T11:05:17.815 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-13T11:05:17.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-13T11:05:17.882 INFO:teuthology.task.internal:Not uploading archives. 2023-11-13T11:05:17.883 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-13T11:05:17.893 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-13T11:05:17.894 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-13T11:05:17.900 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-13T11:05:17.955 INFO:teuthology.orchestra.run.smithi016.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 13 11:05 /home/ubuntu/cephtest 2023-11-13T11:05:17.961 INFO:teuthology.orchestra.run.smithi184.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 13 11:05 /home/ubuntu/cephtest 2023-11-13T11:05:17.964 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-13T11:05:17.976 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-13T11:05:18.000 INFO:teuthology.nuke:Checking targets against current locks 2023-11-13T11:05:18.027 DEBUG:teuthology.nuke:shortname: smithi016 2023-11-13T11:05:18.028 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-13T11:05:18.054 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 10:28:47.875774', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOmDbJx+U9GmeFR4wwiA9x34wyOtfCZEFig5NaMF0vaUMTTjcj/SfrLykbTjtRWCfPw3oIGybvtHApcHSzVhUZg='} 2023-11-13T11:05:18.056 DEBUG:teuthology.nuke:shortname: smithi184 2023-11-13T11:05:18.056 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-13T11:05:18.078 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-11-13_10:27:15-fs-wip-mchangir-testing-20231110.052303-testing-default-smithi/7456203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 10:28:47.874354', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJpsKYQqUYDjhRd5uNMPwBwJ509eIRR2VHJ+32ddcRdjFkfLmbJY/0edFsI/2jlUR0AIUdrZhPsuw93QGjPHEZY='} 2023-11-13T11:05:18.110 INFO:teuthology.orchestra.console.smithi016:Power off 2023-11-13T11:05:18.111 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T11:05:18.127 INFO:teuthology.orchestra.console.smithi184:Power off 2023-11-13T11:05:18.127 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T11:05:18.136 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-11-13T11:05:18.137 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:05:18.150 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off 2023-11-13T11:05:18.151 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:05:18.159 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-11-13T11:05:18.172 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-11-13T11:05:22.160 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:05:22.173 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:05:22.271 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-11-13T11:05:22.285 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-11-13T11:05:26.277 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:05:26.287 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T11:05:26.388 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-11-13T11:05:26.397 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off 2023-11-13T11:05:26.490 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-11-13T11:05:26.498 INFO:teuthology.orchestra.console.smithi184:Power off completed 2023-11-13T11:05:26.604 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} duration: 1440.3505475521088 failure_reason: 'Test failure: test_scrub_merge_dirfrags (tasks.cephfs.test_scrub_checks.TestScrubChecks)' flavor: default mon.a-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 mon.b-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 owner: mchangir sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a212fc8f00504ae4ad8e7416eadc0050 status: fail success: false 2023-11-13T11:05:26.604 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T11:05:26.691 INFO:teuthology.run:FAIL